[wellylug] Squid, wpad.dat auto-configuration and the Air NZ booking service - RESOLVED

[Daniel Pittman]

David Harrison <david.harrison at stress-free.co.nz> writes:
> On 12/04/2009, at 11:31 PM, David Harrison wrote:
>>
>> I was just wondering if anyone had come across a problem when using
>> proxy auto-configuration files (wpad.dat) with a Squid proxy and the
>> Air New Zealand booking service?  e.g. If you go to
>> airnewzealand.co.nz, and in the left hand panel select a
>> destination+date and press search.
>
> I resolved this problem with Squid and the Air New Zealand booking
> service during the week.
>
> The problem it turned out wasn't with the wpad.dat auto-configuration
> file but a network connection bottleneck on the Squid server that only
> became apparent when the flightbookings.airnewzealand.co.nz site was
> accessed.

How did you discover that was the problem?

> The issue was resolved by tuning the underlying Centos TCP/IP settings
> on the Squid server by adding the following to the /etc/sysctl.conf
> file:

> ----------------------------------------
> # Decrease the time default value for tcp_fin_timeout connection
> net.ipv4.tcp_fin_timeout = 15
>
> # Decrease the time default value for tcp_keepalive_time connection
> net.ipv4.tcp_keepalive_time = 1800

There are extremely unlikely to make a difference to your problem,
whatever it is.

> # Turn off the tcp_window_scaling
> net.ipv4.tcp_window_scaling = 0
>
> # Turn off the tcp_sack
> net.ipv4.tcp_sack = 0
>
> # Turn off the tcp_timestamps
> net.ipv4.tcp_timestamps = 0

You just made your connection extremely unreliable, and dropped
significant performance.  Generally speaking you really *DON'T* want to
turn off the last ten years of TCP development.

These, at least, /might/ have made a difference if there is a faulty
router or firewall between you and the remote system, however.

Window scaling is the most likely candidate: there are some broken
systems out there that assume it must be zero, or ignore it in their
connection tracking, and get out of sync.

Most of the world, however, got with the picture.  Oh.  Um, you didn't,
I imagine, note what it used to be, eh?  Linux defaulted to 7 for a
while, which caused more grief when dealing with broken sites than the
new default of 2, since the former caused a total stall while the later
still crawls on in the face of a broken intermediate system...

SACK has been default-on widely enough around the world that it probably
isn't the problem and timestamps...


I am surprised you didn't also fiddle with ECN, the most likely
candidate for breaking the connection.

> # Increases the size of the socket queue (effectively, q0).
> net.ipv4.tcp_max_syn_backlog = 1024
>
> # Increase the tcp-time-wait buckets pool size
> net.ipv4.tcp_max_tw_buckets = 1440000

These made no difference to your problem.  They don't even have anything
to do with anything in the vague area of your problem.

> # Allowed local port range
> net.ipv4.ip_local_port_range = 16384 65536

This is so unlikely to have been related that you may as well not
bother.


> These configuration settings were from the following site;
> http://www3.wiredgorilla.com/content/view/220/53

What a disastrous site.  It contains mostly bad advice cargo-culted from
who knows where, listed without justification or explanation.  Plus,
they make TCP work vastly worse, making the Internet worse for everyone.

In any case, most of what they suggest is relevant to running a *server*
system, not making client connections, and probably has only made things
worse.

> After this TCP/IP tuning the Air NZ booking site works flawlessly
> through the Squid proxy using the wpad.dat file.

If you take my advice you will enable all those features again, then
test the candidate features in the order of:

1. Set (only) window scaling to zero, test the site.
2. Set (only) SACK to disabled, test the site.
3. Set both window scaling to zero and SACK disabled, test the site.

Then work out which of those features actually causes the problem and,
ideally, report that to the technical people at AirNZ who have a broken
Internet connection.

Regards,
        Daniel