Over the last 4 months I’ve been attempting to track down and solve a problem with ISA Server and an SSL web service we were using. Initial access to the site was fine, but about 3 or 4 pages in, access would become painfully slow and page elements or entire pages would fail to load. This problem was most evident in Internet Explorer, but would also appear in Firefox. It was also most visible on this one particular web service we use, but at times showed up on many other SSL-enabled sites. Last Monday I finally figured out what was happening and solved the problem. It was right in my face the whole time.
Update 14 Jul 2005
It would seem that this entry has become quite popular on the ‘net. It is, in fact, the number 1 hit on Google for anyone searching on "ISA slow SSL", or variations thereof. Therefore, I felt some clarifications are in order. Read on:
- This entry only applies to ISA 2004. I never had this problem on ISA 2000, but I wasn’t using per-user controls under ISA 2000. If you’re having similar issues with ISA 2000, you may want to check out www.isaserver.org or the Microsoft public newsgroups for ISA Server. I don’t have ISA 2000 installed anywhere, so I can’t really offer much assistance on that front.
- Following up on #1, this entry has nothing to do with MS Proxy 2.0. While ISA is the replacement for MS Proxy server, they are two totally different animals. In addition, I haven’t touched Proxy Server 2 since 1999, so I can’t really offer much assistance on that front either. (As an aside, if you’re still running Proxy Server 2, please replace it. It’s really a terrible product).
- This solution really only has an effect if you’re running your ISA Server with per-user access controls. If that’s not the case, I would look at other areas of your configuration. ISAServer.org is a great resource for this type of information.
- Please continue to post comments if you have questions or other problems. I will do my best to help you, but again keep in mind I’m on ISA now. My knowledge of Proxy Server 2 has long since faded (for the better, I assure you).
The background
We run a Microsoft ISA Server 2004 firewall to provide Internet access and security for all computers on campus. ISA Server is known for its strong per-user authentication features, and we make heavy use of them. All outbound connections from our workstations must be authenticated, which is handled by the Firewall Client that is installed through group policy. We also use the web caching feature of ISA, so all of our browsers are configured to use the ISA Server as a proxy through the auto-configuration script.
The symptoms
Initial site access was fine, and load times were quick. Subsequent page loads from the same site would begin to slow down, and some page elements (read: images) would fail to load. Once that happened, future attempts to access the site would fail with timeout errors.
The investigation
When testing this problem, I noticed the problem was much more prevalent in Internet Explorer, so I switched this particular user to Firefox for this one site. Things seemed to work better, but reports of page failures still would come in. Other attempts at resolving the problem, including removing proxy settings from Firefox and disabling caching for the site, provided initial but ultimately false hope. Finally, as a desperation attempt, I added a rule to our ISA server to allow any request to the site without authentication or HTTP content checks. I even bumped this rule to the top of the list, to reduce the amount of time the firewall was involved in the transaction. In short, these blind fix attempts proved fruitless.
It was time to take it up a notch, so I busted out network monitor. Traffic between the ISA server and the site appeared to be normal, with no unusual packets or headers to throw things off. Of course, much of the traffic was encrypted with SSL, so network monitor was next to useless. I then began to monitor the log files from the ISA console while troubleshooting the issue, and initially everything seemed to be normal. Then I saw something.
The realization
I noticed that the connections to the site were staying open, even after all the data for that connection was transferred. After ISA’s connection timeout limit was reached, the connections would then close. Strange. I also began to notice that once I got locked out of this site, I was locked out of any type of Internet access for about two minutes. Two minutes, the exact amount of ISA’s timeout limit. Then the clincher, connection attempts to port 8080 and 1745 on the firewall were being denied from my machine. For those of you not familiar with ISA Server, port 8080 is used for the proxy/cache portion of ISA, and port 1745 is the Microsoft Firewall Client communication port. Every single new connection my machine was making to the ISA server was being denied. I began to pore over my ISA settings looking for something that could explain this. I soon found it.
The solution
ISA ships with a pre-defined per-machine connection limit of 40. Once a machine makes 40 simultaneous connections to the ISA server, any new connections are denied. Because the HTTP and SSL connections were being left open, jumping around various pages on this site quickly brought the number of connections to 40. Raising the per-machine connection limit to 160, 4x the default, as well as reducing the timeout to 60 seconds has solved these problems. I’m not quite certain why connections to this site are being left open after all the data for the connection has been transferred, nor have I determined if this problem is specific to this site. I’ll have to break out my TCP/IP book and do a little RFC reading to get that fully figured out. I’m just glad to have finally solved this problem.
Man you are my god…
Your solution solved the problem.. many thanks
Sorry to be dumb.But are you talking about the settings under outgoing web requests?
The above solution is for ISA Server 2004, which doesn’t have an Outgoing Web Request listener. I assume you’re talking about ISA Server 2000. I haven’t run ISA Server 2000 for about a year, and I hadn’t solved this problem when I was running ISA 2000, so I’m not certain where you would find the client connection limits. Note that this problem on ISA 2004 stemmed from ALL client connections to the server being limited, not just outgoing web requests. I don’t know if ISA 2000 even has such a feature.
Did you deal with the TcpMaxDataRetransmissions TCP/IP Value?
At my box the connection limit is set to unlimited and the time out is set to 120 in the internal networks object web proxy tab under advanced.
There was an article http://support.microsoft.com/default.aspx?scid=kb;en-us;191143 for the older Proxy Servers but the problem is
still the same. We use only plain web proxy feature without having the firewall client installed.
See you…
No, we didn’t need to chnage the retransmit value in the registry, because our connections weren’t timing out, they were being deined by the ISA Server. ISA Server is significantly different from Proxy Server, and I believe ISA Server controls TCP/IP retransmits on its own.
Hm.. we did not have a limitation for connections and the time is 120 seconds, so I am not sure if this what we see is the same like yours.
You will see open connections from clients because HTTP/1.1 is used
and it will have keep alive the connection for a faster re-usage for the next.
You will find informations about this at http://www.faqs.org/rfcs/rfc2616.html section 8.1.3 Proxy Servers.
Cool finally i have more to go on, it partly solved my problems as well requarding one website that was soooo sloooow.
I use the word partly because the website is now browse able, but still slow. Using a direct connection gives a good comparisons in what ISA 2004 is doing whit my speed. Im rather disappointed of some bad “features” of the product and the low flexibility of the configuration options rules ed.
Are there more tricks to boost the speed up as u know of?
Darn its seems to be a client (ie) issue as well, my XP workstation (sp2) works almost fine. But all client workstations (2k sp4, ie 6sp1) still give problems. Wtf has M$ done, Again!
I’m running MS Proxy 2.0 having the same slow SSL connections. Some sites however after a while are running much faster, without a single change in the configuration. In the netherlands two sites to try are: https://secure.postplaza.nl/tracktrace/ and https://www.p3.postbank.nl/sesam/SesamLoginServlet.
I acuse the webhosting side of the connection of having some settings not in order.
I suppose on some SSL servers there are settings for redirected SSL traffic.
Where do you change the timeout setting?
Jacks, in your ISA console, under Server Name -> Configuration -> General, there should be an icon for “Define Connection Limits”. Click there to configuration your connection limits and timeouts.