So, at dealnews, we are getting ready to launch a super secret thing (redesign beta preview) that requires us to use some cookie tricks. What we decided to do was to give our users a link to a page that would set a cookie. Then we configured our F5 BIG-IP load balancers to direct those users with the cookie set to a different pool (back end ip/port pairs). Its not an original idea. Yahoo! was doing something similar with their recent front page beta. In fact, that is where I got the idea.
Well, it worked great in testing with mod_rewrite (buying a $40k device for testing is not in the budget right now) on my local machine and on the test servers. We had no problems. However, when we turned it all on in production using the BIG-IP we got some unexpected results. We could go to the URL to set our cookie and our site would change. On the redesigned page, there is another link to switch you back. It simply deleted the cookie and redirected you. Since the cookie was gone, you would be back to the old design, right? WRONG! You were stuck. But, if you did not click on any links on the site for about, oh, 15 seconds, you would get back to the old design. I should say at this point that Safari was the only browser that did not do this. IE, Mozilla and Opera all had this problem.
Hmm, 15 seconds. That is the default KeepAliveTimeout in Apache. I took a chance and disabled keep alive in Firefox (about:config, search for keep, set to false). BAM! It all worked like a charm. It seemed that IE, FF and Opera all keep your keep alive connection open even after the page is done loading. And because the BIG-IP determined which pool you are connected to at connection time, you stayed connected with the new pool rather than switching back. And, as long as you kept clicking around on our site, you would keep that connection open.
As for a solution, we decided to let Apache do the work for us. We didn’t want to tell the BIG-IP to start disconnecting users on every request. Instead, we used a Location directive and SetEnv to set the nokeepalive environment variable only when users access the page that sets/unsets the cookie. Now Apache sends the Connection: close header and the browsers comply. You can see an immediate difference too. Firefox for example has a noticable pause while it closes the connection and makes a new one. I am going to dig around in the BIG-IP manual some more to see if there is anything we can do to make this work at the load balancer layer. But, I don’t really want my load balancers spending CPU cycles on something that will not be an issue once this redesign is launched.