Web Connection
Web Connection, IIS, Kernel Caching, and Low Latency
03/20/2010
08:56:07 AM
2W501WQPO Show this entire thread in new window
Gratar Image based on email address
From:
Brett Baggott
To:
All 
Attachments:
None
I'm going to lay out a hypothetical here and _please_ understand it's hypothetical only because your first instinct is probably going to be to say, "but why would you do that", I wouldn't.

Here goes.

Let's say you wanted to build MarchMadnessRealtimePlayerStats dot Com and so you build a pretty UI (I'm talking web page here) to send to the client and then you fired an Ajax call _per second_ back to the server where all the stats were stored that would update the user's UI with all the second by second stats.

Now, if the backend to this was Web Connection, and you anticipated having thousand of viewers, you'd want to be as efficient as possible. For example, you'd service your Ajax call via a "static cookieless domain" to help improve latency. After all, if your responses are taking 1 sec+, polling per second is kinda pointless.

Now let's say that 99% of your users were viewing their favorite player 99% of the time. Now you start to think, maybe cache can help me out here. After all, at any given millisecond, with thousands of viewers, and with so may uber popular players, you're _very_ likely to have _identical_ request/response streams.

So, all that to say, in the given hypothetical, where you might hope to one day have hundreds of thousands of users but yet still keep the back-end we all love, what can be done to:

a. improve latency (IIS, don't need any networking pointers)
b. improve possibility of kernel caching kicking in
c. improve things I haven't thought of

In the 'nix world, there are several flavors devoted specifically to this issue. We have IIS (well, at least I'm not interested in running WWWC on Apache unless I have to).

I've seen threads here before on performance, and I got a lot of great info out of them. But I've never seen a thread on _absolute best_ latency performance (course I might have missed it). Again, I'm not talking general all-purpose "defrag your hard drive" type things. I'm talking what is the absolute lowest latency environment to get a data-based response from WC over the wire and is it possible to structure that response such that kernel caching will kick in.

Thoughts?

EDIT: I should add that I've been playing around with this in a test environment over local LAN, over the net to our own datacenter, and over the net to EC2 and without even any extreme tweaking I can get to 100-150ms ranges for non-cached hits, which makes the app a doable situation.

However, same HTML test page with URL set to hit my LAMP Nginx server with same data hit just server via PHP/mySQL hums along at 30-50ms.

Also, I've been looking into kernel caching and it doesn't support querystrings. Trying to keep things simple, rather than using a POST for my tests, I was using a GET like so:

example.com/update.ww?player=1001&time=12345678 (i.e. LTRIM(STR(SECONDS()*1000)))

however, given the querystring issue, couldn't you do this:

example.com/player1001/time12345678/update.ww

and get away with having kernel caching fire?