I was on the road today giving a demo at a site where I couldn’t get onto the network. I was going to demonstrate a few performance comparisons of using various ASP.NET cache techniques to a local client here. For quick tests like this I usually use Application Center Test (that comes with VS.NET Enterprise) or Web Application Stress Tool.

I found out today that neither of these tools work if they are not on a live network connection. Both simply fail. I was doing local tests of course hitting LocalHost or my local machine name. When doing this you get a very unceremonious ‘Results were not created’ error. No further messages.

This has actually bitten me a few times before in the middle of a presentation at conferences, but at the time I didn't make the connection between the machine being offline and the stress testing tools not working. I just chalked it up to the flakiness of these tools (which of course isn't flaky, but consistent in its failure <g>).

While testing a local machine like this is probably not the greatest idea it strikes me as a pretty lame tool that can’t deal with an all local network connection. Especially a tool like Application Center Test which is part of a ‘premium’ product like VS.NET Enterprise.

Speaking of Web Stress Testing tools from Microsoft, both WAST and ACT are pretty limited. ACT is fairly powerful with its script driven tests, but once the scripts are generated you can’t easily modify your test by removing requests other than editing the single generated script block.

I do like to use it for quick demos and quick perf tests of individual pages in a Web site though. And it demos well in presentations of course because you can see the results at a glance without waiting for a test to complete. Testing of course is where you can easily run into problems using these tools because giving a presentation it can happen more frequently than not that I’m not online… Arrgh.

WAST is OK but it has a number of bugs and lacks support for SSL. Act is Ok but it only installs with VS.NET which makes it pretty much useless for a production environment or real hard core testing where more than one controller is needed. One can only hope Microsoft will provide something that is free standing and more configurable than either of these tools. Or pay a bucketload of $$$ to some of the third party vendors.

As far as MS goes I would guess that stress testing will get rolled into Team System which if anything only makes this tool more coupled to VS.NET which is probably not a good thing.