I mentioned a week or so ago before I left for LA, that I had put up a Sandbox version of the West Wind Web Store on the site so people can play around with the full version of the store including the Admin interface.

I had spent a fair amount of time trying to disable most of the features that are either too destructive or dangerous, by using a DemoMode configuration flag which gets stored along with the rest of the Configuration settings that the West Wind Web Store writes out into its configuration file (web.config actually).

Nothing will bring out problems like making something public and watching your error logs <g>. Actually I was quite surprised how few errors were being generated. The Admin backend is nearly as rigorously tested as the front end o the store as the backend is usually considered to be used by experienced users.

Anyway, one night while I was on the road I noticed that items were disappearing from the store while I was showing a demo to a friend of mine. Of course, demos always bring out the worst… <g> What was happening is that Google got into the Sandbox store and started firing away at the Admin interface. The result was that Google was going systematically through my demo Inventory and deleting each item which is listed from the item listing page. Ooops…

So two things obviously needed to happen: add the Sandbox to robots.txt in the root of the site to make sure that search engines are staying out (a little too late, but for the future) and make sure that people can’t delete items. Items can be added but deletion is not allowed to minimize some of the hard links for front page specials.

For those not familiar with Robots.txt: It’s a file that you can place on your server that good bots will look for to check what content they have access to. It’s a good idea to keep bots out of some content – specifically demos and other stuff that either you don’t want indexed because of load on the system, because the content is not static and meaningless to indexing or simply because the data presented is dummy data as in my case.

# robots.txt for http://www.west-wind.com/

#

User-agent: Mediapartners-Google*

User-agent: *

#Disallow: /wconnect/   # keep robots out of the demo trees

#Disallow: /scripts/

#Disallow: /webstoresandbox/

The mediapartners thing is required (supposedly) if you place Google Ads on your site only. The * means all user agents – ie. all robots.

BTW, no big deal about the data being wiped out by Google. I had built-up my data locally before sending the data up to the server and one quick pass with Sql Data Compare from Red Gate and the database was back to its original demo state. The Red Gate SQL tools are awesome that way...