Search Engine Optimization : How directories and search engines function
Jean-Marc Herrelier
Prepare a Spider Friendly Website
The
spider is the software that crawls down your website to collect information on your pages. It will appear once you have submitted
your website and will visit it regularly.
The spider does not view your page like a browser. For instance, it disregards the JavaScript code. And since its signature
is different from that of a browser, the lines of JavaScript you could place at the beginning of your pages to reject too
outdated versions of browsers could also reject the spiders and your website would consequently disappear from search engine
listings.