Sunday, September 25, 2005

Anatomy of a Search Engine Crawler
(Search Engine Journal)

“When you go to a search engine and perform a search many people don’t understand how those results end up there. Some people may think that sites are submitted while others know that a piece of software finds the pages.

This article explains one piece of that puzzle: The search engine crawler.”