Search Engine Optimization ― Robots, Agents and Spiders - Identifying Search Engine Crawlers If you've been surfing search engine optimization web sites, you've no doubt come across the above being mentioned on many occasions. Crawlers, Agents, Bots, Robots and Spiders Five terms all describing basically the same thing, but in this article they'll be referred to collectively as spiders or "agents". A search engine spider is an automated software program used to locate and[...]