Differences
This shows you the differences between two versions of the page.
— |
crawler [2014/08/11 13:15] (current) stephan created |
||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== Crawler ====== | ||
+ | ---- | ||
+ | |||
+ | ===== Definition ===== | ||
+ | |||
+ | A **crawler**, also called //robot//, //bot//, //search bot// or //spider//, is a software that analyzes [[website|websites]] automatically and repeatedly in order to gather information and save it to the index. They find websites by visiting directories and following the links from websites. | ||
+ | |||
+ | You can check how often crawlers visited your website in the log file of the internet provider. | ||
+ | |||
+ | ---- | ||
+ | |||
+ | ===== Importance for SEO ===== | ||
+ | |||
+ | Analyzed websites are saved in the search engine's index and will be visited regularly by crawlers and analyzed again. |