Search engines like well organized websites. Especially on large websites with many sub-pages it's important that crawlers find all pages quickly and don't miss anything.
This means that the most important pages should be linked already on the home page. Files like the sitemap.xml and the robots.txt also make it easier for crawlers to crawl a website.