SEO
Crawl
When Google's bots visit and read your website pages.
💡
The Tradie Take
Googlebot is like the apprentice who walks through your workshop to take stock of everything. He opens every cupboard, reads every label, and reports back on what you've got. If you lock a cupboard (block a page), he can't see it. If your workshop is a mess, he gives up early. Make it easy for him to look around.
Definition
Crawling is when Google's automated bots (called Googlebot) visit your website and read its pages. They follow links from one page to another, discovering what content you have. If Google can't crawl your site properly, it can't index your pages, and you won't show up in search results. It's the first step in getting found on Google.
How It Works
- 1 Googlebot starts with a list of known URLs from previous crawls and sitemaps
- 2 It visits each URL, reads the content, and follows links to find new pages
- 3 The information is sent back to Google's index for processing
- 4 Crawl budget = how many pages Google will crawl on your site in a given timeframe
Tips for Tradies
- Submit a sitemap to Google Search Console so Google knows all your pages
- Fix broken links and errors - they waste your crawl budget
- Use robots.txt to tell Google which pages NOT to crawl (admin pages, thank-you pages)