@gusw
Technical SEO Nov 04, 2026

what does crawling a website actually mean?

a client asked me about crawling a website and i waffled. i've checked the obvious settings and nothing looks wrong. what is it actually for?
1

Leave A Reply

Log in to post an answer.

1 Answer

kamal_serp Nov 05, 2026
gusw
It's Google's bot (and other search engines' bots) fetching your pages, following links from one page to the next to find out what exists. Crawling is just the fetching. Indexing is the separate step where Google decides whether to store the page and show it in results, which is why a page can be crawled and still not be indexed. If you want to see it happening, the Crawl stats report in Search Console (under Settings) shows how many requests Googlebot makes to your site each day.