Crawl and Indexation
We crawl the site the way a search engine would, checking status codes, redirect chains, robots directives, and which pages are actually indexed versus which ones exist. For sites under six months old, it's common to find pages that were never submitted for indexing at all, or pages accidentally blocked during development.