The first time you export a crawl of a real site you get a spreadsheet with forty columns and several thousand rows, and the honest reaction is to close it.
I have been doing this for fifteen years and I still read almost none of it. Here is the order I actually go in, which takes about twenty minutes and answers most of the question before the detailed reading starts.
Before you crawl: give it the sitemap
This is the step that changes the output more than any setting in the tool.
A crawler starting from the home page follows links. By definition it will never reach a page nothing links to. So on a site with eight thousand orphan pages, a plain crawl reports zero orphans and you get a clean bill of health that is completely wrong.
Feed it the XML sitemap as a second URL source and let it compare the two lists. Anything in the sitemap the crawl never reached is an orphan. On one marketplace that comparison produced 8,317 pages nothing on the site linked to, out of about 9,220 indexed. None of them would have appeared otherwise.
The five things I look at, in order
- 1. Total indexable URLs versus what you think you have. One number against the client’s expectation. If they say “about two hundred pages” and the crawl says 1,630, stop and find out what is generating the difference. That gap is often the entire finding and you can have it in ninety seconds.
- 2. Status codes, grouped. Not each one. How many 200s, 301s, 404s, 5xx. What I want is the shape: a site with 68 redirects out of 174 URLs has a redirect problem; a site with three has housekeeping. Any 5xx at all gets looked at immediately.
- 3. Indexability, and why not. Every crawler has a column for this. Sort by the reason. Noindex, canonicalised elsewhere, blocked. What you are looking for is a page you care about sitting in a bucket you did not intend.
Two figures worth carrying while you read that column, both from the 2025 Web Almanac: a canonical tag is present on just 67% of mobile pages, and 10.3% have an invalid element inside the <head>. The second one matters more than its position on any issues list, because the head closes at the first invalid element, so a canonical or a meta robots sitting below the break is not being read at all, while your crawler cheerfully reports it as present.
4. Word count, sorted ascending. Crude and I use it every time. It surfaces the empty templates, the placeholder pages, the category pages nobody finished. On one platform this showed 466 of 812 pages under the threshold, and the useful part was not the count, it was that they were all the same page type.
Resist benchmarking that against the web. The 2024 Web Almanac put the median mobile home page at 364 words, so almost anything you find will look respectable by comparison. The median page on the internet is not your competition. The four already ranking for your query are, and they are not median or they would not be there.
5. Crawl depth. How many clicks from the home page. Anything important sitting at depth five needs an explanation, and usually the explanation is that it is only reachable through pagination.
That is it for the first pass. Titles, headings, meta descriptions and alt text: all of that is real and none of it is where I start, because none of it explains why a site that should rank does not.
The habit that saves the most time
Look for numbers that match.
When two findings report the same count, they are describing the same pages. 8,317 orphans and 8,317 bad canonicals are not two problems. 1,347 pages missing a canonical and 1,347 missing a meta description are one page-set with one template behind them.
I check for this deliberately now, before reading anything in detail, because it collapses a long report into a short list of causes. A four-hundred-row report is usually six to twelve actual problems.
What the crawl will not tell you
It will not tell you whether a page should exist, whether the home page says what the business is, or which of the fifty flagged issues is the one costing money. Those need somebody to open the site and form an opinion.
Which is why I run the crawl in the background and start by looking at the site myself. The crawl is corroboration. It is not the diagnosis.
The short version
- Feed the crawler your sitemap or it cannot find orphans, and will report zero.
- Compare total indexable URLs to what the client believes. The gap is often the whole finding.
- Read status codes as a shape, not row by row.
- Sort indexability by reason and look for pages you care about in the wrong bucket.
- Sort word count ascending. Crude, fast, reliable.
- Matching counts mean one cause. Check for them before reading anything closely.
- The crawl corroborates a diagnosis. It does not make one.
If a site should be ranking and it isn’t, that’s the work I do. What I check first is the sequence the crawl runs alongside. If you’re not sure which of several plausible problems is costing you, that’s what an SEO audit is for.
