Faceted navigation: index, crawl or block, per facet

Written by

in

Faceted navigation is the single largest generator of URLs on most ecommerce sites, and the decision about it is almost never made. It is inherited from the theme and then discovered two years later in a crawl.

The maths is the problem. Six filters with five options each is not eleven pages. It is every combination anyone can click, and the number runs into the tens of thousands on a catalogue that has a few hundred products in it.

The wrong question

“Should we index faceted URLs?” has no answer, because it treats every facet as the same thing. They are not.

Some facets describe how people actually search. “Waterproof walking boots” is a query with volume behind it, and if you sell waterproof walking boots then that filtered view is a legitimate landing page, arguably a better one than the category above it.

Other facets describe how the warehouse is organised. “Sort by newest”, “12 per page”, “colour: seven shades of grey”. Nobody searches those and nobody ever will.

So the decision is per facet, and it takes an afternoon with somebody who knows the products. That is the actual work and there is no shortcut through it.

It is worth doing because of how much of the web this describes. HTTP Archive’s 2025 Web Almanac found ecommerce software on 19.2% of mobile sites, with Shopify at 25.3% of those and WooCommerce at 44.4%. Faceted filtering ships as standard on all of them, which means the URL explosion is the default state and the tidy version is the exception somebody had to choose.

The three buckets

  • Index it. The facet matches real demand, the resulting page has enough products on it to be useful, and you are willing to write an intro and a title for it. Usually this is one or two facets: brand, or a core product attribute. Treat those views as category pages, because that is what they are.
  • Crawlable but not indexed. Let Google follow the links so it can reach the products, but keep the filtered view out of the index with a noindex. This is the right default for most combinations, and it is the bucket people forget exists. The choice is not binary between “index” and “block”.
  • Not crawlable at all. Sort orders, pagination sizes, session parameters, anything combinatorial. These should not be links Google can follow.

Why noindex rather than robots.txt

This is the mistake I see most and it is worth being precise about.

Blocking a URL in robots.txt stops Google fetching it. It does not remove it from the index, and if the URL is already indexed, blocking it means Google can never read the noindex you have also added, so the page stays in there indefinitely with no snippet.

Worse on a faceted site: blocking the filter paths can cut off the route to products that are only reachable through a filter. You have then made products invisible in order to tidy up a report.

HTTP Archive’s 2025 Web Almanac found meta robots tags on 47.9% of mobile pages but a noindex directive on only 2.4%. The mechanism that should be doing this work is barely used.

The tell that you have a problem

Compare three numbers: products in the catalogue, pages you think you have, and pages Google has indexed. On the sites where these diverge badly, facets are usually most of the gap.

Facets are also a common source of pages nothing links to, because a filtered view can be generated, indexed, and then dropped from the navigation when the filter set changes.

The other tell is in the crawl: filter parameters appearing in URLs that also carry no canonical, or a canonical pointing at themselves rather than the unfiltered category.

What I would do first

Not a facet strategy document. Count them. Export the URL list, group by parameter, and see which facets are producing the volume. On most sites two or three parameters account for nearly all of it, and dealing with just those gets you most of the benefit for a fraction of the argument.

Then decide those two or three properly, and set the rest to the safe default.

The short version

  1. Facets are combinatorial. Six filters is not six pages.
  2. The decision is per facet, not per site. Some match real demand; most describe the warehouse.
  3. Three buckets: index, crawlable-but-noindexed, not crawlable.
  4. The middle bucket is the default and the one people forget exists.
  5. Use noindex, not robots.txt. Blocking makes indexed pages permanent and can hide products.
  6. Count before you strategise. Two or three parameters usually cause nearly all of it.

If a site should be ranking and it isn’t, that’s the work I do. Technical SEO covers the crawling, indexing and architecture layer. If you’re not sure which of several plausible problems is costing you, that’s what an SEO audit is for.

Headshot of Mehul Dedhia, SEO consultant

Written by Mehul Dedhia

I’ve been doing SEO for about fifteen years. Most of my work isn’t building new websites. It’s figuring out why sites that should rank don’t. Local businesses, ecommerce and Shopify brands, affiliate sites, small brands and enterprise. If a site is stuck or has lost rankings, that’s usually where I come in.

Connect with me on LinkedIn