Open the Page Indexing report in Search Console and the “not indexed” number is almost certainly bigger than it was a year ago. Pages that used to sit comfortably in the index have quietly dropped out. New content gets crawled and then just sits there, unindexed, for weeks.
You are not imagining it, and you are not alone. This has been one of the most consistent complaints across the SEO community for months now.
Daniel Foley Carter published a thorough breakdown of every non-indexed status in Search Console that is worth bookmarking as a reference. It walks through all of them, what causes each one, and how to fix it.
I want to do something different here. Rather than define every status, I want to cover how to actually read the report: which parts of that number are fine and need nothing from you, which parts are Google delivering a verdict on your content, and two audits almost nobody runs that are worth an hour of your time.
The Squeeze Is Real. The Explanation Is a Theory.
Two things worth separating.
The pattern is real and widely observed. Google is dropping previously indexed content more aggressively than it used to, and it is more selective about what it indexes in the first place. Site owners, agencies, and in-house teams have all been reporting the same thing.
The explanation for it is a theory. The popular version says Google is shrinking its index to cut storage and processing costs, and that its AI products depend less on a comprehensive index than traditional search did. That is a reasonable hypothesis. It is not something Google has confirmed, and you should treat anyone stating it as fact with the usual skepticism.
Here is why the distinction matters practically rather than academically.
If you believe non-indexing is a technical fault, you go hunting for a technical fix. You resubmit URLs, you rebuild sitemaps, you tinker with settings, and you get nowhere, because there was never a broken thing to repair.
If you accept that Google’s bar for what earns an index slot has risen, then a page sitting unindexed is not always a bug report. Sometimes it is a verdict. And the response to a verdict is different from the response to a bug. You either make the page worth indexing, or you accept that it is not and stop spending resources on it.
That reframing is the single most useful thing in this note.
Two Different Problems That Look Identical
Before diagnosing anything, work out which of these you have, because they are not the same problem.
Google dropped pages it had already indexed. These pages were in, ranking, maybe earning traffic, and now they are out. This points at a quality reassessment, a loss of trust signals, content that has gone stale or factually wrong, or pages Google has decided it is unlikely to ever serve.
Google refuses to index new pages. You publish, Google crawls, nothing happens. This points at the bar for entry: thin or unoriginal content, mass-generated pages, weak internal linking, or a domain that has not earned enough trust for Google to bother.
The fastest way to tell them apart is the Performance report. Take a sample of the affected URLs and check whether they have any impression history. If they used to get impressions and now sit in the non-indexed bucket, you have the first problem. If they have never had a single impression, you have the second.
Conflating these wastes weeks. The first is a content decay and trust problem on pages you already own. The second is an entry-standards problem on what you are producing now.
Most of That Number Is Fine
The instinct when the not-indexed count climbs is to treat the whole thing as a backlog to clear. It is not. A large portion of that number is Google correctly reporting that your site is configured the way you configured it.
These statuses are usually healthy and need nothing from you:
Page with redirect. The URL redirects, so the destination gets indexed instead. This is Google confirming your redirect works.
Alternative page with proper canonical tag. You pointed a canonical somewhere, Google honored it. Working as designed.
Excluded by noindex tag. If you noindexed thank-you pages, internal search results, or thin archives on purpose, this is the tag doing its job.
Not found (404). A page that genuinely no longer exists returning a 404 is a correct, healthy state. Google keeps requesting long-dead URLs for ages, so seeing 404s here that do not exist anywhere on your live site is normal.
Blocked by robots.txt. If the block is intentional, nothing to fix.
Scan these for accidents rather than working through them as tasks. A section noindexed by a template rule, a canonical pointing somewhere it should not, an important page 404ing after a migration. The accidents are what matter here. The intentional configurations are noise in the report, not work.
The Two Statuses That Are Actually Verdicts
Two statuses are different in kind from everything else, and they are where your attention belongs.
Crawled, currently not indexed. Google fetched the page, read it, and decided against indexing it. There is no technical blocker. It is a quality and value judgment.
Discovered, currently not indexed. Google knows the URL exists and has not bothered to crawl it. This is usually a mix of perceived value and crawl prioritization, and on large sites, weak internal linking is often the driver.
Here is the detail from Daniel’s article that I think matters most, because it changes how you read the trend: “Crawled, currently not indexed” is not exclusively a queue of new pages awaiting judgment. It also catches pages that were previously indexed and have since been dropped.
So a climbing count in that status is not necessarily a backlog. It can be Google steadily revoking index slots from content you already had. If that number is trending up while your published volume is flat, that is a quality signal about your existing library, and it is worth taking seriously.
The causes are the ones you would expect: thin or unoriginal content, near-duplicates, pages where the content never renders for Googlebot, and mass templated or programmatic pages with nothing distinct to say. That last one connects directly to what I wrote about where the doorway page line actually is. Google not indexing your 400 city pages is the same judgment expressed a different way.
The fix is not resubmission. Requesting indexing is a hint, not a command, and it does nothing about the reason. The honest options are to make the page genuinely worth an index slot, consolidate several weak pages into one strong one, or prune it and stop asking. For “discovered,” add real internal links from pages Google crawls often, because a page nothing links to looks unimportant, and it is. If you are signaling that the page is not important, why would Google think otherwise?
The Link Equity Audit Nobody Runs
Your non-indexed report is a list of URLs Google will not serve. Some of those URLs have external links pointing at them. That is link equity arriving at a dead end.
The workflow:
- In the Page Indexing report, open a status worth checking. Start with “Excluded by noindex tag,” then “Not found (404),” then “Blocked by robots.txt.”
- Export the URL list.
- Run those URLs through a bulk backlink check. Semrush’s Backlink Analytics has a bulk analysis view where you can paste a list of URLs, and most other link tools have an equivalent batch feature. Set it to check exact URLs, not domains.
- Sort by referring domains.
What you are looking for, and what to do about it:
404s with backlinks. Someone is linking to a page you deleted. Redirect it to the closest genuine equivalent and recover the equity. If it has no backlinks and no traffic, leave it 404 or serve a 410, which I covered in more depth in an earlier note.
Noindexed pages with backlinks. Worth a second look. If a page has earned real external links, ask whether noindexing it is still the right call.
Robots.txt-blocked URLs with backlinks. This is the worst version, because a blocked page cannot be crawled, so whatever equity arrives there is stuck. It cannot pass anything onward. If meaningful links point at blocked URLs, unblock them.
That last point is worth stating on its own, because it is the most common robots.txt mistake in the wild: robots.txt is not a deindexing tool. A blocked page cannot be crawled, which means Google never sees the noindex tag you put on it. If you want something out of the index, allow the crawl and use noindex.
Three Silent Killers Worth Checking
The X-Robots-Tag you cannot see. A noindex can be set as an HTTP response header at the server, CDN, or middleware layer instead of in the HTML. It will not appear in page source, which is why people hunt for a noindex they cannot find. Check response headers directly with curl -I or your browser’s network tab. This one has quietly killed entire sections of sites.
Robots.txt returning the wrong status code. Daniel’s article says anything unfetchable means your rules are not obeyed, and it is worth being more precise, because it fails in two opposite directions.
First, the obvious question: how does robots.txt ever return anything other than 200? It is fetched over HTTP like any other resource, so it carries a status code, and there are several ordinary ways it goes wrong.
On WordPress and plenty of other platforms, robots.txt is often not a static file at all. If no physical file sits at the root, WordPress generates one dynamically through PHP, and SEO plugins hook into that to add their directives. A dynamically generated robots.txt fails when the application fails, so a PHP error or a database problem returns a 500 for robots.txt along with the rest of the site. Add plain server outages, overloaded hosting, and WAF rules that challenge automated requests, and this is a routine occurrence rather than an exotic one. Google built a robots.txt report into Search Console for a reason.
Now the two directions it fails in.
A 4xx means Google treats the file as missing and assumes no crawl restrictions. If you never had a robots.txt, that is entirely normal and harmless. The problem case is when you do have rules and the file becomes unreachable anyway, say a security rule starts serving 403 to bots. Your blocks silently stop applying, and everything you meant to exclude becomes fair game.
A 5xx does the opposite. Google postpones crawling the site. Search Console’s own “Unreachable: robots.txt” message says exactly that: because it could not confirm what you wanted excluded, it held off crawling. It falls back to a cached copy for up to 30 days, and only after that behaves as if no file exists.
So a broken robots.txt either strips controls you meant to enforce or halts your crawling, depending on how it breaks. The 5xx version is the more dangerous one, and it is invisible unless you look. Search Console has a robots.txt report under Settings showing fetch status for each file it found. Check it.
Pages that are 200 for you and 403 for Googlebot. A WAF, firewall, or aggressive anti-scraping rule can serve you a perfectly normal page while blocking Google entirely. You cannot catch this by loading the page in your browser. Use the URL Inspection live test, which requests as Googlebot, and allowlist verified Googlebot by reverse DNS rather than by user-agent string.
One more worth knowing: if you set up a domain property in Search Console rather than a URL-prefix property, you can filter with a regex to catch staging and dev subdomains Google may have discovered and indexed. Daniel’s article includes a ready-made regex for this, and it is a quick check that occasionally turns up something ugly.
The Takeaway
Treat the Page Indexing report as a diagnostic instrument, not a task list.
Most of the number is your own configuration reflected back at you. Scan it for accidents, not for chores. The two statuses that carry real information are “crawled, currently not indexed” and “discovered, currently not indexed,” and both are Google telling you something about quality and importance rather than reporting a fault you can patch.
Work out first whether you are losing pages you had or failing to get new ones in, because those are different problems. Run the backlink check on your non-indexed URLs at least once, because equity pointing at dead ends is free money on the floor. And check your response headers and robots.txt status, because the silent failures are the ones that persist.
If Google’s bar has risen, and everything visible suggests it has, then the durable response is not more pages. It is fewer pages that each deserve to be there.
