Blocked Due to Other 4xx Issue: Find the Real Code

Google Search Console has a report called Page Indexing, and one of its rows reads “Blocked due to other 4xx issue”. A 4xx is the family of HTTP status codes a server sends when it is refusing your request rather than failing at it. This row is the one that never says which refusal it was.

Every other blocked status in that report names a cause. Not found (404). Blocked by robots.txt. Blocked due to access forbidden (403). This one names a range. Three completely different situations can sit inside it at the same time:

410  https://example.com/jobs/senior-engineer/   deleted on purpose
400  https://example.com/search?q=%%%            malformed URL
405  https://example.com/api/checkout/           POST-only endpoint

Three causes, three different fixes, one row. One of them is not even a problem. Google’s own definition does not narrow it down either. And the tool its documentation sends you to repeats the same non-answer back, unless you know which panel to open.

Where you are seeing this

If you landed here from an error string and are not certain which screen produced it, it is this one.

• Open Google Search Console and pick your property.
• In the left sidebar, under Indexing, click Pages.
• Scroll to the table headed “Why pages aren’t indexed”.
• The row reads “Blocked due to other 4xx issue”. Click it for example URLs.

You can also reach it one page at a time. Paste any address into the search bar at the top of Search Console and the URL Inspection tool reports the same status for that single URL.

Google’s definition is one sentence long

Here is the whole thing. Google’s Page Indexing report documentation defines the status as the server encountering a 4xx error not covered by any other issue type described there, and then suggests debugging the page with the URL Inspection tool.

That is it. No cause, no list of codes, no shortlist of usual suspects. Compare it with the neighbors. The 403 entry explains that the code means credentials were provided and rejected, and points out that Googlebot never provides credentials. The 401 entry tells you to remove the authorization requirement or verify Googlebot’s identity. Both of those hand you a next step. This one hands you a category.

The reason is visible in a second document. Google’s page on HTTP status codes and network errors describes exactly seven 4xx codes by name: 400, 401, 403, 404, 410, 411 and 429. The 4xx range is much larger than that. Everything Google did not write a paragraph about arrives in this row, undescribed, which is why the row exists at all.

It also tells you the consequence, and the consequence is blunt. All 4xx errors except 429 are treated the same way, with Google’s crawlers informing the next processing system that the content does not exist. Not “retry later”. Does not exist.

Which codes actually land here

The report will not tell you, but the API will, and the definition is precise enough to work from.

Google’s URL Inspection API reference publishes the enum behind the Page fetch field. Three of its values matter here.

So the exclusion set is 401, 403 and 404, and nothing else in the range gets special handling. The codes you will realistically meet in this bucket, with what each one means when a crawler gets it:

• 410 Gone. The page was deliberately removed and is not coming back. The most common member of this row by a distance, and covered in its own section below.
• 400 Bad Request. The server could not parse the request. Usually a URL Google discovered from a broken link somewhere, carrying malformed encoding or a query string your router chokes on.
• 405 Method Not Allowed. The path exists but refuses GET. API endpoints and form handlers do this, and they end up in the crawl because something linked to them.
• 406 Not Acceptable. Content negotiation failed. A server configured to require a specific Accept header will refuse a crawler that does not send one.
• 409 Conflict and 451 Unavailable For Legal Reasons. Rarer, and both usually deliberate. A 451 in particular is a geo or legal block that is doing exactly what it was built to do.

The formal definitions of most of these live in RFC 9110, which is the current HTTP specification and worth a bookmark if you are going to be reading status codes for an afternoon.

The 410 case, and why most of this row is fine

This is the part that changes what you do next, so it is worth being careful about the reasoning rather than just asserting it.

Google’s enum defines its not-found state as 404 specifically. It defines the other-4xx state as everything that is not 403 or 404. And 401 holds a separate value of its own. Follow that through and a 410 has exactly one place it can be reported, which is this row.

That matters because 410 is the correct code for a page you meant to delete. A job listing that closed. A product you stopped selling. A campaign landing page that served its purpose. The polite, standards-correct way to say “this is gone and will not return” is 410, and doing the right thing puts the URL into a row labeled as a blocking error.

Google treats 404 and 410 identically for removal purposes, so you gain nothing in ranking terms either way. What you gain is a report that lies to you slightly less, because a 404 lands in a row that says Not found and a 410 lands in a row that reads like a misconfiguration.

So check the intent before you check the config. If the URLs in this row are pages you removed on purpose, the row is not an error report. It is a receipt. Leave it alone. Most guides on this status skip that possibility entirely and send you straight into your server settings, which is a good way to spend an afternoon fixing something that was already right.

The genuine problems in this row are the URLs you expected to rank. Those are the ones to pull out and work on.

A 429 is not in this bucket

A smaller group arrives at this status looking for rate limiting, having deliberately throttled Googlebot and gone hunting for where that shows up.

It does not show up here. Google’s crawler documentation says the 429 status code is treated as a signal that the server is overloaded and is considered a server error, which makes it the one exception to how the whole 4xx family is handled. Everything else in the range means the content does not exist. A 429 means the server is struggling.

The practical consequence is that a 429 to Googlebot reports under Server error (5xx), not under this row. If you throttled the crawler on purpose and cannot find the evidence, you have been reading the wrong line of the table.

There is a second consequence worth knowing while you are here. Because 4xx codes other than 429 have no effect on crawl rate, using a 401 or a 403 to slow Googlebot down does not slow it down. You get the same crawling with your pages dropped from the index. Google’s own documentation says not to do it. If crawl load is a real problem, 429 is the only code in the family that Google reads as a rate signal.

How to get the real status code

The report gives you a bucket. The standard inspection result gives you the same bucket back. Three routes get you an actual number, and they are worth running in this order.

Fetch it yourself first. It takes thirty seconds and it is right most of the time.

curl -s -o /dev/null -w "%{http_code}\n" \
  https://example.com/jobs/senior-engineer/

Then confirm it from Google’s side. Run the URL through Search Console and press Test Live URL. Google’s documentation for the URL Inspection tool says that clicking View tested page shows additional response data including the raw HTML returned, the HTTP headers, the JavaScript console output and any page resources loaded. The HTTP headers panel is where the real status line is. That panel is the answer to this whole status, and it sits two clicks past where most people stop.

Then read your server or edge log if the first two disagree, or if the code makes no sense for that path. A log entry names the handler, which a status code on its own never does.

Why curl works here and lies about a 403

This is the useful difference between this status and its better-documented sibling, and it decides how much work the diagnosis is.

On blocked due to access forbidden (403), your own request is the wrong request. The block is usually identity-based, keyed on IP reputation, bot scoring or request rate, so a fetch from your laptop sails through and tells you nothing. A 200 from curl there means “not a user agent rule”, and people read it as “not blocked”.

The codes in this row mostly are not identity-based. A 410 is 410 for everybody. A 405 refuses GET regardless of who is asking. A 400 is a parsing failure that has nothing to do with reputation. So your fetch usually reproduces Googlebot’s fetch exactly, which is why the cheap route is worth running first here and is close to worthless there.

The exception to watch for is content negotiation. A 406 depends on the headers you send, and curl’s defaults are not Googlebot’s. If you get a 200 on a URL the report insists is blocked, that is the case to suspect, and it is the case where the View tested page panel earns its keep.

Checking it across a whole site

One URL is a diagnosis. The question that decides your afternoon is how many URLs are in this row and whether they share a cause.

The report will not answer that. Google says the example list is limited to 1,000 items and is not guaranteed to show every URL in a status even when there are fewer than 1,000. The count on the row and the URLs you can open are two different numbers.

The URL Inspection API closes most of that gap. You can inspect any URL in your property and read its Page fetch state, at a published quota of 2,000 queries per day per site. Filter on BLOCKED_4XX and you have the real list.

It does not close all of it, and the limit is worth knowing before you build around it. The API’s own reference says only the status of the version in the Google index is available and you cannot test the indexability of a live URL. So the API sweeps and the live test proves. Two jobs, two tools.

Doing that sweep as a script means writing and maintaining a script. Doing it in an agent means typing a sentence.

Copy this prompt

Inspect every URL in my sitemap against Search Console. List only the
ones whose page fetch failed with a 4xx state, group them by the exact
failure state and by the first two segments of the path, and show me
how many are in each group and the oldest last-crawl date.

Grouping by path is the move that does the work. Forty URLs sharing one state and one folder is a single cause, and you will see the shape of it before you open a config file. It is the same approach as running a full site audit that way, narrowed to one field.

How this compares to the neighboring statuses

Five statuses in this report all mean Google did not get a usable page, and they have nothing in common beyond that. Mixing them up is how people go looking for a firewall rule that was never there.

The last row is the one people miss. It is where a deliberate rate limit ends up, and it is the reason a 429 hunt in this row comes back empty.

The contrast worth holding on to is with crawled, currently not indexed. There, the fetch worked perfectly and Google made a judgment about the content. Here, Google never got to make a judgment at all. One is a content problem and one is a plumbing problem, and no amount of rewriting moves this one.

Where the data comes from

Diagnosing this needs one reading the report will not give you, which is the status code itself, on every affected URL rather than on the one you happened to click.

ContextBolt SEO is the tool I build. It puts a read-only Search Console connection and a full SEO toolkit inside the agent you already work in, so inspecting three hundred URLs is a sentence rather than three hundred tabs. The Search Console tools are free and never spend your monthly research credits. It is $35 a month with a 7-day free trial, and there is no dashboard to learn, because every lookup lands on a board that fills itself while you work somewhere else.

On this status the value is entirely in the grouping. One URL tells you a code. Three hundred tell you whether you are looking at a routing bug, a deliberate deletion, or two unrelated problems that Google put in the same row.

What I would actually do

Take five example URLs off the row rather than one. A single URL tells you a code, and five tell you whether there is one problem or several.

Fetch all five with curl and write the codes down. If they are all 410 and they are all pages you remember deleting, you are done. Close the tab. The report is describing your own housekeeping back to you.

If any code surprises you, run that URL through a live test and open View tested page. The HTTP headers panel shows what Google actually received, which settles the question when your fetch and Googlebot’s disagree. A 406 is the usual reason they do.

Then fix the ones that are genuinely wrong, and only those. A 405 on an API path that got crawled is a linking problem more often than a server one. A 400 is nearly always a malformed URL something built and Google discovered.

The thing to resist is treating the whole row as a fault. It is the only status in this report that mixes a mistake and a correct decision in the same bucket, and the first job is to sort one from the other rather than to fix all of it.