Stop guessing why pages are missing from Google. This guide covers how to use the URL Inspection tool to check index status, request indexing, and fix common errors like blocked resources or soft 404s. Includes real error codes and a tactical workflow for agencies and site owners.
The Google Search Console URL Inspection tool is the single most underused diagnostic in SEO. Most site owners open it, see 'URL is on Google', and leave. That is a mistake.
This tool reveals exactly why a page is not indexed — whether it is a server error, a noindex tag, a blocked resource, or a soft 404. Without it, you are debugging blind. In practice, when you run an audit for a site with 10,000 pages and only 1,200 indexed, the URL Inspection tool (via the API) becomes the only scalable way to isolate the 8,800 failures.
A common situation we see: an agency uploads 50 guest posts, waits two weeks, and sees zero indexed. They blame Google. We inspect one URL, find 'Discovered - currently not indexed', and realize the pages have thin content with no internal links. The tool told us the issue. We just had to look.
Paste the full URL into the inspection bar. Ensure it is the canonical version.
Read the large status banner. It is either green (indexed), yellow (pending), red (error), or gray (excluded).
If green, confirm the canonical URL and check the 'User-declared canonical'. Mismatches cause index bloat.
Scroll to the 'Coverage' section. Read the exact error: 'Crawled - currently not indexed', 'Discovered - currently not indexed', 'Page with redirect', 'Soft 404', or 'Blocked by robots.txt'.
Only click this after fixing the issue. Requesting indexing on a broken page wastes the crawl budget.
After 24-48 hours, check the Index Coverage report to see if the error count drops. Repeat for other URLs.
| Status Banner | Root Cause | Recommended Action | Hidden Risk / Failure Mode |
|---|---|---|---|
| URL is on Google Green | Page is indexed. Canonical may differ. | Verify rel=canonical matches the inspected URL. Check sitemap inclusion. | If Google chose a different canonical, your preferred URL may not rank. Inspect the 'Google-selected canonical' field. |
| Crawled - currently not indexed Yellow | Google crawled the page but chose not to index it. Often due to thin content, duplicates, or low perceived value. | Improve content depth (add 500+ words, original research, or multimedia). Build internal links from strong pages. | Requesting indexing without changes will fail. The same error reappears. We have seen this loop 3-4 times before the team fixes content. |
| Discovered - currently not indexed Yellow | Google found the URL via sitemap/link but hasn't crawled it yet. Common on new sites or large sites with crawl budget issues. | Ensure the page has inbound links from indexed pages. Submit a smaller sitemap. Reduce total URLs in the sitemap to under 10,000. | On sites with 50k+ pages, this can be a permanent state. Use the Indexing API for time-sensitive pages. Check this guide on faster indexing for backlinks. |
| Page with redirect Gray | The URL redirects (301/302/ meta refresh). Google may index the target, not the source. | Update internal links to point directly to the final URL. Remove unnecessary redirect chains. | If the redirect is temporary (302), Google may treat it as permanent. Long redirect chains (3+ hops) waste crawl budget. |
| Soft 404 Red | The page returns a 200 status but shows a 'not found' experience (empty page, single sentence, login wall). | Return a proper 404 or 410 status code. Fill the page with valuable content or remove it. | Soft 404s accumulate. A site with 500 soft 404s can lose 30% of its crawl budget. Use the URL Inspection tool to batch-test suspect URLs. |
| Blocked by robots.txt Red | The page is disallowed in robots.txt. Google cannot crawl it. | Edit your robots.txt to allow the path. Then request indexing. The change takes effect after the file is re-crawled (usually 24h). | This often blocks CSS/JS too. Use the 'Blocked Resources' tab in the inspection tool to see exactly which files are blocked. |
Scenario: You publish a guest post on example.com/blog/guest-post-seo-tips. After 7 days, it is not indexed. You run the URL Inspection tool.
Step 1: Paste the URL. Status: 'URL is not on Google'. Coverage: 'Crawled - currently not indexed'.
Step 2: Check the page content. The post has 300 words, one image (no alt text), and zero internal links from the homepage. The site has 20,000 other pages. Crawl budget is tight.
Step 3: Add 700 words of original advice (not rewritten), add alt text, and insert three internal links from high-authority pages on the same domain. Also add a link from the homepage's 'Latest Posts' widget.
Step 4: Click 'Request Indexing' in the URL Inspection tool. Wait 48 hours.
Result: Status changes to 'URL is on Google'. The page ranks for a long-tail term within two weeks. If it had stayed yellow, we would have repeated the content improvement cycle.
The URL Inspection tool is not magic. It fails silently in several scenarios:
1. Infinite redirect loops. The tool shows 'Redirect error' but no details. You must manually curl the URL to confirm. 2. Blocked JavaScript resources. The tool may say 'URL is on Google' but the rendered page is blank because the JS is blocked by robots.txt. The page appears indexed but never ranks. 3. Canonical mismatch on AMP pages. Google indexes the AMP version but ignores the canonical HTML version. The tool shows the AMP URL as indexed, but the desktop version is 'Discovered - currently not indexed'.
For a deeper diagnostic on whether your backlinks are actually indexed, use the check if backlinks are indexed by Google resource. It walks through bulk-checking using the API.
Another hidden issue: the tool does not surface 'Duplicate without user-selected canonical' errors for every duplicate. You must manually check the 'Google-selected canonical' field. If it points to a different URL, your page is not the one ranking.
The manual URL Inspection tool works great for 5 URLs. For 500 or 5000, you need the Indexing API. It allows you to request indexing for up to 200 URLs per day per property (with a verified site). The API returns the same coverage statuses as the web tool.
We recommend this workflow for agencies managing guest posts: export your list of posted URLs, filter out any that are not yet in the sitemap, batch-request indexing via the API, then run a scheduled check 48 hours later to see which ones are still 'Discovered - not indexed'. Those need content fixes before re-requesting.
One operational failure: the API has a 200-URL daily limit. If you have 500 guest posts, you cannot blast them all in one day. Spread requests over 3 days, prioritizing pages with the highest link equity.
Inspect the homepage and top-5 money pages weekly. Track status changes.
After publishing a guest post, inspect the URL 24h later. If status is 'Discovered - not indexed', add internal links immediately.
Before a site migration, inspect 50 critical URLs and record their 'Google-selected canonical'. After migration, reinspect to confirm the new URLs are indexed.
Use the API to batch-check all URLs from your sitemap. Export the ones with 'Crawled - not indexed' and prioritize content improvements.
Check 'Blocked Resources' tab for every URL that is not indexed. A single blocked CSS file can break rendering for thousands of pages.
Verify that <a href="https://developers.google.com/search/docs/crawling-indexing/special-tags">special tags like noindex, canonical, and hreflang</a> are correctly implemented. The URL Inspection tool shows the tags Google found, not the ones you intended.
Open the tool, paste the guest post URL, check the coverage status. If it says 'Crawled - currently not indexed', improve the content and add internal links. Then click 'Request Indexing'. Wait 48 hours and re-inspect. If still not indexed, repeat the content fix.
Google may consider the content thin, duplicated, or low-value relative to other pages on the web. Check for exact duplicate paragraphs, low word count (<300 words), or missing schema. Add original analysis, images, and internal links from strong pages. Request indexing again after changes.
Use the Indexing API (v3) with OAuth 2.0. Send a POST request to https://indexing.googleapis.com/v3/urlNotifications:publish with the URL and type 'URL_UPDATED'. The API returns 'urlNotificationMetadata' with latest status. You get 200 requests per day per property. Use a script to loop through your URL list.
It means Googlebot cannot access the page because your robots.txt file disallows that path. The tool shows which specific directive is blocking it. Fix by editing robots.txt to allow the path (e.g., 'Allow: /blog/'). Wait for Google to re-crawl robots.txt (usually 24h), then request indexing again.
A soft 404 means the page returns a 200 status code but the content signals 'not found' (e.g., empty page, login prompt, or single sentence). Fix by either adding substantial content (500+ words) and a clear 200 response, or returning a proper 404/410 status code. Use the tool to re-inspect after the fix.
Indirectly. You can inspect the page where the backlink lives. If that page is indexed, the backlink is eligible to pass equity. For a direct check, use a tool like the one described in <a href="https://teletype.in/@speedyindex/check-if-backlinks-are-indexed-by-google">this guide on checking backlink index status</a>. It uses the URL Inspection API in bulk.
Discovered - not indexed means Google found the URL (via sitemap or link) but hasn't crawled it yet. Crawled - not indexed means Google crawled the page but decided not to add it to the index. The latter is a content quality issue; the former is a crawl budget issue.
Inspect any URL. Scroll to the 'Indexing' section and look for 'Google-selected canonical'. If it differs from the 'User-declared canonical', Google ignored your tag. This often happens with duplicate or near-duplicate pages. Fix by ensuring the declared canonical is the strongest version and adding unique content.
Indexed does not equal ranking. The page may be in the index but buried due to low relevance, poor content quality, or strong competition. Check the 'Enhancements' tab for structured data errors. Also verify that the page has unique value and is linked from authoritative pages on your site.
Quick calculator. Put in the expected monthly value of a page or link batch and the natural waiting time.