What does "Crawled - currently not indexed" mean in Search Console?

What does "Crawled - currently not indexed" mean?

The status "Crawled - currently not indexed" in Google Search Console means that Google's automated software, known as web crawlers, has visited and downloaded the content of a page on your site. This process is called crawling. After crawling, Google attempts to process the page's content, including text, images, and videos, and store this information in its vast database, which is known as the Google index. This second step is called indexing.

However, when a page is "Crawled - currently not indexed," it signifies that while Google successfully crawled the page, it chose not to add it to its index. Google states that indexing is not guaranteed for every page it processes. Consequently, a page with this status is not eligible to appear in Google Search results or other Google services like Discover.

Why would Google crawl a page but not index it?

Google may crawl a page but choose not to index it for several reasons. Google documents that it does not index pages that are blocked by a robots.txt rule or a noindex tag. These directives explicitly tell search engines not to include the page in their index. Additionally, Google may not index pages that it identifies as duplicates of other pages on your site, or pages that are considered inappropriate for indexing, such as variations of a page with different filters applied.

Other common indexing issues Google identifies include low-quality content on the page, or a website design that might make indexing difficult. Google's indexing process involves determining if a page is a duplicate of another page on the internet or canonical, meaning the preferred version among similar pages. Ultimately, Google states that indexing is not guaranteed, even if a page has been successfully crawled.

How does Google find pages to crawl?

Google Search operates as a fully automated search engine, using web crawlers to regularly explore the web and find pages to add to its index. For Google to crawl a page, it needs a way to discover it. This typically means the page must be linked from another known page on the web, or it must be listed in a sitemap.

Googlebot, the primary crawler, uses an algorithmic process to determine which sites to crawl, how often to visit them, and how many pages to fetch from each site. While Google can usually discover most pages on a site if they are properly linked internally, sitemaps provide information about the pages, videos, and other files on your site, helping search engines like Google crawl your site more efficiently. However, a sitemap helps discover URLs but does not guarantee that all items within it will be crawled and indexed.

When do you need a sitemap?

While a sitemap can help search engines discover URLs on your site, it does not guarantee that all the items listed will be crawled and indexed. Google states that in most cases, a site will benefit from having a sitemap. You might particularly need a sitemap if your site is large, making it harder for Google to find all pages through links alone.

Another scenario where a sitemap is beneficial is if your site is new and has few external links pointing to it, as these links are a primary way Google discovers content. If you use a Content Management System (CMS) such as WordPress, Wix, or Blogger, it is likely that your CMS has already generated a sitemap for search engines, and you may not need to take additional action.

How can you check why a specific page isn't indexed?

To understand why a specific page has the "Crawled - currently not indexed" status, you can use Google's URL Inspection tool in Search Console. This tool provides information about Google's most recently indexed version of a specific page, not the live version on the web. It also allows you to test whether a URL might be indexable.

If the tool reports "URL is not on Google," it means the URL cannot appear in Search results. The reason for this status can generally be found within the tool's report. If indexing is disallowed, the tool will show the reason, confirming that the page will not appear in Google Search results.

It's important to note that if your page is blocked by robots.txt, the "Indexing allowed?" status might still show "Yes" because Google cannot see and respect any noindex directives when robots.txt prevents crawling. To explicitly block indexing, Google recommends using the noindex tag or directive on your URL. The live URL test within the tool only confirms if the Google-InspectionTool can access your page for indexing.

What can you do to get a page indexed?

To encourage Google to index a page that has been crawled but not indexed, you should first address the underlying issues. Google states that it needs a way to find a page to crawl it, so ensure the page is linked from a known page on your site or included in a sitemap. If the page is intentionally blocked, remove any robots.txt rules or noindex tags that prevent indexing.

If the issue is related to content quality or website design, as Google mentions, work on improving these aspects. After you have fixed all instances of a specific issue on your site, you can use Google Search Console to ask Google to confirm your fixes. This signals to Google that the page is ready for re-evaluation and potential inclusion in the index.

Common questions