Imagine you are working on your website or a client’s website, and you notice a technical error in Google Search Console as Page Crawled – currently not indexed, what would you do to fix it?
Crawled – currently not indexed is an important Google index coverage issue that must be fixed depending on the priority. Various factors contribute to this issue, with different priority levels for improvement.
Many webmasters take this coverage issue on the higher side, but the impacts are high regarding crawl budget, indexing, and ranking.
The goal of every SEO professional will be to index the web pages rapidly and rank them on the search engine results page (SERP).
But what if your web pages are not indexed?
Your quality content, research, and many more efforts will go in vain.
This index coverage issue is one of the vital checklists we usually audit in our technical SEO process.
At the end of the blog, you will come to know what causes the “Crawled – currently not indexed” error and how to fix it smoothly, It is not as complicated as you think,
Let’s dive in.
What Is Crawled - Currently not Indexed?
This Google index coverage issue states that Googlebot could
- Discover your web pages,
- Crawl the complete web pages (up to 15 MB),
- Render the contents of the web pages.
However, Google may fail to index the web pages for many reasons. We will be looking at both the reason and solution part later in this content.
As per Google or other search engine ethics, indexing is their choice. It is not mandated for Google to index your web pages post request indexing through URL inspection in the Google search console.
Post rendering, Google undergoes multiple checklists as per their algorithm and ranking factors to set a canonical URL.
If a web page doesn’t meet Webmaster quality guidelines, there is a high chance of Google not indexing or de-indexing the page.
How to Find the Pages That Are Crawled - Currently Not Indexed?
Google Search Console, a free technical SEO tool by Google, helps us identify these errors. To check what web pages come under crawled – currently not indexed, follow these steps:
Step 1 – Login to Google Search Console and choose your property.
Step 2 – On the left side, you will be provided with multiple options, click on pages that comes under index.
Step 3 – On page Indexing dashboard, Click Not Indexed, you will see list of index coverage issues on your website.
Step 4 – If your website has “crawled – currently not indexed” , error then it will appear with the number of web pages affected.
Step 5 – Just click on the option crawled – currently not indexed. You will be provided with the web pages that are excluded for this reason.
What are the Reasons for Crawled - Currently Not Indexed?
Just like the other Google index coverage issue discovered – currently not indexed, this coverage issue is backed by multiple reasons.
Sometimes, Google might take action on the web pages that are indexed earlier.
Here are some reasons that can stop your web pages from indexing after discovering, crawling, and rendering.
- Excluded by Robots tag
- Keyword cannibalization
- Crawl errors
- Duplicate content
- Thin and low-quality content
- Improper internal linking and orphaned page
- Backlinks from spammy websites
- Google Penalty
Usually, when a web page is marked noindex under the robots tag, your web page can get this Google index issues.
“Noindex” robots tag command informs search engines to exclude from indexing.
Generally, details or pages with internal confidential content, tags, terms and conditions, privacy policy, etc., will come under noindex.
If you find any other pages that are important to be indexed in search engines, then you have to change noindex to index.
When two or more web pages have the same search intent, it is termed as keyword cannibalization. Google will get confused about which page to rank, so it excludes all the pages with the same intent from indexing.
Keyword cannibalization is an act of competition between internal web pages, which holds a high chance of web pages being excluded under crawled – currently not indexed.
Sometimes, your web pages won’t be indexed due to the crawl budget. To manage the crawl budget, you should check the crawl errors in the Google Search Console.
As per the above image, go to settings in Google Search Console to find Crawl Stats.
Click on Open Report, you will be getting all the details of the status of HTTP response.
Here, status codes like 200 and 304 are good URLs. Sometimes, you can be fine with redirects like 301 and 302.
The error part comes when you have webpages with HTTP status code 404 (not found), other internal issues 4xx, and server issues 5xx.
These issues consume a lot of crawl budget, and it takes a lot of time to crawl a new valuable web page, as the crawl budget would have vanished.
4.Duplicate Content:
When you have the same content across different web pages of your website without a proper canonical tag, it becomes duplicate content.
In this case, Google will tag all the duplicate web pages as crawled – currently not indexed.
Our best advice would be to avoid using the same content for any other web pages and show the uniqueness and relevancy of the web page.
Still, if you require to copy the same content (franchise, multiple-location businesses), use proper canonical tags to avoid exclusion from indexation.
5.Thin or Low Quality Content:
Often, pages with thin or low-quality content are mapped under crawled – currently not indexed.
Thin content is when you have very low content (less than 600 words) on a web page.
Low-quality content is when your web pages are built by AI, auto-generated, cloaking, keyword stuffing, irrelevant intent in the web page, etc.
6.Improper Internal Link Planning and Orphan Pages
Your web pages would have been discovered and crawled by search engines through sitemap. But, if the web page is not referred from internal web pages, then the chance of indexing goes to zero.
You often see orphan pages in the coverage issue crawled-currently not indexed.
To avoid this, plan your site architecture and internal link each relevant page.
How to Fix Crawled Currently Not Indexed Issue?
There are multiple ways to fix this issue, and we provide some priority levels with different types of URLs.
Looking at the recovery, you should do this to check when Googlebot crawled this web page the last time.
As shown in the above image, please place the URLs (that are excluded from indexing post crawling) in the URL inspection of Google Search Console.
You will get these details, as the URL is not on Google with the exact status on coverage crawled currently not indexed, with the last crawl date.
Once you have done that, then you should follow the upcoming ways to fix this coverage issue,
1.Improve the Content Quality
High Priority
We find the thin or Low-quality content error quite often when we do SEO audits. This error can even de-index the web pages that are indexed in earlier episodes of crawling.
The following actionable steps helps to fix thin or low quality content:
- Create high-quality and original Content. It should provide value to the users.
- Expand the thin content by adding more relevant information.
- The content should provide complete information that addresses the intent of the user.
- Optimize the content structure, formatting and the usage of keywords to offer better user experience
- Avoid Over-optimization. Use relevant keywords naturally.
To enhance a better quality of the content, answer the following questions after completing the content. If not, optimize the content based on the content and quality questions offered by Google Search Central.
Always focus on quality content. Your content should be a solution to your audience query. So, never engage in building content with
- AI tools
- Auto Generating Programs
- Bot contents
- Copy Content
- Cloaking Contents
Just update the content of the web page. If you find this a tedious process, you can get our content writers to work on that.
2.Address Technical Issues
The technical issues on the website are also the potential cause of the Crawled – Currently not Indexed issue. So, it is needed to identify and address Technical Issues on the website.
Some of the key Technical Issues are,
- Crawl Errors – Make use of Google Search Console to identify crawl errors. Verify whether the server is responding correctly and there are no broken links.
- Robots.txt and Meta Tags – Make sure that the robots.txt file is not blocking any important pages from being crawled. Also check there is no “noindex” meta tag on the page.
- XML Sitemap – The sitemap should be accurate and up-to-date. It should be updated regularly and submitted to the Google Search Console to get crawled and indexed. Make use of the “Request Indexing” feature in the GSC for specific URLs.
3.False Positives
Low priority
This case is when your Google search console says that the web page is crawled-currently not indexed. But, your web pages are visible on the search engine result page.
How that is possible, and how to check a web page is a false positives of not indexed?
Take, for example, you need to check the indexation station of the webpage https://the7eagles. com/discovered-currently-not-indexed/
Take this URL and place it in the search engine bar with “site:” placed before the URL.
If you see your web page visible in SERP (search engine results page), then do nothing. This is known as false positives.
Give the Google search console some more time to solve this by itself. The good news here is your web page is already indexed.
You can request manual indexing in Google Search Console to speed up the process. Still, Google takes the decision.
4.URLs with /feed/
Neither do URL inspection and manual requests for indexing nor remove the URL.
These webpages will be navigated to XML documents that are important for the RSS feed and not for the Google index.
5.Optimize Site Structure and Internal Linking
Optimize the structure of the website with a clear and logical hierarchy.
Use categories and subcategories which helps Google to understand the relationship between pages.
Add internal links to the page from other relevant pages on the site.
This helps search engines to discover and understand the context of the page.
6.URLs with Pagination
Low Priority
This is quite a common issue of crawled – currently not indexed. Frequently seen in e-commerce or websites with a high number of pages. Search engines crawl large pages of websites with the help of pagination.
To resolve a number of pagination URLs from the excluded section, you should build with proper architecture and internal links so crawling becomes feasible for search engines.
Besides this make sure, your paginations are free from noindex tag (when you have e-commerce or news website), and contains self-referring canonical tag.
So, when you see these types of URLs under crawled – currently not indexed, then do nothing. Go to bed, just relax.
If you still think on saving crawl budgets from pagination, then block the pagination URLs from Robots.txt
7.URLs with Redirects
High Priority
This is an important issue that should be given high focus. SEO professionals usually do a redirect when creating a destination page instead of an existing page.
This helps transfer both traffic and authority to the destination page. Still, the redirected web page rents in the Google Index.
Redirect URLs consume more crawl budget. That, too, when a page has multiple redirects, the consumption of crawling increases.
We advise you not to change the URL of any web pages. If you are changing the URL for SEO optimization, then you should use the 410 command to redirect the URL and start indexing the destination URL.
Yes, you will lose authority that has been built into the existing page. Yet, that’s the best way of resolving this use, as the redirecting URL will still be in the index database for crawling with no content.
So, it will keep the redirecting URL or destination URL in crawled – currently not indexed.
Just use 410 command to the redirecting URL and move on, this issue will resolve soon.
8.Control Duplicate Content
The Duplicate content issue can be controlled by the following steps,
- Each page should offer unique insights or information. Utilize Canonical Tags to specify the preferred version of the page.
- Maintain a consistent and clear URL Structure.
- If you find any two pages having similar content, consider merging the pages into a single comprehensive page.
Looking to Fix Crawled – Currently Not Indexed? You’re just away from a mail!!!
Conclusion:
- Crawled – currently not indexed is one of the Google Index coverage issues that affects the indexing of any web page.
- This can harm your potential page from indexing.
- This issue can be caused by various factors like thin content, duplicate content, improper internal links, excluded by robots tag, etc.
- Focus on providing valuable content, and don’t duplicate, these are the best way to fix this google coverage issue.
- If you find this process tedious, you can hire us for technical SEO services.
Frequently Asked Questions (FAQs)
Google may crawl a page but not index for the following reasons,
- The page has low-quality content
- The content is considered as duplicate or thin content
- The page has technical issues or
- The Page is not being relevant enough for Indexing
Crawled-Currently not Indexed – Google has visited the page, but not yet added it to the index.
Discover-Currently not Indexed – Google knows about the page but hasn’t crawled it yet.
To fix “Crawled – Currently Not Indexed,”
- improve the page’s content quality,
- ensure it provides value,
- check for technical issues and
- use Google Search Console to request indexing.
