You might notice Duplicate without user-selected canonical under excluded in the Google search console under indexing –> pages section.
Page indexing has only two options Indexed and Not Indexed. This coverage error is a part of not indexed, which could be caused by duplicate content or error in canonical URL.
When you see any pages affected under this issue, you have to look at whether the web page is the potential web page to be indexed? If not, this is just a notification, and not an error.
If yes, the affected page to be indexed, then you have to read this article completely to find all the ways to fix this coverage issue.
What Is Duplicate Without User Selected Canonical?
Duplicate without user-selected canonical is a common index coverage issue/error that excludes a web page from indexing; it is when two or multiple web pages have duplicate content without a user-selected canonical URL.
In this situation, search engine like Google gets confusion in choosing the original page to index, and end with excluding the potential web page as duplicate.
It really hurts the SEO. But the best practice includes selecting a proper and relevant canonical tag for every web page can solve this problem.
In fact in much cases, this could be just a condition depending upon the nature of the URL, that you will learn while the fixing process.
How to Discover Web Pages with this "Duplicate Without User Selected Canonical" Issue?
To discover the web pages excluded by duplicate without user selected canonical is to log on to the Google search console.
Once logged in, click on Page under the Index section. This will take you to the visibility of the Page Indexing dashboard.
Following this, click on Not indexed, and the Google Search Console will provide a list of all the index coverage issues on your website.
If your domain has a duplicate without user-selected canonical, then it will appear with the number of web pages affected.
Export all the web pages affected, and start fixing by categorizing on the type of reason.
Reason that causes this page indexing issue?
The reason behind duplicate without user selected canonical are as follow,
- RSS Feed URLs can cause this issue – Sometimes, rss feed of same web page can be assumed as canonical version when you don’t provide user-selected canonical.
- Pagination – In most case, web pages with pagination like https://yourdomain.com/page-1; https://yourdomain.com/page-2, etc. If such page is excluded under this page indexing issue, you do nothing, it’s just a condition.
- The conflict between HTTPS and HTTP protocol of same web page.
- Sitemap with both canonical and non-canonical versions – Always remove the duplicate web page from XML sitemap
- Targeting various countries with similar content – Always use preferred canonical URL, and avoid duplicating the content with only replacing the locations.
- A self-referencing canonical tag without unique content.
- URL parameters like search strings, tracking UTM parameters, session ids, etc.
- www and non-www web pages.
- Syndicated content – Avoid scraping or syndicating the content of your web page in various other content management or micr0-blog websites like medium, bechance, LinkedIn, etc.
- Multi-language and multilingual web pages.
How to Fix Duplicate without User-Selected Canonical Issue?
The moment you export the URL, you have to validate whether the web page is a potential page or a canonical one. If a canonical page is found excluded under this coverage issue, then it’s high time to fix them.
If you find any other web pages with pagination, RSS feed, or HTTP protocol, then be cool. They are the least priority to fix. Still, you have to fix them by the following steps.
Step 1 - Update Canonical Tag for the Original Page:
Canonical tags are an HTML component that should update in the Header of the HTML file. These tags provide information on the originality of a web page to search engine crawlers.
If an original page that has to be indexed is excluded under duplicate without the user selecting canonical, then you have to update the canonical tag of the web page.
The canonical format is as below:
<link rel="canonical" href="https://yourdomain.com/original-page">
Once you update the canonical tag, you have to remove the web page selected by Google as a canonical web page from the Google index by the upcoming steps.
Step 2 - Remove the duplicate page from XML sitemap
When you update the original page’s canonical tag, you should remove the duplicate pages using the 410 HTTP response code (Content is gone).
You can also use the Removal option from the Google Search console.
Once you remove them using GSC or 410 HTTP status code, remove the URL from the XML sitemap.
Crawlers will read and index on a quick note if a web page is found in an XML sitemap. So, it mandates removing all the duplicate web pages apart from the canonical version in the sitemap.xml.
Step 3 - Redirect to Original Page
In some cases, when HTTP is selected as canonical by Google instead of HTTPS, you can redirect the HTTP version of the web page to the HTTPS version.
This makes Google and other search engines understand that HTTPS web pages are the canonical version.
Besides this, try to remove the HTTP version from XML sitemap. So, it will be excluded from indexing.
Step 4 - Mention Google search console to ignore URLs with parameters
Most often, we get web pages with parameters like session id, tracking codes, search strings.
The best way to avoid such web pages affected under this coverage issue or Google making this URL as original, you can inform Google search console to ignore the parameters.
This will ensure Google to only index the clean URL (without any parameters).
The best way to do so is by adding a disallow: /*?* in robots.txt. This provides the instruction to search engine not to crawl any URL with ? followed by parameters
Step 5 - Consolidate Identical or Nearly-identical contents
The biggest reason for duplicate without user-selected canonical is when a website has one of more similar or near-identical content.
The reason could be offering services on multiple location, multiple language, multiple devices.
The best way to fix is to consolidate the complete content into a single webpage, and incase of multiple language use hreflang.
Also, you can delete the duplicate web pages, so your website can have a unique web pages.
Step 6 - Use Noindex Robots Tag for all the duplicate web pages
Robots tag are important HTML meta tag search engine go through before indexing a webpage and following the links on the web pages.
When you find any web page as duplicate, make sure the web page is marked ‘noindex’ in robots tag, and make sure you remove the same URL from XML sitemap.
<meta name="robots" content="noindex, follow">
Step 7 - Validate Fix
Once, you finish all the procedure to clear the reason behind this issue, you can use Validate Fix option from the Google Search Console.
This makes a call for Googlebot to check the fixation of this issue and clear the web page from duplicate without user-selected canonical dashboard. It usually takes up to 28 days from the appeal date.
Conclusion
Duplicate without user-selected canonical is the coverage issue status that arise when Google found duplicate web pages, and exclude a web page from indexing when the user-selected canonical is not found.
The best way to fix this issue is to provide canonical URL for all the web page, noindex duplicate web pages, consolidate all similar contents, use robots.txt to block URL with parameters.
If you still find it tedious in fixing this issue, you can avail our technical SEO Services to solve it.
