cancel
Showing results for 
Search instead for 
Did you mean: 

Prevent Search engines from indexing sitemap file

Prevent Search engines from indexing sitemap file

Hello,

How do we prevent search engines from indexing our XML sitemap file?

Any ideias?

Thanks!

 

 

4 REPLIES 4

Re: Prevent Search engines from indexing sitemap file

Hey @UNLda

You can use the following tag to achieve that:

 

X-Robots-Tag: noindex

 

Was my answer helpful? You can accept it as a solution.
230+ professional extensions for M1 & M2 with free lifetime updates!

Re: Prevent Search engines from indexing sitemap file

Could we add  <META NAME="robots" CONTENT="noindex"> on a XML file?

Re: Prevent Search engines from indexing sitemap file

Hi

 

I think you can add in robots.txt

Disallow: /sitemap.xml

See http://stackoverflow.com/questions/6552060/should-sitemap-be-disallowed-in-robots-txt-and-robot-txt-...

 

Can you tell us why you want to prevent sitemap indexing ? Does it show up in search engine results ?

Maxime Coudreuse, developer of Product Manager for Magento: User-friendly product editor with customizable grid interface and category tree for Magento 1 and 2

Re: Prevent Search engines from indexing sitemap file

To prevent search engines from indexing your sitemap file, you can use the robots.txt file to disallow it. Here’s how you can do it:

  • Update robots.txt File: Add the following line to your robots.txt file:

    User-agent: *

Disallow:  /sitemap.xml

 

This tells search engines not to crawl or index the sitemap file located at /sitemap.xml.

  • Noindex HTTP Header (Advanced Option): Some web servers allow you to set a X-Robots-Tag HTTP header on the sitemap file, which instructs search engines not to index it. If you have access to server configurations, you can add:


    Header set X-Robots-Tag "noindex"

in your server configuration (e.g., .htaccess for Apache) to prevent indexing.

  • Ensure Proper Access Settings: Double-check that the sitemap is accessible to search engines for reading, as disallowing its crawling might prevent them from discovering your site’s content.