I am trying to find the way to customize the specific page metadata but I cannot find any option to do this. I can customize the NOINDEX, NOFOLLOW from Magento backend as Global change to complete store. But how to change NOINDEX, NOFOLLOW for the specific created page.
For example, www.domain.com/contactus
This page is generated for example from Content > Under Element > Pages and once you created a new page and added content, it will have by default
<meta name="robots" content="INDEX,FOLLOW"/>
I cannot see any way to block search engines from indexing this specific page. Is it possible to do this using any JS Execution on text editor as the source, or using Layout UPDATE XML ?
Add following lines in layout section or layout file for specific pages.
<head> <meta name="robots" content="NOINDEX,NOFOLLOW"/> </head>
Or you can add url in the robots.txt file as well.
I hope it will help you!
I am getting the following error when using the code you mention to use in the Layout section.
Please correct the XML data and try again. Element 'head': This element is not expected. Expected is one of ( referenceContainer, container, update, move ). Line: 1
and regarding your second suggested solution https://docs.magento.com/m2/ce/user_guide/marketing/search-engine-robots.html
how can i tell google that specific url is NOINDEX even it is set to INDEX in the source ?
i mean what about old pages in google that i need to remove from google?
disallow: /abc/ ( this block url from indexing specific page )
Here is conceptual example:
for example above url is in google but no longer available in magento website, and it is set to 404 page and still indexed in google. I have made an page to this end point, but this page also have in source set to INDEX, FOLLOW. How to change INDEX, FOLLOW for this specific endpoint page so google no this page is NOINDEX, NOFOLLOW and i should remove it from google ?
Make sure that you are updating correctly.
Just tried and it is working fine, PFA
yes, doing same as you mentioned
Still getting same error not sure why ?
Just checked, code is not working with magento 2.2 version. There is some bug with this extension.
You can use second approach until you upgrade to latest magento version.
you need to add like you mentioned in robots.txt.
I hope it will help you!
Yes, i am aware of adding disallow on robots.txt but this will not solve the problem. Because links are already on google search engine and to remove them i have to tell google bot to NOINDEX specfic static page, and i have to change it to NOINDEX,NOFOLLOW. Just 1 page. Since XML is not working on this magento version, i have to wait for new update then.