I tried adding this code:
<action method="setRobots"><value>noindex, follow</value><name>robots</name></action>
in the layout update XML of the 404 page but still when viewing the page source the page has "index, follow" meta tags in it.
Can someone show me the right way to add noindex ,follow in the 404 page?
Have you cleared cache to make sure the latest changes from your file are being rendered on the page?
If you are adding this xml in the Magento admin then use following
<reference name="head"> <action method="setRobots"><value>NOINDEX,FOLLOW</value></action> </reference>
and you must clear caches after doing this. Then it should work for you.
I added the code in the 404 not found page in its layout update XML area ,save the page and cleared the cache .
But after checking the page source of the 404 page , it still shows index, follow.