robots.txt generates outside the 'pub' web root

[comport3]

Hi,

We are seeing an issue in Magento 2.0.0 - 2.0.4 where a robots.txt file is generated from Stores > Configuration > General Design > Search Engine Robots > Reset to default > Save Config.

Apart from the very strange UI of 'reset to default & Save' instead of 'Generate default robots.txt'...

This file is generated in <base path on disk>/robots.txt INSTEAD OF <base path on disk>/pub/robots.txt

Which means hitting <SITE URL>/robots.txt doesn't work out of the box.

Easy to workaround with a symlink from pub up a level, or a webserver location block, but doesn't work out of the box.

[Dmitry_Fedyuk]

It is normal, the robots.txt file should be located in the Magento 2 root folder: https://mage2.pro/t/1221

[comport3]

Dmitry,

It's possible this works as expected under the Apache webserver, but as Magento2 advertises full NginX support, there is no provision within the sample included NginX configs to correctly serve the robots.txt from the $MAGE_ROOT where the robots.txt gets generated, as the actual NginX root is defined as 'root $MAGE_ROOT/pub;'.

So if I symlink from /pub up one level it will work, but not otherwise - unless I add an additional location block to match it.

[Dmitry_Fedyuk]

Maybe... I do not use the standard Nginx out-of-box configuration and have written my own:

https://mage2.pro/t/20

https://mage2.pro/t/24

https://mage2.pro/t/23