Magento 1.9.0.1
Hi,
I have since 10 days an increase in 404 errors on webmaster tools, and in the crawl error section i found 150 weird url that make no sense :
http://sitename.com/ (cms page)/index.php/(another cms page or product).
In my memory i haven't change anything, but this has to come from somewhere. I've checked robot.txt (i only deleted prior to that the line User-agent: Googlebot-Image Disallow: ), htaccess , local.xml and friendly url in system/config/web. Looks all good.
Here is a screenshot : http://s15.postimg.org/k82bu8317/image.jpg
In my robot.txt i've got :
Disallow: /index.php/
Disallow: /.css$
Disallow: /.php$
Disallow: /?p=&
Disallow: /?SID=
Disallow: /?___from_store=
Disallow: /___from_store=
Thanks in advance for your help.
Regards,
Marc
I just found out in my access.log that i had an attack to my /downloader folder :
"GET /downloader/index.php?A= HTTP/1.1" 200 2518 "-" "-"65.98.60.146 - - [07/Mar/2016:12:37:45 +0100]
"POST /downloader/ HTTP/1.1" 302 - "-" "-".
And after every 100 attacks i get the URL error that appear :
"GET /bache-tendue/index.php/tendeur-elastique/tendeur-caoutchouc/tendeur-caoutchouc-25cm HTTP/1.1" 404 111180 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.0; +http://ahrefs.com/robot/)"
Hope this helps.
Regards,
Marc
I managed to block the brut force attack, and protect my downloader/ folder with the htaccess file.
But when i look at the access.log, it keeps getting the weird urls and they make absolute no sense. I'm searching for a way to redirect these wrong urls to my homepage via the htaccess but have no idear how. Can anyone help me ?
- [12/Mar/2016:09:10:40 +0100] "GET /bache-tendue/cadre-mural/index.php/revendeur HTTP/1.1" 404 103472 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
- [12/Mar/2016:09:22:59 +0100] "GET /bache-tendue/index.php/revendeur/kit-revendeur HTTP/1.1" 404 103472 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
- [12/Mar/2016:06:43:58 +0100] "GET /info-creation-graphique/index.php/maquette/ HTTP/1.1" 404 103472 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
Regards,
Marc