SEO Indexing

[josh_whelan]

Hey everybody,

I am running into a few indexing issues with my Magento 1.9.3.6. SEO.

They might all be related and I'm hoping somebody has some idea what's going on.

1. My category pages aren't showing up in Google's index.

I've optimized category meta tags, and they're reflected on the page, but they're not indexed.

Product pages do appear to be indexed as normal.

2. My category search pages are showing up in the index.
I think I need to noindex them, but I can't seem to get that to work.

I have noindex code in the robots.txt file & in the local.xml file, but it's not working.

3. My listing URLs are showing up in search results. I've got URLs with ?s and lists indexed in Google. I think I need to block them.

Any ideas?

[Samith]

Looks like you haven't submitted category urls to Google. Did you set up a Google search console account and submitted category sitemap? 

[gelanivishal]

Hello @josh_whelan,

Google Sitemap is an .xml file which serves as a guideline for Google bots (and other search engines’ crawlers as well) on all of the pages on your website so it would be easier than findind and indexing the pages on their own.

Please follow below steps to generate sitemap

1. In your backend, go to System > Configuration. On the left menu, under Catalog section, click Google Sitemap.
   
2. Under Generation Settings, set Enabled to Yes. Then click Save Config.
   
   
3. Click the Add Sitemap button.
   
   Filename: Enter sitemap.xml
   Path:
   To put the sitemap in a folder, enter the path to the folder: /sitemaps
   To put the sitemap at the base path of your store, enter a forward slash: /
   Storeview: The view where the sitemap will be used
4. Click the Save & Generate button to create the sitemap.

After being generated, the sitemap appears in the list, with a link to sitemap.xml file. From here, you can do the following:

• To view the sitemap, click the hyperlink in the Link for Google column. This is the link that you will later submit to Google.
• To update the sitemap, click the Generate link.

Please follow below steps to submit sitemap on google

1. Log in to your Google Search Console account (or create a new one if you don’t have an account yet).
2. Click the Add A Property button. Enter your website homepage into the popup and then click Add. For example: https://www.example.com/
   
3. You will then be asked to verify that you are the owner of this site. You can use your Google Analytics account or one of the alternate methods that Google offer.
4. After verification succeeds, you can access your site property from Search Console Dashboard. Go to Crawl > Sitemaps. Click the Add/Test Sitemap button.
   
5. Return to your store Admin, and copy the path to your Google sitemap. If necessary, log back in and do the following:
   a. Go to Catalog > Google Sitemap.
   b. In the Link for Google column, right-click the link to your sitemap. Then on the context menu, click Copy Shortcut.
   c. Open a text editor and paste the link. Then, copy the part of the link that comes after the “dot com.” It looks like this:
   

   /media/s4d71280645299/sitemaps/sitemap.xml

6. Return to Google Search Console and paste the text into the box to complete the address of your sitemap. Then, click the Submit button.
7. Google Search Console will then show the number of Submitted URLs in your sitemap, along with 0 Indexed URLs. Don’t worry, this is totally common. It takes Google a few days to index your submitted URLs, and the number of Indexed URLs will show up after some time.
   

It’s done! You have successfully created a sitemap for your Magento website.



--
If you've found one of my answers useful, please give "Kudos" or "Accept as Solution"

[sonu_saini2]

Resubmit your XML sitemap.

As well, You are trying to search with "site:" but sometimes Google doesn't show metadata for such type search.

So, you should try to search a particular category webpage on Google or try to search with relevant keyphrases.

I'm sure, it will help you to see... what you want.

[seoeditors40b4]

In general, this issue is caused by problem in robots.txt file.

You may check if your URLs are not blocked by going to sitename.com/robots.txt.

Alternatively, improper canonical tags are also possible reason for your issue.

Kind regards.

Sydney Digital Agency

[linkjuice121dd]

Hi, this is a fairly common issue in Magento 1.9.3.6. For category pages not indexing, make sure canonical tags are enabled and that the pages are linked internally (Google won’t index orphan pages). For category search pages, blocking them in robots.txt won’t remove them from the index — you should use a meta no-index tag directly on those pages or handle it via your layout XML to ensure it’s applied correctly. For listing URLs with parameters (like ?dir, ?mode, ?limit), configure them in Google Search Console’s URL Parameters tool and set them to “No URLs.” Keeping a clean XML sitemap with only canonical product, category, and CMS pages also helps. It can take some time, but with these adjustments Google should gradually clean up your index.

Hope so it works for you!

[mateengh03921a]

Indexing challenges in Magento, especially with category and search pages, are quite common. For category pages not showing up, it often comes down to crawl budget and internal linking—Google may prioritize product pages unless your category pages have strong, unique content and proper linking signals. For search result pages and parameterized URLs, relying only on robots.txt isn’t always enough; setting noindex via meta tags in the template and using canonical tags to consolidate duplicates can help.

These types of issues are part of the bigger picture of how businesses adapt to changes in SEO, technology, and digital strategy. If you’re interested in keeping up with the latest developments in these areas, you can check out myreadignamga.com<a href="https://myreadignamga.com/">myreadignamga</a> for Technology News and Business Updates.

[mateengh03921a]

Indexing challenges in Magento, especially with category and search pages, are quite common. For category pages not showing up, it often comes down to crawl budget and internal linking—Google may prioritize product pages unless your category pages have strong, unique content and proper linking signals. For search result pages and parameterized URLs, relying only on robots.txt isn’t always enough; setting noindex via meta tags in the template and using canonical tags to consolidate duplicates can help.

These types of issues are part of the bigger picture of how businesses adapt to changes in SEO, technology, and digital strategy. If you’re interested in keeping up with the latest developments in these areas, you can check out myreadignamga.com for Technology News and Business Updates.