the site is not indexed

[macstuff]

robots.txt is taken from one of the forums, before adding an exception / ajax_cart items indexed, but not all, but only what was extracted through ajax_cart those variations. No description of the goods, etc. for a month by the index is not gone. Only indexed categories of goods, what is the problem?
The following information from the master Yandex and Robots.
Thank you in advance.

 

User-agent: * 
   
## Crawl-delay parameter: number of seconds to wait between successive requests to the same server. 
## Set a custoskim crawl rate if you\'re experiencing traffic problems with your server. 
# Crawl-delay: 30 
   
## Magento sitemap: uncomment and replace the URL to your Magento sitemap file 
Sitemap: http://urbanprojects.ru/sitemap.xml 
Host: urbanprojects.ru
   
## DEVELOPMENT RELATED SETTINGS 
   
## Do not crawl development files and folders: CVS, svn directories and dump files 
Disallow: /CVS 
Disallow: /*.svn$ 
Disallow: /*.idea$ 
Disallow: /*.sql$ 
Disallow: /*.tgz$ 
   
## GENERAL MAGENTO SETTINGS 
   
## Do not crawl Magento admin page 
Disallow: /admin/ 
Disallow: /myadmin/ 
   
## Do not crawl common Magento technical folders 
Disallow: /app/ 
Disallow: /downloader/ 
Disallow: /errors/ 
Disallow: /includes/ 
Disallow: /lib/ 
Disallow: /pkginfo/ 
Disallow: /shell/ 
Disallow: /var/ 
Disallow: /ajaxcart/
Disallow: /quickview/

   
## Do not crawl common Magento files 
Disallow: /api.php 
Disallow: /cron.php 
Disallow: /cron.sh 
Disallow: /error_log 
Disallow: /get.php 
Disallow: /install.php 
Disallow: /LICENSE.html 
Disallow: /LICENSE.txt 
Disallow: /LICENSE_AFL.txt 
Disallow: /README.txt 
Disallow: /RELEASE_NOTES.txt 
   
## MAGENTO SEO IMPROVEMENTS 
   
## Do not crawl sub category pages that are sorted or filtered. 
Disallow: /*?dir* 
Disallow: /*?dir=desc 
Disallow: /*?dir=asc 
Disallow: /*?limit=all 
Disallow: /*?mode* 
   
## Do not crawl 2-nd home page copy (example.com/index.php/). Uncomment it only if you activated Magento SEO URLs. 
## Disallow: /index.php/ 
   
## Do not crawl links with session IDs 
Disallow: /*?SID= 
   
## Do not crawl checkout and user account pages 
Disallow: /checkout/ 
Disallow: /onestepcheckout/ 
Disallow: /customer/ 
Disallow: /customer/account/ 
Disallow: /customer/account/login/ 
Disallow: /firecheckout/
   
## Do not crawl seach pages and not-SEO optimized catalog links 
Disallow: /catalogsearch/ 
Disallow: /catalog/product_compare/ 
Disallow: /catalog/category/view/ 
Disallow: /catalog/product/view/ 
   
## SERVER SETTINGS 
   
## Do not crawl common server technical folders and files 
Disallow: /cgi-bin/ 
Disallow: /cleanup.php 
Disallow: /apc.php 
Disallow: /memcache.php 
Disallow: /phpinfo.php 
   
## IMAGE CRAWLERS SETTINGS 
   
## Extra: Uncomment if you do not wish Google and Bing to index your images 
# User-agent: Googlebot-Image 
# Allow: / 
# User-agent: msnbot-media 
# Allow: / 

Disallow: /checkout/cart/  

[baljitimark]

Please add below mentioned code in robots.txt file

# Group 1
User-agent: Googlebot
Disallow: /subfolder/
Disallow: /nogooglebot/
Disallow: /subfolder/page-url.html

# Group 2
User-agent: *
Allow: /

# Directories & files
Disallow: /wishlist/
Disallow: /customer/
Disallow: /customer/account/
Disallow: /catalog/product_compare/
Disallow: /catalog/category/view/
Disallow: /catalog/product/view/
Disallow: /customer/account/login/
Disallow: /review/product/
Disallow: /catalogsearch/
Disallow: /catalog/
Disallow: /catalogsearch/
##Disallow: /checkout/
Disallow: /catalogsearch/
Disallow: /downloader/
Disallow: /razorpay/
Disallow: /order/
Disallow: /rma/
Disallow: /sendfriend/
Disallow: /returns/
Disallow: /sales/
Disallow: /review/
Disallow: /invitation/
Disallow: /newsletter/
Disallow: /install/
Disallow: /enable-cookies/
Disallow: /skin/
Disallow: /js/
Disallow: /directory/
Disallow: /var/
Disallow: /private/
Disallow: /admin/
Disallow: /api/
Disallow: /app/
Disallow: /bin/
Disallow: /dev/
Disallow: /lib/
Disallow: /phpserver/
Disallow: /pub/
Disallow: /tag/
Disallow: /review/

Disallow: /lib/
Disallow: /includes/
Disallow: /composer.json
Disallow: /composer.lock
Disallow: /CONTRIBUTING.md
Disallow: /CONTRIBUTOR_LICENSE_AGREEMENT.html
Disallow: /COPYING.txt
Disallow: /Gruntfile.js
Disallow: /LICENSE.txt
Disallow: /LICENSE_AFL.txt
Disallow: /nginx.conf.sample
Disallow: /package.json
Disallow: /php.ini.sample
Disallow: /RELEASE_NOTES.txt
Disallow: /*.Zip$
Disallow: /*.php$

# Parameters
Disallow: /*?cat=
Disallow: /*&cat=
Disallow: /*?carrier=
Disallow: /*&carrier=
Disallow: /*?color_filter=
Disallow: /*&color_filter=
Disallow: /*?display_size=
Disallow: /*&display_size=
Disallow: /*?network=
Disallow: /*&network=
Disallow: /*?os=
Disallow: /*&os=
Disallow: /*?price=
Disallow: /*&price=
Disallow: /*?timestamp=
Disallow: /*&timestamp=
Disallow: /*?___store=
Disallow: /*?SID=
Disallow: /*?___SID=
Disallow: /*?name=
Disallow: /*&name=
Disallow: /*?top_seller=
Disallow: /*&top_seller=
Disallow: /pkginfo/
Disallow: /report/