WordPress friendly Robots.txt

Robots.txt is a great way of controlling what the search engine’s index about your website. This way you can control that only important information that is beneficial to visitors is index and your SEO is well worth the effort. At Website Designers R Us, our favorite Robots.txt structure looks something like

User-agent: *
Allow: /
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content
Disallow: /e/
Disallow: /show-error-*
Disallow: /xmlrpc.php
Disallow: /trackback/
Disallow: /comment-page-
Allow: /wp-content/uploads/

User-agent: Mediapartners-Google
Allow: /

User-agent: Adsbot-Google
Allow: /

User-agent: Googlebot-Image
Allow: /

User-agent: Googlebot-Mobile
Allow: /

Sitemap: https://www.websitedesignerrus.com/sitemap_index.xml