Wordpress kullanıyorsanız Q2A robots.txt dosyası aşağıdaki gibi olmalıdır.
User-agent: *# Q2A WordPressDisallow: /WP/feed/Disallow: /WP/tag/Disallow: /WP/trackback/Disallow: /WP/cgi-bin/Disallow: /WP/wp-admin/Disallow: /WP/wp-content/pluginsDisallow: /WP/wp-content/cacheDisallow: /WP/wp-Disallow: /WP/wp-includes/Disallow: /WP/trackback/Disallow: /WP/xmlrpc.phpDisallow: ?wptheme=Disallow: /*.php*Allow: /WP/media-files/Sitemap: http://sizinsiteniz.com/sitemap-index.xml
Yukarıdaki robots.txt dosyasını ftp aracılığı ile sitenize atın.
User-agent: *# For Q&A SiteDisallow: /loginDisallow: /askDisallow: /forgotDisallow: /registerDisallow: /questions?sortDisallow: /admin# allow google image bot to search all imagesUser-agent: Googlebot-ImageAllow: /*# disable duggmirrorUser-agent: duggmirrorDisallow: /Sitemap: http://_________/sitemap.xml
User-agent: *
# For Q&A Site
Disallow: /login
Disallow: /ask
Disallow: /forgot
Disallow: /register
Disallow: /questions?sort
Disallow: /admin
# allow google image bot to search all images
User-agent: Googlebot-Image
Allow: /*
# disable duggmirror
User-agent: duggmirror
Disallow: /
Sitemap: http://_________/sitemap.xml