User-agent: Mediapartners-Google Disallow: User-agent: * Disallow: /usertag/ Disallow: /search/ Disallow: /settings/ Disallow: /people/ Disallow: /me/ Disallow: /response/ Disallow: /comment/ Disallow: /tag/ User-agent: AhrefsBot Crawl-Delay: 10 Sitemap: http://sitemaps.mylot.com/sitemap_index.xml