User-agent: Scrapy Allow: / # START YOAST BLOCK # --------------------------- User-agent: * Disallow: Sitemap: http://bibapublishing.com/sitemap_index.xml Sitemap: https://bibapublishing.com/sitemap_index.xml Sitemap: https://spinalnewsinternational.com/sitemap_index.xml Sitemap: https://cardiacrhythmnews.com/sitemap_index.xml Sitemap: https://interventionalnews.com/sitemap_index.xml Sitemap: https://cardiovascularnews.com/sitemap_index.xml Sitemap: https://venousnews.com/sitemap_index.xml # --------------------------- # END YOAST BLOCK User-agent: * Disallow: /wp-login.php Disallow: /wp-admin/ Crawl-delay: 600