User-agent: turnitinbot User-agent: NPBot User-agent: ia_archiver User-agent: webzip User-agent: ccbot Disallow: / User-agent: * Disallow: /downloads/da/ Disallow: /downloads/dd/ User-agent: SemrushBot User-agent: PetalBot User-agent: Baiduspider User-agent: MJ12bot User-agent: Yandex User-agent: SemanticScholarBot User-agent: AhrefsBot Disallow: /downloads/d/