theo_66 TH
@theo_66
Technical SEO Nov 12, 2026

how are people stopping ahrefs from crawling certain pages in 2026?

tried to stop ahrefs from crawling certain pages last month and saw no difference at all. it looked like it was working for about two weeks and then flattened out. is there an order that works better?
stop ahrefs crawling
1

Leave A Reply

Log in to post an answer.

1 Answer

S
sanjay.nigam Nov 12, 2026
theo_66
robots.txt still does it, AhrefsBot respects it: User-agent: AhrefsBot Disallow: /private-folder/ Worth knowing that this only stops it crawling your site. Links pointing to you from other sites will still show up in Ahrefs, because they find those by crawling the other sites.
Reply