Or… More bullshit that doesn’t work.
This note was inspired by this thread I saw on LinkedIn:
While I’m sure all of us would love an opportunity to “terrorize” our competitors, this frankly just does not work.
Nathan is not the first person I have seen suggest this over the years.
But it doesn’t actually hide anything from your competition.
Let’s start with what you are ranking for…
Crawling your webpages does not give tools like Semrush or Ahrefs any information about what those pages are ranking for.
They get this information from crawling the SERPs for all the different keywords they have data on.
Blocking them from crawling your site does not block them from seeing your site show up in the search results on Google. They see which URLs are ranking. They don’t need to crawl them.
In other words, they are scraping Google, not your site.
As for reverse engineering your link profile…
It’s the same issue.
So you block Semrush, Ahrefs, Majestic, and whoever else you want to from crawling your site… what does that do exactly?
In order to find your link profile, they do not crawl YOUR site. They are crawling the rest of the internet and finding sites that link to you.
Unless you control all of the sites that are linking to you and can block the crawlers from crawling them, you cannot hide your link profile.
Should you block SEO tools from crawling your site…
If you have a really large website, I’m talking millions of pages, and managing server resources is important to you, then blocking these bots might be a good idea.
However, you will be causing yourself additional headaches if you use any of these tools for auditing and monitoring your website.
At the end of the day, you are not gaining any benefit by blocking Semrush, Ahrefs, Majestic, etc.
