Testing signals
Test your signal on 30 websites first
We were about to build a campaign around sites blocking AI crawlers. Thirty websites and two small scripts later, we dropped the idea. What we learned about testing cheaply.
Signals are easy to come up with and expensive to build. Once a signal is sitting in an enrichment table, a set of email variants and a sending schedule, nobody wants to hear that it only applies to one company in thirty.
That is why we test first, on a small sample, with simple scripts.
The idea
In October we were preparing outreach to SEO agencies. AI search is the topic every SEO agency is talking about. So we looked at the websites of their potential clients for a verifiable gap in "AI readiness", in the same way the missing HTML sitemap had worked in an earlier campaign.
We had two candidates:
- robots.txt: does the online shop block AI crawlers such as GPTBot, PerplexityBot or ClaudeBot in its robots.txt?
- llms.txt: does the online shop publish an llms.txt, the proposed standard that tells AI tools what is on a site?
Both are yes or no. Anyone can check both. On paper, both are good signals.
What 30 online shops told us
We ran both checks on 30 Dutch online shops on 7 October 2026.
| Check | Yes | No | Couldn't be checked |
|---|---|---|---|
| Blocks AI crawlers | 1 | 22 | 7 |
| Has a genuine llms.txt | 7 | 19 | 4 |
The first signal was dead on arrival. One in thirty is too rare to build a campaign on. A block is also usually a deliberate choice rather than a mistake. Asking someone to reverse a decision they made on purpose is a poor way to open a conversation.
The second signal was alive. Most shops in the sample didn't have the file, the gap is easy to explain, and it ties to work an SEO agency can sell.
Thirty sites is a small sample, and ours wasn't random. So we treat the result as a direction, not a statistic. It was enough to make the decision, and that is all a first test needs to do.
Three ways the data could have gone wrong
The biggest lesson wasn't the result. It was how close we came to measuring the wrong thing.
- Pages that say "found" when there is nothing there. Several online shops return a normal "page found" response for /llms.txt and simply show their homepage. A script that only looks at the status code counts them as shops that have the file. We now judge the content of the response, never just the status code or the declared file type. One site served a perfectly valid robots.txt labelled as an HTML page.
- Sites that block scripts. Some sites refuse requests that don't look like an ordinary browser. In our first set-up, that produced 4 false "couldn't be checked" results out of 30. Having our script pass itself off as an ordinary browser fixed it.
- One rule for several crawlers. One robots.txt listed six crawlers above a single blocking rule. A simple parser reads that as if only the last one is blocked.
None of these errors threw an error message. They would have quietly filled a column with wrong values, and those wrong values would have ended up in emails.
How we test a signal now
- Pick the smallest sample that can kill the idea. Thirty is usually enough.
- Count the "couldn't be checked" results separately, because technical problems hide behind them.
- Read ten raw responses by hand before you trust a script.
- Note the date. Adoption of something like llms.txt changes within months, so a percentage from today will be out of date by spring.
- Decide in advance which result kills the signal.
A day of testing costs less than a month of campaign built on a column of bad data.
Looking for the signal in your market?
Tell us what you sell and to whom. After the intro call you'll know whether we can help.