AI crawlers now outnumber human visitors
Over five days in September 2026, across nine small sites, AI crawlers generated 2,878 hits against 2,050 human pageviews — and 23× more than every search engine put together.
We did not go looking for this. We found it because one of our own sites reported 3,088 visitors from the United States and 22 from Spain, in a directory of Spanish businesses written in Spanish. That is not an audience. That is a crawler.
The numbers
Nine sites, 17–21 September 2026, five complete days. Every hit below reached a JavaScript tracker and was classified by the user agent the visitor declared.
| What hit the sites | Hits | Share of bot traffic |
|---|---|---|
| AI crawlers — any of GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, PerplexityBot, CCBot, Bytespider or meta-externalagent. See the caveat below about which ones actually showed up | 2,878 | 95.1% |
| Search engines (Googlebot, Bingbot, Applebot, Yandex, Baidu…) | 124 | 4.1% |
| Everything else — self-declared crawlers we could not name (13), unidentified user agents (9), SEO tools (1) | 23 | 0.8% |
| Human pageviews, same nine sites, same days | 2,050 | — |
It is not evenly spread, and that matters
The average hides the interesting part. Two sites carry 98% of it, and one site shows the exact opposite pattern:
| Site | AI crawlers | Search engines |
|---|---|---|
| A local business directory (~42,000 listings) | 2,316 | 0 |
| An e-commerce blog | 515 | 61 |
| A free online-tools site | 1 | 46 |
| Six other small sites, combined | 46 | 17 |
A big directory of structured, factual content is exactly what a model wants to ingest, and it got scraped thousands of times while receiving zero measurable search-engine crawling. A site of small interactive tools — nothing to extract, everything to use — got the reverse. If you run a content site, the first pattern is likelier to be yours than the average is.
What is wrong with this data
Quite a lot, and anyone quoting it should quote this part too.
It measures bots that execute JavaScript. These hits were counted by a tracker script in the page, not by reading server logs. A crawler that fetches your HTML and never runs the script is invisible here. Googlebot renders JavaScript on a second pass and often not at all, so the search-engine column is a floor, not a crawl count. The honest version of the headline is: among bots that run JavaScript, AI crawlers dominate — and, as the named breakdown below shows, more of them run JavaScript than this field generally assumes.
Nine sites is not a study. They are small, they belong to one portfolio, and one of them produces most of the AI traffic. Five days is a snapshot. Treat this as a reading from a thermometer we happened to be holding, not as a market-wide figure.
It trusts the user agent. These crawlers announce themselves — GPTBot says it is GPTBot. A scraper that disguises itself as Chrome is not in the AI column; it is somewhere in the human numbers, which is a good reason to distrust anybody's human numbers, including ours.
Which crawlers, by name
The five days above recorded the type of each bot but not its name. We started recording names on 23 September. Updated with a full week now that one is available — 23-29 September, nine sites:
| Crawler | Run by | Hits | Share of AI traffic |
|---|---|---|---|
meta-externalagent | Meta | 1,954 | 81.6% |
OAI-SearchBot | OpenAI | 404 | 16.9% |
GPTBot | OpenAI | 23 | 1.0% |
ClaudeBot | Anthropic | 9 | 0.4% |
Bytespider | ByteDance | 5 | 0.2% |
| All AI crawlers | — | 2,395 | 100% |
| Every search engine, combined | — | 115 | — |
| Human pageviews, same nine sites, same days | — | 1,609 | — |
The pattern from the first window holds, and firms up with a full week of names attached: AI crawlers outnumbered humans 1.5× and search engines ~21×. Still heavily concentrated — Meta alone is 82% of it — but it is four companies now, not one: OpenAI runs two separate agents, and Anthropic and ByteDance both showed up this week where they hadn't in the first three days. The concentration ratio is stable across both windows; the roster of who else shows up keeps growing the longer we look.
Every one of these executed JavaScript. That is the part worth pausing on. The
standard advice in this field is that AI bots do not run JavaScript, so client-side analytics cannot
see them and only server logs will do. These hits were counted by a script in the page. Whatever the
general rule may be, meta-externalagent, OAI-SearchBot, GPTBot
and ClaudeBot all reached one. Server logs would still show more — a crawler that never
runs the script is invisible here — so treat this as a floor, not a census.
Why almost nobody can tell you this about their own site
Every analytics tool filters bots. Almost none of them show you what they filtered:
- Google Analytics 4 states it plainly: you "cannot… see how much known bot traffic was excluded".
- Plausible, Simple Analytics, Pirsch and Umami filter bots and publish no report of what was filtered.
- GoatCounter stores it in a table its own changelog describes as "never accessed".
- Matomo excludes bots by default; a separate report needs a third-party plugin.
- Fathom does show how much it blocked and by which detection method — datacenter IP, user agent, headers — but not which bots they were.
- Logly counts them apart and labels them by type, so "AI crawlers" is a line you can read instead of a number that quietly vanished.
See the full breakdown on the comparison pages.
Want this number for your own site? Add the Logly script, wait a day and read the bot line on your dashboard. Free plan, no card, under 1 KB, no cookie banner — and if the answer turns out to be "almost none", you will have learned that for free too.
Method, in full
Traffic is classified at the moment the tracker fires, from the User-Agent header, against
a published list of self-declaring crawlers grouped by type: search engines, AI crawlers, social link
previews, SEO tools, monitors, headless browsers and scripted clients. Anything that does not match a
pattern and does not look like a browser is counted as unidentified and reported separately rather than
guessed at — the numbers above put that bucket at nine hits in five days. Bots are excluded from visitor,
session and pageview counts; they are never quietly folded into them.
Period: 17–21 September 2026 inclusive. Sites: nine, all tracked by Logly. Figures are raw hits, not deduplicated by session, because a crawler does not have sessions.
Questions, or a data set of your own you want compared against this? Use the chat widget in the bottom-right corner of this page and we will answer with the query we ran.