We did this on two dates, 13 and 15 October. In 4.8% of the answers on the first date and 5.4% on the second, at least one of the five facts was wrong. The most common error was the CEO, and almost every time it was the previous one. Many errors came back exactly the same on both dates.
What this means in practice
This is the reassuring finding in the report, and it matters for how the others read. AI isn't making these companies up. It knows what they do, where they are and who runs them, nearly all the time.
What it gets wrong is the recent past. A new CEO, a new address, a new company name, a business sold or split off: the change has happened, the company has announced it, and some engines are still repeating the old version. In the cases we found, the change was anything from six weeks to four years old. Combined with Finding 2, where answers about restructured companies kept describing the old structure, it points the same way. The engines are slower to update than the companies are to change.
The bigger gap is elsewhere. The facts are right, and the description around them, who the company is and why it matters to a buyer, is still mostly written by someone else (Finding 1). AI knows the facts about you. It takes the story from other sources.
Where the errors are
Five engines, 100 companies, asked on 13 and 15 October. "Given" is the number of answers that stated that fact at all.
| Fact | Wrong, 13 Oct | Wrong, 15 Oct | Share wrong, 13 Oct | Share wrong, 15 Oct |
|---|---|---|---|---|
| CEO | 13 of 456 | 10 of 496 | 2.9% | 2.0% |
| Headquarters | 5 of 448 | 9 of 489 | 1.1% | 1.8% |
| Part of a larger group or not | 4 of 450 | 4 of 485 | 0.9% | 0.8% |
| Revenue | 4 of 458 | 6 of 497 | 0.9% | 1.2% |
| What the company does | 0 of 477 | 0 of 499 | 0% | 0% |
Most of the CEO errors named a predecessor. In two cases the change was less than seven months old when we asked, and both were still wrong two days later. In one, an engine named an interim CEO who had held the post four years earlier. Once, the chair of the board was named as the CEO. The headquarters errors were old addresses, a registered office placed in the wrong city, and three answers that put a company headquartered in Espoo in Helsinki next door. Two companies were described as subsidiaries of what is in fact their own former name, and some revenue figures were for a quarter or an interim period, given as the annual figure.
The same errors repeat. 13 of the wrong facts were exactly the same on both dates: the same company, the same engine, the same fact. An old fact doesn't go away by asking again.
28 of the 100 companies got at least one wrong fact on at least one of the two dates, 12 of them on both. 17 of the 28 are in the smaller half by revenue. That leans towards smaller companies, less clearly than the first date alone suggested (14 of 18). Less is written about smaller companies, so a stale fact has less to compete with.
The engines differ, and the order held on both dates. Share of answers with at least one wrong fact:
| Engine | 13 Oct | 15 Oct | Facts not given, 13 Oct | Facts not given, 15 Oct |
|---|---|---|---|---|
| Claude | 12% of 100 | 11% of 100 | 2% | 1% |
| ChatGPT | 7% of 100 | 9% of 100 | 1% | 1% |
| Perplexity | 2% of 100 | 1% of 100 | 1% | 1% |
| Gemini | 1% of 100 | 1% of 100 | 0% | 0% |
| Google AI Mode | 2% of 99 | 5% of 99 | 37% | 0% |
Read the last row with care. On 13 October Google AI Mode often gave a short summary without the details we asked for. On 15 October it gave nearly every detail, and its share of answers with a wrong fact went from 2% to 5%. An engine that says less can be wrong less.
Markup as a whole made no clear difference. Over the two dates, companies without markup had 0.62 wrong facts each and companies with markup 0.51. We decided in advance what difference would count, and this one falls well short of it: a gap this size could come from chance about one time in three. On the first date the gap looked larger, and among companies without an English Wikipedia article it looked clear. Neither held on the second date. And when we asked the engines to read a page's markup directly, none of those that could open the page saw it (see "What the markup says, and who reads it").
Markup that states facts is worth a closer look. Most of the markup in the study says nothing about the company (see "What the markup says, and who reads it"), so we also looked at the 18 companies whose markup states at least one fact about themselves. Over the two dates they had 0.28 wrong facts each, against 0.61 for the other 82. Three of the 18 had a wrong fact in any answer, against 25 of the 82. We didn't plan this comparison in advance, and a gap this size could come from chance about one time in six, so read it as a lead worth testing properly. It fits "What the markup says, and who reads it", though. Markup can only help an engine get the facts right if it states them, and the page and the rest of the web say the same.