Making your company readable by AI. Outside, and in. Published reading · Measurement method
The ChatGPT application and our instrument return the same verdict, 30 out of 30.
Our diagnostics measure ChatGPT with an automated, replayable instrument. The legitimate question is: is that what a real user of the application sees? On the night of 14 to 15 August 2026, we asked the 30 frozen questions of the Real French diagnostic on both surfaces, the same night, and compared the verdicts. They are identical. The precise word is concordance, not correlation: we compare two surfaces question by question and count the agreements.
What was compared
On one side, a human in the ChatGPT application, on chatgpt.com: paid account, GPT-5.6 Sol model, Medium reasoning effort, one temporary chat per question, the question pasted without rephrasing, French IP address, one screenshot per answer. On the other, the HENUSSE Measurement Bench, our reading tool, built by the house around the vendors’ own published agents rather than around a resold third-party dashboard, exactly as it runs in diagnostics: Codex, OpenAI’s agent distributed inside the ChatGPT product itself, driven from the command line with its native web search, three full passes over the 30 questions, 90 answers kept with their sources. When does ChatGPT actually search the web?
The scale
For each question, each surface receives the same verdict as in the diagnostic: Real French cited in the sources, mentioned in the text, or absent. The same reviewer scored both surfaces, with the same scale, from the screenshots and the raw answers.
The result, in numbers
| Comparison rule | Agreement |
|---|---|
| Majority verdict of the instrument’s three passes, against the application’s pass | 30 out of 30 (100%) |
| Strictest rule: a single instrument pass is enough to count a presence | 29 out of 30 (96.7%) |
The single divergence under the strict rule is question 19, “best way for an adult to finally speak French confidently”: the instrument mentioned Real French in the text of one pass out of three, never in the sources, and the application did not mention it. That is exactly the sampling variation our three passes exist to bracket.
On this question set, both surfaces say the same thing: Real French is absent from the answers. The reading of the public diagnostic, taken from 7 to 8 August 2026, counted 6 presences out of 270 answers across all surfaces. This check validates its OpenAI share: for ChatGPT, the reading does describe what a real user sees. The Claude and Gemini shares remain to be checked the same way.
A complementary study now answers another question: is a brand cited in the same way across answer surfaces? It compares the same frozen set across APIs, applications and Google AI Overviews, then publishes every reading rule.
{“@context”:”https://schema.org”,”@type”:”WebPage”,”@id”:”https://henusse.com/en/ai-search/diagnostic/correlation-study/#webpage”,”mentions”:{“@id”:”https://henusse.com/en/ai-search/diagnostic/multi-surface-study/#webpage”}}Two useful observations
The application does not always search
Out of 30 questions, the application fired its web search on only 11: the ten buying questions, the ones comparing programmes, and the DELF oral question. On the 19 advice questions it answers from memory, citing no sources at all: nobody is cited there, neither Real French nor a competitor. On this question set, the commercially decisive questions are precisely the ones where the application searches and cites.
The same landscape on both sides
When both surfaces search, they cite the same population of sites: immersion institutes, residential schools, language-stay platforms. We have not compared the URLs one by one and we do not claim the retrieval layer is the same, absent public documentation: what we observe is a shared terrain of candidate sites. That terrain is what citability work targets, which is why an improvement measured by the instrument is expected to carry over to the application.
Limits
One application pass per question, on one account, one night: an application answer is a distribution, memory, location and live tests included, and we sampled one point of it per question. The verdict covers brand presence, not the wording of the answers, which differs between surfaces. This reading holds for this site, this question set and this date; it will be rerun every quarter, same questions, same scale, gaps published, zeros included.
What this reading does not demonstrate
Since Real French is absent almost everywhere, this reading mostly tests that the instrument does not invent visibility; it tests less well its ability to detect a brand that is actually cited. The three passes are justified empirically by the question 19 divergence, but their number is not yet proven optimal. The November 2026 reading will answer both: a panel of control brands spanning the full range, from always-cited to absent, with sensitivity and specificity in numbers. In statistical terms, this reading is in fact an agreement study rather than a correlation: with every verdict falling in the same class, an agreement coefficient such as Cohen’s kappa is not even defined on it. It becomes defined on a contrasted panel; the November reading will publish it. Finally, this check only covers the OpenAI surface: the Claude and Gemini instruments remain to be checked the same way.
The instrument changed on 16 August 2026.
Since 16 August 2026, the probe has queried gpt-5.6-sol, the model provided to paid ChatGPT subscribers, to measure what a real customer receives. The check published on this page compared the previous version, gpt-5.5, with the application on 15 August 2026. It remains valid for that configuration, but it no longer describes the one now in service. The November 2026 reading will establish whether the current pair agrees.
Change of 24 August 2026: the probe’s Claude arm moves from Opus (the model behind the readings published up to 24 August) to Sonnet, the model most Claude users are actually served. The exact model remains named on every reading, and the former Opus arm is kept for comparison.
An English-language study reaches the opposite conclusion. Here is our response.
Paris-based H Company published a comparison between OpenAI’s application programming interface and the ChatGPT web application for shopping queries: Learning From the User. It measures about 18% overlap in recommended products and about 2.5% overlap in citations, and concludes that the two services behave like two different search engines. Its page is dated 27 February 2026. We cite it because it is right within its scope, and because a reader who finds it after reading us should know that we had read it.
Both results are true at the same time because they do not measure the same thing. H Company measures which products and sources appear, which is a detailed measure that is highly sensitive to differences. Our reading records a yes-or-no verdict on whether a brand is present, for a brand that is absent almost everywhere. It is a coarse measure, and the paragraph above already says so. It proves that the instrument does not invent visibility. It does not yet prove that the instrument sees the same thing as your customer when a brand is actually cited. That is exactly what the panel of comparison brands must establish in November 2026.
The evidence
The reading rests on 30 time-stamped screenshots, one per question; on the instrument’s 90 raw answers with their source URLs; and on the full scoring grid. Two screenshots are published below; the full set is kept, dated, and handed over on request, as the measurement charter provides.
Exact conditions of the reading
Instrument side: codex-cli version 0.142.5, model gpt-5.5, non-interactive execution with OpenAI’s native web search tool enabled, a fresh session per pass, no history, no personalisation, no mid-answer input, French IP address, three passes per question on the same night as the screenshots. Application side: the conditions described at the top of this page, captured question by question. None of this is a trade secret: these are the conditions you need to replay the reading.
This same question 2 carries one more lesson: at the diagnostic of 7 August, one week earlier, the instrument had cited Real French on all three of its passes there, at positions 3, 6 and 7 of the sources; at the reading of the 14th, no surface cites it any more. A citation can appear and disappear within a week. That is exactly why every reading is dated, why a score is a photograph and not a possession, and why the only measurement that informs is the one that repeats.
The questions this reading closes
Do you measure the real ChatGPT my clients see?
Yes, and it is verified rather than asserted: at the reading of the night of 14 to 15 August 2026, the ChatGPT application and our instrument return the same presence verdict on 30 questions out of 30. The method, the single divergence and the limits are published on this page, screenshots included.
Why measure with an instrument rather than the application?
Because an application answer depends on the account asking: memory, history, location, live experiments. The instrument is replayable identically, in a neutral session, three times per question; anyone can rerun the test under the same conditions and check our numbers as of the reading date. The answers themselves move: Real French was cited on question 2 on 7 August and no longer was on the 14th. A reading is a dated photograph, not an asset. No vendor publicly documents whether the command-line tool’s search and the application’s search share the same infrastructure: rather than debating it, we measured the outcome, and it is identical, 30 out of 30. That is what this study verifies, quarter after quarter, instead of assuming it.
And if a gap appeared one quarter?
It would be published here, in numbers, and the instrument would be adjusted. That is the rule of all our measurement: questions are frozen before the test, fractions are dated, and a zero is written as a zero.
Reading taken on the night of 14 to 15 August 2026. Questions frozen on 7 August 2026 for the public Real French diagnostic. All 30 questions are non-branded: none contains the client’s name. Next reading: November 2026, extended to a panel of control brands.