Your AI visibility score measures the part OpenAI can change overnight

Reddit's share of ChatGPT citations fell 86% over four days in August, and the two firms that measure this for a living disagree on whether the model changed what it reads or only what it shows

by Markus Busch

The event. Reddit is not a hotel, and that is the point of what happened to it. Between July 18 and August 7, Reddit averaged 3.83% of ChatGPT's citations, according to Promptwatch, a platform that sells brands a running score for how often the AI names them. By August 14 the share was under 1%. Across August 14 to 17 it averaged 0.52% — a relative fall of 86.4% in four days. Stacker's chief executive pointed to a second tracker, Petra Labs, which put the drop at 80% on August 16, PhocusWire reported on September 4. Two vendors selling into the same market, agreeing roughly on the size of it.

They do not agree on what happened. Promptwatch says the machine changed what it reads: on August 8, the share of ChatGPT's background searches that named a specific website went from about 0.37% to 16.8%, which it takes as the engine going straight to sites it has decided to trust instead of casting around. Nicolas Sitter, head of AI labs at Lighthouse and co-founder of Hotelrank.ai, told PhocusWire the opposite: "What has dropped are the citations (what you and I can see in the answer) but not the retrieval part (the source). ChatGPT still heavily 'consults' Reddit. It just doesn't cite it in the answer." One firm says the machine changed what it reads. The other says it changed only what it shows. Neither can see inside OpenAI, and both sell the number that moved.

That is the whole problem. What the machine reads is invisible from outside. The links it lists under an answer are not. So every AI visibility product a hotel can buy counts the listed links, because they are the only part there is to count. The number in your monthly report is a tally of those links, sold to you as a measure of whether a machine will recommend your hotel. Nobody has ever shown the two move together.

One source, three engines, three answers. Over the same weeks Promptwatch measured Reddit down 11.3% in Google's AI Overviews and 30.5% in AI Mode, both of them gradual. Same forum, same content, same period, three unrelated stories depending on which product you happened to be watching. There is no such thing as your hotel's AI visibility. There is a number per engine, and each moves on decisions made inside that engine.

Monday. Your agency or a platform sends you one of these numbers every month. The question to put to them is not what the score is. It is what the score does when the answer your guest reads stays the same and only the links under it change — and whether they can tell you which of the two happened the last time your number moved. A Reddit spokesperson gave the honest version to PhocusWire: "No one is able to promise what a large language model (LLM) will deliver in a response."

Reddit did not change anything during those four days. Neither will your hotel, when its turn comes.

Read also: Your hotel's AI visibility score is mostly noise

Enjoying this analysis?* Hospitality.today delivers daily insights on hotel distribution, AI trends, and travel commerce — straight to your inbox. Subscribe for free at Hospitality.today →