We removed sentiment from our dashboard. Here is the data.

Almost every AI visibility tool ships a sentiment score. We built one too, put it on our landing page as a headline metric, and then took it off.

This is the measurement that made us remove it, including the part where our own first attempt at the number was wrong.

What we measured

AI answers are not deterministic. Ask the same question twice and you can get a different answer, so every number in this category is really a number about a distribution.

We store every raw response, which means we can ask a blunt question of our own archive: how often does the same question, asked again, produce a different verdict?

The unit is a cell — one run × prompt × engine with two or more samples in it. Inside a cell, everything a tool controls is held fixed. Anything that changes is the engine changing its mind.

Across 11 brands and 1,075 parsed answers:

Whether a brand is named flips 11.8% of the time (314 cells). Whether the mention reads positive flips 37.7% (61 cells). Sentiment is about three times less stable than the thing everyone treats as the noisy metric.

Why that gap exists

Mention detection is string matching. The brand name is either in the text or it is not. There is very little room for a classifier to wobble.

Sentiment is a second model reading the first model's prose and assigning a label. You have a model's output being judged by a model. Two layers of variance stacked on each other, and only one of them is the thing you meant to measure.

A published study found the same shape at larger scale. Kumar 2026 measured 6.8% for mention flips and 45.5% for sentiment flips across 102 brands. The useful part is the author's own caveat, quoted exactly:

Sentiment is classified by a model on the spans that mention the brand, so the 45.5%-sentiment-flip rate mixes genuine output variance with classifier noise; we have not separated the two.

We can't separate them either. Neither can any tool currently shipping a sentiment score, unless it re-runs its own classifier on identical text and publishes the disagreement rate. We have not seen one do that.

The part that decided it

Then we counted the labels themselves.

Across 1,075 parsed answers, negative was assigned zero times. Neutral 947, positive 128.

So we were never comparing positive against negative. We were comparing positive against neutral — a two-value scale where both values move 37.7% of the time.

That is not a spectrum. That is a coin with a slight bias, redrawn as a gauge.

There is a plausible reason. Assistants are tuned to be even-handed about third parties. If you ask one to compare tools, it will hedge rather than say a product is bad. The absence of negatives may be telling us something real about how these models talk — but that makes it a fact about the model, not a measurement of your brand.

The mistake we made counting it

Our first pass at this produced 12.7% for sentiment flips, which would have made sentiment look about as stable as mentions and this whole post unnecessary.

The error was the denominator. We counted flips across all 314 cells, including the ones where the brand was never mentioned at all. Those cells are trivially stable — no mention, no span to classify, neutral every time. They cannot disagree, so they pad the bottom of the distribution with agreement that exists by construction.

Sentiment only exists where the brand appears. Restricted to cells with two or more mentions, the denominator drops to 61 and the flip rate is 37.7%.

Worth stating plainly, because it is the kind of thing that produces a confident dashboard: a stability number is only as honest as the rows it is allowed to count. If a vendor reports that their sentiment is stable, the first question is what fraction of their denominator had a mention in it at all.

That mistake is general enough to deserve its own post, including why the wrong version was the comfortable one and how to run the check on a number somebody is selling you: Check the denominator before you trust an AI metric.

What this means if you are buying one of these tools

We are not going to tell you sentiment is noise. We do not know that. What we know is narrower and more useful:

• Ask for the flip rate, not the score. Any vendor storing raw responses can compute how often their own metric disagrees with itself. If they can't, they are not storing enough to check.
• Ask what the denominator is. Specifically: of the cells behind the sentiment number, how many contained a mention?
• Ask how many negatives they have ever recorded. If the answer is near zero, the scale they are charting has two working values.
• Watch the sample count. At three samples per prompt and 30% visibility, a sentiment average is built on roughly one labelled span per cell. That is not an average. That is a single draw with error bars drawn around it.

What we do instead

We report whether a brand was named, how often, and which sources the engine read instead. Those are the parts we can measure and re-measure without the number moving under us.

That leaves one denominator question standing, and it is the larger of the two: whether a brand is named depends heavily on whether the question contained its name. In the same archive, four brands were named in 100% of the answers to questions carrying their own name and in 4.2% to 37.1% of the answers to their category questions — Branded vs unbranded prompts.

We kept the raw answers. If the models start producing negatives, or if someone publishes a way to separate classifier noise from real variance, the data is still there and we will re-run this and post whatever it says — including if it says we were wrong to remove it.

Method. 1,075 parsed answers across 11 brands, two engines (ChatGPT and Gemini), three samples per prompt, measured 28 August 2026. A cell is one run × prompt × engine with N ≥ 2. Mention flips: cells where brand_mentioned was not constant, 314 cells. Sentiment flips: cells with two or more mentions where the label was not constant, 61 cells. Sentiment labels are assigned by an LLM parser over the spans naming the brand.

Limits. Single vendor's pipeline, our own classifier, no control group, and no way to separate classifier disagreement from genuine engine variance — the same limit the paper below reports. Eleven brands is a small sample and the sentiment denominator is 61 cells.

Source. Kumar 2026, arXiv:2606.20065 — 102 brands, ~102k responses, vendor-collected data, no control group.