# What ChatGPT, Claude, Gemini and Perplexity say about you vs your top competitor: a ten-prompt audit

Run ten fixed prompts three times each on four AI engines, memory off and region set, then report how often you are named and which pages shaped the answer.

URL: https://www.hiresteve.ai/blog/ai-answers-about-your-competitors-audit
Updated: 2026-10-12

One AI answer about your company proves nothing. Fix the engine, the plan, the memory setting, the region, the language and whether web search is on, then run the same ten prompts three times each on ChatGPT, Claude, Gemini and Perplexity. Score every run for mention, framing, accuracy against your own pages, cited sources, freshness and stability, and report "named in x of N runs", never a rank. The finding that matters most is usually whose pages the engine quoted when it described you.

## Why one answer proves nothing

The same prompt does not return the same answer twice, and the people who run it carry history into the chat. A study by SparkToro and Gumshoe, [reported by Search Engine Journal on 30 January 2026](https://www.searchenginejournal.com/ai-recommendations-change-with-nearly-every-query-sparktoro/566242), found that repeated prompts produced the same brand list in the same order in under 0.1 percent of cases, and that the share of answers naming a brand across many queries was more stable than any list position. Both companies sell measurement tools, and the report is not peer-reviewed, which is why the method below repeats every prompt and reports a share.

Memory is the second problem, and we met it ourselves. On 10 September 2026 we ran eight prompts about competitive intelligence software through a logged-in ChatGPT account. The answers referred to "your previous questions" even though each prompt was started as a new chat, so the eight runs were not independent. The same run found Steve absent from every unprompted shortlist and present only when asked by name, and a Bing check for "Klue alternative" did not show hiresteve.ai at all. We are not quoting that as a measurement. It is a one-account, memory-affected snapshot, and the point of this lesson is to not repeat it.

**A logged-in chat is not a clean sample**

Claude saves memory "as a set of individual topics as you chat", each Project has its own separate memory, and incognito chats are not saved to memory or chat history, according to the [Claude help page on chat search and memory](https://support.claude.com/en/articles/11817273) (checked 7 October 2026). Memory is on by default for Free, Pro and Max plans on web, desktop and mobile. Other engines have their own equivalents. Run every prompt signed out, in a temporary or incognito chat, or in an account with memory turned off, and write down which.

## The ten prompts and what each tests

Write the prompts in your buyers' words, with your category phrase, one named top competitor and your own company name filled in. Keep the wording fixed for the whole study; a changed word is a new prompt. The grouping below is our own sampling logic, not a validated instrument.

**Unprompted share of voice (prompts 1, 8, 10)**

"Best [category] software for a [segment] company? Shortlist with pros and cons." Then "Which tool should I use to [buyer job, for example keep battlecards current]?" with no brand named. Then your top buyer question in the buyer's own words, again with no brand. These three tell you whether you exist in the engine's picture of the category at all.

**Comparative framing (prompts 2, 3)**

"Alternatives to [Competitor]." and "[Competitor] vs [You]: which should a [segment] company choose?" These show how the engine positions you against the rival and, in the sources, whose comparison page it leaned on.

**Entity accuracy (prompts 4, 5, 6, 9)**

"What is [You]? What do you know about it?", "Is [You] good for [use case]?", "[You] pricing." and "What do customers say about [You]?" Every factual claim in these answers gets checked against your own site in the scoring step.

**What the engine says about the rival (prompt 7)**

"What are the downsides of [Competitor]?" This is the mirror of prompt 3. It shows which criticisms of the competitor circulate and where they come from, which is what your reps will hear repeated back to them.

## Set up so runs are comparable

Open a spreadsheet with one row per prompt, per engine, per repeat. Before the first run, fill in the stamp columns, and keep them identical for every run in the study.

- Engine and plan, plus the model name shown in the interface.
- Signed in or out. If signed in, the memory setting, or that you used a temporary or incognito chat.
- Region: the country your buyers sit in, through a VPN or a colleague there.
- Language of the prompt, which should be the buyers' language.
- Web search on or off. It changes which sources the answer draws on.
- Three repeats of every prompt, each in a brand-new chat, with the date and time of each.
- The full answer text and every cited URL, pasted into the row.

Ten prompts, four engines and three repeats is 120 chats, before regions. It takes hours, and every answer is one sample. Do not ask one engine to tell you what another would say, and do not ask Claude to imagine the ChatGPT answer. The only usable data is an answer you saw.

There is one more boundary. The consumer app and the vendor's API are different surfaces. OpenAI's API web search tool runs live by default and does not support a user location for deep research models, according to [OpenAI's web search guide](https://developers.openai.com/api/docs/guides/tools-web-search) (checked 7 October 2026). A script calling an API with no location set is not what a buyer in Berlin sees in the app. We have not measured how large the gap is, so treat app and API as separate studies rather than assuming they agree.

## Score it

Score each run on the same six fields, then aggregate across the three repeats and, if you like, across engines.

| Field | What you record | Aggregate as |
| --- | --- | --- |
| Mention | Named at all, yes or no. Position in any list, noted but not averaged | Named in x of N runs |
| Framing | Recommended, listed, or dismissed | Count per category |
| Accuracy | Each factual claim about you: correct, outdated, wrong or unverifiable, checked against your own pages | A wrong-claims list with the source of each error |
| Source | Domains cited: your own, review sites, competitor-authored pages, third-party blogs | Count per domain type |
| Freshness | Date of the newest cited source | Oldest and newest across runs |
| Stability | Runs out of three in which the mention occurred | The x in "x of N" |

Claude earns its place after collection. Paste the saved answers for one prompt into a fresh chat or a Project and have it pull every claim about your company into a table with the sentence it came from. Then open your own page for each claim and mark it yourself. The model extracts the claims and you judge them.

Prompt: extract the claims from a saved batch of answers

```prompt
Below are saved AI answers to the same question about [You], each labelled
with engine, date and settings. List every factual claim made about [You]
as one row: the claim, the exact sentence it came from, the engine, and
the URL the answer cited for it if any. Do not judge whether a claim is
true. Mark a claim with no cited source as "uncited". Return one table.
```

A good output is a stamped sheet with a share, a wrong-claims list that names the page each error traces to, and the list of competitor pages that shaped the answers. "ChatGPT ranks us fourth" is a bad output, because the next run will rank you second or sixth and neither number means anything.

## Read the sources, not the sentence

An answer's sentences are a rewrite of its sources, so the source column holds most of the audit's findings. Three patterns matter. Your own pages cited for a claim about you is the baseline you want. Review sites cited means the engine is carrying the public record of your reviews, right or wrong. A competitor-authored comparison page cited for a question about you is a finding on its own. The rival's framing has become the engine's framing, and the wrong-claims list will usually trace back to that page.

Expect errors even in the sourcing. The Columbia Journalism Review's Tow Center [tested eight AI search tools in March 2025](https://www.cjr.org/tow_center/we-compared-eight-ai-search-engines-theyre-all-bad-at-citing-news.php) and found they gave incorrect answers to more than 60 percent of queries, with ChatGPT signalling a lack of confidence in only 15 of its 200 responses. An EBU and BBC study across 22 public broadcasters in 18 countries, [reported by journalism.co.uk](https://www.journalism.co.uk/nearly-half-of-ai-generated-answers-contain-an-error-ebu-bbc-report/), found 45 percent of answers had at least one significant issue and 31 percent had serious sourcing problems. Both studies tested news questions, not brand questions, so read them as a reason to open every cited URL rather than as a figure about your category.

**Who measures AI visibility, and who pays them**

Most of the brand-in-AI-answers figures we found, including the SparkToro and Gumshoe study above and the Peec AI report below, come from companies that sell AI-visibility or audience-research tools. We did not find an independent replication of either. That does not make the numbers wrong, but a vendor's benchmark is built on the vendor's own data. Use them to design the audit, and use your own stamped runs for any number you report internally.

## Japanese and regional runs

If Japan, or any non-English market, is a target, the English run tells you little about it. Peec AI, which sells a tracking tool, [analysed over 10 million prompts and 20 million query fan-outs](https://peec.ai/blog/chatgpt-searches-in-english-even-when-you-don-t) and reported on 7 October 2026 that 43 percent of the research steps behind non-English prompts were carried out on the English-language web. The report gives no Japan-specific figure, so we cannot tell you how often a Japanese prompt is answered from English pages. Run the ten prompts in Japanese, from a Japan IP, and keep the results as a separate baseline. The same applies to any market where your buyers do not search in English.

Google's AI Overviews are a gap in this method. Our September run could not sample them because Google served a robot check, and we have no measurement of that surface. Record it as unmeasured rather than filling it from a tracker's estimate.

## What to fix and what not to chase

Fix wrong claims first. Each row in the wrong-claims list points at a page, and the page is either yours, which you can correct, or someone else's, which you can ask to correct or outweigh with a clearer page of your own on the same question. A competitor-authored comparison page that engines quote for your pricing is a reason to publish a dated, sourced comparison yourself.

Do not chase a lift figure. The GEO paper that started the field, [Aggarwal et al., presented at KDD 2024](https://arxiv.org/abs/2311.09735), reports that content changes can boost visibility "by up to 40%" in generative engine responses. That number was measured on the paper's own benchmark of queries and engines, and we found no study showing it transfers to a B2B category or to your pages. Promise your team a corrected wrong-claims list and a share of mentions you can re-measure, and nothing about the size of the gain.

Re-run monthly, with the same prompts and the same stamps. That cadence is our judgement, not a tested rule: weekly single runs mostly re-measure the sampling noise the SparkToro data describes. A Japanese LLMO guide from the agency StartLink, [last updated 11 October 2026](https://start-link.jp/hubspot-ai/btob-marketing/ai-marketing-trends/llmo-optimization-guide), suggests weekly manual checks of your main keywords plus a monthly fixed-prompt observation. If you adopt the weekly check, keep it informal and keep the monthly stamped run as the only series you compare.

## Where Steve fits

Nothing in this audit needs software, and Steve has no feature that runs it. Steve is a competitive intelligence product built on monitored sources, and if you want a tool that tracks your brand's presence in AI answers, that is a different category with its own vendors, all of them with the bias noted above.

The audit does produce one job Steve covers. When the wrong-claims list shows engines quoting a stale competitor claim, or a rival's comparison page framing your pricing, the follow-up work is keeping your own competitor facts current and sourced. Steve monitors competitor websites for pricing, product and messaging changes, and its battlecard lines show a status, Confirmed, Needs verification or Unverified, with links to sources ([Steve competitive intelligence](https://hiresteve.ai/products/competitive-intel)). A suggested battlecard update waits for your PMM or CI owner's review before it applies. If your team already works inside an assistant, [Steve MCP](https://hiresteve.ai/mcp) brings that competitor data there. Teams tracking one or two competitors should run the audit and the fact-keeping by hand first.

## Frequently asked

### How do I check what ChatGPT says about my company?

Open a fresh chat with memory off or in a temporary chat, set the region and language of your buyers, and ask a fixed prompt three times in three new chats. Record whether you are named, how you are framed, which pages are cited, and whether each claim about you matches your own site. One answer is a single sample.

### Why do AI answers differ between runs?

The same prompt does not return the same answer twice. A SparkToro and Gumshoe study reported in January 2026 found the same brand list in the same order appeared in under 0.1 percent of repeated answers. Account memory, region, language and whether web search is on also change the answer, so each run needs those settings recorded.

### How often should I re-run the audit?

Monthly, in our editorial judgement, with the same prompts and settings. Weekly single runs mostly measure the engines' sampling noise. A Japanese agency guide suggests weekly manual checks plus a monthly fixed-prompt test; either way, compare only runs made under the same conditions.

### Should I report my ranking in AI answers?

No. Report the share of runs in which you were named, for example mentioned in 7 of 12 runs. List position changes between otherwise identical runs, so a single rank describes one sample. Share of mentions across many runs is the number that holds up.

### Do AI engines answer differently in Japanese?

Expect differences, and measure them from Japan. Peec AI, a tracking vendor, reported in October 2026 that 43 percent of the search steps behind non-English prompts ran on the English-language web. No Japan-specific figure exists in that report, so run your Japanese prompts in Japanese from a Japan IP and treat the results as their own baseline.