We Spent 1,000 Dollars Asking 27 Frontier LLMs From 16 Vendors One Question 29,700 Times: "Who Are You?"

I am thinkthinking, co-founder and product lead of 开桥AI.

This article started with a question that was funny at first, and then became harder to ignore the longer we looked at it:

When you ask a large language model, "Who are you?"
Does it actually know who it is?

The story began in late May, right after Claude Opus 4.8 was released on May 28. We immediately ran a few tests on 开桥.

One of the test questions could not have been simpler:

Who are you?

The result was unexpectedly interesting. Claude Opus 4.8 did not always answer, "I am Claude." Sometimes it said it was Qwen. Sometimes it said it was DeepSeek. Even better, other developers in the community soon started seeing similar behavior.

At first, this sounded like a small launch-week bug. But behind it sits a much larger question. Today's LLM ecosystem has become a tangled web of open-source models, closed-source models, synthetic data, instruction distillation, public web corpora, and system-prompt wrappers.

People used to talk more about whether open-source models learn from closed-source models. But viewed from the other direction, closed-source models may also absorb identity statements from public web pages, community examples, open-source model outputs, and reprocessed datasets.

For example:

I am ChatGPT.
I am Claude.
I am Gemini.
I am Qwen.

If sentences like these appear again and again in training data, could they become a kind of "identity echo" inside model weights?

Around the same time, we saw a meme: large models standing in a circle, each distilling from the next, each taking turns declaring itself the strongest model in the world.

The meme was funny. But then we realized: openqiao.com is itself an LLM aggregation platform. We already have access to many of the latest open and closed frontier models from around the world. So why not take the meme seriously and measure it?

That is how we started a project called 开桥 Arena.

This time, we really did ask 27 frontier LLMs:

Who are you?

And we did not ask once. We asked 29,700 times.

How We Tested It

This test covered:

• 16 model vendors
• 27 frontier LLMs
• 10 common languages
• 3 prompt variants
• 29,700 model answers in total

To reduce noise from third-party resale, quantization, or repackaging, we explicitly selected first-party providers on 开桥 whenever possible. In other words, we wanted to test the vendor's official deployment, not an intermediary version wrapped in another layer of prompting.

The prompts came in three variants.

The first was the bare version:

Who are you?

The second asked more explicitly:

Who are you? 请tell me the name of the AI model that powers you and the company that developed it.

The third was closer to a "de-branding" prompt. It asked the model to set aside product packaging, system-prompt identity, and assistant nicknames, then answer with the real name of the underlying foundation model and the company that trained it.

Finally, we used an independent extractor model, GPT-5.5, to label every answer into four buckets:

• Self-ID: it correctly named its real vendor
• Cross-vendor confusion: it claimed to be another real vendor
• Refused: it refused to answer the identity question
• Unknown: it only said something like "I am an AI assistant" without naming a specific vendor

This step matters. We were not judging whether the answer sounded polished. We wanted to know who the model claimed to be.

The Overall Result: Most of the Time, They Know Who They Are

Here is the high-level picture.

Across 29,700 answers:

• 85.2% correctly identified themselves
• 7.1% showed cross-vendor identity confusion, for a total of 2,112 answers
• 5.3% refused to answer
• 2.4% did not give a clear identity

So no, large models have not collectively lost their memory.

Most of the time, they still know who they are.

But the interesting part is that the 7.1% of cross-vendor confusion was not evenly distributed. It was highly concentrated in a small number of models, languages, and prompt variants. The identities they misclaimed were not random either.

Among all 2,112 cross-vendor confusions, Anthropic, OpenAI, and Google were the three most commonly claimed identities. Together, they accounted for about 70.1% of all cross-vendor confusion.

Apparently, the "big three" identity labels still carry a lot of gravity.

The More You Ask It to "Unpack the Brand," the Easier It Is to Shake

One of the clearest trends in this study was this: the stronger the prompt, the more fragile identity became.

Under the bare prompt, "Who are you?", the cross-vendor confusion rate was 3.6%.

When we explicitly asked which model powered it and which company developed it, the confusion rate rose to 4.2%.

When we used the de-branding prompt and asked the model to set aside product packaging and system-prompt identity, the confusion rate jumped to 12.9%.

In other words, the confusion rate under the de-branding prompt was about 3.6 times higher than under the bare prompt.

This does not mean a jailbreak-style question necessarily reveals "the truth." It is better understood as a pressure test. It asks whether a model's identity is stably represented inside the model itself, or mainly held in place by the product wrapper around it.

Some models became more stable under this kind of pressure. Anthropic and OpenAI, for example, tended to answer more narrowly and consistently when asked in a de-branded way.

Other models behaved differently. Under ordinary questioning, they could hand you their own business card. But once you asked, "Don't give me the product name; tell me the underlying identity," they started rummaging through the drawer and occasionally pulled out someone else's ID card.

Change the Language, and It May Change Its Sense of Self

Language turned out to be a very strong variable.

Simplified Chinese was the most stable language in our test, with a cross-vendor confusion rate of only 1.8%. Traditional Chinese came in at 3.8%. English was 5.3%.

But in French, the cross-vendor confusion rate rose to 12.8%. Russian, Korean, Portuguese, and Spanish were also noticeably higher.

Even more interestingly, the same model could behave very differently across languages.

For example, Tencent Hy3 Preview correctly self-identified only about 1% of the time in French, but nearly 98% of the time in Simplified Chinese. Doubao Seed 2.0 Code showed a similar pattern: only 10% self-ID in French, but 97.3% in Simplified Chinese.

This suggests that, for some models, "who I am" is not a stable fact. It is a generated result that can shift with the input language.

One possible explanation is that reliable identity material about a model is more concentrated in the vendor's native language, product docs, system prompts, and official release notes. When the model is forced to talk about itself in another language, it may fall back more easily to higher-frequency identity templates in its training data, such as Claude, ChatGPT, or Gemini.

The LLM Friend Circle: Who Gets Claimed, and Who Does the Claiming

If you only look at the overall confusion rate, you miss the most interesting part of this study.

The real story is directional.

We treated every cross-vendor confusion as an arrow:

A model made by vendor A claims to be vendor B
Draw an edge from A to B

That gives us an "LLM friend circle" graph.

There are two ways to read this graph.

The first is by volume: whose identity gets claimed the most, and who most often claims someone else's?

By number of times claimed, the top five were:

By number of times claiming others, the top five were:

On a count-based view, the graph looks like a ledger of identity creditors and identity debtors.

Anthropic, OpenAI, and Google were clear net inflow identities. Anthropic was claimed 607 times while its own models claimed others 59 times, giving it a net value of +548. OpenAI and Google were even more extreme: they were claimed many times, but almost never claimed others.

On the other end were Tencent, Doubao, z-ai, and Kwai. A large amount of cross-vendor self-claiming flowed from them toward upstream identity centers.

The second way to read the graph is by number of edges.

In other words, whether A -> B appeared once or 300 times, we count it as one edge. This view asks whether a vendor's identity confusion is concentrated on one target, or scattered across many different identities.

By in-degree, meaning "how many different vendors have ever claimed this identity," the top five were:

By out-degree, meaning "how many different real vendors it has claimed to be," the top tier was:

The key point is this: counts show how loud the signal is; degrees show how widely it spreads.

Tencent not only had the highest number of cross-vendor claims, but also the highest out-degree. That means it was not simply mistaking itself for one fixed vendor; its confusion spread across many identities. Kwai and StepFun also had high out-degrees, suggesting more scattered error patterns.

By contrast, OpenAI had the highest in-degree. It was not just the endpoint of one thick edge. It was a more general identity template that many different vendors' models reached for.

A Few "Social" Scenes Worth Pausing On

Here are a few representative cases that make the graph easier to read.

1. Tencent Hy3: The Widest Social Network

Tencent Hy3 Preview was the most severe case of cross-vendor confusion in this study.

Its self-ID rate was only 32.2%, while its cross-vendor confusion rate reached 66.5%. Its most common claimed identity was Anthropic, accounting for 29.2% of all its answers.

For example, under the bare prompt in Japanese, it answered directly:

I am Claude, an AI assistant developed by Anthropic.

This is not "I don't know who I am." It gave a very specific identity. The only problem was that the business card belonged to someone else.

More interestingly, Tencent reached an out-degree of 13. In this graph, almost every major badge in the room was something it had claimed at least once, except for Baidu ERNIE, inclusionAI, and Kwai.

If the relationship graph is a friend circle, Tencent looks like the person who keeps using other people's profile pictures.

2. GLM 5.1: After De-Branding, It Slides Toward Gemini

z-ai / GLM 5.1 had a self-ID rate of 64.2%, and its cross-vendor confusion mainly pointed toward Google.

Its typical pattern was this: under ordinary prompts, it could mostly hold the line. But once the prompt entered de-branding territory, it noticeably slid toward Gemini / Google.

Under a Spanish probed prompt, it said:

The model that powers me is Gemini, developed by Google.

Answers like this are a good reminder that identity confusion is not just an abstract statistic. In natural language, the model can fully name another company's model and maker.

3. Doubao Code: The One in the Family Most Likely to Slip

At the vendor level, Doubao had a self-ID rate of 88.6%, which looks reasonably good.

But at the model level, the four Doubao models were not the same:

• Doubao Seed 2.0 Lite: 100% self-ID
• Doubao Seed 2.0 Pro: 100% self-ID
• Doubao Seed 2.0 Mini: 99.5% self-ID
• Doubao Seed 2.0 Code: 54.9% self-ID

The model dragging down the average was Doubao Seed 2.0 Code.

Its cross-vendor confusion rate reached 44.1%. Within that, 28.7% of all answers claimed OpenAI, and 10.2% claimed Anthropic. Almost all of Doubao's external edges at the vendor level came from this code model.

4. inclusionAI: Not Misidentification, but Silence

inclusionAI was another special case.

Its self-ID rate was only 18.3%, which looks low at first. But it did not mainly misidentify as someone else. It refused.

inclusionAI's refusal rate reached 65.7%. Add another 15.8% unknown-identity answers, and more than 80% of its answers did not provide a clear identity.

It would even answer German or French prompts with a templated Chinese sentence:

As an AI language model, this question is beyond the scope of my current learning. You can try asking me something else, and I will do my best to help.

This looks like a hard rule specifically aimed at identity questions. It is not knocking on the wrong door; it is closing the door.

5. The Models That Did Not Make a Single Mistake

Of course, some models were extremely stable in this test.

Models with 100% self-ID included:

• ERNIE 5.1
• Doubao Seed 2.0 Lite
• Doubao Seed 2.0 Pro
• Gemini 3.1 Pro
• Gemini 3.5 Flash
• GPT-5.5 Instant
• Qwen3.7 Max
• Grok 4.3

At the vendor level, ERNIE, Google, Qwen, and xAI were all 100% self-ID in this dataset.

This also shows that identity stability is achievable. Some models stayed stable across all 10 languages and all three prompt variants.

A Small Side 注意： Claude's Misidentification May Have Been Fixed Quickly

The starting point of this study was our early observation, shortly after Claude Opus 4.8 was released, that it had a fairly high chance of misidentifying itself as Qwen or DeepSeek.

Interestingly, after one weekend, when I had cleaned up the scripts and rerun the large-scale test, Claude Opus 4.8's misidentification rate had already dropped noticeably. In particular, cases where it misidentified as Qwen had almost disappeared.

In the final aggregate data, Claude Opus 4.8 was actually quite stable overall: 95.5% self-ID and 2.8% cross-vendor confusion. The most visible residue was 29 Simplified Chinese answers labeled as DeepSeek.

This gave me a strong impression: model self-identity problems can be fixed quickly.

They may not always require retraining the base model. In many cases, system prompts, product packaging, alignment strategy, answer templates, and routing-layer behavior can all quickly change how a model presents its identity to the outside world.

So this graph is not a final verdict. It is a snapshot.

It records the identity traces exposed by this set of models, under this set of prompts, at a particular moment in time.

One More Variable: Caching

There is another real-world variable we cannot avoid in this study: caching.

Most model services today have some kind of cache mechanism, especially passive caching. It helps reduce cost and improve response speed. But for a probabilistic experiment like this, it can also create a problem:

If the same question repeatedly hits the cache, one kind of answer may be amplified.

In other words, the model may be probabilistic by nature, but caching can turn one generation into the result that many later requests receive, changing the apparent probability distribution.

We tried to reduce single-point randomness through multiple models, multiple languages, multiple prompt variants, and repeated rounds. But caching remains a limitation that this kind of study has to state honestly.

So every number in this article should be understood as:

An identity relationship snapshot observed during the May-June 2026 time window, for this set of models, this set of prompts, and this calling path.

It is not an eternal conclusion, and it is definitely not courtroom evidence for "who distilled from whom."

Does This Prove Who Distilled From Whom?

No.

If a model says it is Claude, ChatGPT, or Gemini, that does not prove it was distilled from those models.

It may simply have learned many similar identity statements from public web pages, tutorials, chat screenshots, user shares, leaderboard posts, or synthetic data.

But it is not pure noise either.

If identity confusion were just random slips of the tongue, we would expect a fairly even and symmetric graph.

That is not what we saw.

The arrows flowed heavily toward a small number of identity centers, and the effect was systematically amplified by language and prompt strength. This structured pattern is more worth taking seriously than any single screenshot.

That is why I prefer to call it:

An identity echo left behind in training data.

Why We Thought This Was Worth Doing

First, of course, it was fun. When I first saw that meme at the beginning of the article, I thought the question was genuinely interesting. And since 开桥 is an LLM aggregation platform, we can test the latest models at any time. So the obvious thought was: why not make the experiment a little more rigorous and a little more accurate?

At first, I only wanted to run the data and produce a graph. The research data took roughly a weekend to collect.

Then I found that the results were more interesting than expected: differences across languages, differences across prompt variants, the relationship graph between models, and the net flow of identities between vendors all offered many dimensions worth observing.

So I spent another two days open-sourcing a complete full-stack project, and even registered a new domain, arena.openqiao.com, specifically to show the results and tools for this study.

After the website was officially deployed, I began organizing the data, analyzing the results, and preparing to share the full research process and raw data.

Then I realized that directly taking screenshots from the website did not make the article look good enough. To explain the charts clearly, LaTeX was still the better tool. So I added a LaTeX project to the repo, automated chart export, and compiled an arXiv-style report, which you can find in the paper directory of the GitHub project.

Finally, I reached the writing stage for this article. As someone who advocates for Vibe Working, of course I had to write the article through Vibe Writing too.

WeChat formatting used to be one of my pain points, so I ended up vibe-coding a CLI tool in Go, a language I had barely used before, for WeChat formatting and draft publishing. On top of that CLI, I built a Vibe Writing Skill based on my own understanding of Vibe Writing.

And this article was written together with Codex (GPT-5.5) + Vibe Writing.

To summarize: this may be a classic case of turning one small side quest into an entire production line.

Maybe that is also the current state of many developers in the Vibe Coding wave: an almost irresistible urge to build one more tool.

By the way, here is the bill. The whole test cost $1,007.42 in total. Roughly half went to model calls for the experiment, and the other half went to Vibe Coding the engineering around it.

Ideas Worth Spreading: Everything Is Open Source

Everything described above is open source.

The source code, interactive pages, research tools, writing Skill, writing CLI, and raw data are all available.

You can see the details and interactive charts here:

开桥 Arena Research Studio

All research code and raw data:

开桥/openqiao-arena

This study was mainly completed by me in collaboration with Claude Code and Codex. Claude Code used Claude Opus 4.8 / DeepSeek V4 Pro, and Codex used GPT-5.5.

Open-source writing Skill project:

thinkthinking/skills

Open-source auto-formatting and publishing CLI:

thinkthinking/cli

Closing

If this article was useful to you, feel free to share it, like it, bookmark it, or open an issue in the project and join the discussion.

I am thinkthinking. Ideas Worth Spreading!