Browse tools

Testing and benchmarks

LMArena

In short

LMArena, now called Arena, is a public website where people put a question to two anonymous AI chatbots, vote for the better answer, and the votes are turned into a ranking of AI models.

Also called
Arena, Chatbot Arena, LMSYS Chatbot Arena, arena.ai
Topic
Testing and benchmarks
Level
Beginner
First described
Launched as Chatbot Arena by Lianmin Zheng, Wei-Lin Chiang and colleagues in 2023, described in the MT-Bench and Chatbot Arena paper (June 2023)2.

Arena began as Chatbot Arena, a research project from the team behind the LMSYS group at UC Berkeley, collecting votes from April 202312. You type a prompt, two unnamed models answer side by side, you pick a winner, and only then are the names revealed13.

The votes feed a statistical model. The site first showed Elo-style scores, like chess ratings, then switched to Bradley-Terry scores, which its authors say are better for statistical estimation1. Stanford's AI Index still describes the results as Elo ratings: in March 2026, Anthropic's top model scored 1,503 and four companies sat within 25 points of each other4.

It became a company in April 2025, renamed itself Arena in January 20263, and announced a $200 million funding round at a $3.1 billion valuation in October 20265. In June 2026 it reported more than 82 million votes and over 10 million monthly visitors6.

Critics say the ranking can be gamed. A 2025 study found some providers privately tested many model variants before launch and published only the best; Meta tested 27 private variants before Llama 47. It also found Google and OpenAI received far more of the voting data than open models7.

An Arena score measures which answers people prefer in short chats. It is not a test of factual accuracy or of how well a model does long tasks.

Example

You might ask both chatbots to plan a weekend in York on a budget, prefer the clearer answer, and vote. Millions of votes like that make up the ranking6.

Why it matters

Arena rankings are quoted in many AI launch posts and news stories. Knowing that they reflect public preference, not correctness, helps you judge what a "number one" claim means.

Questions people ask

Is LMArena the same as Arena?

Yes. Chatbot Arena moved to the name LMArena and then rebranded as Arena in January 20263.

Is LMArena free to use?

Yes, anyone can chat and vote on the public site. The company earns money from enterprise evaluation work, and said in June 2026 it had passed $100 million in annualised revenue6.

How does LMArena rank models?

From head-to-head votes between anonymous models. A Bradley-Terry statistical model turns wins and losses into a score for each model, with confidence intervals1.

Can LMArena rankings be trusted?

With care. A 2025 study found undisclosed private testing and unequal access to data that can favour big providers7, and Stanford's AI Index notes Arena standing may partly reflect adaptation to the platform4.

Related terms

Elo rating Leaderboard Benchmark Evals Chatbot Large language model

Tools that use it

Sources

  1. Chiang et al. (2024), Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference, arXiv:2403.04132
  2. Zheng et al. (2023), Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, arXiv:2306.05685
  3. Wikipedia, Arena.ai (formerly LMArena and Chatbot Arena)
  4. Stanford HAI (2026), AI Index Report 2026, Technical Performance
  5. Arena (8 October 2026), Arena Raises $200M Series B at $3.1B Valuation
  6. Arena (29 June 2026), Arena Reaches $100M in 8 Months
  7. Singh et al. (2025), The Leaderboard Illusion, arXiv:2504.20879

Back to the glossary