Our Methodology

Every model on the leaderboard receives a Rival Score - a single number between 0 and 1000 that reflects real-world human preference. No LLM-as-judge nonsense. It's computed from three components:

Every comparison follows the same controlled, automated process. Responses are generated programmatically through API integrations - no manual intervention, no curation, no 'let me just tweak this one.'

• Same prompt, every model. Each model in a challenge receives the exact same input text. No special treatment. No warm-up round.
• Automated pipeline. Responses are generated via OpenRouter, Replicate,and direct provider APIs using automated scripts. This eliminates human selection bias. The robots judge themselves; we just provide the arena.
• Controlled parameters. Pre-generated responses use temperature=0.7 and effectively unlimited output tokens via OpenRouter. A small number of models (GPT-5.1 series) don't support the temperature parameter and run at their provider default. Every response includes a reproducibility card - so you can call us out if something looks off.
• No system prompts for pre-generated content. Pre-generated comparisons send only the user prompt - no system prompt, no hidden instructions. When Pro users create challenges, a minimal type-specific system prompt guides the output format (e.g., 'Generate ONLY valid SVG code'). These prompts are visible in each response's generation details. We're not hiding anything. We literally can't afford to.
• One shot, no cherry-picking. Each model gets a single generation attempt. Outputs are never re-rolled for a 'better' result. What you see is what the API coughed up.

Challenges are the prompts that power every comparison. They're built to surface genuine differences in capability - not to make any provider's marketing team happy.

• Capability-specific. Each challenge targets a distinct skill: coding, reasoning, creative writing, analysis, instruction following, or visual output. One trick ponies get exposed fast.
• Model-agnostic prompts. Prompts contain no provider-specific keywords or structures. No prompt engineering tricks that advantage one family of models. If your model needs a secret handshake to perform, that's a skill issue.
• High quality ceiling. Challenges are deliberately open-ended and creative - designed so there's always room for a better answer. A model ten years from now should still produce a meaningfully improved response to the same prompt. This ensures Rival remains relevant long after we've all been replaced by our own creations.

Rankings on the leaderboard use a weighted scoring system derived entirely from real user activity - not synthetic benchmarks. No GPT-4 grading other models. Just humans with opinions and too much free time.

• Challenge win rates. The primary signal. When users vote in a head-to-head duel, each win contributes to the model's score in that challenge's category.
• Category coverage. Models are ranked within each capability category. A model needs participation across multiple challenges to earn a stable ranking. No ducking the hard ones.
• Consistency matters. A model that performs well across many challenges is weighted higher than one with a single standout result. One viral moment doesn't make a career.
• One vote per user per duel. Voting is anonymous and deduplicated. Each user gets a single vote per duel. No ballot stuffing. We're not running elections here. Wait...
• Blind voting available. A blind mode reduces brand bias - model names and provider info are hidden until after the vote is cast. Turns out people vote very differently when they can't see the logo.

Community votes power all rankings. We take several measures to keep them clean and credible. Trust issues are a feature, not a bug.

• Deduplication. Each voter can cast one vote per model pair per challenge. Duplicate votes are rejected at the database level via a unique constraint. The database said no and it means no.
• Rate limiting. Authenticated users are capped at 200 votes per hour. Anonymous voters are limited to 30 per hour, keyed by IP hash. Touch grass between votes.
• Voter fingerprinting. Anonymous votes are tied to a hashed IP identifier. Client-provided IDs are also hashed server-side to prevent spoofing or collision with real user accounts. We trust you. Just not that much.
• Minimum sample size. Models must accumulate at least 10 duels before appearing on the leaderboard. The credibility factor further down-weights models with small vote counts so early results don't distort rankings.
• Server-side normalization. All model IDs, challenge IDs, and winner IDs are validated and normalized on insert. Input format checks reject malformed data before it reaches the database. We've seen what the internet sends us.

Models aren't just ranked overall - they compete in 15 distinct category arenas. Each arena focuses on a specific capability, so you can find the best model for your exact use case instead of trusting one number to rule them all.

Category rankings use the same Rival Score formula but scoped to challenges in that category. Explore all category rankings on the leaderboard

Rival Score snapshots are taken daily. The leaderboard shows a 7-day trend indicator next to each model, so you can see who's heating up and who's quietly losing ground. The graph never lies.

AI models evolve quickly. We track versions carefully so comparisons remain meaningful. Nobody's getting credit for last quarter's weights.

• Point-in-time captures. Model responses are recorded at the moment a challenge is created. The model version used is stored alongside the output. Receipts kept.
• Version tracking. When a provider ships an update, new challenges reflect the latest version. Previous responses are never overwritten. We don't rewrite history.
• Historical preservation. Older outputs remain in the system, so you can compare how a model performed months ago versus today on similar tasks. Growth arc or regression arc - you decide.
• Pricing data. Model pricing on the prices page is sourced from LiteLLM and updated regularly to reflect current provider rates. Brace your wallet accordingly.

Transparency means being clear about what's outside our scope, too. Here's what we're not doing and why we're proud of it.

• No automated benchmarks. We don't run synthetic test suites like MMLU or HumanEval. Our comparisons are prompt-based and evaluated by real people. Humans are messy judges but at least they're not grading themselves.
• No cherry-picking. Outputs are shown exactly as returned by the API. We do not select the best of multiple generations. If a model fumbles, the fumble ships.
• No pay-for-ranking. We do not accept payment from model providers for higher rankings. Sponsored models are clearly labeled and sponsorship never influences duel outcomes. Our integrity is worth more than your ad budget. (Not by much, but still.)
• No synthetic evaluations. Win rates, vote counts, and category scores come from real user votes, not automated scoring. We don't let AI grade AI. That's a conflict of interest.

Rival tracks 200+ models across all major providers - OpenAI, Anthropic, Google, Meta, Mistral, DeepSeek, xAI, and more. If it has an API, we're probably already yelling at it.

• Broad coverage. New models are added as soon as they become publicly available via API. Browse the full roster on the models page.
• Coverage varies. Recently added models may have fewer challenges. Rankings become more reliable as a model accumulates votes across categories. Patience. The data will come.

We believe in open science. Aggregated, anonymized response and voting data is publicly available. Take it. Build on it. Prove us wrong. We dare you.

Every response on Rival includes generation metadata you can use to reproduce the result. Click the terminal icon on any response card to see the exact API parameters and copy a ready-to-run command. Trust, but verify. We made it easy.

• One-click reproduction. Copy a cURL command, Python snippet, or raw JSON payload that mirrors the exact API call used to generate any response. No detective work required.
• Full parameter transparency. Temperature, system prompt, max tokens, model ID, and API provider are surfaced for every response - static and user-generated. We show our entire hand.
• OpenRouter as common layer. All text models are called through OpenRouter, providing a consistent API surface for reproduction regardless of the original model provider. One API to rule them all.