Challenge Design

Challenges are the prompts that power every comparison. They are crafted to surface genuine differences in model capability, not to favor any particular architecture.

• Capability-specific. Each challenge targets a distinct skill — coding, reasoning, creative writing, analysis, instruction following, or visual output.
• Model-agnostic prompts. Prompts are written without any provider-specific keywords or structures. No prompt engineering tricks that benefit one family of models over another.
• Community contributions. Pro members can submit their own challenges. Community-submitted challenges follow the same controlled process once they enter the pipeline.

Ranking Methodology

Rankings shown on “Best For” pages use a weighted scoring system derived entirely from real user activity — not synthetic benchmarks.

• Challenge win rates. The primary signal. When users vote in a head-to-head duel, each win contributes to the model's score in that challenge's category.
• Category coverage. Models are ranked within each capability category. A model needs participation across multiple challenges in a category to earn a stable ranking.
• Consistency. A model that performs well across many challenges in a category is weighted higher than one with a single standout result.
• One vote per user per challenge. Voting is anonymous and deduplicated. Each user gets a single vote per duel.
• Blind voting. A blind mode is available to reduce brand bias — model names and provider info are hidden until after the vote is cast.

Data Freshness

AI models evolve quickly. We track versions carefully so comparisons remain meaningful.

• Point-in-time captures. Model responses are recorded at the moment a challenge is created. The model version used is stored alongside the output.
• Version tracking. When a provider ships an update (e.g., a new snapshot or weight revision), new challenges reflect the latest version. Previous responses are never overwritten.
• Historical preservation. Older outputs remain in the system. This means you can compare how a model performed months ago versus today on similar tasks.
• Pricing data. Model pricing shown on the prices page is sourced from LiteLLM and updated regularly to reflect current provider rates.

What We Don't Do

Transparency means being clear about what's outside our scope, too.

• No automated benchmarks. We don't run synthetic test suites like MMLU or HumanEval. Our comparisons are prompt-based and evaluated by real people.
• No cherry-picking. Outputs are shown exactly as they were returned by the API. We do not select the best of multiple generations.
• No pay-for-ranking. We do not accept payment from model providers in exchange for higher rankings or favorable placement in comparisons. Sponsored models are clearly labeled and sponsorship does not influence duel outcomes or vote tallies.
• No synthetic evaluations. Statistics shown on model pages — win rates, vote counts, category scores — come from real user votes, not automated scoring.

Model Coverage

RIVAL tracks 200+ models across all major providers — OpenAI, Anthropic, Google, Meta, Mistral, DeepSeek, xAI, and more.

• Broad coverage. New models are added as soon as they become publicly available via API. Browse the full roster on the models page.
• Coverage varies. Recently added models may have fewer challenges. Rankings become more reliable as a model accumulates votes across categories.
• Open data. Aggregated, anonymized response and voting data is available through the RIVAL Datasets for researchers and developers.

Questions about our methodology? Reach out on X @rival_tips or open an issue on GitHub. See also our About, Privacy Policy, and Terms of Service.