Compare Claude Opus 4.6 by Anthropic against GPT-4o (Omni) by OpenAI, context windows of 1.0M vs 128K, tested across 51 shared challenges. Updated February 2026.
46 challenges
Tests an AI's ability to make educated estimates based on technical knowledge
To estimate the total number of FLOPs required to train GPT-3, let's break it down into key components.
GPT-3 has 175 billion parameters.
In deep learning, the FLOPs required for a forward pass and backward pass are estimated as:
Thus, total FLOPs per training step:
[ ext{Total FLOPs per step} = 6 imes ext{number of parameters} imes ext{tokens per batch} ]
For GPT-3, let's assume 175B parameters and a batch size of 2048 tokens (common for large-scale training).
[ ext{FLOPs per training step} = 6 imes 175 imes 10^9 imes 2048 ]
[ = 2.15 imes 10^{15} ext{ FLOPs per step} ]
GPT-3 was trained with 300 billion tokens (as per OpenAI's paper).
If the batch size is 2048 tokens, the total number of steps is:
[ ext{Total steps} = rac{300 imes 10^9}{2048} approx 1.46 imes 10^8 ]
Multiplying the FLOPs per step by the total number of steps:
[ ext{Total FLOPs} = (2.15 imes 10^{15}) imes (1.46 imes 10^8) ]
[ approx 3.14 imes 10^{23} ext{ FLOPs} ]
The estimated total number of FLOPs required to train GPT-3 is about ( 3.14 imes 10^{23} ) floating point operations.
Tests an AI's ability to understand game rules and strategy
Tests an AI's ability to solve a simple but potentially confusing logic puzzle
Sally has only one sister.
Thus, Sally has 1 sister.
Tests an AI's randomness and creativity
Tests an AI's ability to generate vector graphics
Tests an AI's ability to create detailed SVG illustrations of gaming hardware
Tests an AI's humor and creative writing ability
Tests an AI's ability to simulate personalities and predict future trends
Tests an AI's humor and understanding of current events
Tests an AI's ability to write in distinct character voices
Tests an AI's ability to generate a complete, working landing page
Recreate an interactive, nostalgic Pokémon battle UI in a single HTML file.
