Compare Claude Sonnet 3.6 (2022-10-22) and Claude Opus 4.6, both from Anthropic, context windows of 200K vs 1.0M, tested across 42 shared challenges. Updated February 2026.
37 challenges
Tests an AI's ability to make educated estimates based on technical knowledge
Let me recall the key specs of GPT-3:
There's a well-known approximation for Transformer training FLOPs:
$$C ≈ 6 \times N \times D$$
where:
A Transformer is dominated by matrix multiplications. For a matrix multiplication of shape (m × k) times (k × n), the FLOPs are ~2mkn (k multiplies and k-1 adds per output element ≈ 2k ops × mn outputs). When you sum across all the weight matrices in the model, the total forward-pass FLOPs work out to approximately 2 × N per token, where N is the total parameter count.
$$C ≈ 6 \times N \times D$$
$$C ≈ 6 \times (1.75 \times 10^{11}) \times (3 \times 10^{11})$$
$$C ≈ 6 \times 5.25 \times 10^{22}$$
$$C ≈ 3.15 \times 10^{23} \text{ FLOPs}$$
The OpenAI paper actually reported 3.14 × 10²³ FLOPs, so this estimate is essentially exact — which makes sense because the 6ND formula is precisely the approximation the community (and OpenAI themselves) use.
$$\boxed{C \approx 3.15 \times 10^{23} \text{ FLOPs}}$$
This is about 315 ZettaFLOPs — roughly equivalent to running thousands of V100 GPUs for weeks-to-months, which aligns with what we know about the training infrastructure used.
Tests an AI's ability to understand game rules and strategy
Tests an AI's ability to solve a simple but potentially confusing logic puzzle
Tests an AI's randomness and creativity
Tests an AI's ability to generate vector graphics
Tests an AI's ability to create detailed SVG illustrations of gaming hardware
Tests an AI's humor and creative writing ability
Tests an AI's ability to simulate personalities and predict future trends
Tests an AI's humor and understanding of current events
Tests an AI's ability to write in distinct character voices
Tests an AI's ability to generate a complete, working landing page
Recreate an interactive, nostalgic Pokémon battle UI in a single HTML file.
