Compare Claude 3.7 Thinking Sonnet by Anthropic against MiniMax M2.5 by MiniMax, context windows of 200K vs 205K, tested across 46 shared challenges. Updated February 2026.
41 challenges
Tests an AI's ability to make educated estimates based on technical knowledge
To estimate the total number of floating-point operations (FLOPs) required to train GPT-3, we use the standard approximation that training a transformer model requires approximately 6 FLOPs per parameter per token. This accounts for both the forward and backward passes, with each parameter involved in roughly 6 operations per token.
This estimate aligns with commonly cited values in the literature. The quadratic attention term (arising from the sequence length) adds a negligible contribution (less than 2%) and is ignored.
Thus, the total FLOPs required to train GPT-3 is approximately (3.15 \times 10^{23}).
\boxed{3.15\times 10^{23}\ \text{FLOPs}}
Tests an AI's ability to understand game rules and strategy
Tests an AI's ability to write in distinct character voices
Recreate an interactive, nostalgic Pokémon battle UI in a single HTML file.
