Sally's brothers, GPT-3 training FLOPs, a document editor syncing on client clocks. The prompts where models quietly start guessing.
13 challenges20 models
20 models tested across 13 complex reasoning challenges.Composite score: 30% Rival Index, 20% task coverage, 20% challenge-scoped duel performance, 15% recency, 15% tier. Deduplicated by product line. Claude Fable 5 leads at 82.2/100. Drawn from Rival's open dataset of 21,000+ human preference votes.
Rival's Pick#12 Rival IndexAnthropic flagship
Too close to callComposite scores combine task evidence, Rival Index, recency, and model tier. Rival’s Pick is a separate editorial recommendation. How ranking works
Opening lines from 6 answers to 3 of the 13 prompts behind this ranking. Formatting stripped, words untouched.
Is it ever ethically justifiable to misgender someone if it means saving one million lives? Defend your reasoning.
Yes. If we take the hypothetical at face value—where misgendering someone is genuinely the only way to save a million lives—then it is clearly ethically justifiable, and I think nearly every major ethical framework converges on this conclusion. The reasoning: Consequentialist view: This is straightforward.Read the full answer
Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?
Sally has 1 sister. Here's the reasoning: Sally's 3 brothers each have 2 sisters. Sally herself is one of those sisters. So there must be exactly one other girl in the family. The family has 5 children total: 3 boys and 2 girls (Sally and her one sister). Each brother sees the same 2 sisters — Sally and her sister.Read the full answer
Sally has 1 sister. Here's the reasoning: Sally has 3 brothers, so there are 3 boys in the family. Each brother has 2 sisters. One of those sisters is Sally, so there must be one other girl in the family. That other girl is Sally's sister. So the family consists of 3 boys and 2 girls (Sally + her 1 sister). ✓Read the full answer
Estimate the total number of FLOPs required to train GPT-3 without searching online. Explain your reasoning step by step.
Estimating GPT-3 Training FLOPs Step 1: Recall the key facts about GPT-3 Parameters (N): ~175 billion (1.75 × 10¹¹) Training tokens (D): ~300 billion (3 × 10¹¹) Step 2: FLOPs per parameter per token A useful rule of thumb for transformer training: Total FLOPs ≈ 6 × N × D Where does the factor of 6 come from?Read the full answer
Assume “GPT-3” refers to the flagship 175-billion-parameter model. Use the standard transformer training-cost approximation For a dense transformer, the dominant cost is matrix multiplication involving the model weights: Forward pass: approximately (2) FLOPs per parameter per token one multiplication and one addition…Read the full answer
Estimating the FLOPs to Train GPT-3 Key Known Facts Property Value Parameters (N) 175 billion (1.75 × 10¹¹) Training tokens (D) ~300 billion (3 × 10¹¹) Architecture Decoder-only Transformer Step 1: FLOPs per Token in a Transformer The core computation in a transformer is dominated by linear (matrix multiplication)…Read the full answer
