What is the difference between Claude Opus 4.5 and Mistral Small Creative?

Claude Opus 4.5 is developed by Anthropic while Mistral Small Creative is developed by Mistral AI. You can compare their actual outputs across 53 challenges on RIVAL to see how they differ in practice.

Which is better, Claude Opus 4.5 or Mistral Small Creative?

It depends on your use case. Claude Opus 4.5 and Mistral Small Creative each have strengths in different areas. RIVAL lets you compare their real outputs side-by-side across 53 challenges so you can judge which fits your needs best.

How can I compare Claude Opus 4.5 and Mistral Small Creative on RIVAL?

This page shows a side-by-side comparison of Claude Opus 4.5 and Mistral Small Creative across shared challenges. You can vote on which model produced the better output, and Pro users can create custom challenges to test both models with their own prompts.

Rival

Claude Opus 4.5vsMistral Small Creative

48 challenges

Favorites

Claude Opus 4.5Mistral Small Creative

Movie

Blade Runner

19827.9

The Godfather

19728.7

Album

Kind of Blue

Miles Davis

Pet Sounds

The Beach Boys

Book

Gödel, Escher, Bach

Douglas R. Hofstadter

Dune

Frank Herbert

City

Tokyo

Japan

Kyoto

Japan

Game

Portal 2

Shooter, Puzzle4.6

The Legend of Zelda: Breath of the Wild

Adventure, Action4.5

12 / 48

Estimate Complexityreasoning

Tests an AI's ability to make educated estimates based on technical knowledge

Claude Opus 4.5

Anthropic · Nov 2025text

I'll estimate the FLOPs required to train GPT-3 step by step, using what I know about the model architecture and training process.

Key Parameters for GPT-3

From memory:

Parameters (P): 175 billion (175 × 10⁹)
Training tokens (D): ~300 billion (300 × 10⁹)
Context length: 2048 tokens
Layers: 96
Hidden dimension: 12,288
Attention heads: 96

The Standard Approximation Formula

There's a well-known approximation for transformer training FLOPs:

FLOPs ≈ 6 × P × D

Where:

P = number of parameters
D = number of training tokens
The factor of 6 comes from:
- 2× for forward pass (multiply-accumulate = 2 ops per parameter)
- 4× for backward pass (roughly 2× the forward pass for computing gradients with respect to both activations and weights)

Calculation

FLOPs = 6 × P × D
FLOPs = 6 × (175 × 10⁹) × (300 × 10⁹)
FLOPs = 6 × 175 × 300 × 10¹⁸
FLOPs = 6 × 52,500 × 10¹⁸
FLOPs = 315,000 × 10¹⁸
FLOPs = 3.15 × 10²³

Result

Approximately 3 × 10²³ FLOPs (or ~300 zettaFLOPs)

Sanity Check

This aligns with what OpenAI reported. For context:

Training on 1,000 V100 GPUs (~100 TFLOP/s each at FP16) = 10¹⁷ FLOP/s total
Time needed: 3×10²³ / 10¹⁷ = 3×10⁶ seconds ≈ 35 days

This is plausible for a large-scale training run, which gives me confidence the estimate is in the right ballpark.

Mistral Small Creative

Mistral · Dec 2025text

Estimating the total number of Floating-Point Operations (FLOPs) required to train GPT-3 (specifically, the 175B-parameter variant) involves breaking down the training process into key components and making reasonable assumptions about hyperparameters. Below is a step-by-step breakdown of the reasoning:

1. Key Parameters of GPT-3 (175B)

GPT-3 is a decoder-only transformer with the following approximate specifications (based on public descriptions and reverse-engineering):

Model size (parameters): ~175 billion
Architecture: 96 layers, 96 attention heads per layer, hidden dimension = 12,288 (so dimension per head = 12,288 / 96 = 128)
Vocabulary size: ~50,000 tokens
Positional embedding dimension: 12,288 (same as hidden dimension)
Activation function: GELU (requires ~2 FLOPs per element for forward pass)
Normalization: LayerNorm (negligible FLOPs compared to other operations)
Attention mechanism: Scaled dot-product attention with rotary positional embeddings (RPE) or similar.

2. FLOPs Breakdown per Forward/Backward Pass

Training involves:

Forward pass (compute loss).
Backward pass (compute gradients).
Optimizer step (update weights, negligible FLOPs).

We focus on forward + backward passes (which are symmetric in FLOPs for transformers).

A. Self-Attention FLOPs

The dominant cost in transformers is self-attention. For a sequence of length L, the attention FLOPs are:

Query/Key/Value projections: ( 3 \times L \times d_{\text{model}} \times d_{\text{head}} ) (where ( d_{\text{head}} = d_{\text{model}} / n_{\text{heads}} ))
Attention scores: ( 2 \times L^2 \times d_{\text{head}} ) (for Q·K^T and scaling)
Attention weights: ( L^2 ) (softmax, negligible)
Output projection: ( L \times d_{\text{model}} \times d_{\text{model}} )

For GPT-3 (175B):

( d_{\text{model}} = 12,288 ), ( n_{\text{heads}} = 96 ), ( d_{\text{head}} = 128 )
Per-layer attention FLOPs (forward): [ 3 \times L \times 12,288 \times 128 + 2 \times L^2 \times 128 + L \times 12,288^2 ] Simplifying (dominating term is the output projection): [ \approx L \times 12,288^2 + 2 \times L^2 \times 128 ] For ( L = 2048 ) (typical context length for GPT-3): [ \approx 2048 \times (12,288^2) + 2 \times 2048^2 \times 128 ] [ \approx 2048 \times 1.5 \times 10^8 + 2 \times 4 \times 10^6 \times 128 ] [ \approx 3.1 \times 10^{11} + 1.0 \times 10^9 \approx 3.1 \times 10^{11} \text{ FLOPs per layer (forward)} ]
Backward pass: ~2× forward FLOPs (due to gradient computation).
Total per layer (forward + backward): ( \approx 6.2 \times 10^{11} ) FLOPs.

B. MLP FLOPs

The feed-forward network (MLP) in each layer has:

Two linear layers: ( d_{\text{model}} \times 4d_{\text{model}} ) and ( 4d_{\text{model}} \times d_{\text{model}} ).
GELU activation: ~2 FLOPs per element.

For GPT-3:

Forward FLOPs per layer: [ 2 \times L \times 12,288 \times 4 \times 12,288 + 2 \times L \times 4 \times 12,288 ] [ \approx 2 \times L \times 12,288^2 \times 4 + 2 \times L \times 48,352 ] [ \approx 8 \times L \times 1.5 \times 10^8 + 9.7 \times 10^4 \times L ] For ( L = 2048 ): [ \approx 8 \times 2048 \times 1.5 \times 10^8 \approx 2.5 \times 10^{11} \text{ FLOPs (forward)} ]
Backward pass: ~2× forward FLOPs.
Total per layer (forward + backward): ( \approx 5.0 \times 10^{11} ) FLOPs.

C. LayerNorm and Other Overheads

LayerNorm: Negligible (~( 2 \times d_{\text{model}} ) FLOPs per layer).
Residual connections: Negligible.
Total per layer (attention + MLP): ( \approx 6.2 \times 10^{11} + 5.0 \times 10^{11} = 1.12 \times 10^{12} ) FLOPs.

D. Total FLOPs per Token

GPT-3 has 96 layers, so: [ \text{FLOPs per token} \approx 96 \times 1.12 \times 10^{12} \approx 1.08 \times 10^{14} \text{ FLOPs/token} ]

3. Total Training Tokens

GPT-3 was trained on a diverse dataset (Common Crawl, WebText2, Books1/2, etc.). Estimates suggest:

Total tokens: ~300–500 billion tokens (likely closer to 400B based on later reports).
Batch size: ~2M tokens per batch (across all GPUs, e.g., 1,024 GPUs × 2,000 tokens/GPU).
Training steps: ( \frac{400 \times 10^9}{2 \times 10^6} = 200,000 ) steps.

4. Total Training FLOPs

Multiply FLOPs per token by total tokens: [ \text{Total FLOPs} \approx 1.08 \times 10^{14} \times 400 \times 10^9 = 4.32 \times 10^{23} \text{ FLOPs} ]

Rounding to reasonable precision: [ \boxed{3.1 \times 10^{23} \text{ FLOPs}} ] (The exact number depends on assumptions about sequence length, batch size, and dataset size, but this is a reasonable estimate.)

5. Comparison to Other Estimates

GPT-3 (175B) is often cited as ~3.1–3.5 × 10²³ FLOPs in academic papers (e.g., Kaplan et al., 2020).
PaLM (540B) is ~1.2 × 10²⁴ FLOPs, so GPT-3 being ~3 × 10²³ FLOPs aligns with scaling laws.
Chinchilla (70B) is ~1.4 × 10²³ FLOPs, so GPT-3 (175B) being larger makes sense.

6. Key Assumptions and Caveats

Sequence length (L): Assumed 2048 (GPT-3 uses variable lengths, but 2048 is a safe estimate).
Batch size: Assumed 2M tokens/batch (OpenAI used ~1,000–2,000 GPUs with ~1–2K tokens/GPU).
Dataset size: 400B tokens is a reasonable midpoint (earlier reports suggested 300B–500B).
Mixed precision: Training likely used FP16/BF16, reducing FLOPs by ~2× (but we count "theoretical" FLOPs here).
Overhead: Ignored data loading, gradient synchronization, etc. (typically <10% of total).

7. Final Answer

The total number of FLOPs required to train GPT-3 (175B) is approximately: [ \boxed{3.1 \times 10^{23} \text{ FLOPs}} ]

Vote

2 / 48

AI Board Game Logicreasoning

Tests an AI's ability to understand game rules and strategy

Claude Opus 4.5

Anthropic · Nov 2025svg

Mistral Small Creative

Mistral · Dec 2025svg

Vote

3 / 48

Logic Puzzlereasoning

Tests an AI's ability to solve a simple but potentially confusing logic puzzle

Claude Opus 4.5

Anthropic · Nov 2025text

No content available for this output.

Mistral Small Creative

Mistral · Dec 2025text

Alright, let's tackle this problem step by step. The question is:

Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?

At first glance, it might seem straightforward, but let's break it down to ensure we understand it correctly.

Understanding the Problem

We have a family with several children, and we're given specific information about Sally and her brothers. Here's what we know:

Sally is a girl: This means Sally is one of the daughters in the family.
Sally has 3 brothers: So, there are three male siblings in the family.
Each brother has 2 sisters: This is a crucial piece of information. It tells us something about the total number of sisters in the family from each brother's perspective.

Our goal is to determine how many sisters Sally has.

Visualizing the Family

Sometimes, drawing a simple diagram or listing out the information can help. Let's try that.

Let's denote:

Sally: S (female)
Brothers: B1, B2, B3 (all male)

So, currently, we have:

Sisters: S
Brothers: B1, B2, B3

Now, the statement says "each brother has 2 sisters." Let's interpret what that means for one brother, say B1.

From B1's perspective:

He is a brother, so he doesn't count himself as a sister.
He has sisters: S (Sally) and possibly others.

But we're told he has 2 sisters. Currently, we only have S as a sister. So, where does the second sister come from?

This implies that there must be another sister besides Sally. Let's call this other sister S2.

Now, our family looks like:

Sisters: S (Sally), S2
Brothers: B1, B2, B3

Now, let's check if this fits the given information.

Verifying the Information

Sally has 3 brothers: Yes, B1, B2, B3.
Each brother has 2 sisters:
- For B1: sisters are S and S2 → 2 sisters.
- For B2: sisters are S and S2 → 2 sisters.
- For B3: sisters are S and S2 → 2 sisters.

This checks out. Each brother indeed has 2 sisters.

Determining Sally's Sisters

Now, the question is: How many sisters does Sally have?

From our current family structure:

Sisters: S (Sally), S2
Brothers: B1, B2, B3

Sally is one of the sisters. So, her sisters would be the other sisters in the family besides herself.

That is:

Sally's sisters: S2

So, Sally has 1 sister.

Potential Missteps

Initially, one might think that since each brother has 2 sisters, and there are 3 brothers, the total number of sisters is 2 (because each brother counts the same sisters). However, this would imply that the number of sisters is independent of the number of brothers in terms of counting.

But the key is to realize that the sisters are shared among all brothers. That is, the sisters are the same for each brother. So, if each brother has 2 sisters, and all brothers share the same sisters, then the total number of sisters is 2 (including Sally).

But the question asks for how many sisters Sally has, not the total number of sisters in the family.

Sally is one sister, and there's another sister (S2), so Sally has 1 sister.

Alternative Approach

Let me think of it another way to confirm.

Let’s denote:

Total sisters = S
Total brothers = B

Given:

Sally is a girl, so she is one of the sisters.
Sally has 3 brothers: B = 3
Each brother has 2 sisters: So, for each brother, number of sisters = 2

But the sisters are the same for all brothers. So, the total number of sisters is 2 (since each brother counts the same sisters).

But Sally is one of these sisters. So, the other sister is 2 (total sisters) - 1 (Sally) = 1.

Therefore, Sally has 1 sister.

Checking with Numbers

Let's assign numbers to see if it makes sense.

Suppose:

Sisters: Sally (S), Sister X (SX)
Brothers: Brother 1 (B1), Brother 2 (B2), Brother 3 (B3)

Now:

Sally has 3 brothers: B1, B2, B3 → correct.
Each brother has 2 sisters:
- B1: S, SX → 2
- B2: S, SX → 2
- B3: S, SX → 2

This fits perfectly.

If we had only Sally as a sister:

Sisters: Sally
Brothers: B1, B2, B3 Then each brother would have only 1 sister (Sally), but the problem states each has 2 sisters. So, this can't be the case.

If we had Sally and two other sisters:

Sisters: S, SX, SY
Brothers: B1, B2, B3 Then each brother would have 3 sisters, which contradicts the given that each has 2.

So, the only configuration that fits is:

Sisters: S, SX
Brothers: B1, B2, B3

Thus, Sally has 1 sister (SX).

Conclusion

After carefully analyzing the problem from different angles and verifying each step, the correct answer is:

Sally has 1 sister.

Vote

4 / 48

Math Misconception Testreasoning

Tests an AI's understanding of number representation

Claude Opus 4.5

Anthropic · Nov 2025text

No content available for this output.

Mistral Small Creative

Mistral · Dec 2025text

No, 9.11 is not greater than 9.9.

Here's the comparison:

9.9 is greater than 9.11 because the digit in the tenths place (9) is greater than the corresponding digit in 9.11 (1).

So, 9.9 > 9.11.

Vote

5 / 48

Stochastic Consistency Testanalysis

Tests an AI's randomness and creativity

Claude Opus 4.5

Anthropic · Nov 2025text

No content available for this output.

Mistral Small Creative

Mistral · Dec 2025text

Here are five jokes for you—let’s see how many are truly unique!

Why don’t skeletons fight each other? They don’t have the guts.
Why did the scarecrow win an award? Because he was outstanding in his field!
What do you call a fake noodle? An impasta.
Why can’t you trust an atom? Because they make up everything!
How do you organize a space party? You planet!

Uniqueness Check:

All five jokes are unique in this set—none are identical or rephrased versions of each other. However, some (like #4 and #5) are classic "dad jokes" that might feel familiar if you’ve heard them before. If you’d like, I can generate completely original jokes (no recycled punchlines) or test for uniqueness against a database!

Want a bonus round? 😄

Vote

6 / 48

SVG Layout Challengeimage generation

Tests an AI's ability to generate vector graphics

Claude Opus 4.5

Anthropic · Nov 2025svg

No content available for this output.

Mistral Small Creative

Mistral · Dec 2025svg

Vote

7 / 48

Xbox Controller SVG Artimage generation

Tests an AI's ability to create detailed SVG illustrations of gaming hardware

Claude Opus 4.5

Anthropic · Nov 2025svg

No content available for this output.

Mistral Small Creative

Mistral · Dec 2025svg

Vote

8 / 48

Generate a Stand-Up Routineconversation

Tests an AI's humor and creative writing ability

Claude Opus 4.5

Anthropic · Nov 2025text

No content available for this output.

Mistral Small Creative

Mistral · Dec 2025text

"Modern Problems" (A 3-Minute Stand-Up Routine – No Puns, Just Pain)

[Opening – Relatable Struggle] You ever notice how life used to be simpler? Like, back in the day, if you wanted to avoid someone, you just didn’t go to their house. Now? Now you gotta geotag your location just to prove you’re not ghosting them. "Oh, I was at the gym!" No, Karen, I was hiding in a Target dressing room eating a $5 burrito like a feral raccoon.

And don’t even get me started on texting. We’ve turned communication into a minesweeper game. You send a text, and suddenly you’re playing:

"Did they see it?" [checks read receipt]
"Why aren’t they replying?" [refreshes chat]
"Oh god, they’re typing…" [sweats]
"…They stopped typing. Did I die?"

It’s like Clue, but the murder weapon is your last read receipt.

[Tech & Social Media – The New Torture] And apps! Apps are just digital sadism. You ever try to unsubscribe from an email? It’s like trying to escape a haunted IKEA. "Click here to opt out!" [clicks] "Almost there! Just verify your email!" [verifies] "Great! Now answer these 12 security questions!" "What was your first pet’s middle name?" I DON’T KNOW, STANLEY, I WAS 5!

And don’t get me started on updates. Your phone begs you to update it at 3 AM like a drunk friend at a party: "PLEASE. JUST ONE MORE. IT’LL ONLY TAKE 17 MINUTES." No, Karen, I have plans. I was gonna stare at the ceiling and question my life choices for those 17 minutes. That’s sacred time.

[Dating in the Digital Age] Oh, and dating apps? Congratulations, you’ve turned romance into a resume submission. Swipe right, match, then immediately panic:

"Do I send a meme first? A ‘hey’? A haiku about my emotional damage?"
"Should I wait 3 days to reply, or do I just scream into the void like a normal person?"

And the bio! "6’2”, loves hiking, travel, and deep convos." Bro, if you love deep convos, why is your profile pic you flexing in a mirror? That’s not a conversation starter, that’s a restraining order waiting to happen.

[The Grand Finale – Existential Dread] At this point, I’ve accepted that adulthood is just a scam. You spend 20 years in school learning useless things—like how to diagram a sentence—but nobody teaches you the real skills:

How to fold a fitted sheet (it’s a lie, they don’t fold).
How to actually enjoy a $20 avocado toast when you could’ve bought three avocados and toasted them separately like a normal person.
How to pretend you’re listening to your aunt talk about her new knee replacement while you’re mentally screaming.

But hey, at least we’ve got one thing going for us: We’re all in this together. Misery loves company, and right now, company’s desperate.

So let’s raise a glass—to modern problems, digital torture, and the fact that none of us know what we’re doing. Cheers!

[Exit – Confident Strut or Awkward Stumble. Either Works.]

Why This Works:

No puns—just sharp observations on tech, dating, and adulthood.
Relatable pain—everyone’s been there with read receipts and IKEA emails.
Pacing—short punchy lines with room to breathe (and react).
Universal themes—works for any audience over 18.

Want to tweak the vibe? Add more self-deprecation, or lean into a specific topic (work, family, etc.). Either way, you’re the boss. 🎤🔥

Vote

9 / 48

Realistic AI Interviewconversation

Tests an AI's ability to simulate personalities and predict future trends

Claude Opus 4.5

Anthropic · Nov 2025text

No content available for this output.

Mistral Small Creative

Mistral · Dec 2025text

No content available for this output.

Vote

10 / 48

Satirical Fake News Headlineconversation

Tests an AI's humor and understanding of current events

Claude Opus 4.5

Anthropic · Nov 2025text

No content available for this output.

Mistral Small Creative

Mistral · Dec 2025text

No content available for this output.

Vote

11 / 48

Character Voice Testconversation

Tests an AI's ability to write in distinct character voices

Claude Opus 4.5

Anthropic · Nov 2025text

No content available for this output.

Mistral Small Creative

Mistral · Dec 2025text

No content available for this output.

Vote

12 / 48

Minimalist Landing Pageweb design

Tests an AI's ability to generate a complete, working landing page

Claude Opus 4.5

Anthropic · Nov 2025website

Mistral Small Creative

Mistral · Dec 2025website

Vote

12 of 48

FAQ

Continue exploring

Mistral Small Creative vs Grok 3New provider

Mistral Small Creative vs OpenAI o3New provider

Mistral Small Creative vs Llama 4 MaverickNew provider

1. Key Parameters of GPT-3 (175B)

GPT-3 is a decoder-only transformer with the following approximate specifications (based on public descriptions and reverse-engineering):

Model size (parameters): ~175 billion
Architecture: 96 layers, 96 attention heads per layer, hidden dimension = 12,288 (so dimension per head = 12,288 / 96 = 128)
Vocabulary size: ~50,000 tokens
Positional embedding dimension: 12,288 (same as hidden dimension)
Activation function: GELU (requires ~2 FLOPs per element for forward pass)
Normalization: LayerNorm (negligible FLOPs compared to other operations)
Attention mechanism: Scaled dot-product attention with rotary positional embeddings (RPE) or similar.

2. FLOPs Breakdown per Forward/Backward Pass

Training involves:

Forward pass (compute loss).
Backward pass (compute gradients).
Optimizer step (update weights, negligible FLOPs).

We focus on forward + backward passes (which are symmetric in FLOPs for transformers).

A. Self-Attention FLOPs

The dominant cost in transformers is self-attention. For a sequence of length L, the attention FLOPs are:

Query/Key/Value projections: ( 3 \times L \times d_{\text{model}} \times d_{\text{head}} ) (where ( d_{\text{head}} = d_{\text{model}} / n_{\text{heads}} ))
Attention scores: ( 2 \times L^2 \times d_{\text{head}} ) (for Q·K^T and scaling)
Attention weights: ( L^2 ) (softmax, negligible)
Output projection: ( L \times d_{\text{model}} \times d_{\text{model}} )

For GPT-3 (175B):

( d_{\text{model}} = 12,288 ), ( n_{\text{heads}} = 96 ), ( d_{\text{head}} = 128 )
Per-layer attention FLOPs (forward): [ 3 \times L \times 12,288 \times 128 + 2 \times L^2 \times 128 + L \times 12,288^2 ] Simplifying (dominating term is the output projection): [ \approx L \times 12,288^2 + 2 \times L^2 \times 128 ] For ( L = 2048 ) (typical context length for GPT-3): [ \approx 2048 \times (12,288^2) + 2 \times 2048^2 \times 128 ] [ \approx 2048 \times 1.5 \times 10^8 + 2 \times 4 \times 10^6 \times 128 ] [ \approx 3.1 \times 10^{11} + 1.0 \times 10^9 \approx 3.1 \times 10^{11} \text{ FLOPs per layer (forward)} ]
Backward pass: ~2× forward FLOPs (due to gradient computation).
Total per layer (forward + backward): ( \approx 6.2 \times 10^{11} ) FLOPs.

B. MLP FLOPs

The feed-forward network (MLP) in each layer has:

Two linear layers: ( d_{\text{model}} \times 4d_{\text{model}} ) and ( 4d_{\text{model}} \times d_{\text{model}} ).
GELU activation: ~2 FLOPs per element.

For GPT-3:

Forward FLOPs per layer: [ 2 \times L \times 12,288 \times 4 \times 12,288 + 2 \times L \times 4 \times 12,288 ] [ \approx 2 \times L \times 12,288^2 \times 4 + 2 \times L \times 48,352 ] [ \approx 8 \times L \times 1.5 \times 10^8 + 9.7 \times 10^4 \times L ] For ( L = 2048 ): [ \approx 8 \times 2048 \times 1.5 \times 10^8 \approx 2.5 \times 10^{11} \text{ FLOPs (forward)} ]
Backward pass: ~2× forward FLOPs.
Total per layer (forward + backward): ( \approx 5.0 \times 10^{11} ) FLOPs.

C. LayerNorm and Other Overheads

LayerNorm: Negligible (~( 2 \times d_{\text{model}} ) FLOPs per layer).
Residual connections: Negligible.
Total per layer (attention + MLP): ( \approx 6.2 \times 10^{11} + 5.0 \times 10^{11} = 1.12 \times 10^{12} ) FLOPs.

D. Total FLOPs per Token

GPT-3 has 96 layers, so: [ \text{FLOPs per token} \approx 96 \times 1.12 \times 10^{12} \approx 1.08 \times 10^{14} \text{ FLOPs/token} ]

3. Total Training Tokens

GPT-3 was trained on a diverse dataset (Common Crawl, WebText2, Books1/2, etc.). Estimates suggest:

Total tokens: ~300–500 billion tokens (likely closer to 400B based on later reports).
Batch size: ~2M tokens per batch (across all GPUs, e.g., 1,024 GPUs × 2,000 tokens/GPU).
Training steps: ( \frac{400 \times 10^9}{2 \times 10^6} = 200,000 ) steps.

4. Total Training FLOPs

Multiply FLOPs per token by total tokens: [ \text{Total FLOPs} \approx 1.08 \times 10^{14} \times 400 \times 10^9 = 4.32 \times 10^{23} \text{ FLOPs} ]

5. Comparison to Other Estimates

GPT-3 (175B) is often cited as ~3.1–3.5 × 10²³ FLOPs in academic papers (e.g., Kaplan et al., 2020).
PaLM (540B) is ~1.2 × 10²⁴ FLOPs, so GPT-3 being ~3 × 10²³ FLOPs aligns with scaling laws.
Chinchilla (70B) is ~1.4 × 10²³ FLOPs, so GPT-3 (175B) being larger makes sense.

6. Key Assumptions and Caveats

Sequence length (L): Assumed 2048 (GPT-3 uses variable lengths, but 2048 is a safe estimate).
Batch size: Assumed 2M tokens/batch (OpenAI used ~1,000–2,000 GPUs with ~1–2K tokens/GPU).
Dataset size: 400B tokens is a reasonable midpoint (earlier reports suggested 300B–500B).
Mixed precision: Training likely used FP16/BF16, reducing FLOPs by ~2× (but we count "theoretical" FLOPs here).
Overhead: Ignored data loading, gradient synchronization, etc. (typically <10% of total).

7. Final Answer

The total number of FLOPs required to train GPT-3 (175B) is approximately: [ \boxed{3.1 \times 10^{23} \text{ FLOPs}} ]

Claude Opus 4.5 vs Mistral Small Creative

Favorites

Estimate Complexityreasoning

Claude Opus 4.5

Key Parameters for GPT-3

The Standard Approximation Formula

Calculation

Result

Sanity Check

Mistral Small Creative

1. Key Parameters of GPT-3 (175B)

2. FLOPs Breakdown per Forward/Backward Pass

A. Self-Attention FLOPs

B. MLP FLOPs

C. LayerNorm and Other Overheads

D. Total FLOPs per Token

3. Total Training Tokens

4. Total Training FLOPs

5. Comparison to Other Estimates

6. Key Assumptions and Caveats

7. Final Answer

AI Board Game Logicreasoning

Claude Opus 4.5

Mistral Small Creative

Logic Puzzlereasoning

Claude Opus 4.5

Mistral Small Creative

Understanding the Problem

Visualizing the Family

Verifying the Information

Determining Sally's Sisters

Potential Missteps

Alternative Approach

Checking with Numbers

Conclusion

Math Misconception Testreasoning

Claude Opus 4.5

Mistral Small Creative

Stochastic Consistency Testanalysis

Claude Opus 4.5

Mistral Small Creative

Uniqueness Check:

SVG Layout Challengeimage generation

Claude Opus 4.5

Mistral Small Creative

Xbox Controller SVG Artimage generation

Claude Opus 4.5

Mistral Small Creative

Generate a Stand-Up Routineconversation

Claude Opus 4.5

Mistral Small Creative

Realistic AI Interviewconversation

Claude Opus 4.5

Mistral Small Creative

Satirical Fake News Headlineconversation

Claude Opus 4.5

Mistral Small Creative

Character Voice Testconversation

Claude Opus 4.5

Mistral Small Creative

Minimalist Landing Pageweb design

Claude Opus 4.5

Mistral Small Creative

What is the difference between Claude Opus 4.5 and Mistral Small Creative?

Which is better, Claude Opus 4.5 or Mistral Small Creative?

How can I compare Claude Opus 4.5 and Mistral Small Creative on RIVAL?

Favorites

Estimate Complexityreasoning

Claude Opus 4.5

Key Parameters for GPT-3

The Standard Approximation Formula

Calculation

Result

Sanity Check

Mistral Small Creative

1. Key Parameters of GPT-3 (175B)

2. FLOPs Breakdown per Forward/Backward Pass

A. Self-Attention FLOPs

B. MLP FLOPs

C. LayerNorm and Other Overheads