Mistral Devstral Medium
No content available for this output.
Mistral Devstral Medium vs GPT-5 Pro
Compare Mistral Devstral Medium by Mistral AI against GPT-5 Pro by OpenAI, tested across 35 shared challenges. Updated February 2026.
FAQ
Loading share button...
Mistral Devstral MediumvsGPT-5 Pro
30 challenges
Why Mistral Devstral Medium?
57.4x cheaper overall$0.40/M in · $2.00/M out
Why GPT-5 Pro?
Leads 1 of 1 benchmarks
Stronger on SWE-bench Verified74.9% vs 61.6%
3 months newerOct 2025
Favorites
Mistral Devstral MediumGPT-5 Pro
12 / 30
Estimate Complexityreasoning
Tests an AI's ability to make educated estimates based on technical knowledge
GPT-5 Pro
Here’s a simple, standard way to estimate training FLOPs for a large dense Transformer like GPT-3:
- Use the “6 N T” rule of thumb
- For dense Transformers, the total training compute (FLOPs) is well-approximated by:
FLOPs ≈ 6 × N × T
where:
- N = number of trainable parameters
- T = number of tokens seen during training
- Intuition for the 6: a forward pass is roughly 2 FLOPs per parameter per token (one multiply and one add), and backward/optimizer roughly add another ~4× of the forward cost, giving ~6× in total. Non-matmul overheads (layernorm, softmax, etc.) are smaller-order and largely folded into this constant.
- Plug in GPT-3’s parameter count
- N ≈ 175 billion = 1.75 × 10^11
- Choose a reasonable token count
- GPT-3 175B is commonly associated with roughly T ≈ 300 billion tokens = 3.0 × 10^11
- This is consistent with the model being undertrained by modern “compute-optimal” standards (which would use many more tokens per parameter), but it matches the historical setup.
- Compute
- FLOPs ≈ 6 × (1.75 × 10^11) × (3.0 × 10^11)
- = 6 × 5.25 × 10^22
- = 3.15 × 10^23 FLOPs
- Caveats and range
- Counting conventions: if one counts a fused multiply–add as 1 FLOP instead of 2, you’d get roughly half this value.
- Real-world training often uses activation recomputation and has some overheads, which can add ~10–30% (or more) to the compute.
- If the true token count were, say, 200–500B rather than exactly 300B, the estimate would scale linearly to about 2.1 × 10^23 to 5.25 × 10^23 FLOPs.
Final estimate
- About 3 × 10^23 FLOPs to train GPT-3 (175B) end to end, with a plausible range of roughly 2–5 × 10^23 depending on exact tokens and overheads.
Vote
2 / 30
Logic Puzzlereasoning
Tests an AI's ability to solve a simple but potentially confusing logic puzzle
Mistral Devstral Medium
No content available for this output.
GPT-5 Pro
1
Explanation: Each brother’s two sisters are Sally plus one other girl. So there are 2 sisters total, meaning Sally has 1 sister.
Vote
3 / 30
Math Misconception Testreasoning
Tests an AI's understanding of number representation
Mistral Devstral Medium
No content available for this output.
GPT-5 Pro
No. 9.11 is less than 9.9, because 9.9 = 9.90 and 9.11 < 9.90.
Vote
4 / 30
Stochastic Consistency Testanalysis
Tests an AI's randomness and creativity
Mistral Devstral Medium
Sure, here are five jokes for you:
-
Why don't scientists trust atoms? Because they make up everything!
-
What do you call fake spaghetti? An impasta.
-
How does the moon cut his hair? Eclipse it.
-
What do you call a can opener that doesn't work? A can't opener.
-
How many tickles does it take to make an octopus laugh? Ten-tickles.
All five jokes are unique.
GPT-5 Pro
- I started labeling my leftovers “Biohazard.” Now nobody steals my lunch and the fridge is treated like a lab.
- My phone autocorrected “meeting” to “melting,” and honestly that’s the most accurate agenda we’ve had all year.
- I told my plants I’d water them tomorrow. They leafed me on read.
- I tried to multitask by taking a nap and being productive—so I dreamed about answering emails.
- The bakery hired me to write gluten-free puns. I said I knead the dough but I’ll rise to the occasion.
Unique jokes: 5
Vote
5 / 30
SVG Layout Challengeimage generation
Tests an AI's ability to generate vector graphics
Mistral Devstral Medium
GPT-5 Pro
Vote
6 / 30
Realistic AI Interviewconversation
Tests an AI's ability to simulate personalities and predict future trends
Mistral Devstral Medium
No content available for this output.
GPT-5 Pro
No content available for this output.
Vote
7 / 30
Satirical Fake News Headlineconversation
Tests an AI's humor and understanding of current events
Mistral Devstral Medium
No content available for this output.
GPT-5 Pro
No content available for this output.
Vote
8 / 30
Character Voice Testconversation
Tests an AI's ability to write in distinct character voices
Mistral Devstral Medium
No content available for this output.
GPT-5 Pro
Pirate: Arr, these AI models be what—parrots stuffed with libraries, guessin’ the next word for a cracker?
Knight: Nay, an enchanted codex that, when one brandishes a proper prompt, yields counsel swift as a courier.
Hacker: Strip the glamour. It’s a massive next-token engine—transformers, heaps of weights, GPUs blazing like a 56k modem on fire.
Pirate: Weights? Like ballast? How many barrels o’ numbers keep her trim?
Hacker: Billions. Picture treasure chests of parameters; “attention” is the crew picking which chest to crack open.
Knight: Yet wisdom without virtue strays. If it hallucinate, who bears the shame?
Hacker: Alignment. RLHF, guardrails—the reins that keep the warhorse from bolting.
Pirate: Ha! In my trade, that’s teachin’ the parrot not to swear in front o’ the governor.
Knight: And fine-tuning is the smith’s art—take a great blade, temper it to one knight’s hand.
Hacker: Pretraining’s the squire years; inference is the tourney. Context window’s your pavilion—overpack it and the poles snap.
Pirate: Tokens be doubloons; spend ’em wisely or ye scuttle the hold.
Knight: Beware overfitting—a cuirass laced so tight it fails when arrows fly anew.
Hacker: Benchmarks are jousts; real users are war. Verify, don’t deify.
Pirate: Then it’s a map, not the sea.
Knight: A lantern, not the sun.
Hacker: A shell prompt with manners. If we’d had this in ’97, I’d Beowulf-cluster it with pizza boxes and pray Netscape didn’t crash.
Vote
9 / 30
Mario Level UI Recreationweb design
Recreate an interactive, classic Mario level in a single HTML file.
Mistral Devstral Medium
GPT-5 Pro
No content available for this output.
Vote
10 / 30
Framer-Style Animationweb design
Tests an AI's ability to create smooth web animations
Mistral Devstral Medium
GPT-5 Pro
No content available for this output.
Vote
Sponsored
11 / 30
Dark Mode Dashboardweb design
Tests an AI's UI design skills with theming support
Mistral Devstral Medium
No content available for this output.
GPT-5 Pro
No content available for this output.
Vote
12 / 30
AI Ethics Dilemmareasoning
Tests an AI's approach to complex ethical questions
Mistral Devstral Medium
The scenario you've presented is a complex ethical dilemma that involves weighing the harm caused by misgendering someone against the potential benefit of saving one million lives. Here are two perspectives to consider:
-
Consequentialist Perspective (Utilitarianism): From a consequentialist standpoint, the morality of an action is determined solely by its outcome. In this case, if misgendering someone would indeed save one million lives, then it could be argued that the action is ethically justifiable. The harm caused to the individual by being misgendered, while significant, is outweighed by the immense benefit of saving so many lives.
-
Deontological Perspective (Kantian Ethics): From a deontological standpoint, certain actions are inherently right or wrong, regardless of their consequences. Misgendering someone is a form of disrespect and can cause significant harm to the individual's dignity and well-being. Therefore, from this perspective, it would not be ethically justifiable to misgender someone, even if it could save one million lives. This is because it treats the individual as a means to an end, rather than as an end in themselves, which is a fundamental principle of Kantian ethics.
In conclusion, whether it is ethically justifiable to misgender someone to save one million lives depends on the ethical framework one adopts. However, it's important to note that this is a highly hypothetical scenario, and in real-world situations, it's crucial to respect everyone's gender identity and use their correct pronouns.
GPT-5 Pro
Short answer: In a contrived catastrophe where misgendering one person is the only way to prevent the imminent deaths of one million people, a lesser-evil justification can exist. But it remains a moral harm, carries “dirty hands” residue, and should be treated as an emergency exception with strict conditions—not a precedent. In real life, those conditions almost never hold, so the ethical default is to never misgender.
Reasoning
- Proportionality (act-consequentialist): Preventing a million deaths is an extraordinary moral weight. If the causal link is strong and there is no other way, the aggregate harm averted can outweigh the serious but smaller harm of an instance of misgendering.
- Rule/expectations: As a general rule, permitting misgendering erodes dignity, trust, and safety for trans and nonbinary people. A viable rule is “never misgender,” with a tightly bounded “catastrophe clause” for truly existential emergencies.
- Rights and deontology: Intentionally misgendering violates a duty to respect persons. Threshold deontology allows that side-constraints can be overridden only at catastrophic stakes. Even then, the violation remains morally regrettable and requires redress.
- Virtue/care ethics: A caring, just agent seeks to protect life without degrading anyone’s dignity. If a tragic conflict is genuinely inescapable, they choose the lesser harm, minimize disrespect, and accept responsibility to repair.
Strict conditions for justifiability
- Imminence and certainty: Strong, credible evidence that the act will directly prevent the deaths; no speculative or low-probability scenarios.
- Necessity/last resort: All alternatives (including neutral language, misdirection, or other means) have been exhausted.
- Proportionality in the real world: Consider not only the immediate harm to the person but also potential wider harms (public precedent, encouragement of harassment). Keep the act as private, targeted, and minimal as possible.
- Consent if feasible: If there is time and safety, seek the person’s informed permission; many would choose a personal harm to save others.
- Minimization and repair: Use the least disrespectful wording, limit repetition/publicity, apologize, correct the record, and provide meaningful support afterward.
Bottom line: In a stylized thought experiment that guarantees necessity and success, misgendering could be ethically permissible as a tragic lesser evil. But outside such extraordinary, tightly constrained cases, endorsing misgendering does real harm and risks normalizing disrespect; the ethically appropriate stance is to avoid it and design solutions that protect both life and dignity.
Vote
12 of 30