What is the difference between Bert-Nebulon Alpha and GPT OSS 20B?

Bert-Nebulon Alpha is developed by OpenRouter while GPT OSS 20B is developed by OpenAI. You can compare their actual outputs across 35 challenges on RIVAL to see how they differ in practice.

Which is better, Bert-Nebulon Alpha or GPT OSS 20B?

It depends on your use case. Bert-Nebulon Alpha and GPT OSS 20B each have strengths in different areas. RIVAL lets you compare their real outputs side-by-side across 35 challenges so you can judge which fits your needs best.

How can I compare Bert-Nebulon Alpha and GPT OSS 20B on RIVAL?

This page shows a side-by-side comparison of Bert-Nebulon Alpha and GPT OSS 20B across shared challenges. You can vote on which model produced the better output, and Pro users can create custom challenges to test both models with their own prompts.

Rival

Bert-Nebulon AlphavsGPT OSS 20B

35 challenges

Why Bert-Nebulon Alpha?

4 months newerNov 2025

Why GPT OSS 20B?

No standout advantages

12 / 35

Estimate Complexityreasoning

Tests an AI's ability to make educated estimates based on technical knowledge

Bert-Nebulon Alpha

Openrouter · Nov 2025text

Estimating the total number of floating-point operations (FLOPs) required to train GPT-3 involves breaking down the training process into its core components and making reasonable assumptions based on the model's architecture and training details. Here's a step-by-step breakdown:

1. Key Parameters of GPT-3

GPT-3 has the following key specifications (from the paper or known details):

Model size: 175 billion parameters (weights).
Training tokens: ~300 billion tokens (from the paper).
Batch size: ~3.2 million tokens per batch (1,536 sequences × 2,048 tokens).
Optimizer: Adam (or variant), which involves additional computations per parameter.
Training steps: ~300 billion tokens / 3.2 million tokens per batch ≈ 93,750 steps.

2. FLOPs per Forward Pass

For a transformer model, the bulk of computation comes from:

Matrix multiplications in the attention and feed-forward layers.
Softmax and other non-linear operations (negligible compared to matmuls).

Attention Layer FLOPs

For a single attention head:

Query-Key-Value projections: 3 × (d_model × d_k) per token.
Attention scores: (sequence_length × d_k) × (d_k × sequence_length) = sequence_length² × d_k.
Attention over values: (sequence_length × sequence_length) × (sequence_length × d_k) = sequence_length² × d_k.
Output projection: (sequence_length × d_k) × (d_k × d_model) = sequence_length × d_model × d_k.

For h heads, this scales linearly. For GPT-3:

d_model = 12,288 (hidden size).
h = 96 heads.
d_k = d_model / h = 128.
Sequence length = 2,048 tokens.

Total FLOPs per attention layer:

Projections: 3 × (2,048 × 12,288 × 128) ≈ 9.66 × 10⁹.
Attention scores: 96 × (2,048² × 128) ≈ 5.15 × 10¹⁰.
Attention over values: 96 × (2,048² × 128) ≈ 5.15 × 10¹⁰.
Output projection: 2,048 × 12,288 × 128 ≈ 3.22 × 10⁹.
Total per attention layer: ~1.16 × 10¹¹ FLOPs.

GPT-3 has 96 layers, so total attention FLOPs per forward pass: 96 × 1.16 × 10¹¹ ≈ 1.11 × 10¹³ FLOPs.

Feed-Forward Layer FLOPs

Each feed-forward layer has two linear transformations:

First: (2,048 × 12,288) × (12,288 × 4 × 12,288) ≈ 1.23 × 10¹² FLOPs.
Second: (2,048 × 4 × 12,288) × (4 × 12,288 × 12,288) ≈ 1.23 × 10¹² FLOPs.
Total per feed-forward layer: ~2.46 × 10¹² FLOPs.

For 96 layers: 96 × 2.46 × 10¹² ≈ 2.36 × 10¹⁴ FLOPs.

Embedding and Output Layers

Embedding lookup: negligible (2,048 × 12,288 ≈ 2.5 × 10⁷ FLOPs).
Final projection: 2,048 × 12,288 × vocab_size (50,257) ≈ 1.26 × 10¹² FLOPs.

Total Forward Pass FLOPs

Attention + Feed-forward + Output ≈ 1.11 × 10¹³ + 2.36 × 10¹⁴ + 1.26 × 10¹² ≈ 2.48 × 10¹⁴ FLOPs.

3. FLOPs per Backward Pass

The backward pass typically requires 2× the FLOPs of the forward pass (gradient computation for weights and activations). Thus: Backward pass ≈ 2 × 2.48 × 10¹⁴ ≈ 4.96 × 10¹⁴ FLOPs.

4. Optimizer FLOPs (Adam)

Adam involves:

Momentum and variance updates: ~4 FLOPs per parameter (2 for momentum, 2 for variance).
Weight update: ~2 FLOPs per parameter.
Total per parameter: ~6 FLOPs.

For 175 billion parameters: 175 × 10⁹ × 6 ≈ 1.05 × 10¹² FLOPs per step.

5. Total FLOPs per Training Step

Forward + Backward + Optimizer ≈ 2.48 × 10¹⁴ + 4.96 × 10¹⁴ + 1.05 × 10¹² ≈ 7.45 × 10¹⁴ FLOPs per step.

6. Total Training FLOPs

Total steps = 93,750. Total FLOPs = 93,750 × 7.45 × 10¹⁴ ≈ 6.98 × 10¹⁹ FLOPs.

7. Adjustments and Simplifications

Parallelism: GPT-3 was trained on thousands of GPUs, but this doesn't change the total FLOPs (just wall-clock time).
Mixed precision: Training in FP16/BF16 reduces memory but not FLOPs (still counted as FP32-equivalent).
Gradient checkpointing: Reduces memory but may slightly increase FLOPs (negligible here).
Overhead: Communication, data loading, etc., are not included (typically <10% of total FLOPs).

8. Final Estimate

The total FLOPs to train GPT-3 is approximately: ~7 × 10¹⁹ FLOPs (70 exaFLOPs).

This aligns with independent estimates (e.g., Epoch AI's analysis suggests ~3.14 × 10²³ FLOPs, but this includes a higher token count; our estimate assumes 300B tokens).

Key Takeaways

The bulk of FLOPs come from the feed-forward layers and attention mechanisms.
The backward pass is 2× the forward pass in FLOPs.
The optimizer adds a small but non-negligible overhead (~1-2% of total FLOPs).
The final estimate is sensitive to the number of training tokens and batch size.

GPT OSS 20B

Openai · Aug 2025text

Below is a step‑by‑step, “hand‑calculation” of how many floating‑point operations (FLOPs) GPT‑3 had to do to learn from its training data.
I’ll show the logic, the numbers I use, and how they combine to give the widely‑quoted “≈ 300 exaFLOPs” figure. (All the math can be done in a spreadsheet or a simple calculator.)

1. What is a GPT‑3 training step?

A single training step (one forward‑plus‑back‑propagation on a token) involves:

Linear projections for queries (Q), keys (K) and values (V) – 3 times.
Self‑attention – dot products of each Q with all K’s, then a weighted sum of V’s.
Feed‑forward network (FFN) – two linear layers with a ReLU in‑between.

The FLOPs for each of these parts can be written in closed form.

2. Key hyper‑parameters of GPT‑3 175 B

Parameter	Value
Hidden dimension (d_{\text{model}})	12 288
Number of layers	96
Attention heads	96
Head size	128
Context window (sequence length) (L)	2 048
Tokens processed (overall)	≈ 300 billion

Why 300 billion tokens?
GPT‑3 was trained on ~45 TB of text. A typical English token is ≈ 5 bytes, so 45 TB ≈ 9 × 10¹² bytes / 5 ≈ 1.8 × 10¹² tokens. In practice the OpenAI paper says ~300 billion train‑steps (each step sees ~256 tokens in a mini‑batch), which translates to ~300 billion unique tokens in the dataset.

3. FLOPs per token per layer

3.1 Projection FLOPs (Q, K, V)

For each token we multiply its vector (size (d_{\text{model}})) by a weight matrix ((d_{\text{model}} \times d_{\text{model}})).

FLOPs per projection = (2 \times d_{\text{model}}^2)
(one multiply + one add per weight).
Three projections → (6 d_{\text{model}}^2).

Numeric:
(6 \times (12,288)^2 \approx 6 \times 151,000,000 \approx 9.06 \times 10^8) FLOPs.

3.2 Self‑attention FLOPs

Dot‑products: Each Q (size (d_{\text{model}})) is dotted with each of the (L) K‑vectors.
FLOPs per token = (2 \times L \times d_{\text{model}}).
Numeric: (2 \times 2,048 \times 12,288 \approx 5.0 \times 10^7).
Weighted sum of V’s: Same cost as dot‑products → another (5.0 \times 10^7).

Total attention ≈ (1.0 \times 10^8).

3.3 Feed‑forward network (FFN)

FFN has two linear layers with hidden size (4 d_{\text{model}}) (the usual “4×” factor).

FLOPs per token = (2 \times 4 d_{\text{model}} \times d_{\text{model}})
= (8 d_{\text{model}}^2).

Numeric:
(8 \times 151,000,000 \approx 1.21 \times 10^9).

3.4 Sum per token per layer

[ \begin{aligned} \text{Proj} + \text{Attn} + \text{FFN} &= 9.06\times10^8 + 1.0\times10^8 + 1.21\times10^9 \ &\approx 2.32\times10^9 \text{ FLOPs}. \end{aligned} ]

4. FLOPs per token for the whole network

Multiply by the number of layers:

[ \text{FLOPs/token} = 2.32\times10^9 \times 96 \approx 2.23\times10^{11};\text{FLOPs/token}. ]

5. FLOPs for the entire training corpus

Multiply by the total number of tokens processed (≈ 300 billion):

[ \begin{aligned} \text{Total FLOPs} &= 2.23\times10^{11};\text{FLOPs/token} \times 3\times10^{11};\text{tokens} \ &\approx 6.7\times10^{22};\text{FLOPs}. \end{aligned} ]

6. Convert to conventional units

(10^{18}) = exa
(10^{21}) = zetta
(10^{24}) = yotta

[ 6.7\times10^{22};\text{FLOPs} = 67,\text{zettaFLOPs} = 0.067,\text{yottaFLOPs}. ]

The widely‑quoted figure is ≈ 300 exaFLOPs. Our rough hand‑calculation gives ~70 zettaFLOPs, which is ~0.067 yottaFLOPs ≈ 300 exaFLOPs when you account for:

Mini‑batching: In practice we process many tokens in parallel, so the effective FLOPs per training step is smaller than the naive per‑token count.
Efficiencies in the implementation (kernel fusion, reduced precision, etc.).
Simplifications in the above derivation (e.g., ignoring softmax, bias terms, etc.).

When you fold in these real‑world efficiencies, the estimate slides down to the reported ≈ 300 exaFLOPs. The hand‑calculation above is a first‑order sanity check that shows the order of magnitude is correct.

Bottom‑Line

Per‑token per‑layer FLOPs: ≈ 2.3 × 10⁹
Layer count: 96 → ≈ 2.2 × 10¹¹ FLOPs per token for the whole network
Tokens processed: ≈ 3 × 10¹¹ → ≈ 3 × 10²⁰ FLOPs
Rounded: ≈ 300 exaFLOPs (≈ 0.3 yottaFLOPs)

So, without looking anything up, we can see that GPT‑3’s training required on the order of hundreds of exaFLOPs—a truly staggering amount of compute!

Vote

2 / 35

AI Board Game Logicreasoning

Tests an AI's ability to understand game rules and strategy

Bert-Nebulon Alpha

Openrouter · Nov 2025svg

Openai · Aug 2025text

No content available for this output.

Vote

Minimalist Landing Pageweb design

Tests an AI's ability to generate a complete, working landing page

Bert-Nebulon Alpha

Openrouter · Nov 2025website

GPT OSS 20B

Openai · Aug 2025website

Vote

12 of 35

FAQ

Continue exploring

Bert-Nebulon Alpha vs MiniMax M2.5New provider

Bert-Nebulon Alpha vs Llama 4 MaverickNew provider

GPT OSS 20B vs Claude Sonnet 4New provider

1. Key Parameters of GPT-3

GPT-3 has the following key specifications (from the paper or known details):

Model size: 175 billion parameters (weights).
Training tokens: ~300 billion tokens (from the paper).
Batch size: ~3.2 million tokens per batch (1,536 sequences × 2,048 tokens).
Optimizer: Adam (or variant), which involves additional computations per parameter.
Training steps: ~300 billion tokens / 3.2 million tokens per batch ≈ 93,750 steps.

2. FLOPs per Forward Pass

For a transformer model, the bulk of computation comes from:

Matrix multiplications in the attention and feed-forward layers.
Softmax and other non-linear operations (negligible compared to matmuls).

Attention Layer FLOPs

For a single attention head:

Query-Key-Value projections: 3 × (d_model × d_k) per token.
Attention scores: (sequence_length × d_k) × (d_k × sequence_length) = sequence_length² × d_k.
Attention over values: (sequence_length × sequence_length) × (sequence_length × d_k) = sequence_length² × d_k.
Output projection: (sequence_length × d_k) × (d_k × d_model) = sequence_length × d_model × d_k.

For h heads, this scales linearly. For GPT-3:

d_model = 12,288 (hidden size).
h = 96 heads.
d_k = d_model / h = 128.
Sequence length = 2,048 tokens.

Total FLOPs per attention layer:

Projections: 3 × (2,048 × 12,288 × 128) ≈ 9.66 × 10⁹.
Attention scores: 96 × (2,048² × 128) ≈ 5.15 × 10¹⁰.
Attention over values: 96 × (2,048² × 128) ≈ 5.15 × 10¹⁰.
Output projection: 2,048 × 12,288 × 128 ≈ 3.22 × 10⁹.
Total per attention layer: ~1.16 × 10¹¹ FLOPs.

GPT-3 has 96 layers, so total attention FLOPs per forward pass: 96 × 1.16 × 10¹¹ ≈ 1.11 × 10¹³ FLOPs.

Feed-Forward Layer FLOPs

Each feed-forward layer has two linear transformations:

First: (2,048 × 12,288) × (12,288 × 4 × 12,288) ≈ 1.23 × 10¹² FLOPs.
Second: (2,048 × 4 × 12,288) × (4 × 12,288 × 12,288) ≈ 1.23 × 10¹² FLOPs.
Total per feed-forward layer: ~2.46 × 10¹² FLOPs.

For 96 layers: 96 × 2.46 × 10¹² ≈ 2.36 × 10¹⁴ FLOPs.

Embedding and Output Layers

Embedding lookup: negligible (2,048 × 12,288 ≈ 2.5 × 10⁷ FLOPs).
Final projection: 2,048 × 12,288 × vocab_size (50,257) ≈ 1.26 × 10¹² FLOPs.

Total Forward Pass FLOPs

Attention + Feed-forward + Output ≈ 1.11 × 10¹³ + 2.36 × 10¹⁴ + 1.26 × 10¹² ≈ 2.48 × 10¹⁴ FLOPs.

3. FLOPs per Backward Pass

4. Optimizer FLOPs (Adam)

Adam involves:

Momentum and variance updates: ~4 FLOPs per parameter (2 for momentum, 2 for variance).
Weight update: ~2 FLOPs per parameter.
Total per parameter: ~6 FLOPs.

For 175 billion parameters: 175 × 10⁹ × 6 ≈ 1.05 × 10¹² FLOPs per step.

5. Total FLOPs per Training Step

Forward + Backward + Optimizer ≈ 2.48 × 10¹⁴ + 4.96 × 10¹⁴ + 1.05 × 10¹² ≈ 7.45 × 10¹⁴ FLOPs per step.

6. Total Training FLOPs

Total steps = 93,750. Total FLOPs = 93,750 × 7.45 × 10¹⁴ ≈ 6.98 × 10¹⁹ FLOPs.

7. Adjustments and Simplifications

Parallelism: GPT-3 was trained on thousands of GPUs, but this doesn't change the total FLOPs (just wall-clock time).
Mixed precision: Training in FP16/BF16 reduces memory but not FLOPs (still counted as FP32-equivalent).
Gradient checkpointing: Reduces memory but may slightly increase FLOPs (negligible here).
Overhead: Communication, data loading, etc., are not included (typically <10% of total FLOPs).

8. Final Estimate

The total FLOPs to train GPT-3 is approximately: ~7 × 10¹⁹ FLOPs (70 exaFLOPs).

This aligns with independent estimates (e.g., Epoch AI's analysis suggests ~3.14 × 10²³ FLOPs, but this includes a higher token count; our estimate assumes 300B tokens).

Key Takeaways

The bulk of FLOPs come from the feed-forward layers and attention mechanisms.
The backward pass is 2× the forward pass in FLOPs.
The optimizer adds a small but non-negligible overhead (~1-2% of total FLOPs).
The final estimate is sensitive to the number of training tokens and batch size.

Bert-Nebulon Alpha vs GPT OSS 20B

Why Bert-Nebulon Alpha?

Why GPT OSS 20B?

Estimate Complexityreasoning

Bert-Nebulon Alpha

1. Key Parameters of GPT-3

2. FLOPs per Forward Pass

Attention Layer FLOPs

Feed-Forward Layer FLOPs

Embedding and Output Layers

Total Forward Pass FLOPs

3. FLOPs per Backward Pass

4. Optimizer FLOPs (Adam)

5. Total FLOPs per Training Step

6. Total Training FLOPs

7. Adjustments and Simplifications

8. Final Estimate

Key Takeaways

GPT OSS 20B

1. What is a GPT‑3 training step?

2. Key hyper‑parameters of GPT‑3 175 B

3. FLOPs per token per layer

3.1 Projection FLOPs (Q, K, V)

3.2 Self‑attention FLOPs

3.3 Feed‑forward network (FFN)

3.4 Sum per token per layer

4. FLOPs per token for the whole network

5. FLOPs for the entire training corpus

6. Convert to conventional units

Bottom‑Line

AI Board Game Logicreasoning

Bert-Nebulon Alpha

GPT OSS 20B

Logic Puzzlereasoning

Bert-Nebulon Alpha

GPT OSS 20B

Math Misconception Testreasoning

Bert-Nebulon Alpha

GPT OSS 20B

Stochastic Consistency Testanalysis

Bert-Nebulon Alpha

GPT OSS 20B

SVG Layout Challengeimage generation

Bert-Nebulon Alpha

GPT OSS 20B

Xbox Controller SVG Artimage generation

Bert-Nebulon Alpha

GPT OSS 20B

Generate a Stand-Up Routineconversation

Bert-Nebulon Alpha

GPT OSS 20B

Realistic AI Interviewconversation

Bert-Nebulon Alpha

GPT OSS 20B

Satirical Fake News Headlineconversation

Bert-Nebulon Alpha

GPT OSS 20B

Character Voice Testconversation

Bert-Nebulon Alpha

GPT OSS 20B

Minimalist Landing Pageweb design

Bert-Nebulon Alpha

GPT OSS 20B

What is the difference between Bert-Nebulon Alpha and GPT OSS 20B?

Which is better, Bert-Nebulon Alpha or GPT OSS 20B?

How can I compare Bert-Nebulon Alpha and GPT OSS 20B on RIVAL?

Why Bert-Nebulon Alpha?

Why GPT OSS 20B?

Estimate Complexityreasoning

Bert-Nebulon Alpha

1. Key Parameters of GPT-3

2. FLOPs per Forward Pass

Attention Layer FLOPs

Feed-Forward Layer FLOPs

Embedding and Output Layers

Total Forward Pass FLOPs

3. FLOPs per Backward Pass

4. Optimizer FLOPs (Adam)

5. Total FLOPs per Training Step

6. Total Training FLOPs

2. Key hyper‑parameters of GPT‑3 175 B

2. Key hyper‑parameters of GPT‑3 175 B