A common back-of-the-envelope estimate for transformer training compute is:

[ \text{Training FLOPs} \approx 6 \times N \times D ]

where:

• (N) = number of model parameters
• (D) = number of training tokens

---

For GPT-3 (175B), reasonable known assumptions are:

1. Parameters: (N \approx 175 \text{ billion} = 1.75 \times 10^{11})
2. Training tokens: (D \approx 300 \text{ billion} = 3.0 \times 10^{11})

Now plug in:

[ 6 \times (1.75 \times 10^{11}) \times (3.0 \times 10^{11}) = 6 \times 5.25 \times 10^{22} = 3.15 \times 10^{23} \text{ FLOPs} ]

Estimate

[ \boxed{\text{Total training compute} \approx 3 \times 10^{23}\ \text{FLOPs}} ]

A reasonable range (depending on exact token count, sequence-length effects, optimizer/update overhead, and hardware efficiency) is roughly:

[ \boxed{(2.5\text{ to }4)\times10^{23}\ \text{FLOPs}} ]

So the headline answer is: on the order of (10^{23}) FLOPs, about (3.1\times10^{23}).