1) For an experienced software engineer

Think of an LLM less like a database of facts and more like a gigantic learned program that has been trained to compress the patterns of text, code, and conversations into its weights. During training, it sees trillions of token sequences and is repeatedly asked: “given everything so far, what token is most likely next?” That sounds like fancy autocomplete, but the prediction target is hard enough that the model has to internalize syntax, semantics, APIs, naming conventions, error patterns, argument structure, user intent, and a lot of world knowledge. If it’s trying to continue try { ... } catch ( in Java, or explain why a 503 might happen in a microservice chain, it can’t do that well without building a latent model of how software and language work.

Architecturally, a transformer is basically a stack of functions that turns a sequence of tokens into contextual representations, where each token can “look at” relevant earlier tokens through attention. You can think of attention as dynamic dependency resolution: for the current position, the model computes which prior pieces of context matter and how much. Training is just gradient descent on prediction error, over and over, until the weights become a compressed statistical map of how human-written sequences tend to continue. No one hard-codes rules like “JSON usually closes braces this way” or “a stack trace mentioning connection reset often implies network or timeout issues”; those regularities get baked into the parameters.

At generation time, the loop is simple: take your prompt, compute a probability distribution for the next token, choose one, append it, and repeat. The reason this can produce surprisingly coherent design docs, code, or debugging advice is that “next token” is the interface, not the capability. To predict the next token in a useful way, the model has to maintain an internal state about what problem is being discussed, what constraints have been established, what style is expected, and what consequences follow from earlier text. It’s still fallible—it has no built-in truth checker or live system state unless you connect tools to it—but “it only predicts the next word” is a bit like saying “Postgres just writes bytes to disk”: true at one level, but it misses the abstraction where the real behavior lives.

---

2) For a PhD physicist

Formally, a language model defines a conditional probability distribution over token sequences: [ p_\theta(x_{1:T})=\prod_{t=1}^T p_\theta(x_t \mid x_{<t}). ] Training minimizes the negative log-likelihood [ \mathcal{L}(\theta) = -\sum_t \log p_\theta(x_t \mid x_{<t}) ] over a very large corpus. In a transformer, each token is mapped to a vector, positional information is added, and layers apply self-attention plus nonlinear mixing. The central attention operation is content-dependent coupling: [ \alpha_{ij} = \mathrm{softmax}j!\left(\frac{q_i \cdot k_j}{\sqrt d}\right), \qquad h_i' = \sum_j \alpha{ij} v_j. ] So yes: at base, it is linear algebra composed with nonlinearities, trained by stochastic gradient descent. There is no mystery there.

At inference time, generation is autoregressive: given a prefix (x_{<t}), compute (p_\theta(\cdot \mid x_{<t})), select or sample a token, append it, and iterate. The interesting part is why this objective yields capabilities that look broader than “word prediction.” If the next token depends on latent variables—topic, speaker intent, syntax, discourse structure, factual associations, code semantics—then minimizing predictive loss forces the network to infer those latent variables from context. In that sense, the hidden state functions as a distributed, approximate sufficient statistic for the posterior over latent causes of the observed prefix. Translation, summarization, code completion, dialogue, and some forms of reasoning all reduce to conditional sequence modeling, so competence on next-token prediction transfers surprisingly far.

What is genuinely novel is not the mathematics in isolation; most ingredients are decades old. The novelty is the empirical discovery that the transformer architecture, trained at large scale on diverse data, exhibits smooth scaling behavior and unexpectedly general task transfer, including in-context learning, where the prompt itself specifies a task without parameter updates. What is overhyped is the leap from “excellent statistical predictor” to “understands truth” or “reasons like a scientist.” These models do not optimize for factuality or causal validity unless you explicitly add mechanisms for that; they optimize for likelihood under the training distribution. The result is powerful and nontrivial, but it is still best understood as high-capacity probabilistic sequence modeling, not machine metaphysics.

---

3) For a venture capitalist

A large language model is best understood as a general-purpose prediction engine trained on enormous amounts of text and code. In pretraining, the model consumes massive corpora and learns to predict the next token in sequence. That simple objective turns out to be commercially potent because most knowledge work is expressed as sequences: emails, support chats, contracts, code, medical notes, sales calls, queries, and reports. At runtime, the model takes a prompt, estimates the most likely next token, emits one, and repeats; product systems then wrap that core loop with retrieval, tool use, guardrails, and fine-tuning so the outputs are useful inside a real workflow.

The key diligence question is where value accrues. The foundation model layer is increasingly concentrated among a small number of labs and increasingly accessible through APIs or open-weight alternatives, so “we have AI” is not a moat. For most startups, the defensible asset is not the raw model but the system around it: proprietary workflow data, integrations into systems of record, evaluation infrastructure, feedback loops from user actions, latency/cost optimization, and product design that inserts the model at a high-value decision point. In other words, the best businesses are not selling a chatbot; they are owning a workflow where model performance compounds as more real usage data flows through the system.

Founders’ claims are credible when they can decompose performance clearly: what comes from the base model, what comes from fine-tuning, what comes from retrieval or tool invocation, and how they measure quality against incumbent workflows. Red flags include hand-wavy claims about a “secret model,” no answer on inference economics, no proprietary data flywheel, and demos that ignore failure modes. A strong team will understand both the upside and the limits: LLMs are powerful enough to create real product discontinuities, but durable moats usually come from distribution, embedded workflow, and data advantage—not from wrapping a commodity API and hoping the model remains scarce.