1) For an experienced software engineer

A large language model is easiest to understand as a very large function that maps a sequence of tokens to a probability distribution over the next token. “Tokens” are subword chunks, not necessarily words. During training, the model is shown enormous amounts of text and repeatedly asked: given everything so far, what comes next? It starts with random weights, makes terrible guesses, and uses gradient descent to adjust billions of parameters so that its predicted distribution puts more mass on the actual next token. Architecturally, the key innovation is the transformer: instead of processing text strictly left-to-right like an old RNN, it uses attention to let each position dynamically pull information from relevant earlier positions. That gives it an efficient way to model long-range dependencies, syntax, code structure, and latent patterns like “this variable was defined 30 lines earlier” or “this answer should stay consistent with the question.”

The skeptical reaction—“but it only predicts the next token”—is reasonable, but a bit like saying a CPU “only flips bits.” The objective is simple; the behavior that emerges from optimizing it at scale is not. To predict the next token well across internet-scale data, the model has to compress a huge amount of structure about the world: grammar, facts, conventions, reasoning patterns, APIs, common bugs, argument forms, even human dialogue norms. If the prompt is “Here’s a Python traceback…”, the best next-token predictor is one that has internalized what stack traces mean, what typical fixes look like, and how programmers explain them. It’s not executing a symbolic reasoning engine in the classic sense; it’s more like a learned, high-dimensional program synthesizer over text. Inference is then just an autoregressive loop: feed in prompt → get next-token distribution → sample or choose a token → append it → repeat. The surprising part is that when the model is large enough and trained on enough diverse data, “next-token prediction” becomes a general interface for many tasks because so many tasks can be represented as “continue this text in the right way.”

What matters in practice is that the base model is usually only step one. After pretraining, labs often do supervised fine-tuning and preference optimization so the model follows instructions, refuses some requests, formats outputs usefully, and behaves more like an assistant than a raw text completer. So if you’re evaluating intelligence claims, don’t picture a magic chatbot database or a brittle rules engine; picture a gigantic distributed compression-and-generalization system that has learned statistical programs from text. Its strengths and failure modes look like that too: great at pattern completion, abstraction, and interface adaptation; unreliable when precise grounding, state tracking, or guaranteed correctness matter unless you add scaffolding like retrieval, tools, verification, or constrained decoding.

2) For a PhD physicist

At core, a modern language model defines a conditional distribution (p_\theta(x_t \mid x_{<t})) over token sequences, where (\theta) are learned parameters and training minimizes empirical cross-entropy: [ \mathcal{L}(\theta) = - \sum_t \log p_\theta(x_t \mid x_{<t}). ] So yes, in one sense it is “just” high-dimensional function approximation trained by stochastic gradient descent. The novelty is not the loss function itself, which is conceptually straightforward, but the regime: transformer architectures with attention scale unusually well in parameter count, data volume, and parallel training. Self-attention lets the representation at each position depend on content-addressed interactions with all earlier positions, which is a much more expressive inductive bias for language than older sequence models. The resulting system learns internal representations that are useful for many latent tasks because minimizing predictive error on natural language requires modeling syntax, semantics, discourse, world regularities, and patterns of reasoning encoded in text.

The strongest version of the skeptical critique is that this is interpolation in a vast statistical manifold, not “understanding.” That critique is partly right and partly incomplete. These systems do not possess grounded semantics in the human sense merely by virtue of training on text, and they do not infer truth conditions from first principles. But “mere next-token prediction” understates what the objective demands: if your training corpus contains proofs, code, explanations, negotiations, and scientific arguments, then the sufficient statistics for good prediction include abstractions that look functionally like concepts, procedures, and heuristics. In physics language, the model is learning a compressed representation of a highly structured distribution; the surprise is that the representation supports nontrivial generalization far outside rote memorization. One can reasonably view this as an emergent phenomenon from scale plus architecture, though “emergence” here should be understood operationally, not mystically.

What is genuinely novel is therefore less “we discovered intelligence by linear algebra” and more “we found a scalable recipe by which generic differentiable systems trained on next-step prediction acquire broad competence across many cognitive-linguistic tasks.” What remains overhyped is the tendency to anthropomorphize that competence. The models are impressive because a single objective produces transfer across translation, coding, summarization, tutoring, and question answering. They are limited because the learned distribution is not the same thing as a calibrated world model tied to reality. This is why they can display mathematically sophisticated behavior on one prompt and hallucinate confidently on the next. If you want the precise framing: the field’s progress is real, the rhetoric about general intelligence is often ahead of the evidence, and the central empirical fact is that predictive modeling on human-generated data appears to recover a surprisingly rich basis of cognitive behavior.

3) For a venture capitalist

A large language model is a foundation model trained on massive amounts of text to predict the next token in a sequence. That sounds narrow, but it creates a very general engine: if you prompt it with an email draft, it completes like an email assistant; if you prompt it with code, it behaves like a coding copilot; if you prompt it with customer support history, it acts like a support agent. The underlying reason is that many commercially useful tasks can be expressed as language transformation: summarize this, classify that, answer in this style, extract fields, generate code, reason over documents, call tools. The training process has two broad phases: pretraining, where the model absorbs broad linguistic and factual patterns from internet-scale corpora, and post-training, where it is tuned to follow instructions and behave usefully in products. Generation is then iterative: the model reads the prompt, predicts the next token, appends it, and repeats very quickly.

For investment purposes, the key question is not “is the model intelligent?” but “where does durable value accrue?” The foundation model layer has some moats—capital, compute access, research talent, data pipelines, optimization know-how—but it is brutally competitive and increasingly concentrated. If a startup says they are “building their own LLM,” you should ask whether they truly mean a frontier base model, a domain-adapted open model, or an application wrapped around an API. Those are very different businesses. Real defensibility often sits above the raw model: proprietary workflow integration, hard-to-replicate distribution, trusted customer data loops, domain-specific evaluation suites, fine-tuning data generated from product use, and systems that combine models with retrieval, tools, human review, and compliance infrastructure. In many categories, the model itself is becoming a component, not the moat.

A credible founder will be precise about what the model can and cannot do. Red flags include claims of unique AGI, “our secret prompts are the moat,” or benchmark numbers without customer outcome metrics. Green flags include a clear story about why this workflow needs AI, what accuracy threshold matters commercially, how they handle hallucinations and security, and whether they can improve performance through proprietary data exhaust. You should also distinguish demo quality from production quality: many AI products look magical in a 5-minute demo but fail on latency, reliability, edge cases, or unit economics. The best framing is that LLMs are a powerful new primitive, similar in importance to search or mobile platforms, but most venture outcomes will come not from owning the primitive itself, but from building the best system, product, and data flywheel around it.