1. Experienced software engineer

Think of a large language model as a very large function that maps a sequence of tokens to a probability distribution over the next token. It’s implemented as a deep neural network (usually a Transformer). During training, the model is shown billions of text sequences and repeatedly asked: “given these previous tokens, what token comes next?” The training loop is straightforward gradient descent on cross‑entropy loss. Over time the model adjusts billions of parameters so that its predicted probability distribution matches the real next token in the dataset.

What makes this more than trivial autocomplete is scale and architecture. Transformers use a mechanism called attention that lets every token in a sequence directly read information from every other token, building contextual representations. Internally the model learns abstractions like syntax, entities, relationships, and even patterns of reasoning because those abstractions help reduce prediction error. If predicting the next token requires modeling “how a Python stack trace works” or “how a legal argument unfolds,” the network ends up learning those patterns because they improve the loss.

Generation is just running this function repeatedly. Start with a prompt, compute the next‑token distribution, sample a token, append it, and repeat. Even though the training objective is local (next token), the model learns compressed representations of language, code, and world knowledge. That’s why it can do tasks like summarization or coding: those behaviors fall out as useful intermediate structures for predicting text that humans wrote while performing those tasks.

2. PhD physicist

Formally, a large language model parameterizes a conditional probability distribution P(x_t | x_<t). The model (typically a Transformer) is trained by minimizing the negative log likelihood of tokens in a very large corpus. If θ are the network parameters, the objective is:

L(θ) = −E_data Σ_t log P_θ(x_t | x_<t)

So at a high level it is indeed “just” maximum likelihood estimation of a huge autoregressive model. The novelty is not the objective but the architecture and scale: a Transformer builds contextual representations of each token through stacked attention and feed‑forward layers. Self‑attention effectively allows every position to compute weighted interactions with every other position, which makes long‑range dependencies tractable.

What emerges empirically is that when the parameter count, dataset size, and compute cross certain thresholds, the model develops internal representations that encode surprisingly rich structure: syntax, semantics, factual associations, program structure, and sometimes multi‑step reasoning patterns. None of this is explicitly supervised; it arises because modeling the joint distribution of human text requires capturing those latent variables. In physics language, the network is learning a highly compressed representation of the manifold that human-generated language occupies.

Generation is simply sampling from the learned distribution. Given a prefix, the network computes logits over the vocabulary, converts them to probabilities, samples a token, appends it, and repeats. The interesting scientific question isn’t the training objective—it’s why this particular architecture trained at scale yields representations that generalize across tasks. Much of current research is essentially probing and characterizing the structure of these learned representations and the scaling laws governing them.

3. Venture capitalist

A large language model is essentially a system trained on massive amounts of text to predict the next token in a sequence. During training it reads trillions of tokens—books, websites, code, papers—and adjusts billions of parameters so that its predictions match what humans actually wrote. Because human text contains reasoning, instructions, explanations, and problem solving, the model ends up learning the patterns behind those behaviors. When you prompt it, it generates new text by repeatedly predicting the most plausible continuation.

What’s economically interesting is that once a model has learned this representation of language and knowledge, it becomes a general interface for many tasks: writing, coding, summarizing, customer support, research assistance, and more. The same base model can perform many tasks with only prompting or light fine‑tuning. That’s why people call these “foundation models.” The core capability is general text reasoning, and applications are layered on top.

From a defensibility standpoint, the key variables are compute, data, and distribution. Training frontier models requires enormous compute budgets and specialized infrastructure, which creates barriers to entry. But raw models commoditize quickly, so durable companies tend to build moats through proprietary data, workflow integration, or product ecosystems built around the models. When evaluating a startup, the important question is whether they are just wrapping an API or whether they are accumulating unique data, user lock‑in, or infrastructure advantages that improve the model or product over time.