What an LLM actually is (and isn't)

If you have used ChatGPT, Claude, or any of the chatbots that exploded into business conversations over the last three years, you have used a Large Language Model — or LLM, which is just the technical name for the kind of system underneath those chat boxes.

Most people who use LLMs every week could not, if you asked them point-blank at dinner, explain what one actually is. That is not a knock on anyone. The marketing has been deliberately mystical — "intelligence," "thinking," "reasoning," "agents" — and the actual mechanics are surprisingly simple. The gap between how it gets sold and how it actually works is wide enough to drive every bad AI decision through.

This piece closes that gap. It is short on purpose. You do not need a computer science background to understand what is going on under the hood, and a basic understanding will save you from most of the mistakes owners are making with this technology right now.

The whole trick, in one sentence

An LLM predicts the next word.

That is it. That is the entire mechanic.

You feed it a stretch of text. It looks at the text, considers an enormous number of possibilities for what word would come next based on patterns it has seen before, picks the most likely one, and adds it to the text. Then it does the same thing again. And again. One word at a time, all the way down to the end of the response.

That is what is happening when you watch a chatbot type out an answer in front of you. It is not "thinking" in any sense that resembles human thinking. It is not consulting a database of facts. It is not reasoning step by step. It is predicting the next word, then the next, then the next, until it predicts that the response is over.

The reason it can do this convincingly is that it has been trained on an enormous amount of human writing — books, articles, websites, conversations, code. Through that training, it has built up a statistical sense of which word is likely to come next given everything that came before. The depth and breadth of that training is why the predictions feel coherent across an entire essay, not just a single sentence.

That is the trick. The whole trick.

Why "just predicting the next word" is more impressive than it sounds

When people first hear "an LLM just predicts the next word," they often dismiss the technology as hype. It is a fair instinct. But the next-word framing actually understates what is happening.

To predict the next word convincingly, the model has to have some functional grasp of:

• Grammar (the next word usually has to fit the sentence structure)
• Vocabulary (the word has to be a real word, used correctly)
• Topic (the word has to be relevant to what is being discussed)
• Tone (the word has to match the register — formal, casual, technical, plain)
• Context across the whole conversation (the word has to fit not just the sentence but the whole thread)
• The kind of answer the user is asking for (a question wants a different next word than a request for a list)

A model that can do all of that, well enough to produce paragraphs that read as if a thoughtful person wrote them, is genuinely impressive. The fact that it works by predicting one word at a time does not make it less impressive. It makes it more impressive — because the trick scales up to behavior that looks a lot like thinking, even though the underlying mechanic is simple.

So: the mechanic is humble. The behavior on top of the mechanic is remarkable. Both things are true.

Why "just predicting the next word" is also less impressive than it sounds

The other side of the same coin: because the mechanic is "predict the next word," the model is doing exactly that, even when it looks like it is doing something else.

It is not actually reasoning. It is producing text that looks like reasoning, because it has read a lot of reasoning and learned what the patterns of reasoning look like in writing.

It is not actually checking facts. It is producing text that looks like fact-checked statements, because it has read a lot of fact-checked statements and learned what they sound like.

It is not actually researching your question. It is producing text that looks like a researched answer.

This is why an LLM will sometimes confidently state something that is completely wrong. Not because it is lying. Because it is doing what it was built to do — produce text that fits — and "the text that fits here" is not the same thing as "the text that is true." Sometimes those overlap. Sometimes they do not. The model has no internal way to tell the difference.

This is the single most important thing for an owner to understand about LLMs. The output looks like answers. It is actually next-word predictions that are usually but not always correct. Treating the output as answers without verification is the source of more AI failures than every other cause combined.

What LLMs are not

A few things that LLMs are not, despite frequent claims to the contrary:

They are not search engines. A search engine retrieves documents. An LLM produces text that resembles what a document on that topic might say. Different mechanism, different reliability profile.

They are not databases. A database holds facts and returns them when queried. An LLM holds patterns and generates text that follows those patterns. It does not "know" anything in the database sense.

They are not reasoning engines, in the way a human reasons. Humans hold a problem in mind, manipulate it, check intermediate steps. An LLM produces a stream of tokens that — when the model is good and the topic is in its training — happens to resemble the output of reasoning.

They are not learning from your conversation in the way a person would. Within a single conversation, the model uses everything you have said as input for predicting the next word. The moment that conversation ends, that context is gone. The model does not "remember" you next week unless something else (a separate system) was set up to keep that memory.

They are not agents — at least not by themselves. An LLM by itself only produces text. To take actions in the world — send an email, update a record, write a file — it has to be connected to other systems that can do those things. The connection is where the real engineering lives.

Each of these confusions costs owners money. Treating an LLM as a search engine produces wrong answers presented as facts. Treating it as a database produces phantom records. Treating it as an agent without the right wiring around it produces actions taken on bad assumptions.

What LLMs actually are good at

Now the useful half. Once you accept the mechanic — predicts the next word, looks like thinking but is not — a clear set of strengths comes into focus.

LLMs are good at:

• Drafting. First-pass writing where a human will review it before it goes anywhere.
• Summarizing. Compressing a long thing into a shorter thing.
• Classifying. Sorting items into categories based on patterns.
• Translating. Between languages, but also between formats (turning a transcript into a structured note, turning a contract into a plain-language summary).
• Extracting. Pulling specific pieces out of unstructured text — names, dates, dollar amounts, deadlines.
• Drafting structured output. Generating a list, a table, a form, a quote, an email — given enough context.

What ties these together is that they are all low-stakes-per-output, high-volume tasks where a human can review the output before it leaves the building. The model is not the final say. The model is a fast first pass that a person edits, approves, or rejects. That is the shape of the job an LLM is built to do.

What an owner should take from this

A few things that change once you understand what an LLM actually is:

Stop being impressed by the model's confidence. Confidence in an LLM output is a property of the writing, not a property of the truth. A confidently wrong answer is just as confident as a correct one.

Stop expecting the model to do tasks that require holding a lot of internal state. Long, multi-step reasoning where each step depends on the last is where LLMs slip the most. They will produce text that looks like the reasoning is sound, while the reasoning is not.

Always have a human review LLM output that matters. This is not a hedge against future improvement. This is the structural fact of how the technology works. The model produces drafts. People decide.

Spend your money on what is around the model, not on the model itself. Choosing between providers is a smaller decision than most owners think. The model is the bones of the building. The leverage is in the wiring, the walls, and the safety check around it.

Treat the next round of "this changes everything" announcements with caution. The mechanic of LLMs is not changing. The models get better at the mechanic — bigger training, more efficient inference, better fine-tuning. But the load-bearing fact — predicts the next word, often right but never guaranteed — is going to be true of LLMs for the foreseeable future. Plan accordingly.

The technology is real. The capability is real. The savings are real. But the mechanic is humble, and the owner who understands the humble mechanic underneath the impressive output will make better decisions than the owner who buys the marketing.

For more on where this fits in the bigger picture, read An AI system, walked through like a building. For the line between what AI should draft and what a person should decide, read The Enabler's Playbook.