What AI is for, and what it isn't

The single most consequential decision in any AI project is where you draw the line between what the AI does and what a person does.

Get the line right and the AI feels like a competent assistant — a fast first pass that frees the operator to focus on the parts of the work that actually require their judgment. Get the line wrong and the AI feels like a fight. Either the model is being asked to decide things it has no business deciding, or the operator is babysitting outputs the model could have just generated.

We have watched dozens of AI projects, our own and other people's, and the pattern holds. Every time an AI feature stops feeling useful and starts feeling exhausting, the line is in the wrong place. This piece is about where to draw it.

The shape of the line

There are four kinds of work an LLM is genuinely good at:

• Drafting. First-pass writing — a quote, an email, a project summary, a description, a memo.
• Retrieving. Finding something specific inside a large pile of unstructured text — a clause in a contract, a phrase in a transcript, a number buried in a report.
• Classifying. Sorting things into buckets — flagging which emails need attention, tagging which support tickets are urgent, categorizing which receipts belong to which job.
• Summarizing. Compressing a long thing into a shorter thing — a meeting transcript into action items, a thread into a recap, a contract into the parts that matter.

You will notice these all have something in common. They produce outputs that a human can review before they take effect. A drafted email gets read before it goes out. A retrieved clause gets verified before it gets cited. A classified item gets glanced at before it gets actioned. A summary gets scanned before someone trusts it.

That is not an accident. That is the load-bearing characteristic of every job an LLM is good at. The output is a candidate. A person is in the loop.

There is a fifth kind of work that LLMs are not good at:

• Deciding.

Deciding is when a choice is final, has consequences, and cannot easily be undone. Approving an invoice. Sending a quote to a customer. Scheduling a job. Hiring a sub. Booking a flight. Closing a ticket. Sending a contract.

Decisions are the hinge points of a business. The owner's judgment lives in the decisions. The expert's expertise lives in the decisions. The customer's trust depends on the decisions. Decisions are not the part of the job you should be eager to give to a system that predicts the next word.

The line between AI and human, on every screen of every AI system you build, is the line between drafting / retrieving / classifying / summarizing (AI) and deciding (human). Almost every AI design problem you will face is, at heart, a question about where this line falls on a specific screen.

How the line breaks

There are two failure modes, both common.

Mode one: the AI is asked to decide something a person should own.

A scheduling AI that automatically books the next appointment without a person reviewing it. A pricing AI that auto-applies a discount based on customer profile. A support AI that closes a ticket because it judged the issue resolved. An email assistant that auto-replies on your behalf to a thread it thinks is routine.

Each of these is the same mistake. A decision with real consequences is being made by a system that has no internal way to know whether the decision is correct. When it is wrong — and it will be wrong some percentage of the time — the consequences land on the customer, the employee, or the bottom line, with no human in the loop to catch it.

The signs that this failure mode is in play:

• Customers complaining about decisions nobody on the team made
• Employees describing the AI as "going rogue" or "doing weird things"
• Owners pulling the system back to manual after a few months
• The phrase "we just need to add more guardrails" appearing in internal conversations

The fix is not more guardrails. The fix is moving the decision back to a person and letting the AI do the part it is good at — drafting the proposed decision, surfacing it for review, and making the review fast.

Mode two: the AI is asked to do work a person could trivially do, and the person is asked to babysit the AI.

A summary feature that produces a one-paragraph recap of an email the person could have read in thirty seconds. A "smart" autofill that requires the operator to correct it three times before the field is right. A draft that takes longer to fix than it would have taken to write from scratch.

This mode does not crash the business. It just slowly erodes trust. The operator stops believing the AI is saving time. They start working around it. Six months in, nobody is using the AI feature, and nobody is sure when they stopped.

The signs:

• Employees saying "it's faster if I just do it"
• Adoption numbers that look fine in week one and decline every week after
• Operators copy-pasting AI output without reading it (because reading it does not catch the errors)
• A feature that is technically working but never gets praised

The fix here is the opposite — reduce the AI's role until it is doing something the operator actually wants help with, or remove it from that screen entirely. Not every screen needs AI on it. Adding AI to a workflow that did not need it is one of the most common ways to make the workflow worse.

The right line, drawn well

The right place for the line varies by industry, by workflow, by the maturity of the team, and by the cost of an AI mistake on that screen. There is no universal rule. There is a discipline.

The discipline:

1. For every screen where AI is being introduced, write down what the AI produces and what the person decides. Out loud. In a sentence. If the answer is fuzzy, the line is in the wrong place.
2. Keep the AI's part as far from "deciding" as possible. Drafting is safer than recommending. Recommending is safer than deciding. Deciding is the part the person should own.
3. Make the human's review fast. A line drawn well is one where the human review is a click, not an investigation. If the operator has to dig into the AI's reasoning to verify the output, the AI is doing the wrong job.
4. Capture the corrections. When the operator overrides the AI, that override is signal. Feed it back into the system so the next draft is closer to right. Without that loop, accuracy does not improve and trust degrades.
5. Accept that some screens should not have AI on them. A decision that depends on judgment, relationship, or context the model cannot see is a decision the person should make without a draft. Adding AI here adds noise, not value.

The operators we trust most are the ones who have learned to spot when the AI is being asked to do too much, and who push back. The AI is a tool. Tools have shapes. A tool used outside its shape produces frustration in the short run and bad outcomes in the long run.

What this means for the work

If you are building an AI feature, the first question is not "which model do we use?" or "what does the screen look like?" or "what does the prompt say?" The first question is: for this workflow, where does the AI's part end and the person's part begin?

If you cannot answer that in a sentence, you are not ready to build the feature. You are ready to do the design work that makes the feature buildable.

If you are using an AI feature someone else built, the same question applies. What is this AI deciding for me, and is that something I should let it decide? If the answer is uncomfortable, push back on the vendor or push back on your own setup until the line moves to where it should be.

The line between AI drafts and a person decides is not a hedge. It is not a workaround for the current generation of models. It is the structural fact of how this technology works, and the businesses that respect it will out-compete the businesses that don't.

For more on the technology underneath this line, read What an LLM actually is (and isn't). For where this line sits in the bigger picture of an AI build, read An AI system, walked through like a building.