Build notes · agentic design
The WhatsApp assistant that could explain booking but not book
At 13:00 we deployed a screen that cross-references two things nobody had ever compared: what our assistant can explain, and what it can send. It immediately reported that scheduling sat in the first list and not the second. Fifty-six minutes later somebody typed “Book a meeting for me” into our live WhatsApp number. Ten seconds after that they were told: “Good question, and I would rather not guess at it. I have flagged this for someone here.”
The meeting scheduler was working. It proposes times as tappable buttons, renders them in the contact’s own timezone, checks for conflicts with what is already booked, and sends reminders. We had shipped a timezone correction to it that same morning. None of that mattered, because the part of the system talking to the customer had no way to reach it.
It is worth sitting with what that message is, because it is easy to read “Book a meeting for me” as one more row in a log. It is the highest-intent sentence anybody sends a business. Nobody types it while browsing. They have read enough, they have decided, and they are asking to be sold to. Every other message in the funnel is somebody making up their mind. That one is somebody done making it up.
And we answered it with a shrug. A polite, well-engineered, honestly-worded shrug that had been carefully built not to promise a time it could not keep, delivered in ten seconds to somebody who wanted a time. If they had been buying jewellery from one of our customers at eleven at night, in the twenty minutes their attention lasted, the shop would have lost them to whoever answered with a button.
This is digital distance, the term these notes keep returning to: the gap between what a person believes a business can do for them and what its software actually does. Its defining property is that it is never reported. Nobody writes in to say “your assistant understood me and could not help, so I went elsewhere.” They just come back less often, and no error rate, support queue or satisfaction survey ever shows it. Which is why the only way to find it is to go looking structurally, which is what the screen at 13:00 was for.
TL;DR
An assistant that can only produce words answers a request for an action with a description of that action. The corpus said we could explain booking; the response registry had a product card, a plan list and a handoff, and nothing that proposed times. Closing it meant giving the model a fourth outcome: choose one id from a list the server already filtered by consent, the WhatsApp 24-hour window, entitlement and who is asking. It gained reach and gained no new sentence it can invent. In the same afternoon the same design caught a shadowed question that would have told that prospect the product cannot do a thing it shipped that morning.
The screen that predicted it
Every AI assistant in this category is built on two lists, and almost nobody joins them. The first is what it knows: pages, saved replies, product rows, whatever has been ingested. The second is what it can actually put in front of somebody: a card, a list, a form, a link. The first list is treated as the interesting one because it is where retrieval lives. The second is treated as plumbing.
Joining them on topic produces four states, and only three are worth a human’s attention. Both is the healthy one. Corpus only means words exist and nothing sendable does, which is fine and a hint. Asset only means the reverse and is the one that hurts: the assistant can put a thing in front of you and cannot say a word about it, so anyone who asks gets handed to a person while the thing that answers them sits unused.
| Can send it | Cannot send it | |
|---|---|---|
| Can explain it | BothAnswer the question and offer the thing. The strongest move available. | Corpus onlyWords are all you have. Fine, and a hint that something sendable might be worth building. |
| Cannot explain it | Asset onlyYou can put it in front of somebody and cannot say a word about it. Anyone who asks gets handed to a person while the thing that answers them sits unused. | NeitherA real gap. Worth recording rather than discovering twice. |
Scheduling sat in the top right. We could describe booking. We could not book.
Our own map, live, on the deployment that sells this software:
scheduling → corpus_only · corpus: [product_knowledge] · assets: []
We can describe how booking works. We cannot book. Nobody had been able to see that sentence before, because it is not visible from either list alone. Fifty-six minutes is a small sample and we are not going to pretend a screen with one hour of history is validated. But the sequence is worth recording exactly as it happened: the failure was written down before it occurred, by software, in the place somebody would look.
Explaining is not doing
The distinction sounds obvious written down and is very easy to lose in code. Retrieval makes a model better at answering questions, and a well-grounded assistant handles “how does booking work” beautifully. It will handle “book me a meeting” beautifully too, in the sense that it will produce a fluent, well-sourced, entirely useless paragraph about how booking works.
There is no corpus entry that fixes this. You can write ten more pages about scheduling and the answer gets more detailed and no more useful, because the person did not ask a question. They issued an instruction. The only fix is a mechanism by which the assistant can do something, and the entire difficulty is doing that without also giving it the ability to do the wrong thing.
What the model is allowed to decide
Our last-resort router had three outcomes: answer from grounded facts, recognise a greeting and let the operator’s own sentence send, or show a three-button chooser somebody had written by hand. That chooser was the interesting one. It proved a model can safely pick which of several options a person sees while contributing none of the words. It had exactly one entry.
The fourth outcome generalises it. The model may return
{"action": "send_asset", "assetId": "meeting_slots"} and nothing else. It does
not write the message. It does not choose the times. It picks one id.
The model, since people reasonably ask and almost nobody in this category says: Llama
3.3 70B Instruct, served by Cloudflare Workers AI as
@cf/meta/llama-3.3-70b-instruct-fp8-fast, running in the same Worker that handles
the webhook. No third-party inference vendor, no key of ours in the path, no message text
leaving the customer’s own Cloudflare account to be scored by somebody else. That last
clause is the entire reason this is the model rather than a larger one: an assistant reading
every inbound WhatsApp message a business receives is the worst possible thing to route
through an intermediary, and a self-deployed product that quietly did so would not deserve the
word.
Naming it also lets us be specific about the risk. On 30 May our test suite failed with
5028: This model was deprecated. A pinned Workers AI model expiring is a when and
not an if, and it is the reason the model sits at the very end of the chain, after every
deterministic rule. Under that ordering, a model that dies quietly costs latency on the tail
and nothing else. Put the same model in front of payments and consent and the same event is
an outage in the parts of the system that must never have one.
The enforcement is not in the prompt. Prompts are requests. Before the prompt is built, the server resolves which assets can genuinely reach this person right now, and only those are described to the model. The parser then accepts an id only if it appears in that exact list. Anything else is a handoff rather than a lookup, which means a hallucinated id, a remembered id from training, or an id belonging to a different deployment are all the same outcome, and that outcome is safe. This is the property worth having:
Widening what a model can send must never widen what it can make up.
Everything that ships is assembled server-side. The product card reads live prices from the database at send time, so it cannot quote a stale figure the way a page ingested last week can. The meeting proposal reads the calendar. The model contributed one string.
Four gates, and one of them is physics
An asset is only offered if it survives four checks, and the order matters because the cheap ones run first.
Consent
Opted out or suppressed means nothing is sent, of any kind. This is checked first and short-circuits, and it is worth noting that a friendly acknowledgement is still speaking.
Reach, which is not a product preference but a property of WhatsApp
This one is not ours to choose, it is how WhatsApp works. Meta allows freeform messages, which includes text, interactive lists and Flows, only inside a 24-hour service window opened by the customer’s own last inbound message. Outside it, an approved template is the only thing that lands. A freeform send outside the window does not fail gracefully: it is a guaranteed-failing call and a quality signal against the number. So an undeliverable asset is absent from the menu rather than offered and then refused. A choice that can be made and then rejected is one somebody will eventually make.
Entitlement
Our own plan list exists only on our own deployment. A customer running Klaros for a jewellery shop is never told the outcome exists, so a stray reference to it reads as a handoff.
Substance
A product card for a workspace with no products has nothing to show. That is a normal decline, not an error, and it falls through to words. Which produced a rule we now hold to: “I could not build the thing I chose” must never look, from the customer’s side, the same as being ignored.
One, never a menu
The model may pick at most one asset per reply. This is a product decision rather than a technical limit, and it is the one we would defend hardest.
A reply offering three things to look at is not more helpful than a reply offering the right one. It is a system declining to decide, in public, and asking the customer to do the work instead. What advances a conversation is a single next step. The same instinct governs the rest of the product: we have deliberately not accumulated buttons in the inbox, and the proper version of most features is invisible.
The thing it almost said
Now the part that matters more than the feature.
When the assistant cannot answer, it can generate a discovery question instead of an acknowledgement: a capability gap is the best moment a business ever gets to learn what somebody actually wants, because they have just told you unprompted and they are still typing. That question is generated but not sent until a deployment has reviewed a batch of them and switched it on deliberately. Shadow by default.
For “Book a meeting for me”, it generated this:
“Not something Klaros does today, and I would rather tell you straight than dress it up. Would you mind if I asked one question about what you would use it for?”
That is false. Klaros schedules meetings, and we had shipped an improvement to that scheduler four hours earlier. The opener asserted a fact about the product, and the module producing it runs off a handoff, which means exactly one thing: the model could not ground an answer. That is a statement about the corpus, never about the software. Three different causes, a capability gap, a corpus gap and a registry gap, are indistinguishable from where that code stands, and it had been confidently reporting all three as the first one.
It reached nobody, because it was shadowed. That is the entire argument for shadowing, and it is the second time in one week that a rollout mechanism has paid for itself. The line now says something true whatever the cause, and a test pins the claim out so it cannot come back.
Learning when not to speak
Every guard above bounds what the assistant says. None of them stopped it saying anything at all in a conversation a human colleague had already taken over.
Our inbox has written an assignment field since the day it shipped, and nothing in the AI path had ever read it. An agent could claim a thread, be halfway through something delicate, and the assistant would answer the customer’s next message straight over the top of them. Nobody had reported it, because the AI only reaches threads nothing else claimed and an assigned thread usually gets a human reply first. The race was quiet rather than absent, which is the worst way for a defect to wait.
There is now a fourth outcome that produces no message at all, checked before the model runs so a held conversation costs nothing:
Assigned: somebody claimed this thread. Held with no expiry, because assignment is deliberate and the inbox already has an unassign button. Expiring it on a timer would override a person with a guess.
A colleague replied: the newest outbound came from a human, so the customer is answering them. Held for two hours, because ownership nobody declared should not make a thread permanently manual.
Still typing: a further message within thirty seconds of our reply is part of one thought. Three messages in twenty seconds deserve one answer, not three.
These are recorded as their own outcome rather than folded into handoffs, and the distinction is the whole reason to bother. A review queue full of handoffs says the corpus has gaps. A queue full of holds says the assistant is correctly staying out of the way. Conflating them loses the only signal worth reading back.
Who is allowed to ask
All of this was tolerable while everything the assistant could do was show a price list. It stopped being tolerable the moment it could create something.
The missing piece was identity, and WhatsApp hands it to you. Team members have been keyed by email since the day the feature shipped, which is right for a dashboard and useless on the inbound path, where a message arrives carrying a phone number and nothing else. That number is not self-asserted: Meta verified the account owns it before the webhook fired. So a colleague is an active team member whose WhatsApp number matches the sender, and nothing softer. Not a contact tagged internal, not somebody in the operator’s address book, not a number that has messaged a lot. Those are all things a person can arrange to be true about themselves.
Assets now declare who they are for, and the operator overrides that per workspace from a screen rather than by editing configuration. It fails closed in every direction: a missing number, a failed lookup, a disabled colleague, an agent from another workspace, and an unrecognised permission value all land on the public side. An unrecognised value is refused at registration rather than defaulted, so a typo cannot publish an internal command.
The first staff-only command creates the sender’s own digital business card and replies with the link. It is keyed on their own verified number, which means there is no parameter naming whose card to make, which means “make me a card for someone else” cannot be asked for. Removing a capability is a better guarantee than validating it.
The arguments behind all of that
Everything above is written in the tense of a thing already decided, which is a dishonest tense. None of it arrived clean. Four arguments produced it, and the first one we got wrong.
“This is over-built for the traffic.” The case against spending a day on any of this was concrete and, on the face of it, strong: ten AI decisions in the period, sixteen trial signups, exactly one connected WhatsApp number, which is our own. On volume like that, hardening a reply path looks like polishing a hallway nobody walks down. The counter-argument was that we had picked the wrong measure entirely. The value of this work is not proportional to traffic that has already happened; it is proportional to the cost of the first bad impression. At small volume with high stakes per conversation, you cannot retrofit restraint after the prospect churns, and you never get told which message was the one that mattered. Fifty-six minutes later somebody typed “Book a meeting for me”. The argument was settled by the event rather than by the argument, which is the only kind of settling worth reporting.
“Should it propose times at all?” This was the genuine disagreement, and we stopped and put it down rather than coding through it. Every other thing this assistant can do assembles words. Proposing times writes a row. Three options: propose and let the tap book; build real availability checking first and add the command on top; or keep booking human on purpose. The middle option was the responsible-sounding one, and we rejected it for a reason that turned out to be wrong in an instructive direction. We assumed the cautious path cost a second piece of work. Then we went and looked, and found that the conflict-avoiding slot generator already existed, built months earlier for the reschedule flow: business hours in the operator’s zone, weekends skipped, every candidate checked against what is booked, and an empty result when the calendar is too full. The safe option had already been paid for and nobody had noticed. We had been about to accept a risk we did not actually have to take, because estimating is cheaper than reading.
“Fix the alarm, or fix the class of alarm?” The coverage screen
shipped and immediately reported a gap called contact. It was not real: the word
was a synonym for a topic our corpus already covers, declared by an asset and mapped to no
source, so it would have sat there permanently. We trimmed the word. An hour later the
scheduling asset was about to reintroduce the identical shape with three more synonyms, and we
nearly trimmed those too. Second time, we asked what the class was rather than what the
instance was, and the answer was cleaner: a staff-only command has no customer to explain it
to, so it does not belong on that map at all. Excluding a whole category removed the false
alarms we had and the ones we had not written yet. The screen exists to be worth acting on,
and one wolf teaches a reader to ignore the next.
“Who else might be talking?” Nobody asked for the hold outcome. It came from a question with no interesting answer expected: if the assistant is about to reply, who else might already be handling this? The inbox has written an assignment field since the day it shipped, and nothing in the AI path had ever read it. That defect had been live for months, had never been reported, and could not have been, because it only fires in a race an agent would experience once and shrug at. The uncomfortable part is that no test, no customer and no dashboard would have surfaced it. Somebody had to wonder.
Some things were argued for and deliberately not built, which is worth listing because the absences are decisions too. No response-time targets or automatic assignment: measuring the promise is not keeping it, and a countdown nobody honours is worse than no countdown. No permissions matrix, because the screen holding it gets opened twice and a matrix is a thing nobody reads. No expiry on an assignment, because there is already an unassign button and overriding a person with a timer is not a feature. And no second asset in a reply, ever, which we will keep refusing.
Why this shape needs the stack it runs on
Three constraints made this design available rather than merely desirable, and they are all properties of where it runs.
Meta gives you verified identity and a hard delivery contract. The phone number is trustworthy enough to gate on, and the 24-hour window is unambiguous enough to compute. A messaging channel with softer rules would have produced softer guards, and softer guards are the ones that get relaxed.
WhatsApp’s interactive primitives make a tap the unit of consent. Every risky thing this assistant can do resolves to buttons somebody else wrote. The model proposes three times; the customer’s thumb books one. Nothing reaches a calendar, a consent ledger or a payment until a person taps. That is not a policy we enforce with prompting, it is the shape of the medium, and building on it means the safe version and the native version are the same version.
Cloudflare makes the whole thing yours. Klaros is self-deployed: the Worker, the D1 database and the queues live in the customer’s own account. Every decision this note describes is written as a trace on a row you own, listing the facts retrieved, the assets offered, the guard that fired and the threshold it compared against. You can query it. You can read what your assistant decided last Tuesday and why, without asking us and without us being able to quietly change the answer. An assistant that acts on your behalf and keeps its reasoning on somebody else’s infrastructure is asking for a kind of trust we would rather not have to request.
What this means for somebody using it
Concretely, and in the order a customer would notice.
Asking to book gets times instead of a promise. Asking what is for sale gets the card with today’s price from the database rather than a paragraph about products. Writing three times in twenty seconds gets one considered reply instead of three fragments. Writing to a thread your account manager is already handling gets your account manager, not a machine talking over them. Asking something nobody has written down gets an honest sentence rather than silence, and a person actually flagged.
And for whoever runs it: one screen now says what the assistant knows, what it can do, and who it will do each thing for, with a switch beside every row. That last part is the one we expect to matter most in a year, because the interesting question about an AI assistant stops being what it can do quite quickly, and becomes what it will do without asking.
What we are not claiming
This is one deployment, one live number, and small volume. The mechanism is tested (3,900-odd unit tests, and a growing suite of evaluations built from messages that actually failed in production, including this one), but a test suite proves that if the model picks the right asset the right thing happens. Whether it reliably picks the right asset across a thousand phrasings is not something we can claim from where we are standing.
The scheduler proposes business-hour times in the operator’s zone and skips anything already booked. It does not read your real calendar. If your Tuesday morning is full and Klaros does not know that, it will offer Tuesday morning.
The permission model identifies a handset. A colleague who loses their phone is a colleague until somebody clears the field, which is the same exposure as any phone-based identity and worth saying rather than implying.
And the honest limit behind all of it: a handoff still depends on a human keeping the promise the software just made. We have written the sentence that admits we cannot answer, and it remains the sentence most likely to become a lie if nobody is watching the inbox. Making that promise something the system can keep by design, rather than by somebody remembering, is the next thing we are building. When it exists it will be described here, with the same amount of detail about what it fails to do.
If you are evaluating anything in this category, including this, the useful questions are not about what the AI can do. They are: what is it structurally incapable of doing, who decided that, and where can you read back why it did what it did?
Klaros is a self-deployed WhatsApp Business platform: you run it on your own Cloudflare account, connect your own Meta WhatsApp Cloud API number, and pay Meta directly with no markup added. More build notes · What is next · Start free
