Build notes · conversational AI
The WhatsApp auto-reply that answered a hello with a price list
Someone we had been talking to for five weeks opened WhatsApp and typed “Hello”. Nine seconds later our software replied: “Klaros offers a one-time or annual license depending on the plan.” Nobody had asked about licensing. Nobody had asked anything at all.
The obvious diagnosis is that the model hallucinated, or that the prompt was weak, or that a small model is simply not up to the job. All three are wrong, and chasing any of them would have wasted a week. The model was behaving correctly. It had been handed the wrong conversation.
TL;DR
The thread query was ORDER BY created_at ASC LIMIT 40, which
selects the oldest 40 rows. On a 46-message conversation the model saw
messages 1 to 40, then a later slice cut that to 21 to 40. It never received the greeting,
and the newest message it could see was part of a pricing discussion from three weeks
earlier. Every timestamp had also been stripped, so it could not tell that three weeks had
passed. Investigating that one reply surfaced seven further ways the same assistant
was failing to listen, ending with a customer who asked to speak to a person and was
ignored twice.
What the model was actually given
The router that decides what an unclaimed inbound message deserves fetches the conversation
so the model has context. That fetch read direction, type and
body, ordered ascending, limited to 40. Then the prompt builder took the last 20
of those.
On a short thread this is invisible. On a real relationship it inverts: the longer somebody has been a customer, the further back the window slides, until the model is answering today’s question with last month’s conversation. The better the customer, the less the system knew about them. That is the exact opposite of what a product built on remembering is supposed to do, and it had been true since the feature shipped.
The second half was the clock. The transcript was passed as direction and text only, with
created_at discarded. So even with the right messages, the model would have read
a pricing exchange from 31 July and a greeting on 12 August as consecutive turns of one
conversation. A person who says “hi” after three weeks is starting something. A
person who says “hi” ninety seconds after you quoted them a price is continuing
something. We had removed the only information that distinguishes those two.
The fix is unglamorous: fetch descending, reverse for display, carry created_at,
and render a marker between messages when real time has passed, so the transcript contains
lines like — 12 days passed —. The prompt now says that anything
after a gap of a day or more is a new conversation and not the next turn of the old one.
Then the deterministic layer, which was switched off
A model should never have been the first thing to see “Hello”. There is a greeting rule, a keyword rule and an away rule, all of which run before the model and all of which are free. On this deployment all three were empty, so every inbound message from every stranger fell through to the model. It was not a fallback. It was the front door.
Worse, the greeting could not have fired even if it had been switched on. It was gated on
isFirstInbound, derived from whether the conversation row already existed, which
means a greeting could fire at most once per contact, ever. Somebody
returning after three weeks was, by that definition, mid-conversation.
We replaced it with a session boundary. The first attempt used the 24-hour service window, because that mapped neatly onto something WhatsApp already defines. That was tidy reasoning and wrong reasoning, and it took one more real message to find out: a contact wrote “What’s up ?” at 05:40 after eleven and a half hours of overnight silence, and 11.5 is less than 24, so the system treated the next morning as the same conversation. No human alive agrees with that. The service window is a billing construct, not a conversational one. The boundary is now six hours.
The lazier the message, the worse the answer
People type the smallest thing that will do. They send voice notes. They send a sticker and wait. The matcher in front of the model began by checking that the message text was non-empty:
if (!String(text || '').trim()) return false;A voice note has no text. Neither does a sticker, or a photo without a caption. So the model never ran, the greeting rule required text that looked like a greeting, and the result was total silence in response to the laziest thing a person can send, which in the markets this product serves is also one of the most common. A session-opening message with no text at all is now answered like a greeting. Mid-conversation it still stays quiet, because a person is presumably already engaged.
The spelling problem is the same shape. A lookup table handles “hi”, “hello” and “hey” for free and correctly. It will never handle “hlo bhai”, “vanakkam anna”, “gm sir” or whatever gets invented next, and enumerating them is a losing game. Recognising them is exactly what a model is good at. Writing the reply to a stranger is exactly what it should not be trusted with.
So the model got a new outcome that carries no words of its own. It can classify a message as a greeting; the sentence that sends is the operator’s. Any prose the model attaches to that verdict is discarded. Non-deterministic recognition, deterministic emission. That split is the whole design, and it is the same one that lets the model choose which of three pre-written buttons a person sees without ever composing what happens when they tap one.
It needed a second pass. “What’s up ?” is interrogative in form and contentless in substance, and the model classified it as an unanswerable question and handed off. The instruction now names question-shaped greetings explicitly, and just as explicitly excludes the ones that only look casual: “any update?” is a status request, and answering it with “how can I help?” would be the same deafness pointing the other way.
The compliance bug underneath
While tracing which code claims an inbound message, we found one that is not about tone at
all. The opt-out keyword list contains cancel and remove, and short
messages are matched on whole words. So:
Customer: “cancel my order” → three words, matches
cancel→ consent status flipped to opted out, suppression row written, order untouched.
The customer believed they had cancelled a purchase. The business believed they had refused marketing. Both were wrong, and neither would find out until something else went strange.
The instinct is to remove cancel from the list. That trades one silent failure
for another, because plenty of people do mean the subscription. The actual answer is that
the word is genuinely ambiguous and only live state can resolve it. When the
contact has an order in flight, Klaros now asks: two buttons, My order and Stop
messages. A tap is unambiguous, and because a button payload is read before any keyword
matching, the same word cannot be re-misread on the way back in. stop,
unsubscribe and optout are untouched and still absolute, because
those mean the subscription in every context and weakening them would trade a real compliance
guarantee for a hypothetical one.
This has history. In July a contact typed “extend” and was
unsubscribed, because opt-out matching was substring-based and ext-END contains
end. That was fixed the same day with word boundaries. The word-boundary fix
treated the symptom; this one treats the disease, which is a lexicon making a decision that
needs context.
Whose business is this, anyway
Every automatic reply was being generated under the instruction “You are answering a
WhatsApp message on behalf of…” followed by the founder’s personal
name. The prompt read a display_name column, which holds whatever free text was
typed at signup.
The number itself had four different identities in circulation at once: a Meta
verified_name that had passed review, a business profile pointing at this site, a
signup column holding a person’s name, and an inbound advertising referral claiming
something else entirely. The rule we settled on is that the identity of record is the
one that survived review. A Meta-reviewed name outranks a local free-text column,
which outranks an assertion made by something outside the system. It is read on a schedule and
cached rather than copied once, because a name under review moves.
That last category deserves its own paragraph, because it produced the strangest message of
the week. A Facebook post for an unrelated business, a saree shop, points its
wa.me link at this number. So a shopper arrived asking about cotton sarees, was
offered our product funnel by a sales matcher that could not see where they had come from,
and then received a perfectly accurate description of a WhatsApp CRM.
The referral data explaining all of this had been captured on the message since a migration in July and read by nobody in the chain. It now reaches the model, but deliberately as provenance and never as fact: the prompt states that it describes where the person came from, that it is not a description of this business, and that it must not be adopted as an identity. An advertisement can explain why somebody is confused. It can never redefine who you are.
The one that actually matters
At 18:12 the assistant correctly reached the limit of what it could ground and offered a menu. At 18:13 the contact tapped “Talk to a person”. The handoff was recorded as a milestone. The thread was flagged as needing a reply.
Then nothing happened, because nothing consumed either signal.
Eleven and a half hours later the same person wrote “What’s up ?” and got silence again. Every individual decision in that sequence was defensible. The model was right that it could not answer. The handoff was right to be recorded. The greeting was right not to fire mid-conversation. The sum of all those correct decisions was a business ignoring somebody twice.
Two things came out of it. The first is a sweep that asks the only question that matters: is somebody waiting who has not been answered? Not “did a handoff fire”, because there are already four ways a thread ends up needing a human and there will be more, and a detector that enumerates causes is guaranteed to miss the fifth. How long counts as too long is derived rather than configured: three times that business’s own median reply time, floored and capped, because a fixed threshold is either noise for a business that replies in four hours or useless for one that replies in five minutes. An explicit request for a person gets a fixed fifteen minutes instead, because that is a stated request rather than an inference.
It found something on its first run. A different contact, from that same saree advertisement, had asked for a human and been waiting 62 hours. Nobody knew.
The second is a rule we would now defend in any code review:
Silence is a reasonable answer to a message nobody can parse. It is never a reasonable answer to somebody asking why they were ignored.
When nothing claims a message and that thread is already flagged as waiting, Klaros now says so. It admits fault, promises nothing it cannot keep, and is debounced so that a frustrated person writing five times does not get five apologies. It deliberately does not say “shortly”, because the last promise made on that thread went unkept for eleven hours.
What we are not claiming
This note describes a system that listens better than it did last week. It does not describe one that remembers you.
Everything above is about a single conversation: the right messages, in the right order, with the clock attached, answered by the right identity, or honestly handed to a person. That is a floor, not a ceiling. A window is still a truncation, and on a long relationship the model is still structurally incapable of knowing how it began. Fixing that properly means compaction at session boundaries and a durable record of what was said and promised, which is designed and not built. When it exists it will be described here, with the same amount of detail about what it fails to do.
We would rather publish the eight ways our own assistant was not listening than a page about relationship intelligence we have not shipped. If you are evaluating any vendor in this category, including this one, the useful question is not what the AI can do. It is what happens when it cannot, and who finds out.
Klaros is a self-deployed WhatsApp Business platform: you run it on your own Cloudflare account, connect your own Meta WhatsApp Cloud API number, and pay Meta directly with no markup added. More build notes · What is next · Start free