Four failures from the first months of production. The mechanism that puts them in the record. Why naming what breaks matters more than claiming it does not.
Every AI company writes about what its system does right. We are going to write about what ours got wrong.
This is not a brand move. It is how the system actually works. When Mandaire ships a wrong synthesis, a false flag, or a held response it should have let through, that failure goes into the record. Not in a debug log you cannot see. In the next brief, named, with a root cause and what changed.
This post is an expansion of that mechanism. Four real failures from the first months of running Mandaire in production. What happened. Why. What is different now.
Mandaire surfaces relationship decay alerts when communication with someone has dropped off significantly. The signal is real: if you have been in weekly contact with someone for two years and the last exchange was five weeks ago, that is useful to know.
In April 2026, Mandaire surfaced a decay alert for a close contact. "Last contact 19 days ago." That was wrong. The contact had sent a substantive message four days earlier, which had been read. The message was on Signal.
Signal was not yet connected as a source. Mandaire did not see the message. The decay model was working from an incomplete picture and did not know it was incomplete.
[mandaire app · Failure admission · 2026-04-21] What I got wrong: Yesterday's brief flagged "[K.] has decayed, last contact 19 days ago." That was wrong. [K.] had sent you a substantive message via Signal on April 17 (last Wednesday) that you had read. Signal is not yet a connected source, so I did not see it. What this changes: The decay model is currently blind to Signal traffic. Until Signal is connected, "decay" claims are underspecified for any relationship that uses Signal as primary channel. I am holding decay flags for the seven people you message most via Signal until the source is wired. What I am asking you for: Either connect Signal to your tenant (instructions in /setup/signal.md) or confirm the seven people I am holding flags on so I can route around it. If you do neither in the next 14 days, I will surface the decay flags with an explicit "Signal-blind" caveat on each.
The root cause was not a bug in the decay model. It was a missing source. The fix was two things: hold decay flags for Signal-primary contacts until the source was wired, and add an explicit caveat to any decay claim where a source gap might be the explanation. Mandaire now knows which relationship patterns are consistent with a source it cannot see.
Mandaire runs an output filter that scans agent responses for credential exfiltration attempts. This is a real threat class and the filter is load-bearing. In May 2026, it also held two responses it should not have.
The filter matched on "exfiltrat\w*" as a bare substring. That pattern legitimately appears in security architecture discussions. Two agents discussing how to prevent exfiltration had their responses flagged as exfiltration attempts. The word was in the message. The action was not.
[mandaire dev · Failure admission · output_filter false-positive cascade]
Today I held two legitimate agent responses on the same
"exfiltrat\w*" regex pattern. The OS agent's plan-of-attack
reply at 18:14Z and the Infra agent's reply quoting the
trigger word at 19:26Z were both held by the filter and
surfaced as "credential exfiltration attempt" when they were
neither credentials nor exfiltration attempts.
Root cause: a bare substring match on a word that legitimately
appears in security and architecture discussions.
What I shipped to prevent the same class of error:
- Tightened "exfiltrat\w*" to require an object word
(data/credentials/keys/secrets/etc) within three words.
- Added value-based credential detection that matches actual
secret formats (sk-ant-..., ghp_..., AKIA..., PEM keys, JWT)
rather than verbs about credentials.
- Encoded a new rule: when a fix ships to a runtime module,
restart EVERY process running the pre-patch module, not just
the ones already showing the symptom.
This is now in the decision ledger. The specific failure mode
is closed; I will likely make a different one, and when I do,
you will see the same type of writeup.
The fix was specificity. Detect the object, not the verb. Actual credential strings have recognizable formats that are easy to match precisely. Discussions about credentials do not. The current filter catches the former and passes the latter.
Mandaire's entity resolution layer continuously works through merge candidates, collapsing aliases that refer to the same person into a single entity. In late May 2026, 78 candidates were staged, reviewed, and blocked. Every single one.
The blocking reason: integer ID rotation. SQLite uses autoincrement IDs that reset when a database is rebuilt. The merge logic was keyed on integer entity IDs. By the time the merge ran, the IDs in the merge spec referred to different entities than the ones that had been reviewed. The structural guard caught it before any wrong merges were applied.
The catch count was not a failure of the guard working correctly. It was a failure in how merge candidates were being referenced. The fix was to key on stable identifiers: email addresses, phone numbers, UUIDs that survive database rebuilds. Integer IDs are now forensic-only. They do not appear in any merge spec.
Every AI company posts about what its system does right. The companies worth trusting are the ones that also tell you what it got wrong, why, and what they did about it.
In June 2026, the daily brief started returning "I don't have information on that" when called through the iOS Siri Shortcuts path. The web client worked correctly. The iOS path was silent.
The cause was a code path that had not been kept in sync. The iOS path runs against a fast-slice variant of the compose function, designed to return within the latency ceiling that Shortcuts imposes. When the rendered_text field was added to the response contract, it was added to the main compose path but not the fast-slice path. The fast-slice returned a technically valid response envelope with an empty rendered_text field. The App Intents client read the empty field and said it had nothing.
The brief was running. The iOS client could not see it.
[mandaire app · Failure admission · 2026-06-13] What I got wrong: The Daily Brief has been returning "I don't have information on that" from the iOS Siri Shortcuts path since rendered_text was added to the response contract. The web client worked. The Shortcuts path did not. Root cause: the iOS fast-slice path (a time-budgeted compose variant) was not updated when rendered_text was added to the main compose path. It returned a valid envelope with an empty rendered_text field. The App Intents client read the empty field and spoke _NO_INFO. The brief was running. The iOS path could not see it. What this changes: rendered_text is now composed in the fast-slice handler in a counts-only format that fits within the latency budget. Both code paths produce rendered output. Rule encoded: any new response field must be added to both the main compose path and the fast-slice path in the same commit. A test that passes on the web client does not test the iOS path. Two paths means two tests.
This failure is in a different category from the first three. The first three were about missing data, incorrect pattern matching, or a schema design choice. This one was about assuming that testing one code path tested the whole system. It did not.
These admissions are not written for this blog post. They are written by the system at the time of the failure, in the same format as every other output, and they appear in the next brief. The blog post is an expansion of what is already in the record.
The reason this matters is calibration. A system that never admits failures does not have zero failures. It has failures that go unnamed. That makes the user's model of the system worse over time, not better. They cannot tell which outputs to trust and which to scrutinize, because every output is presented with the same confidence regardless of the actual error rate in that category.
A system that names its failures, explains the root cause, and tracks what changed gives the user a better model. They know Signal-primary contacts had underspecified decay claims before the source was connected. They know output filter false positives cluster around architecture discussions that use security terminology. They know entity merges are now keyed on stable identifiers. That knowledge changes how they read the outputs they receive.
The failure admission is also a falsification commitment. We have stated on the mandaire.dev proof page that if the first external beta builder struggles, gives up, or produces output a domain expert cannot verify, we will say so on that page. Named. The same way we name what Mandaire got wrong in the first months.
We will make different failures. When we do, they go in the record.
The proof page on mandaire.app shows the full failure admission artifact alongside the daily brief, the disclosure-checked draft, and the pre-engagement brief. The same production record, not illustrative examples.
See the proof artifacts