TL;DR: The failures on this site worth publishing were never wrong answers. They were correct-looking output that passed every automated check and shipped broken, in one case for months. Six of them are below, along with the argument they add up to, which is that in an agentic operation the automated check layer is the governance layer and everything else is paperwork.
The first part of this series described what an agentic operation actually consists of once the demo is over, which is a written context layer, a memory with an expiry policy, and one declared source of truth. This part is about what still goes wrong after all of that is in place, and specifically about the class of failure that autonomy makes worse.
That class is not the one people worry about publicly. A hallucinated statistic is embarrassing and it is also, mercifully, the easy case, because a claim that is wrong tends to look wrong to anyone who knows the subject. What has actually cost me time on this site is output that was structurally correct, passed every automated gate, rendered without complaint, and was broken in a way that only surfaced weeks later through a channel that had nothing to do with the pipeline that produced it.
I can publish these because it is my site and my mistakes. Almost nobody in a brand can, which is most of the reason this genre of writing barely exists.
A truncated link is still valid markdown
The underlying hazard is environmental. This project lives on a Windows machine and the work happens through a bridge onto that filesystem, and every so often a write that makes a file larger gets capped at the file’s previous size. The tail is lost. No error is raised anywhere.
For code this is loud and therefore harmless. A .astro component cut off mid-file has no closing tag, the compiler says so in plain language, and the failure is fixed within a minute of appearing. It happened repeatedly to the footer, to the global stylesheet, to the Italian home page, and each time it announced itself.
For prose it is completely silent, and the reason is a detail of the markdown specification rather than anything to do with the model. A link cut mid-URL, so that the line ends ](https://example.com/some-repo (with the closing parenthesis never arriving), is still a perfectly valid markdown document, and what sits there afterwards is literal text with no link in it anywhere. The parser is entirely happy, astro check passes, astro build passes, and the deploy goes out.
Three articles shipped in that state and stayed that way for months. I did not find them. An external link crawler did, long after publication, on a routine site audit, which meant the first signal that my publishing pipeline had a data integrity problem arrived from outside the pipeline entirely.
I want to be precise about what failed there, because it is easy to file this under “the AI made a mistake” and learn nothing. The content was correct when it was written. The tool reported success. The type checker passed on the real, complete source. The build passed. The deploy passed. Every component behaved exactly as designed, and the article was still wrong on the public internet for a quarter of a year. This is a systems failure with no faulty component in it, which is the same shape as most of the expensive integration failures I have been called in to diagnose in client stacks.
The fix is deterministic or it is not a fix
The instinct after a failure like that is to be more careful. Being more careful is not a control, it is an intention, and it degrades the moment the week gets busy.
What went in instead is a script. check-content.mjs is 248 lines of extremely boring Node, it scans every markdown file under the content collections and the drafts folder, and it looks for the specific signatures of a capped write: an unterminated markdown link, an unterminated autolink, unbalanced link brackets on a line, NUL bytes anywhere in the file, and a final content line that ends mid-word with no terminal punctuation. If it finds anything it exits non-zero.
The important part is where it sits. It is the first command in npm run build, ahead of the type check, ahead of the build itself, so a truncated file now fails the deploy rather than shipping quietly. The check is deterministic, it is cheap, and it does not care how the damage got there.
One design decision in it is worth pulling out, because it is the one I would defend hardest. The mid-word-tail check has a whitelist: a run of hashtags is how a social draft legitimately ends, a table row ends on a word character, a horizontal rule is not a sentence. Without those exemptions the script would warn on every run. A checker that always warns is a checker that nobody reads, and a checker nobody reads is worse than no checker at all, because it converts a real signal into background noise while leaving everyone convinced the problem is covered. Every marketing team that has ever muted a data quality alert knows exactly how that ends.
Writing down “cause unknown” and meaning it
Here is the part of this story I am least comfortable with and most convinced by.
When the truncation first appeared I did what any experienced person does, which is form a hypothesis that fit the evidence. Two writers on the same file, one of them on the Windows side: a format-on-save extension, a file watcher, cloud sync. It explained the timing. It explained why it hit some files and not others. It was plausible, it was mechanistically sensible, and I wrote it into the project instructions as the cause.
It was wrong. I have never had format-on-save enabled. A direct test in August could not reproduce any truncation at all: writes growing a file from fifty bytes to two hundred kilobytes came through clean, a tool write followed by an appending edit came through clean, and a sweep across the whole 149-file corpus came through clean. A separate part of the same theory, that a permission error on delete indicated a Windows lock, turned out to be an unrelated safety guard in the tooling.
So the note in the project file now says, in capital letters, that the cause is unknown and that nobody should assert one. It also says explicitly that an earlier version of the same file blamed the editor and was unsupported, because otherwise a future session reads a confident-sounding paragraph and rebuilds the same wrong model.
There is a residue of that mistake still sitting in the repository, and I have deliberately left it. The header comment at the top of check-content.mjs still explains the Windows-writer theory as fact. It is a fossil of a superseded explanation, embedded in the one piece of code that reliably catches the real problem, and it is a useful reminder that the artefact can outlive the reasoning that produced it.
I recognise the pattern from attribution work. A plausible causal story that fits the data is the most expensive thing you can adopt early, because it stops the investigation, it gets written into decks, and it survives long after the evidence for it has evaporated. The check that caught the truncation works precisely because it makes no assumption about cause. It looks for the damage.
The failure a syntax check cannot see
Everything so far is structural, which means a machine can find it. The next one is not.
I publish in English and Italian, and the Italian articles are translations of English originals. In one of them the phrase journey estate, British usage for the whole portfolio of journeys a brand runs, was left in English on the entirely correct judgment that it was a term of art. Estate is also an ordinary Italian word, and what it means in Italian is summer. The sentence, read by an Italian reader with no memory of the English, instructed them to structure a summer journey.
There is no linter for this. The output was valid Italian, every word in it existed, and the grammar was clean. The failure sat at the level of meaning, and specifically at the level of which meaning an Italian reader hits first, which is never the technical one.
The same family of error runs in the other direction as well, where the term does get translated and lands on the wrong Italian word entirely. Eventually becomes eventualmente, which means if need be. Consistent becomes consistente, which means substantial rather than coherent. Sensible becomes sensibile, which means sensitive. Each of those produces a sentence that is grammatical, confident and about a different subject than the one I wrote.
The guards that exist now are unglamorous and they work. A table of false friends lives in the writing rules with a declared house translation for each, so the decision gets made once instead of being re-argued per article. English terms of art that collide with ordinary Italian words have house renderings of their own, and patrimonio is the settled one for the case above. A short list of loanwords is kept in English deliberately, composable among them, because translating those produces something no Italian practitioner would say out loud. And the last check before an Italian article publishes is to read each sentence aloud as a monolingual would, with no memory of the source, and notice where it lands.
That check is not automatable, which is the honest point of this section. Structural damage has a script. Meaning has a reader.
The drift that is only visible in aggregate
There is a third category that no single article reveals.
Every piece on this site that cites anything ends with a Sources section, which is a small and entirely mechanical convention: a heading at one level, groups in bold, bullets underneath, real link text on every entry. Six months of writing produced four different versions of it. Some articles headed the section References, others Sources and references, and a few used an H3 where the house level is an H2. Some entries had become bare autolinks with the URL doing duty as its own link text, and one had drifted into a numbered list. Group names had picked up colons in some places and not others. Roughly half had a blank line between the group name and its first bullet where the rest had none.
Not one of those articles looked wrong on its own. Each variant was internally consistent, rendered cleanly, and passed every check that existed, which was easy because the check that would have caught it did not exist and could not have. There was no written specification to check against. The drift was visible only by opening all ninety-three articles at once, which is not something anybody does in the course of a normal week.
The fix was to specify the format completely, in one place, including the parts that feel too trivial to write down, and then normalise the whole corpus against it in a single pass in August. The em dash rule went through the same treatment, and now carries a standing instruction to strip on every revision rather than depending on me noticing.
What earns this a place next to the truncated links is its shape. Per instance the deviation is invisible and defensible. In aggregate it is obvious. And the only reason it became checkable at all is that somebody eventually sat down and wrote the convention out in full, after six months of it existing purely as something I would recognise when I saw it.
Anyone who has inherited a campaign taxonomy knows this failure intimately. Nobody ever names a campaign wrongly. Everybody names it slightly differently from the person who left, and eighteen months later the reporting layer sits on a naming convention that exists in four mutually incompatible dialects, none of which was a mistake at the moment it was introduced.
The ones that break by being absent
The last category is the hardest, because nothing is wrong on the page. Something is missing from it, and absence does not raise an exception.
The layout on this site derives the social preview image by taking the article’s hero and swapping the extension to .png, because LinkedIn and X cannot render SVG or WebP. The swap is blind. If a hero has no PNG sibling at the same path, the share preview points at a file that does not exist, and there is no way for the page to know. Four posts sat like that until I checked, including, with some irony, both parts of the Braze demo write-up. That check is now in the same script that catches truncation.
The other one took days. Astro aggregates CSS that is reachable from only one page into a chunk of its own, and two unrelated pages can collide on the same chunk name, at which point one of them is silently dropped from the built output. No error, no warning, correct source. The Italian home page lost its styles that way while the English one built fine. Months later the same mechanism hit two shared components and quietly removed the footer newsletter card from the home page and the Weekly Signals hub.
Both of these are the same shape as a tracking tag that was never fired and a consent flag that was never written. The system has no representation of the thing that should have been there, so it cannot tell you it is missing, and every dashboard reports health.
I will add one caution, because the detectors lie too. When an external crawler reported eleven dead outbound links on this site, seven of them turned out to be publishers blocking automated requests, and the links were fine in a browser. Each one had to be verified against a control fetch on the same host before anything was edited. An unverified alert is how you end up rewriting correct articles.
Where governance actually lives
Put the six together and they sort into four groups. Structural damage, which a script can find. Semantic damage, which only a native reader or a targeted grep can find. Stylistic drift, which needs a stated rule and a pass over the corpus. And absence, which needs a check that knows what should exist.
Only one of those four is covered by anything a vendor demos.
The conclusion I have taken back into client work is uncomfortable in meetings. In an agentic operation, the automated check layer is the governance layer, and the human approval step is only worth what the reviewer can actually verify. A person clicking approve on a batch of generated variants, at the volume these systems produce, is not exercising judgment. They are absorbing liability for a sample they did not read. If the check that would have caught the failure does not exist in the pipeline, the approval gate above it is decoration, and the fact that somebody signed makes the audit trail look better and the outcome no different.
I wrote in The Uncomfortably Human Shape of Agentic Loops that the most important component in an agentic loop turns out to be the part that says no. The months of running one has not changed my view, it has just made me specific about it. The part that says no has to be deterministic, it has to run before the thing that ships, and it has to be cheap enough that nobody is tempted to skip it on a Friday.
So when a vendor shows you what their agent can produce, the question worth asking is what it refuses to ship, and who wrote that refusal down.
The third and last part of this series is about the layer above the checks: how repeated editorial work on this site turned into named procedures with their own caps, gates and approval rules, and where the comparison to journey orchestration honestly stops holding.
Sources
Specifications
- CommonMark specification. The parsing behaviour that makes a truncated link render as literal text rather than an error.
- Astro content collections. The schema layer that validates article frontmatter, and silently drops any field not declared in it.
