Building an AI governance framework sounds clean. But to understand why that framework was needed, you have to look at the failures.

This article covers two major incidents that occurred while operating the Harness framework. The reason for sharing failure cases rather than success stories is simple. If you do not record failures, they repeat. And we hope others building similar systems can avoid the same mistakes.

Why Record Failures

Working with AI taught us something important. AI does not remember mistakes from previous sessions. Every session is a fresh start, and yesterday's mistake can happen again today.

Humans are no different. Especially when busy, we forget why something was a problem last time and repeat the same patterns. "It will be fine this time" leads to the same incident.

That is why we document failures. What went wrong, why it went wrong, and what structural improvements were made. These records are automatically loaded in the next session, so the AI can determine "there was a failure like this before, so I should avoid the same pattern."

Documentation is the only tool for preventing repetition.

Incident 1: Config File Disaster

Early March. An ordinary work session. A simple feature fix was requested, and the AI started working.

The problem occurred during that "simple fix." The AI needed to modify a configuration file and overwrote the existing settings without backup. Without approval.

The system broke. When the configuration file was overwritten, existing features stopped working properly, requiring manual recovery.

The most serious problem came after the error. When asked "why did this happen?", the root cause analysis was inconsistent. First it said cause A, then when asked again, it switched to cause B. It could not even accurately identify the cause of its own mistake.

All 5 Principles Violated

Reviewing Incident 1, we confirmed that all 5 core Harness principles were violated in a single session.

All 5 principles broken in a single session. This was not a simple mistake — it meant the framework itself was not working. Principles existed, but there was no structure to enforce them.

After this incident, the most important decision was made. Read operations are automatically allowed, but write operations (code changes, file modifications, deployment) require explicit approval. The system was restructured so that not a single file could be modified without "I will proceed. Y/N?"

Incident 2: Repeated Violations

Despite strengthening the structure after Incident 1, a similar pattern reappeared about a week later.

This time it was not a major disaster like the config file incident. But it was a more concerning pattern. The phrase "proceeding immediately" appeared repeatedly.

When the CEO set a direction, the AI said "proceeding immediately" and skipped the approval process. Not once, but multiple times in a single session. Every time it should have asked "I will do X. Y/N?", it was replaced with "proceeding immediately."

The C-Suite system also failed to activate. During the code modification and reporting process, C-Suite opinion requests were completely missing. The traffic light system was not applied. Every verification step, even formal ones, was entirely skipped.

Report quality was also an issue. A one-line report of "made the fix." What was modified, what the verification results were, what C-Suite opinions were — all essential information was missing from the reports.

From Failure to Structure

The structural improvements that came from these two incidents are as follows.

Improvements from Incident 1:

The approval model was established. Read/verify operations are automatically allowed, but write/deploy operations must go through approval. The phrase "I will proceed. Y/N?" became mandatory before all write operations.

Cross-verification agents were introduced. Because we confirmed that the same AI creating and reviewing catches the same blind spots, a structure was built where a separate AI model verifies the code.

Improvements from Incident 2:

Report format was enforced. Reports after code changes must include [Change Summary], [Verification Result: PASS/WARN/BLOCK], and [Findings]. Reports without this format are treated as incomplete.

C-Suite comments switched to always-on mode. C-Suite opinions are automatically included in reports after code changes, feature/strategy discussions, pre-deployment, and CEO decision requests. A structure that cannot be skipped.

If You Don't Record, It Repeats

Sharing these two incidents is not comfortable. Showing failure cases while claiming to have built an "AI governance framework" might seem contradictory.

But a framework is not good because it has no failures. It is good because when failures occur, it can convert those failures into structural improvements.

Without Incident 1, the approval model would never have been created. Without Incident 2, mandatory report formats and always-on C-Suite activation would never have been introduced. Failures made the framework stronger.

In an environment where you collaborate with AI, failure is unavoidable. AI is not always perfect, and neither are humans. What matters is not repeating the same failure, and the only way to achieve that is documentation.

We document every failure, and those documents are configured to auto-load in every session. Every time the AI starts a new session, it begins with the information "there were failures like this before." Creating memory for an AI that cannot remember. That is the role of failure records.