Why Agents Need Five Gates, Not One: The Satisfaction Evaluator

The Premature Conclusion Problem

Most agent frameworks use a single exit condition: did the last tool call succeed? If the code compiles, if the test runner exits with code 0, if the file write returned OK — the agent concludes and hands back to the user.

This is insufficient for four reasons:

Tests can pass without covering the changed code. The test suite was green before — it is green after — but no test actually exercises the new function.
The diff can be technically correct but architecturally wrong. The function compiles but violates the module's established patterns, introduces a naming inconsistency, or adds a dependency that conflicts with existing constraints.
The agent can satisfy the letter of the task but miss the intent. "Add a retry mechanism" — the agent adds three lines that technically retry but with incorrect backoff semantics.
Rules can be violated without compilation failing. Security constraints, formatting standards, and API contract requirements are not enforced by the compiler.

The Myelix Agents Satisfaction Evaluator addresses this with five independent gates, each checking a different dimension of correctness. A task can only conclude when all five pass.

The Five Gates

GATE 01 — PRE_EDIT

Rule Adherence

Before any code is written, the planner's proposed approach is evaluated against the project's rules, constraints, and conventions. This gate checks:

Does the proposed change violate any explicit rules in the agent's operating protocol?
Does it introduce a pattern that is explicitly prohibited in this project?
Does it attempt to modify files that are outside the agent's authorised scope?
Does it use a library or API that is on the project's denylist?

This gate runs before the code is written — not after. A rule violation at this stage costs zero tokens of tool execution and saves the entire subsequent implementation round.

Why not just check rules at the end? Because at the end, the agent has already written 200 lines of code in the wrong direction. Catching the violation before the first file write saves 3–5 rounds of rework and the entire cost of the implementation attempt.

GATE 02 — PLAN_ADHERENCE

Plan Adherence

After implementation, the completed work is checked against the original plan the agent committed to executing. This gate verifies:

Did the agent complete every step it declared it would execute?
Are there any steps from the plan that have no corresponding implementation?
Did the agent add work that was not in the plan without flagging it?
Is the execution contract (the commitment made at plan approval) satisfied?

The execution contract is a structured record of what the agent explicitly promised to deliver, created at plan approval and used as the reference for this gate. It is not derived from the conversation — it is a separate data structure that persists independently of the context window.

Why does this need its own gate? Agents drift. A plan with 6 steps often completes 4. Without an explicit check against the committed plan, the agent has no mechanism to detect its own omissions. This gate is the structural equivalent of a code review checklist — it ensures nothing was silently dropped.

GATE 03 — TEST_COVERAGE

Test Coverage Assessment

This gate uses the Synapse knowledge graph to assess whether the new or modified code paths are exercised by tests. It does not simply check if tests pass — it checks if the right tests exist and cover the right paths:

What new functions or modules were added by the diff?
Do existing tests call into those functions?
Are there test files linked to the modified modules in the knowledge graph?
If no tests exist or coverage is inadequate, has the agent been instructed to add tests?

If coverage is insufficient and the task scope includes test creation, the gate fails and the planner receives a specific gap report — not a generic "add tests" instruction but a structured list of uncovered code paths derived from the impact analysis.

The green test suite fallacy. A test suite that was green before the change and is green after the change proves nothing about the new code. This gate exists specifically to catch the case where the agent added a new function that is never called by any test — the green suite is a false signal.

GATE 04 — DIFF_SAFETY

Diff Safety Review

The complete diff — every file modified, added, or deleted — is reviewed against a structural safety checklist:

Does the diff introduce any hardcoded secrets, API keys, or credentials?
Does it modify configuration in a way that would break deployment?
Does it delete files that are referenced by other modules not in the diff?
Does it change a public API surface (function signatures, module exports) without updating all callers?
Does it add a new dependency without updating the lock file?
Does it contain debug code, commented-out blocks, or TODO markers?

Synapse's blast-radius analysis provides the caller/callee graph for every modified symbol — the gate checks that no callers exist outside the diff that are now broken by signature changes.

The diff review that never happens. Human developers frequently skip diff review when they are confident the implementation is correct. Agents are even less likely to self-review their own output. This gate makes the review mandatory and structural — not an optional last step.

GATE 05 — FINAL_SATISFACTION

Holistic Satisfaction Assessment

The final gate is a holistic evaluation of whether the task is actually done — from the perspective of the original user intent, not just the technical implementation. It considers:

Does the completed implementation match the intent of the original task description?
Are there edge cases or error conditions mentioned in the task that have not been addressed?
Is there anything in the conversation history that the agent committed to doing but has not yet done?
Would a senior engineer reviewing this work consider it complete and ready for merge?

This gate is intentionally qualitative. The previous four gates are structural and deterministic — they check specific measurable properties. This gate checks the things that cannot be reduced to a checklist: coherence, completeness, and intent alignment.

The last line of defence. An agent can pass all four structural gates and still have produced something that misses the point. This gate exists to catch the "technically correct, practically wrong" outcome that is the most frustrating failure mode in autonomous development.

Gate Failure: What Happens Next

When a gate fails, the failure is not logged and ignored — it is fed back into the planner as structured input for the next round.

Gate violations are stored in a dedicated gate_violations slot in the volatile tier of the prompt (see the 3-tier prompt architecture). This slot is never pruned. The planner receives the specific gate ID, the violation description, and a structured remediation hint.

%GateViolation{
  gate: :test_coverage,
  severity: :blocking,
  description: "3 new functions in UserAuthenticator have no test coverage",
  uncovered_paths: [
    "UserAuthenticator.verify_token/2",
    "UserAuthenticator.refresh_session/1",
    "UserAuthenticator.invalidate_all/1"
  ],
  remediation: "Add ExUnit tests covering the happy path and error cases for each uncovered function"
}

The planner uses this structured input to generate a targeted remediation plan — not a generic "fix the tests" instruction, but a specific implementation of what needs to happen to clear the gate.

The Gate Loop

Gates run after every tool execution batch. The flow is:

Planner proposes actions
Gate 1 (pre_edit) evaluates the plan — if it fails, the planner revises without executing
Actions execute
Gates 2–5 evaluate the results
If any gate fails → structured violation → planner receives it → next round
If all gates pass → task concludes

There is a round budget. If the agent has not cleared all gates within the budget, the session concludes with a partial completion report rather than an incorrect completion report. The developer receives a transparent accounting of exactly what was and was not achieved.

Why Five? Why Not More?

The five gates were designed to cover the five distinct failure modes that appear most frequently in autonomous coding sessions, based on internal trajectory data. Each gate addresses exactly one of these failure modes:

Gate	Failure Mode It Prevents
Rule Adherence	Agent implements something technically valid but prohibited by project constraints
Plan Adherence	Agent silently omits steps from its committed plan
Test Coverage	Agent passes existing tests but new code is never exercised
Diff Safety	Agent introduces structural breakage (missing callers, secrets, lock file skew)
Final Satisfaction	Agent misses the intent despite technical correctness

A sixth gate would require a sixth distinct failure mode that is not already covered by one of these five. Based on current data, additional gates would be redundant — the five collectively catch >95% of the failure patterns observed in trajectory analysis.

Comparison With Single-Condition Agents

Most agent frameworks expose a single is_done signal: did the last action succeed? The pattern appears in LangChain, CrewAI, and every IDE-based agent. It is the minimal viable completion check.

The Myelix Agents gate system is not an incremental improvement on this pattern. It is a different architectural layer — a dedicated verification module that the planner is structurally prohibited from bypassing. The planner cannot declare success. Only the SatisfactionEvaluator can.

This is the structural guarantee that allows us to claim: Myelix Agents does not stop when it thinks it is done — it stops when it can prove it is done.