- AI Software Factory
- Evidence Spec
Evidence on every pull request:the spec we ship against.
Verify AI-written code without taking our word for it.
What every pull request carries
- E1Acceptance-criteria traceTraced back to each criterion written with you
- E2Independent AI code reviewA different model from the one that wrote it
- E3Tests and logsThe test output itself, not a summary
- E4Visual evidenceAnything on screen, shown on screen
- E5CI, judgedA skipped job never counts as a pass
- E6Merge policy and risk classWho approves is set before work starts
- E7Audit trailEvery decision and every action recorded
What is an evidence report?
- 01Criteria
- YouAgree the criteria before the goal is Ready
- Forward deployed engineersWrite each goal's acceptance criteria with you
- 02Build
- Forward deployed engineersSteer the session while it runs
- Steer + AI agentsA coding session writes it, runs tests, takes screenshots
- Review + checksCI runs on the change, judged by a model
- 03Review
- Forward deployed engineersRead the verdict, the findings and the evidence
- Review + checksA different model checks each criterion
- 04Merge
- YouApprove what your merge policy gates
- Review + checksPolicy decides: merge on evidence or ask a person
- 05After merge
- YouRead the audit trail from your Viewer seat
- Steer + AI agentsKeeps every decision and approval on record
- Review + checksVerified once CI is green on the default branch
Findings go back to a coding session, and the change is reviewed again.
Seven clauses on every pull request.
- E1
Acceptance-criteria trace
Every goal carries acceptance criteria, written with you before the goal is Ready. The pull request is traced back to each one.
- What's checked
- Each acceptance criterion on the goal, one at a time, against the change set.
- Checked by
- The review model, then a Pento forward deployed engineer.
- On the pull request
- Each criterion marked confirmed or to fix, with the review it came from: review 11:05 · 2 of 3 confirmed.
- E2
Independent AI code review
The change set is reviewed by a different model from the one that wrote it, so the coder never grades its own work.
- What's checked
- Spec conformance, coding standards, security, tests and UX.
- Checked by
- A review session on a different model than the coding session, then a Pento forward deployed engineer.
- On the pull request
- A verdict (approve, approve_with_findings or request_changes) and every finding with its kind, severity (low, medium or high), file:line and the criterion it touches.
- E3
Tests and logs
The evidence includes the test output itself, with no summary standing in for it.
- What's checked
- Which tests ran in the session's sandbox, which failed, and on which lines.
- Checked by
- The coding session runs them; the review reads the log.
- On the pull request
- tests.log with its failing-line count (4 failing lines of 478). The session's brief.md and transcript.jsonl sit beside it in the workspace.
- E4
Visual evidence
If a change puts something on screen, the evidence shows it on screen.
- What's checked
- Each screen and state the change touches, against the acceptance criteria.
- Checked by
- The coding session captures them; the review model files ux findings and a forward deployed engineer looks.
- On the pull request
- Screenshots of each screen the change touches.
- Applies to
- Changes that render anything on screen.
- E5
CI, judged
A model reads every CI run and judges it, so a skipped job never counts as a pass.
- What's checked
- Each CI run on the change, and the run on the default branch after merge.
- Checked by
- A model judges each run: passed (proves the commit builds), failed (broken, or could not prove it is not) or ignore (replaced or skipped).
- On the pull request
- The judged result. A merged goal becomes Verified only when CI is green on the default branch.
- Applies to
- Repositories with CI.
- E6
Merge policy and risk class
Who approves a merge is set by policy before work starts, not by the agent that wrote the code.
- What's checked
- The change's risk class (Low, Data or External), the project's autonomy level, and how safe the review model rates the change to merge unattended.
- Checked by
- Steer's merge policy, set by the project's Owner. A change rated under 0.80 safe asks a person at every level.
- On the pull request
- A strip with all three: Risk class: External · sends through Postmark / Safe to merge unattended: 0.62 · below the 0.80 threshold / Autonomy: Autonomous · external risk asks.
- E7
Audit trail
Every decision Steer makes and every action a person takes is recorded.
- What's checked
- Each orchestrator decision: the project state it read, the question, the action, its probability, the model, the latency and the cost. Each person's approvals, answers and steering messages.
- Checked by
- Recorded automatically on every orchestrator turn and every person action.
- On the pull request
- The Audit view in the workspace, which your Viewer seat can read, with Export JSON.
Who approves a merge is decided before the code exists.
Risk class follows what the change touches
Low covers UI and logic inside the repository. Data covers migrations and stored data. External covers outside APIs, secrets, billing and deployment, such as sending email through Postmark.
The 0.80 rule holds at every level
The review model rates how safe each change is to merge unattended. A change it rates under 0.80 asks a person, even under Autonomous.
A policy merge still needs its evidence
A change merges by policy only on an approving review and, where your repository has CI, a passing run. The policy decides who signs off, not whether the checks run.
- web #41Fix plan picker labelMergeRisk class Low · safe 0.96because low risk
- web #45Cart totals in EURMergeRisk class Low · safe 0.91because low risk
- api #18Index invoices by customerAsk a personRisk class Data · safe 0.88because data risk
- api #19Backfill invoice totalsAsk a personRisk class Data · safe 0.62because data risk
- web #44Receipt emails via PostmarkAsk a personRisk class External · safe 0.93because external risk
Gated: 2 merge3 ask a person0 report
Models make the judgment calls; below 0.70, a person does.
- 01
Events in, decisions out
A Slack message, a finished session, a review verdict or a CI run wakes the orchestrator, and it answers one question: what happens next.
- 02
Low confidence goes to a person
Under 0.70 the decision becomes a question for someone specific, such as a merge for Dana Kim to approve, with the context she needs to decide.
- 03
Questions don't stall the project
Each question carries a default and a deadline, 48 hours unless you set another. If nobody answers, the default applies, and anyone can reopen it.
Every decision is on the record.
A seat in the workspace from day one
The Viewer role shows progress, spend, speed and the record: Home, Goals, Changes, Numbers and Audit. You read the work itself.
People are on the record too
Approvals, answers to questions and steering messages are logged with who acted and when, next to the decisions they changed.
It exports
The trail exports as JSON, so your auditors or your own tools can read it.
What evidence proves, and what it doesn't.
Evidence shows
- Which acceptance criteria the review confirmed, and which it sent back to fix.
- Which tests ran, and the exact lines that failed.
- What the change looks like on screen.
- Whether CI proved the commit builds, and later that the default branch is green.
- Who approved the merge, or which policy did.
Evidence doesn't show
- That a test exists for a case nobody wrote down.
- That the acceptance criteria were the right ones to write.
- How the change behaves against production data or traffic it never saw.
- That nothing can break after merge.
The rules around the evidence
- Before work startsReady
Acceptance criteria are written with you before a goal is Ready.
- Data and outside systemsGated
Changes that touch data or outside systems still go to a person under Gated.
- Low confidence0.70
Any decision under 0.70 becomes a question instead of an action.
- After mergeVerified
A merged goal only counts as Verified once CI is green on the default branch.
How to trust AI-generated code: the questions we get
Not covered here? Bring the question to a discovery call. An engineer reads every request.
Book a discovery callContact us
Read the evidence on your next project
Bring a project you've been putting off. We scope it in a free audit and quote it in writing, as a monthly team or at a fixed price. Every pull request then ships with the evidence on this page. An engineer reads every request.