No semantic correctness proof
CEK shadow review does not prove semantic correctness.
LEDGER BENCHMARK · 30 TASKS · ONE RUN
Your agent wrote the patch. AdaptOrch decides if it’s right.
It runs the same task several ways, checks the answers against each other, and ships only the one that survives.
On our ledger benchmark the gate caught 8 wrong answers out of 30 — and introduced 1 of its own. We publish both numbers.
Unverified
Verified
correctness_claim: false —
we report evidence, not semantic correctness.
- MODEshadow-only
- CLAIMcorrectness_claim=false
- SELECTORmutation = 0
- LABOntos Lab
AI coding agents move faster than your review process.
Coding agents can generate patches in seconds. But teams still need to know whether a patch applies cleanly, changes unrelated files, hides regressions, confuses environment failure with candidate failure, or merely looks correct.
AdaptOrch turns each candidate patch into auditable evidence before it becomes a PR decision.
-
Apply is not enough
A patch that applies cleanly may still be semantically wrong. AdaptOrch separates can apply from should submit.
-
Runner errors are not candidate errors
Broken installs, missing dependencies, plugin failures, and service issues are classified as runner health, not automatic patch failure.
-
Repair is not correctness
A repaired patch is only more testable. It still needs executable evidence before promotion.
-
AI rationale is not evidence
Constitutional review can flag risk, but it never overrides hard execution evidence.
A reliability kernel for AI-generated patches.
AdaptOrch is not another coding agent. It is the evidence layer between coding agents and merge decisions.
Use Claude Code, Codex, Cursor, Devin-style agents, internal tools, or open-weight generators. AdaptOrch receives their patch candidates, canonicalizes them, tests them, reviews them against an explicit constitution, and records the decision path.
Docker-Faithful Apply Gate
Run patch application through a harness-aligned gate before any downstream decision. Non-applicable patches are blocked or routed to repair.
Canonical Clean Diff
Convert fuzzy or tool-generated patches into realized git diffs. Only canonical clean patches flow downstream.
Strict Re-Check
Re-apply canonical patches under strict checks. No silent fuzzy acceptance. No hidden mutation.
Runner Causality
Classify failures as environment-caused, candidate-caused, or ambiguous. Environment health never becomes a fake candidate failure.
Constitutional Shadow Review
Apply an explicit software-engineering constitution to each evidence card. Flag broad exceptions, unrelated edits, hardcoding, provenance issues, and repair drift.
Evidence DAG
Every candidate, gate, receipt, risk label, and review decision is replayable as a structured evidence DAG.
Shadow review is not a correctness claim.
AdaptOrch CEK S3 is a shadow integration lane. It emits CEK shadow reports only when explicitly enabled. It does not change the baseline AEG-selected candidate, and every CEK shadow record keeps correctness_claim=false.
AI critique is subordinate
AI critique does not override Docker, test, or provenance evidence.
Runner health is isolated
Environment-caused errors are runner-health signals, not hard candidate blocks.
Forbidden selector inputs
Official outcomes, hidden tests, gold patches, and expected answers are forbidden selector inputs.
Active promotion needs paired evidence
Real Docker paired correctness evidence is required before active promotion.
Read the full boundary
See what AdaptOrch does and does not claim on the Claim Boundary page.
Projected benchmark scenarios across frontier coding models.
These plates are confidence-weighted scenario projections, not measured AdaptOrch benchmark results. They show where an evidence-gated reliability layer may create uplift; validate against your own repositories before making a deployment decision.
From AI patch to replayable evidence in ten gates.
AdaptOrch converts raw agent output into a structured review decision. Every gate is typed, source-labeled, and replayable.
- 01
INTAKE
Receive raw, repaired, synthesized, or multi-agent patch candidates.
- 02
CLUSTER
Group similar candidates and preserve lineage across raw, repair, and synthesis paths.
- 03
APPLY
Check whether the patch can be applied by a harness-faithful path.
- 04
CANONICALIZE
Materialize the realized git diff and remove fuzzy patch ambiguity.
- 05
STRICT
Verify that the canonical patch applies cleanly without hidden mutation.
- 06
PROVENANCE
Detect unrelated edits, broad exception swallowing, large drift, missing source labels, and forbidden evidence.
- 07
REPRO
Run public or generated tests when available. Base-fail / patch-pass evidence is strong. Public pass alone is weak.
- 08
TAXONOMY
Separate runner health, candidate errors, ambiguous failures, and candidate-caused regressions.
- 09
CEK
Review the evidence card against the AdaptOrch constitution. Emit risk labels without promoting correctness.
- 10
EXPORT
Write a replayable shadow report for audit, review, and customer-facing explanation.
A software-engineering constitution for AI patch review.
AdaptOrch CEK reviews each candidate against explicit engineering rules. The constitution is not a vibe check. It is a product policy layer over executable evidence.
Docker-faithful apply gate
A patch that cannot pass the Docker-faithful apply gate is not selectable.
Only clean diffs flow
Only canonical clean diffs may flow downstream. Strict re-check failure blocks selection.
Candidate-caused failures
Candidate-caused syntax, import, collection, or regression failures are hard blocks.
Base-fail / patch-pass is strong
Base-fail / patch-pass evidence is strong positive evidence. Public test pass alone is supportive, not proof.
Repair without repro is higher risk
Repaired candidates without repro evidence are higher risk. Broad exception swallowing, unrelated edits, and hardcoding are penalized.
Every decision is replayable
AI rationale never overrides hard execution evidence. Every final decision must be replayable in the evidence DAG.
Watch one AI patch move through the evidence kernel.
Pick a candidate shape. AdaptOrch applies, canonicalizes, classifies runner vs candidate failure, then emits a CEK shadow review without promoting correctness.
CANDIDATE
Candidate patch modifies astropy/table.py to repair issue behavior.
- APPLYDocker harness apply: pass
- CANONICALIZERealized git diff generated
- REVIEWCEK flags repaired-without-repro risk
DECISION
Needs Review
Applicable, but not correctness-promoted. Runner evidence is inconclusive and CEK shadow review flags repair risk without validated repro evidence.
The review artifact your team can actually inspect.
Every CEK shadow run emits a structured report with candidate states, gate vectors, risk vectors, runner-health counts, hard blocks, receipt digests, and explicit claim boundaries.
run_id · mode · schema_version
mode: shadow · correctness_claim: false · selection_mutated: false
selected_candidate_id · records
Candidate records with lineage, gate_vector, risk_vector, and hard_blocks.
receipt_digests · runner_health
Digest-backed receipts plus runner-health counts separated from candidate correctness.
Evidence your reviewers, SREs, and auditors can read.
The AdaptOrch control plane makes AI patch review observable. It shows which candidates survived, where they failed, whether the runner is unhealthy, and why a patch is selected, blocked, or routed to review.
Run Overview
Track instances, candidates, apply status, canonicalization, test evidence, runner health, and final review state.
Candidate DAG
See raw, repaired, and synthesized candidates flow through evidence gates.
Diff Viewer
Inspect canonical patches, not fuzzy agent output.
Constitutional Review
View CEK violations, risk labels, provenance checks, and claim boundaries.
Runner Health
Separate broken installs, missing dependencies, plugin errors, and service failures from candidate-caused failures.
Audit Export
Export signed JSON reports for security, platform, and engineering review.
Drop it beside your existing coding agents.
AdaptOrch does not own your IDE, prompts, memory, or model provider. It sits beside your agent workflow and verifies patch candidates before review or merge.
Python 3.11 core
Typed candidate and evidence models, deterministic shadow reducer, mypy-gated source tree, focused regression suite.
- Typed candidate and evidence models
- Deterministic shadow reducer
- Mypy-gated source tree
- Focused regression suite
Run detail + evidence timeline
Run detail pages, candidate evidence timeline, CEK shadow report viewer, diff viewer, runner-health dashboard, report export.
- CEK shadow report viewer
- Diff viewer
- Runner-health dashboard
- Report export
API · MCP · workflow hooks
REST API, MCP adapter, GitHub / GitLab workflow hooks, Docker Compose PoC deployment, private runner and enterprise appliance path.
- REST API
- MCP adapter
- GitHub / GitLab workflow hooks
- Private runner / enterprise appliance
Start in shadow. Promote only after evidence.
- DAY 01
Shadow Audit
Connect AdaptOrch beside your existing AI coding workflow. No selector mutation. No merge automation. Evidence only.
- DAY 07
Evidence Review
Inspect candidate DAGs, CEK shadow reports, runner-health signatures, and provenance warnings.
- DAY 14
Policy Calibration
Tune constitution rules, risk thresholds, forbidden evidence checks, and review states.
- DAY 30
PoC Decision
Decide whether to keep AdaptOrch as an audit layer, wire it into PR preflight, or prepare for active gating after real paired evidence.
Start with shadow evidence.
Expand into PR preflight.
AdaptOrch can start as a developer shadow report, a team dashboard, or a scoped B2B audit PoC. Active correctness gating is promoted only after evidence gates pass.
PLAN
POSITIONING
MONTHLY
INCLUDES
ACTION
01STARTER
For local evaluation
$0/month
Self-hosted shadow reports
Basic candidate evidence log
No active selector mutation
Start Free →
02PROSOLO BUILDERS
For solo builders and AI agent developers
$39/month
Hosted shadow dashboard
CEK shadow report export
Patch evidence timeline
Basic constitution templates
Choose Pro →
03TEAM
For startup engineering teams
$149/month
Shared projects
Role-scoped review views
Runner-health dashboard
JSON evidence export
Choose Team →
04SHADOW AUDIT POC
For B2B teams adopting AI coding agents
Directcontract
2–4 week technical validation
1–3 repositories · 30–100 candidates
Evidence DAG + runner-health + CEK report
Final review presentation
Book PoC →
05ENTERPRISE PRIVATE RUNNER
For regulated or private-code environments
Directcontract
Private runner · on-prem or VPC
SSO / RBAC
Custom constitution
GitHub / GitLab workflow integration
Talk to Ontos →
† Active correctness gating is promoted only after real paired evidence gates pass. Shadow mode never mutates baseline selection in CEK S3.
Find risky AI patches before CI and reviewers pay the cost.
Shadow mode shows where AI-generated patches fail: apply, canonicalization, strict checks, provenance, runner health, candidate-caused errors, or constitutional risk.
Expected questions.
If a claim on this page overreaches evidence, tell us. We will correct it.
Q.01Is AdaptOrch another coding agent?
No. AdaptOrch is not a coding agent. It is a reliability kernel for AI-generated patches. It sits beside coding agents and reviews their output.
Q.02Does AdaptOrch prove that a patch is correct?
Not in CEK S3. CEK S3 is shadow-only and keeps correctness_claim=false. It provides evidence, risk labels, and audit trails. Semantic correctness requires stronger executable evidence such as validated paired repro tests.
Q.03Does CEK change which candidate AEG selects?
No. In S3, CEK is a shadow sidecar. It emits cek_shadow_report only when explicitly enabled and does not mutate baseline selected candidate behavior.
Q.04What does shadow-only mean?
Shadow-only means AdaptOrch observes, scores, and reports risk without changing production selection or merge behavior.
Q.05What happens when tests fail because the environment is broken?
AdaptOrch classifies environment-caused failures as runner-health events. They are not treated as candidate-caused hard blocks.
Q.06What is a candidate-caused failure?
A candidate-caused failure is a syntax, import, collection, runtime, or regression failure whose traceback or transition is attributed to the patch candidate.
Q.07What evidence is forbidden?
Official outcomes, hidden tests, gold patches, expected answers, and oracle labels are forbidden selector inputs.
Q.08Can AdaptOrch work with Claude Code, Codex, Cursor, or internal agents?
Yes. AdaptOrch is agent-agnostic. It receives patch candidates from any generator and reviews them through the same evidence pipeline.
Q.09What is the Evidence DAG?
The Evidence DAG is a replayable graph of candidate lineage, apply gates, canonical diffs, test receipts, runner-health events, CEK review labels, and final decision state.
Q.10Can we self-host it?
Yes. Self-host and private-runner deployments are part of the enterprise path. The control plane can be deployed beside your existing code and CI environment.
Q.11Is this safe for private repositories?
AdaptOrch is designed for private-code workflows. Reports use receipt digests and redacted payloads by default. Raw payload inclusion should remain disabled unless explicitly approved.
Q.12When does CEK become active?
Only after active-promotion gates pass: real Docker paired execution, mandatory source labels, expected-answer separation, deterministic reducer checks, causality fixtures, and UI separation of runner health from candidate correctness.
BOOK DEMO
Do not auto-merge AI patches. Audit them first.
Start with shadow mode. Review the evidence. Separate runner health from candidate failure. Promote active gating only when the evidence supports it.
- Works beside any coding agent
- Emits CEK shadow reports
- Keeps correctness_claim=false in S3
- Produces replayable evidence DAGs
- Supports B2B shadow-audit PoCs