Inspect the execution record behind your agent’s work. See available verification results and what still needs review in one report.
Hosted: submit runs and inspect their records. Code checks need a separately configured execution environment. Model-provider usage is billed separately through your own key.
AI-generated illustration
Report layout example, not a customer run. Completion and code verification are separate facts.
Open the report example →Start with the questions you already ask in a PR review. A run report connects the recorded inputs, results and remaining review work.
Changed files and scope, when a configured validation path records them.
The commands and targets that actually ran, including before/after results where available.
Keep completed checks, skipped checks and inconclusive results separate.
Link a finding to its changed file, rule or execution record.
Use the recorded inputs and environment notes to prepare a repeat check.
Review the open items yourself or return the specific context to your agent.
src/table.py+18 −4tests/test_io.py+19 −0config/prod.yaml+4 −0 · outside stated scopeA configuration file changed outside the declared scope. The integration check has not run.
Ask whether the configuration change belongs in this patch, and decide which integration check is needed.
A real report covers the checks that ran, not untested behavior. This example did not execute customer code.
correctness_claim: falseAVAILABLE HOSTED PATH
Use MCP or the API to submit model-backed work and retrieve run records, outputs and available artifacts. Starter is a free hosted plan.
SEPARATE SETUP / SCOPED POC
For repository and command checks, agree the environment, repository access, test commands and evidence handling first. A hosted plan upgrade does not enable this automatically.
BYOK is a billing and credential arrangement, not a promise that code stays on your device. Read the data-flow summary before sending inputs.
Data, security and operating responsibilities →Works beside the tools you already use
Compatible workflow surfaces; logos are not endorsements.
A review record helps you ask the next question: was the scope right, did the check run, did a test change, or did the execution environment fail?
Separate code failures from install, network, and runner failures before CI and review time disappear.
See the execution trail →Read Pass, Needs review, or Block without understanding every line. Send the receipt back to your coding agent as fix context.
Open run history to review status, model attribution and available artifacts. Repository checks require a separate configured execution path.
A repository check depends on the selected environment and commands. Agree file access, network permissions and isolation before execution. Compare recorded results with CI; a pass in one environment is not a guarantee in another.
Temporary workspaces and retained reports are different data. Cleanup follows the runner configuration; execution records, logs and artifacts follow the documented retention policy.
Read the execution and evidence boundaries →Use the available execution evidence to decide what needs another look. These are review questions, not claims that every run detects every problem.
A before/after reproduction makes a fix easier to evaluate. If it is missing, name the other evidence and the behavior that still needs checking.
Compare recorded files and changes with the declared task scope. A configured validation path can make unrelated edits visible.
A removed or weakened test can change what a passing result means. Review test changes alongside the command outcome.
Review setup, network and test outcomes separately. Missing commands and timeouts can be classified, while ambiguous failures still need diagnosis.
These results concern their stated experiments. Read positive results, misses and methodological limits together; they are not a promise about customer repositories.
Improvement and regression counts are available in the evidence JSON.
Every square is one recorded synthetic task. Green marks a passed check; a red outline marks an outcome that regressed from A to C.
Synthetic ledger, 30 tasks: paper baseline A versus robust + verifier C. Eight outcomes improved and one regressed. Both synthesis and verification differ; this is not a verifier-only comparison.
Local execution is measured. Human time savings are not. Keep those findings separate from an assumption-based cost estimate.
This experiment times CommandVerifier construction, execution and classification using trusted local controls, with no model API calls. It does not verify actual AI-generated patches.
Loading local measurement data…
Local data loaded. All outcomes, including mismatches, are shown below.
Measurement data could not be loaded. No results are shown; inspect the downloadable JSON instead.
Five fixed synthetic controls, repeated locally. Repetitions are not independent developers or tasks. Matches are counts of expected classifications, NOT bug-detection accuracy. Scroll the table horizontally if needed; keyboard users can focus it and use arrow keys.
| Control | Expected class | Observed classes (counts) | Median (milliseconds) | p95 (milliseconds) | Matches / attempts (counts) |
|---|---|---|---|---|---|
| Passing assertionClassification mismatch | — | — | — | — | — |
| Failing assertionClassification mismatch | — | — | — | — | — |
| Missing executableClassification mismatch | — | — | — | — | — |
| Deliberate timeoutClassification mismatch | — | — | — | — | — |
| Missing Python dependencyClassification mismatch | — | — | — | — | — |
Classification mismatch: the missing-dependency control expected ENV_ERROR but was observed as FAIL. Mismatched attempts remain in every timing summary and in the downloadable data.
Known limitation: the missing Python dependency was misclassified as an ordinary test failure. The classifier does not distinguish all environment failures from candidate failures.
These are not hosted round-trip latency measurements, a security-isolation test, or actual AI patch verification. Shared hosted command verification is disabled by default. The command component neither copies projects nor applies patches.
Protocol fixed before execution: 20 repeats per control, shuffled with seed 20260905; a fresh verifier per attempt, no cache, one command, and a 0.35-second timeout.
The five fixed controls are passing and failing Python assertions about 2 + 2, an absent executable, a 2-second sleep cut off at 0.35 seconds, and an absent Python import. These trivial fixtures are not AI-generated patches or real project test suites; their millisecond timings are not real project-test latency.
Trusted fixtures use a temporary working directory, default resource limits and no inherited environment. A temporary directory is not a clean project copy or a security boundary.
Monotonic wall-clock time covers verifier construction, command execution and classification. Measurement-program startup, report serialization and network latency are excluded.
All attempts are retained, including slow and error attempts. Median and nearest-rank p95 are descriptive statistics only: no significance test, generalized accuracy estimate or causal savings claim.
The JSON records revision, dirty-tree state, source hashes and environment metadata. This is an internal pre-execution protocol, not external preregistration or release approval.
No human study or customer savings data exist here. Local command timings cannot tell us how much review time a developer saves.
Before claiming savings, measure paired active review time on the same tasks with manual and assisted workflows, wait time separately, independent correctness, and actual subscription, provider, infrastructure, setup and maintenance costs. Include independent developers and repositories, randomize workflow order, and retain unsuccessful or slower runs.
paired_confirmatory_ledger: 30 synthetic ledger tasks, one paired session. Baseline 86.7%, verified 100.0%, difference +13.3pp (95% CI +3.3 to +26.7). The aggregate paired verdict did not reach significance and the family gate did not pass: directional, not proven. This is separate from the A/C ledger comparison.
In this 20-puzzle ARC-AGI-2 run, the best single model scored 14/20 and the exact-answer vote scored 10/20. The vote cost more. This is one internal experiment, not a universal result.
The published aggregates include errors and truncation counts. GPT scored 16 tasks with 4 errors; Qwen scored 19 with 1 truncation. Those denominators differ from the best-model and vote arms.
Recorded model-call cost in USD, summed across the three member arms. Not a customer invoice or full TCO; additional charges are not established by this record.
three-way-20260826T142538Z
Public data contain aggregates, not per-task paired outcomes. They cannot independently establish how many answers were improved or regressed.
n = 20 · descriptive aggregate comparison, not a general performance claim.
Illustrative interactive example. No customer code or model call is executed by this button.
Choose an example scenario and replay the displayed steps. The demonstration uses fixed example data.
Candidate patch modifies astropy/table.py to repair issue
behavior.
The patch applies cleanly, but we cannot call it correct. The test evidence came back inconclusive, and the shadow check flagged repair risk with no reproducible run behind it.
All three plans below are hosted service plans. Start with records, then choose the run allowance and workspace-user capacity you need.
Bring your own key (BYOK) is required on every plan for model execution. Your subscription covers the listed service allowance; your provider bills model usage separately. An AdaptOrch key authenticates the service, not the model provider.
Optional ensemble fan-out calls 2–5 of your own models in a configured run. One admitted run uses one call from your plan quota; your provider bills the tokens for every member. Provider cost depends on the models, tokens and attempts. Workspace users are people, not ensemble model members. Agreement is not an accuracy guarantee.
Model outputs and AI-written explanations are AI-generated and require human review. They are not the same evidence as execution logs or results from checks that actually ran.
Annual amounts are the full yearly charge in USD, not a monthly bill. Pro: $421.20/year ($35.10/month equivalent). Team: $1,609.20/year ($134.10/month equivalent). Starter has no annual discount.
Hosted plans do not automatically apply or merge repository changes. Separately configured checks may apply a candidate in an agreed workspace. Human review and the recorded execution boundaries remain explicit.
The monthly allowance counts admitted runs, not status or artifact reads. It is used at admission rather than at SUCCEEDED. Internal attempts remain in that run; a separately admitted rerun counts again. Provider billing is independent.
Repository command checks need a separately configured execution environment. A Pro or Team upgrade does not turn hosted command verification on; setup and execution-environment costs are agreed separately.
Economics · ESTIMATE ONLY. Compare a monthly hosted subscription with self-hosting under your own labor and cost assumptions. No savings have been measured.
Loading current monthly plan prices…
Plan prices are unavailable. No estimate can be calculated; check the current catalog below.
Choose a plan and complete all valid cost assumptions. Enter 0 explicitly for a cost that does not apply.
Estimate updated from your assumptions. This is not measured savings.
max(0, subscription + extra_provider - self_host_infra) /
hourly * 60
subscription, extra_provider and self_host_infra are monthly USD amounts; hourly is your labor value in USD per hour. The result is minutes per month, floored at zero.
This is the labor-time reduction required each month to break even on these inputs, NOT measured saved time or full TCO. We do not recommend plans from hours or PR counts.
Uses current monthly USD prices, not annual promotions. Taxes and BYOK costs shared by both workflows are excluded. Setup and ongoing maintenance are not fully included; actual total cost may differ.
A passing check covers only the conditions it executed. It does not establish overall correctness, safety, or behavior under untested conditions.
A run receipt reports the evidence that was recorded. A run without executed verification is not a test pass. The local experiment above measures command checks; it does not establish hosted patch execution or developer time savings.
We publish the negative outcomes too. In the synthetic ledger A/C comparison, eight task outcomes improved and one regressed. This compares both synthesis and verification settings, not the verifier alone.
If anything on this page promises more than we can show you, tell us. We will fix the page.
Submit model-backed runs through MCP or the API and inspect their records, outputs and available artifacts. Shared hosted command verification is disabled by default; repository checks need a separate execution environment.
No general correctness proof is provided. Check which verification actually ran and what evidence was recorded. A successful model run without executed tests is not a test pass.
The hosted workflow does not automatically apply or merge changes into your repository. A separately configured validation environment may apply a candidate in its agreed workspace for checking. You retain the review decision.
The current table reports local command-check timings on five fixed controls. Historical ledger timings measure a different operation. Neither establishes latency on your repository, hosted end-to-end latency, or saved developer time.
Missing executables and timeouts were distinguished in the local controls. A missing Python dependency was classified as an ordinary test failure. Failure labels need context; they are not a guarantee that every environment problem is diagnosed.
One admitted run uses one allowance unit. Status, usage and artifact reads do not create a new run. Completion is not the counting trigger, and a separately admitted rerun counts again. Provider token charges are separate.
No. A model can finish without running a code check. Inspect the recorded commands, outcomes and skipped or untested areas. An AI-written explanation does not replace those records.
Use the documented MCP or HTTP connection path for your client. Authentication, a model run, and a repository check are separate steps; each needs its own confirmation.
A report presents the execution record and whatever verification evidence is available. Changed files, before/after tests and rerun details depend on the configured path; missing checks must stay explicit.
The public Starter connects to hosted AdaptOrch. Running a client locally does not make the service self-hosted. Contact us about a separate private-runner deployment for your network.
Hosted requests pass through AdaptOrch and the AI provider you configure. Review the data-transfer and retention policies before submitting private code. A private runner is a separate deployment, not a Starter default.
Automatic merging is not part of the default hosted offer. Any active gating must be separately configured and evaluated against the customer's own environment and review policy.
Tell us what is hard to verify. We will first check the execution constraints, then agree scope, deliverables and price before any repository work.