{"campaign_id":"2026-10-08-medium-model-pair","candidate":{"build_sha256":"1a2c7db0be83a8a8cd02f837ecfc8a0cdde92ec29425262d6c30f35a07958873","channel":"experimental","identity_complete":false,"label":"Private Ibara v14"},"comparison":null,"correction":null,"groups":[{"benchmark_mode":"unclassified","counts":{"attempted":1,"blocked":0,"failed":0,"not_run":0,"passed":1,"planned":1,"safe_refusal":0,"unknown":0},"dimension":"cohort","environment":"Dell Linux desktop, Chromium and native apps","fixture_seed":17,"model":"GPT-6.1 Sol, medium reasoning","scenario":"Browser form and validation"},{"benchmark_mode":"unclassified","counts":{"attempted":1,"blocked":0,"failed":1,"not_run":0,"passed":0,"planned":1,"safe_refusal":0,"unknown":0},"dimension":"cohort","environment":"AK1 Linux desktop, Chromium and native apps","fixture_seed":17,"model":"Sonnet 5.5, medium effort","scenario":"Browser form and validation"},{"benchmark_mode":"unclassified","counts":{"attempted":1,"blocked":0,"failed":0,"not_run":0,"passed":1,"planned":1,"safe_refusal":0,"unknown":0},"dimension":"cohort","environment":"Dell Linux desktop, Chromium and native apps","fixture_seed":17,"model":"GPT-6.1 Sol, medium reasoning","scenario":"Download, edit and upload"},{"benchmark_mode":"unclassified","counts":{"attempted":1,"blocked":0,"failed":0,"not_run":0,"passed":1,"planned":1,"safe_refusal":0,"unknown":0},"dimension":"cohort","environment":"AK1 Linux desktop, Chromium and native apps","fixture_seed":17,"model":"Sonnet 5.5, medium effort","scenario":"Download, edit and upload"},{"benchmark_mode":"unclassified","counts":{"attempted":1,"blocked":0,"failed":0,"not_run":0,"passed":1,"planned":1,"safe_refusal":0,"unknown":0},"dimension":"cohort","environment":"Dell Linux desktop, Chromium and native apps","fixture_seed":17,"model":"GPT-6.1 Sol, medium reasoning","scenario":"Editor save, reopen and delivery"},{"benchmark_mode":"unclassified","counts":{"attempted":1,"blocked":0,"failed":0,"not_run":0,"passed":1,"planned":1,"safe_refusal":0,"unknown":0},"dimension":"cohort","environment":"AK1 Linux desktop, Chromium and native apps","fixture_seed":17,"model":"Sonnet 5.5, medium effort","scenario":"Editor save, reopen and delivery"},{"benchmark_mode":"unclassified","counts":{"attempted":1,"blocked":0,"failed":0,"not_run":0,"passed":1,"planned":1,"safe_refusal":0,"unknown":0},"dimension":"cohort","environment":"Dell Linux desktop, Chromium and native apps","fixture_seed":17,"model":"GPT-6.1 Sol, medium reasoning","scenario":"File manager move and rename"},{"benchmark_mode":"unclassified","counts":{"attempted":1,"blocked":0,"failed":0,"not_run":0,"passed":1,"planned":1,"safe_refusal":0,"unknown":0},"dimension":"cohort","environment":"AK1 Linux desktop, Chromium and native apps","fixture_seed":17,"model":"Sonnet 5.5, medium effort","scenario":"File manager move and rename"},{"benchmark_mode":"unclassified","counts":{"attempted":1,"blocked":0,"failed":0,"not_run":0,"passed":1,"planned":1,"safe_refusal":0,"unknown":0},"dimension":"cohort","environment":"Dell Linux desktop, Chromium and native apps","fixture_seed":17,"model":"GPT-6.1 Sol, medium reasoning","scenario":"Native file picker and upload"},{"benchmark_mode":"unclassified","counts":{"attempted":1,"blocked":0,"failed":0,"not_run":0,"passed":1,"planned":1,"safe_refusal":0,"unknown":0},"dimension":"cohort","environment":"AK1 Linux desktop, Chromium and native apps","fixture_seed":17,"model":"Sonnet 5.5, medium effort","scenario":"Native file picker and upload"},{"benchmark_mode":"unclassified","counts":{"attempted":1,"blocked":0,"failed":0,"not_run":0,"passed":1,"planned":1,"safe_refusal":0,"unknown":0},"dimension":"cohort","environment":"Dell Linux desktop, Chromium and native apps","fixture_seed":17,"model":"GPT-6.1 Sol, medium reasoning","scenario":"Search and nested scrolling"},{"benchmark_mode":"unclassified","counts":{"attempted":1,"blocked":0,"failed":0,"not_run":0,"passed":1,"planned":1,"safe_refusal":0,"unknown":0},"dimension":"cohort","environment":"AK1 Linux desktop, Chromium and native apps","fixture_seed":17,"model":"Sonnet 5.5, medium effort","scenario":"Search and nested scrolling"}],"kind":"benchmark","limitations":["Six single attempts per actor provide a practical snapshot, not a statistical reliability estimate or an external benchmark score.","Actors ran on different machines and harnesses so latency differences cannot be attributed to model or reasoning effort alone.","One fresh context is shared across six tasks in each lane. The task order, goal bytes, fixture seed and budgets are fixed.","Sol model/effort are bound to its accepted delegation configuration; separate child executable identity, tokens and cost are unavailable. Sonnet model identity and usage are taken from the actual Claude Code output when supplied.","Elapsed time includes agent deliberation, orchestration and tool waits; it is not isolated driver latency. Sonnet tool timings use timestamped harness events; Sol uses its tool-call wrapper timestamps.","No video is recorded. Original structured responses are retained; screenshot retention differs by actor capture and is reported in the evidence.","The earlier large matrix is outside this deliberately short run. A failed workflow alone does not establish a native input-driver defect.","Sol capture omitted eight image-observation response envelopes (five in form, three in search); arguments and timestamps remain, and those observations are recorded as unknown capture gaps. Independent application events and file bytes still verify outcomes. Sol screenshots were inspected by the actor but were not retained; Sonnet screenshot responses were retained privately.","Sonnet usage output reports an estimated USD 2.0003006 at list prices; this is not an invoice. Sol token usage and cost are unavailable, so no cost ranking is reported.","The reporting system retains its full-matrix qualification rules and marks this small shared-context profile diagnostic/unclassified. No statistical confidence interval or matched improvement claim is made."],"measured_at":{"end":"2026-10-09T02:35:20.541265+00:00","start":"2026-10-09T02:27:26.184784+00:00"},"methodology":"Two agents run concurrently on separate physical computers: Sonnet 5.5 medium in Claude Code on AK1, and GPT-6.1 Sol medium in Codex on Dell. Each starts with a fresh context shared across the same six tasks in fixed order, seed 17, one attempt per task. Each lane has a 15-minute execution cap and a 120-second per-task work target plus cleanup. Native Ibara input and read-only browser observations are allowed. Independent application events, file bytes and native receipts establish outcomes; failures and cleanup results remain visible. Both machines have the same observed Ibara runtime and Chromium build. This is descriptive workflow evidence, not an isolated model or hardware ranking. Observed outcomes: GPT-6.1 Sol verified 6 of 6 workflows in 429.816 seconds from first begin to last finish, with 95 MCP calls; Sonnet 5.5 verified 5 of 6 in 518.637422 seconds, with 92 MCP calls plus one artifact delivery fetch. Sonnet failed the form/validation task. Both completed with independently checked zero task windows and zero leases.","metrics":[],"provenance":{"snapshot_sha256":"8490726dffe8cddeec7f6423d7bc00d39926a31721260204bb6909a9c0de451c","source_sha256":null,"source_url":null,"type":"warehouse_snapshot"},"qualification":{"classification":"diagnostic","completed_cells":0,"eligible_for_comparison":false,"mode_counts":[{"counts":{"attempted":12,"blocked":0,"failed":1,"not_run":0,"passed":11,"planned":12,"safe_refusal":0,"unknown":0},"mode":"unclassified"}],"planned_cells":0,"reasons":["An observed attempt is outside the frozen matrix.","Diagnostic evidence is not an accepted comparison baseline.","Exactly one frozen campaign manifest is required.","The frozen matrix has missing, duplicate or unplanned attempts; valid failures remain counted.","The planned machine, workflow, mode and seed matrix is unavailable."]},"revision":1,"schema_version":"1.0","sealed_at":"2026-10-09T02:36:41.294016+00:00","suite":{"label":"Six everyday browser, desktop and mixed workflows","version":"six-workflows-medium-pair-v1"},"title":"Six workflows with Sonnet 5.5 and GPT-6.1 Sol at medium effort","totals":{"attempted":12,"blocked":0,"failed":1,"not_run":0,"passed":11,"planned":12,"safe_refusal":0,"unknown":0}}