IBARA LAB
Omarchy native benchmark confirmation
96 attempts recorded. Diagnostic run. Measurement and publication dates are distinct.
Updated
Diagnostic run. Candidate: ibara 0.2.1-1 on Omarchy (released). Identity complete: true.
Measured to . Sealed 2026-10-11T01:30:33.394019+00:00. Revision 1.
Latest revision and correction history · All campaigns · RSS feed
Results and coverage#
96 attempts recorded. The denominator includes every attempted outcome. Unexecuted attempts are listed separately.
| Planned | Attempted | Passed | Failed | Unknown | Blocked | Safe refusal | Not run |
|---|---|---|---|---|---|---|---|
| 96 | 96 | 84 | 12 | 0 | 0 | 0 | 0 |
Qualification and execution modes
0 of 0 frozen cells observed. Execution modes have separate populations and are never pooled into one performance score.
- An observed attempt is outside the frozen matrix.
- Diagnostic evidence is not an accepted comparison baseline.
- Exactly one frozen campaign manifest is required.
- The frozen matrix has missing, duplicate or unplanned attempts; valid failures remain counted.
- The planned machine, workflow, mode and seed matrix is unavailable.
| Mode | Planned | Attempted | Passed | Failed | Unknown | Blocked | Safe refusal | Not run |
|---|---|---|---|---|---|---|---|---|
| scripted | 96 | 96 | 84 | 12 | 0 | 0 | 0 | 0 |
Scenario, model and environment#
Each dimension is a separate partition of the campaign. Do not sum overlapping dimensions together.
| Dimension | Scenario | Model | Environment | benchmark_mode | fixture_seed | Planned | Attempted | Passed | Failed | Unknown | Blocked | Safe refusal | Not run |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| cohort | Frame and shadow controls | Deterministic native recipe | Omarchy representative desktop | scripted | 0 | 3 | 3 | 0 | 3 | 0 | 0 | 0 | 0 |
| cohort | Frame and shadow controls | Deterministic native recipe | Omarchy representative desktop | scripted | 1 | 3 | 3 | 0 | 3 | 0 | 0 | 0 | 0 |
| cohort | Frame and shadow controls | Deterministic native recipe | Omarchy representative desktop | scripted | 2 | 3 | 3 | 0 | 3 | 0 | 0 | 0 | 0 |
| cohort | Frame and shadow controls | Deterministic native recipe | Omarchy representative desktop | scripted | 3 | 3 | 3 | 0 | 3 | 0 | 0 | 0 | 0 |
| cohort | Long Unicode save, checked batches | Deterministic native recipe | Omarchy representative desktop | scripted | 0 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Long Unicode save, checked batches | Deterministic native recipe | Omarchy representative desktop | scripted | 1 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Long Unicode save, checked batches | Deterministic native recipe | Omarchy representative desktop | scripted | 2 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Long Unicode save, checked batches | Deterministic native recipe | Omarchy representative desktop | scripted | 3 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Long Unicode save, single calls | Deterministic native recipe | Omarchy representative desktop | scripted | 0 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Long Unicode save, single calls | Deterministic native recipe | Omarchy representative desktop | scripted | 1 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Long Unicode save, single calls | Deterministic native recipe | Omarchy representative desktop | scripted | 2 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Long Unicode save, single calls | Deterministic native recipe | Omarchy representative desktop | scripted | 3 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Nested scrolling | Deterministic native recipe | Omarchy representative desktop | scripted | 0 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Nested scrolling | Deterministic native recipe | Omarchy representative desktop | scripted | 1 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Nested scrolling | Deterministic native recipe | Omarchy representative desktop | scripted | 2 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Nested scrolling | Deterministic native recipe | Omarchy representative desktop | scripted | 3 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Repeated native observations | Deterministic native recipe | Omarchy representative desktop | scripted | 0 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Repeated native observations | Deterministic native recipe | Omarchy representative desktop | scripted | 1 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Repeated native observations | Deterministic native recipe | Omarchy representative desktop | scripted | 2 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Repeated native observations | Deterministic native recipe | Omarchy representative desktop | scripted | 3 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Short Unicode save, checked batches | Deterministic native recipe | Omarchy representative desktop | scripted | 0 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Short Unicode save, checked batches | Deterministic native recipe | Omarchy representative desktop | scripted | 1 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Short Unicode save, checked batches | Deterministic native recipe | Omarchy representative desktop | scripted | 2 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Short Unicode save, checked batches | Deterministic native recipe | Omarchy representative desktop | scripted | 3 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Short Unicode save, single calls | Deterministic native recipe | Omarchy representative desktop | scripted | 0 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Short Unicode save, single calls | Deterministic native recipe | Omarchy representative desktop | scripted | 1 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Short Unicode save, single calls | Deterministic native recipe | Omarchy representative desktop | scripted | 2 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Short Unicode save, single calls | Deterministic native recipe | Omarchy representative desktop | scripted | 3 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Synthetic browser form | Deterministic native recipe | Omarchy representative desktop | scripted | 0 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Synthetic browser form | Deterministic native recipe | Omarchy representative desktop | scripted | 1 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Synthetic browser form | Deterministic native recipe | Omarchy representative desktop | scripted | 2 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
| cohort | Synthetic browser form | Deterministic native recipe | Omarchy representative desktop | scripted | 3 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
Baseline comparison#
No matched baseline was supplied. This report makes no improvement claim.
Additional measurements#
| Measurement | Value | Unit | Statistic | Population | Samples | Numerator | Denominator |
|---|---|---|---|---|---|---|---|
| Verified task completion | 0.875 | fraction | ratio | scripted attempts | 96 | 84 | 96 |
| Attempt duration | 12223.349333145961 | ms | mean | scripted: Attempt summaries with duration_ms measured | 96 | unavailable | unavailable |
| Tool calls | 14.75 | count | mean | scripted: Attempt summaries with tool_calls measured | 96 | unavailable | unavailable |
| Model tokens | unavailable | count | mean | scripted: Attempt summaries with tokens measured | 0 | unavailable | unavailable |
| Measured model cost | unavailable | USD | mean | scripted: Attempt summaries with cost_usd measured | 0 | unavailable | unavailable |
Method and provenance#
Suite: Native interaction and exact-output workflows; version: native-diagnostic-confirmation-v1.
Three fixed-source confirmation rounds per environment each contain four repetitions of eight workflows;96 executed attempts per environment. Exploratory and changed calibration recipes are retained separately. Exact released target builds, source copies, native receipts, collector timings, independent file or trusted-fixture oracles and cleanup witnesses are retained. Failures remain in scope. The same frozen harness is used throughout each confirmation cohort. This is a scripted diagnostic, not a causal binary performance or agent comparison.
Evidence type: warehouse_snapshot.
snapshot_sha256: abda65b749aa4402e192ab87331f3d767585c22f54c94f142064ee15f039476a
Failures and limitations#
- Scripted system diagnostic; no fresh-agent productivity, model ranking or qualified comparison baseline claim.
- One fixed desktop and bridge route per environment; no cross-platform speed comparison.
- Timings include bridge transport, expectation settlement and returned observations; they are not pure target input latency.
- Frame and shadow targets are probed deliberately; unsupported outcomes remain in the denominator.
- Exact file bytes and trusted fixture events independently verify outputs; observation trials witness the owned native window, not correctness of every captured pixel.
- Continuous video, physical takeover and stream frame-age measurements are absent.
- Original receipts remain private; public output contains sanitized aggregates.
- Public workflow metrics are means and counts; full per-call timing and frame-cost measurements are retained privately.
- Long-text UI settlement uses a bounded prefix; complete independent saved-byte readback is the unchanged correctness oracle.
- Matched save routes use twelve checked native actions. Changing API usage is not a product binary improvement.
- Single-call routes precede batches within each pair; order is not randomized and caches are not reset. Active durations include recording and independent-oracle overhead.
- Confirmation counts include every selected workflow; exploratory and changed harness records have separate source scopes.
- Original exploratory reported-attribution summaries remain unchanged and separate. Confirmation explicitly declares independent attribution.
- Halla is a shared eight-logical-CPU collector. Unrelated project workers and load average near15 were observed during Ubuntu round22; later partial CPU logging captured above90% aggregate busy during round24. Per-attempt overlap is not fully measured. Timings are descriptive, not a controlled host-isolation experiment.
- Per-workflow confirmation p95 is withheld below20 samples; paired saving is computed only for same-round successful single/batch pairs, with full outcome coverage retained.
- Round03 executed two of60 planned workflows before collection interruption. Round04 long-text UI expectations stopped before saving. Round05 retained one refused filename hotkey as unknown.
- Round06 overlapped a Halla warehouse recovery import and is excluded from the clean stability cohort; rounds07,16,17 use the same frozen harness.
- Single-call and checked-batch route timings are separate descriptive measurements. No qualified comparison delta is published; full same-repeat timing analysis remains a separately scoped numerical artifact.
Data and charts#
- Public report JSON
- Outcome counts CSV
- Measurements CSV
- Report schema
- Data dictionary and reuse notes
- Checksums
- Vector chart
- Chart alt text
When citing these measurements, credit ibara and link this dated report revision. Private trajectories and third-party application content are not distributed here. This report does not change stable product documentation or qualify an untested release.