A correction is available. Read revision 2 for the current interpretation. This historical revision and its data files remain preserved.
IBARA LAB
Native interaction benchmark, round 4
60 attempts recorded. Diagnostic run. Measurement and publication dates are distinct.
Updated
Diagnostic run. Candidate: ibara 0.2.1-1 on Omarchy (released). Identity complete: true.
Measured to . Sealed 2026-10-10T22:56:33.653481+00:00. Revision 1.
Latest revision and correction history · All campaigns · RSS feed
Results and coverage#
60 attempts recorded. The denominator includes every attempted outcome. Unexecuted attempts are listed separately.
| Planned | Attempted | Passed | Failed | Unknown | Blocked | Safe refusal | Not run |
|---|---|---|---|---|---|---|---|
| 60 | 60 | 0 | 10 | 50 | 0 | 0 | 0 |
Qualification and execution modes
0 of 0 frozen cells observed. Execution modes have separate populations and are never pooled into one performance score.
- An observed attempt is outside the frozen matrix.
- Diagnostic evidence is not an accepted comparison baseline.
- Exactly one frozen campaign manifest is required.
- The frozen matrix has missing, duplicate or unplanned attempts; valid failures remain counted.
- The planned machine, workflow, mode and seed matrix is unavailable.
| Mode | Planned | Attempted | Passed | Failed | Unknown | Blocked | Safe refusal | Not run |
|---|---|---|---|---|---|---|---|---|
| scripted | 60 | 60 | 0 | 10 | 50 | 0 | 0 | 0 |
Scenario, model and environment#
Each dimension is a separate partition of the campaign. Do not sum overlapping dimensions together.
| Dimension | Scenario | Model | Environment | benchmark_mode | fixture_seed | Planned | Attempted | Passed | Failed | Unknown | Blocked | Safe refusal | Not run |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| cohort | Frame and shadow controls | Deterministic native recipe | Omarchy representative desktop | scripted | 0 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 |
| cohort | Frame and shadow controls | Deterministic native recipe | Omarchy representative desktop | scripted | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 |
| cohort | Frame and shadow controls | Deterministic native recipe | Omarchy representative desktop | scripted | 2 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 |
| cohort | Frame and shadow controls | Deterministic native recipe | Omarchy representative desktop | scripted | 3 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 |
| cohort | Frame and shadow controls | Deterministic native recipe | Omarchy representative desktop | scripted | 4 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 |
| cohort | Frame and shadow controls | Deterministic native recipe | Omarchy representative desktop | scripted | 5 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 |
| cohort | Frame and shadow controls | Deterministic native recipe | Omarchy representative desktop | scripted | 6 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 |
| cohort | Frame and shadow controls | Deterministic native recipe | Omarchy representative desktop | scripted | 7 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 |
| cohort | Frame and shadow controls | Deterministic native recipe | Omarchy representative desktop | scripted | 8 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 |
| cohort | Frame and shadow controls | Deterministic native recipe | Omarchy representative desktop | scripted | 9 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 |
| cohort | Long Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Long Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 1 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Long Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 2 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Long Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 3 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Long Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 4 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Long Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 5 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Long Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 6 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Long Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 7 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Long Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 8 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Long Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 9 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Nested scrolling | Deterministic native recipe | Omarchy representative desktop | scripted | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Nested scrolling | Deterministic native recipe | Omarchy representative desktop | scripted | 1 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Nested scrolling | Deterministic native recipe | Omarchy representative desktop | scripted | 2 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Nested scrolling | Deterministic native recipe | Omarchy representative desktop | scripted | 3 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Nested scrolling | Deterministic native recipe | Omarchy representative desktop | scripted | 4 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Nested scrolling | Deterministic native recipe | Omarchy representative desktop | scripted | 5 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Nested scrolling | Deterministic native recipe | Omarchy representative desktop | scripted | 6 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Nested scrolling | Deterministic native recipe | Omarchy representative desktop | scripted | 7 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Nested scrolling | Deterministic native recipe | Omarchy representative desktop | scripted | 8 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Nested scrolling | Deterministic native recipe | Omarchy representative desktop | scripted | 9 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Repeated native observations | Deterministic native recipe | Omarchy representative desktop | scripted | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Repeated native observations | Deterministic native recipe | Omarchy representative desktop | scripted | 1 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Repeated native observations | Deterministic native recipe | Omarchy representative desktop | scripted | 2 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Repeated native observations | Deterministic native recipe | Omarchy representative desktop | scripted | 3 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Repeated native observations | Deterministic native recipe | Omarchy representative desktop | scripted | 4 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Repeated native observations | Deterministic native recipe | Omarchy representative desktop | scripted | 5 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Repeated native observations | Deterministic native recipe | Omarchy representative desktop | scripted | 6 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Repeated native observations | Deterministic native recipe | Omarchy representative desktop | scripted | 7 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Repeated native observations | Deterministic native recipe | Omarchy representative desktop | scripted | 8 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Repeated native observations | Deterministic native recipe | Omarchy representative desktop | scripted | 9 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Short Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Short Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 1 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Short Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 2 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Short Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 3 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Short Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 4 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Short Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 5 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Short Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 6 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Short Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 7 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Short Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 8 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Short Unicode save and reopen | Deterministic native recipe | Omarchy representative desktop | scripted | 9 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Synthetic browser form | Deterministic native recipe | Omarchy representative desktop | scripted | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Synthetic browser form | Deterministic native recipe | Omarchy representative desktop | scripted | 1 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Synthetic browser form | Deterministic native recipe | Omarchy representative desktop | scripted | 2 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Synthetic browser form | Deterministic native recipe | Omarchy representative desktop | scripted | 3 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Synthetic browser form | Deterministic native recipe | Omarchy representative desktop | scripted | 4 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Synthetic browser form | Deterministic native recipe | Omarchy representative desktop | scripted | 5 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Synthetic browser form | Deterministic native recipe | Omarchy representative desktop | scripted | 6 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Synthetic browser form | Deterministic native recipe | Omarchy representative desktop | scripted | 7 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Synthetic browser form | Deterministic native recipe | Omarchy representative desktop | scripted | 8 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| cohort | Synthetic browser form | Deterministic native recipe | Omarchy representative desktop | scripted | 9 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
Baseline comparison#
No matched baseline was supplied. This report makes no improvement claim.
Additional measurements#
| Measurement | Value | Unit | Statistic | Population | Samples | Numerator | Denominator |
|---|---|---|---|---|---|---|---|
| Verified task completion | 0.0 | fraction | ratio | scripted attempts | 60 | 0 | 60 |
| Attempt duration | 9978.017734081368 | ms | mean | scripted: Attempt summaries with duration_ms measured | 60 | unavailable | unavailable |
| Tool calls | 16.666666666666668 | count | mean | scripted: Attempt summaries with tool_calls measured | 60 | unavailable | unavailable |
| Model tokens | unavailable | count | mean | scripted: Attempt summaries with tokens measured | 0 | unavailable | unavailable |
| Measured model cost | unavailable | USD | mean | scripted: Attempt summaries with cost_usd measured | 0 | unavailable | unavailable |
Method and provenance#
Suite: Native interaction and exact-output workflows; version: v1.
Ten repetitions per workflow on one fixed desktop through the public MCP route. Each attempt retains native receipts, host-monotonic tool timings, independent saved-file or trusted synthetic-fixture witnesses and fresh cleanup observations. Failures and unknown effects are retained. Per-attempt metadata is sealed and imported in a batch; no continuous video. Single and checked-batch variants are reported separately where present.
Evidence type: warehouse_snapshot.
snapshot_sha256: 02fb0b4a182c6df7776edca7fd5e01bbd18dc81d814efba453108fa3c3f291cb
Failures and limitations#
- Scripted system benchmark; no fresh-agent productivity or model ranking claim.
- One Omarchy computer and a fixed bridge route; these results do not qualify Ubuntu performance.
- Timings include bridge transport, expectation settlement and returned observations; they are not pure target input latency.
- Frame and shadow controls are intentionally probed; unsupported outcomes remain in the denominator.
- Exact file bytes and trusted fixture events independently verify output; observation cases independently witness the owned native window, not correctness of every captured pixel.
- Continuous video, physical takeover and stream frame-age measurements are absent.
- All original receipts remain private; public output contains sanitized aggregate results.
- Long-text expectations ended before saving; absent saved-output evidence remains unknown.
- A disposable VM began provisioning on the collector host during this round; this round is descriptive and not a controlled product performance comparison.
Data and charts#
- Public report JSON
- Outcome counts CSV
- Measurements CSV
- Report schema
- Data dictionary and reuse notes
- Checksums
- Vector chart
- Chart alt text
When citing these measurements, credit ibara and link this dated report revision. Private trajectories and third-party application content are not distributed here. This report does not change stable product documentation or qualify an untested release.