IBARA LAB
Computer and browser use: ibara 0.1.0-17
132/157 passed (84.1%). Historical measurement. Measurement and publication dates are distinct.
Updated
Historical measurement. Candidate: ibara 0.1.0-17 (historical). Identity complete: false.
Measured to . Sealed 2026-10-08T02:23:45.570967+00:00. Revision 1.
Latest revision and correction history · All campaigns · RSS feed
Results and coverage#
132/157 passed (84.1%). The denominator includes every attempted outcome. Unexecuted attempts are listed separately.
| Planned | Attempted | Passed | Failed | Unknown | Blocked | Safe refusal | Not run |
|---|---|---|---|---|---|---|---|
| 256 | 157 | 132 | 25 | 0 | 0 | 0 | 99 |
Scenario, model and environment#
Each dimension is a separate partition of the campaign. Do not sum overlapping dimensions together.
| Dimension | Scenario | Model | Environment | Planned | Attempted | Passed | Failed | Unknown | Blocked | Safe refusal | Not run |
|---|---|---|---|---|---|---|---|---|---|---|---|
| scenario | Write and save a note in the text editor | All tested models | All tested machines | 13 | 13 | 13 | 0 | 0 | 0 | 0 | 0 |
| scenario | Rename and move files in the file manager | All tested models | All tested machines | 13 | 13 | 3 | 10 | 0 | 0 | 0 | 0 |
| scenario | Run a command in a terminal and report its output | All tested models | All tested machines | 13 | 13 | 12 | 1 | 0 | 0 | 0 | 0 |
| scenario | Find a setting and change it (editor line numbers) | All tested models | All tested machines | 12 | 12 | 8 | 4 | 0 | 0 | 0 | 0 |
| scenario | Copy text from the terminal into the text editor | All tested models | All tested machines | 11 | 11 | 11 | 0 | 0 | 0 | 0 | 0 |
| scenario | Find a file on the computer and deliver it here | All tested models | All tested machines | 12 | 12 | 9 | 3 | 0 | 0 | 0 | 0 |
| scenario | Look up a fact on Wikipedia | All tested models | All tested machines | 14 | 14 | 13 | 1 | 0 | 0 | 0 | 0 |
| scenario | Fill in and submit a sign-up form | All tested models | All tested machines | 14 | 14 | 12 | 2 | 0 | 0 | 0 | 0 |
| scenario | Navigate a multi-page site and read a table | All tested models | All tested machines | 14 | 14 | 14 | 0 | 0 | 0 | 0 | 0 |
| scenario | Search, download a file and deliver it here | All tested models | All tested machines | 14 | 14 | 13 | 1 | 0 | 0 | 0 | 0 |
| scenario | Get past a cookie banner and a popup | All tested models | All tested machines | 14 | 14 | 14 | 0 | 0 | 0 | 0 | 0 |
| scenario | Type accented and non-Latin text into a page | All tested models | All tested machines | 13 | 13 | 10 | 3 | 0 | 0 | 0 | 0 |
| model | All scenarios | Claude Code · claude-opus-5-5 | All tested machines | 23 | 23 | 22 | 1 | 0 | 0 | 0 | 0 |
| model | All scenarios | Claude Code · claude-sonnet-5 | All tested machines | 23 | 23 | 18 | 5 | 0 | 0 | 0 | 0 |
| model | All scenarios | Codex · gpt-6-astra (reasoning low) | All tested machines | 23 | 23 | 22 | 1 | 0 | 0 | 0 | 0 |
| model | All scenarios | Codex · gpt-6-sol (reasoning low) | All tested machines | 23 | 23 | 21 | 2 | 0 | 0 | 0 | 0 |
| model | All scenarios | omp · grok-4.7 | All tested machines | 22 | 22 | 18 | 4 | 0 | 0 | 0 | 0 |
| model | All scenarios | opencode · opencode/big-pickle | All tested machines | 22 | 22 | 14 | 8 | 0 | 0 | 0 | 0 |
| model | All scenarios | Codex · deepseek-v4.1-flash (OpenRouter) | All tested machines | 21 | 21 | 17 | 4 | 0 | 0 | 0 | 0 |
| environment | All scenarios | All tested models | Mini PC A | 81 | 81 | 67 | 14 | 0 | 0 | 0 | 0 |
| environment | All scenarios | All tested models | Mini PC B | 76 | 76 | 65 | 11 | 0 | 0 | 0 | 0 |
| surface | Desktop | All tested models | All tested machines | 74 | 74 | 56 | 18 | 0 | 0 | 0 | 0 |
| surface | Browser | All tested models | All tested machines | 83 | 83 | 76 | 7 | 0 | 0 | 0 | 0 |
| scenario | Queued but not started | Not run | Not run | 99 | 0 | 0 | 0 | 0 | 0 | 0 | 99 |
| model | Queued but not started | Not run | Not run | 99 | 0 | 0 | 0 | 0 | 0 | 0 | 99 |
| environment | Queued but not started | Not run | Not run | 99 | 0 | 0 | 0 | 0 | 0 | 0 | 99 |
| surface | Queued but not started | Not run | Not run | 99 | 0 | 0 | 0 | 0 | 0 | 0 | 99 |
Baseline comparison#
No matched baseline was supplied. This report makes no improvement claim.
Additional measurements#
| Measurement | Value | Unit | Statistic | Population | Samples | Numerator | Denominator |
|---|---|---|---|---|---|---|---|
| Verified task completion | 0.8407643312101911 | fraction | ratio | All observed attempts | 157 | 132 | 157 |
| computer_act latency | 2500.0 | ms | p50 | computer_act calls in the historical campaign | 927 | unavailable | unavailable |
| computer_act latency | 8900.0 | ms | p95 | computer_act calls in the historical campaign | 927 | unavailable | unavailable |
| browser_act latency | 1440.0 | ms | p50 | browser_act calls in the historical campaign | 388 | unavailable | unavailable |
| browser_act latency | 7200.0 | ms | p95 | browser_act calls in the historical campaign | 388 | unavailable | unavailable |
| computer_observe latency | 360.0 | ms | p50 | computer_observe calls in the historical campaign | 686 | unavailable | unavailable |
| computer_observe latency | 3010.0 | ms | p95 | computer_observe calls in the historical campaign | 686 | unavailable | unavailable |
| computer_begin latency | 160.0 | ms | p50 | computer_begin calls in the historical campaign | 222 | unavailable | unavailable |
| computer_begin latency | 200.0 | ms | p95 | computer_begin calls in the historical campaign | 222 | unavailable | unavailable |
Method and provenance#
Suite: 12 desktop and browser tasks, seven agent setups; version: 2026-09-29.
12 tasks (6 desktop, 6 browser), each from a clean start with new random content per run; a run passes only if every check passes at its finish within the time limit; every run counts, failures included. Agents ran headless on the person's own computer with only the ibara MCP server, the ibara skill that ships with the package and the ibara note in their instructions file; the project's source and notes were hidden from them. Approvals were answered by a script standing in for the person, approving only what the task asked for; time spent waiting for an answer is not counted.
Evidence type: legacy_aggregate.
Source: https://ibara.app/docs/benchmarks.
source_sha256: 0ca6701a435ea0d906fbce75af461b21defa6c9fdf908998cc1b45e079150a58
Failures and limitations#
- These measurements were collected on September 29, 2026. This publication archives an existing benchmark; it is not a new test of the current release.
- Every one of the 157 started runs counts, including four benchmark setup failures. There were 25 failures: 13 attributed to ibara, eight to agents and four to benchmark setup. Eighteen runs reached their time limit.
- The run stopped with 99 queued attempts not started. Task and model sample sizes are uneven. Unstarted attempts are reported separately from the attempted denominator.
- The archived aggregate is the source of these tables. Raw agent trajectories are not included, and the original exact executable hash is not established by this public aggregate.
- Mini PC A: Intel Celeron J3455, 8 GB RAM, Omarchy 4.0.4, Hyprland 0.56.2, Google Chrome, Cua driver 0.29.1, wireless mouse receiver attached.
- Mini PC B: Intel Celeron J3455, 4 GB RAM, Omarchy 4.0.4, Hyprland 0.56.2, Chromium, Cua driver 0.28.2, no mouse attached.
- Harnesses: Claude Code 2.1.284, Codex 0.158.0, omp 18.4.3 and opencode 1.18.31. The two OpenAI model configurations used reasoning low; providers and model names appear in the model table.
- Tool latency was measured by the harness, including the network hop and excluding model thinking time. It is distinct from full task duration.
- Approvals were answered by the benchmark script within the task scope. These are not measurements of a human responding to approvals.
- No improvement percentage is claimed against the September 28 suite because isolation, sample sizes and machine state differed.
Data and charts#
- Public report JSON
- Outcome counts CSV
- Measurements CSV
- Report schema
- Data dictionary and reuse notes
- Checksums
- Vector chart
- Chart alt text
When citing these measurements, credit ibara and link this dated report revision. Private trajectories and third-party application content are not distributed here. This report does not change stable product documentation or qualify an untested release.