Skip the Wrapper
An agent driving a browser through a CLI issues hundreds of sequential commands, so per-call overhead — not per-call work — sets the pace. This study measured five candidate optimizations for one such CLI (Vibium, OSS) across two real journeys under fixed semantics.
Four did not survive. One persistent-process technique lost outright. One was faster only because it stopped checking that elements were actionable — time bought by doing less. One vanished under measurement. The winner turned out not to be an optimization at all but a packaging defect: the command on your PATH is a 41-line Node script that spawns the real binary, so every invocation pays a Node runtime boot first.
under fixed semantics
a packaging change
(118 − 10 = 108)
journey (1.69×)
all verified
patch, 0 mismatches
I.Introduction
Motivation. Agent-driven test automation inverts the usual cost model. A human runs one command and waits; an agent runs a hundred, and the fixed cost of each one dominates. That makes per-invocation overhead a first-class performance property of a CLI — and it is the property nobody profiles, because on a single interactive command it is invisible. The question here is narrow and practical: for one widely used browser-automation CLI, where does the time actually go, and which of the obvious fixes survives contact with a real journey?
Problem Definition. The trap is that almost any change makes a benchmark faster if you let the benchmark change too. Skip an actionability check and clicks get quicker; hold a browser open and startup leaves the total. Neither is a faster tool. So every arm here holds semantics fixed, reports one-time setup apart from the journey, equalizes readiness polling, and is compared against a noise floor measured from a control the change cannot have touched. Under those rules, most of the candidates stopped looking like wins.
II.Approach
A.Review — five candidates, one survivor
Five techniques, two AUTs, semantics fixed. Absolute milliseconds lead: the same underlying factor measured at two dispatch levels produced ratios differing by 1.85× on identical data, so a ratio without its level is not a result.
Only T1 is free. It changes nothing about what the tool does — same verbs, same arguments, same actionability-checked click and fill, byte-identical output — because it is not a change to the tool at all. It is a change to how the tool is installed.
B.Experiment — where the time goes
The wrapper is the whole finding. vibium on PATH is not Vibium: it is a Node script that resolves a platform package and execFileSyncs the real native binary. Every vibium <verb> is two process spawns, and the first is a Node runtime boot — paid before any browser work begins.
On a near-empty command the shim costs ~118ms against ~10ms calling the platform binary directly. The gap is constant whatever the command does, which is what a runtime boot looks like and not what work looks like.
That per-call figure does not multiply into a journey total. On the flat profile 6 × 106.19ms predicts 637ms against 405ms measured, and the 232ms shortfall is 10.3× the campaign's own 22.6ms single-difference floor. The tax is partly self-refunding: the shim arm arrives at each post-navigate command ~107ms later, and a later-arriving call is a cheaper call.
after_poll is the discriminator: a post-navigate position that pays the full tax, which rules out “post-navigate steps are simply different steps” and leaves the arrival-time refund as the explanation.C.Results
The refund shrinks with page weight — but not to zero
A falsifier was registered before running: on a page with nothing to hydrate, the refund must vanish. It shrank monotonically with page weight — 82.5ms on a heavy-JS site, 55.2ms on a lighter one, 19.2ms on a bare static page — the mechanism behaving exactly as claimed. But 19.2ms survived, bootstrap CI [+3.3, +22.6], excluding zero. Hydration explains roughly 77% of the effect and is the right account of why it varies; it is not the whole account of why it exists. The prediction failed, and that is the useful part.
| AUT | page weight | refund | of tax |
|---|---|---|---|
| automation-exercise | heavy JS | 82.5ms | 78% |
| saucedemo | lighter | 55.2ms | 45% |
| local static page | none | 19.2ms | 17% |
The residual is Chrome's BiDi mapper, not the page
Chrome's WebDriver BiDi is not implemented in the browser. It is a JavaScript bundle in its own renderer process, a target Chrome names BiDi-CDP Mapper. Probed causally over CDP — which does not queue behind BiDi's own command handling — the mapper's context costs 10.3ms at t=0 and 0.9ms at t=150ms after a navigation, while the page's context, as a control, stays flat. A fixed CPU loop on the page's own clock runs marginally faster right after load, so the page is not computing. The busy thread belongs to the automation layer.
The wrapper-skipper
The fix is not in the CLI. Resolve bin to the platform binary at install time — hard-link the real binary over the shim in postinstall.js, which is what esbuild already does — and the per-invocation boot disappears. It declines out loud rather than failing: on Windows, under Yarn, or on any link error it prints vibium: keeping the Node shim (<reason>) and leaves a working install alone.
+function maybeReplaceShimWithBinary(vibiumPath) { + if (os.platform() === 'win32') return 'skipped: win32'; + if (isYarn()) return 'skipped: yarn'; + const shim = path.join(__dirname, 'bin', 'cli.js'); + const tmp = `${shim}.${process.pid}.tmp`; + try { fs.linkSync(vibiumPath, tmp); + fs.renameSync(tmp, shim); // atomic swap, keeps the shim on failure + return 'linked'; } catch { /* clean up tmp */ } +}
Guarded off on
win32 · yarn · binary not a file · any link/rename error; on every skip it prints vibium: keeping the Node shim (<reason>) rather than failing the install.13 runs, 710 reps, 0 mismatches, saving 107.4–111.9ms per call at 40×–60× the measured noise floor. Every sweep was also run in reverse, so a direction effect could not hide in the result.
| run | n | saving | ratio | floor |
|---|---|---|---|---|
n15_fwd | 15 | 109.4ms | 11.63× | 2.5ms |
n50_fwd | 50 | 108.5ms | 11.61× | 2.2ms |
n100_fwd | 100 | 108.0ms | 11.44× | 2.7ms |
n15_rev | 15 | 107.6ms | 11.33× | 1.8ms |
n50_rev | 50 | 109.3ms | 11.73× | 2.7ms |
n100_rev | 100 | 108.3ms | 11.59× | 2.4ms |
Re-measured fresh with the reproduction repo's own harness. Each sweep is also run in reverse, so a direction effect cannot hide in the result. Spread 1.7% on the saving, 3.5% on the ratio.
Filed upstream 2026-08-15, currently closed — merged as PR #432.
“Enhancement: vibium CLI pays ~108ms of Node startup on every invocation” — a maintainer asked publicly for 'skip the wrapper' to be filed as an enhancement.
How the numbers were kept honest
- semantics held fixed -- an arm that changes what the journey does is not a faster arm
- one-time setup reported apart from the journey total
- readiness polling equalized across arms and recorded apart
- absolute milliseconds lead; every ratio is stamped with its dispatch level
- a noise floor measured from a control the change cannot have touched
~11.6× is a true measurement of a near-empty command. The journey figures are 1.33× and 1.69×.
[RETRACTED] The persistent-process arm previously measured 55–180× on a trivial command; on a real journey it LOSES.
A published ~2.9ms direction effect re-measured at −0.2ms and is withdrawn, in place.
The floor is the same arm timed twice per rep — 2.2ms on the patch sweep, 22.6ms on the campaign. The saving is 40×–60× the sweep's.
The refund's falsifier was registered first. It fired at 19.2ms, and that located the residual.
III.Conclusion
One technique shipped, and the reason it shipped is the transferable part.
The cheapest performance win is often not in the code. It is in the packaging.
- Profile the invocation, not just the operation. Per-call fixed cost is invisible at n=1 and worth 25–41% of a real journey at n=6–7. Agent workloads live at n=100.
- Hold semantics fixed or you are not measuring speed. The JS-dispatch arm was faster because it stopped checking actionability. Buying time by doing less is not an optimization.
- Measure a noise floor from a control the change cannot have touched. Timing the same arm twice per rep gives what a quiet machine produces when nothing happened — here 2.2ms on the patch sweep and 22.6ms on the campaign. Without it, 108ms is a number, not a result.
- Lead with the absolute difference. ~11.6× is a true measurement of a near-empty command and a false claim about real work. The journey figures are 1.33× and 1.69×. A ratio inherits its denominator.
- Register the falsifier before you re-run. The refund mechanism was pushed to a prediction it could fail. It failed, at 19.2ms, and the failure located the residual.
- Report the technique that lost, and why. The persistent-process arm looked obviously right and came in at −740ms. Publishing that is what stops it being retried.
Filed upstream, at a maintainer's public invitation, as an enhancement in the register the project's own CONTRIBUTING.md asks for — symptom first, patch offered in one line at the end, and an explicit statement of who it does not help: MCP users get no benefit, because that is one long-lived process, and nobody notices 108ms on a single interactive command. It is scripted and agent-driven CLI use where it adds up.