Making a Browser-Automation CLI Faster

Skip the Wrapper

Five optimizations measured under fixed semantics. One ships, and it is a packaging change.
Lana Begunova · Daisy Lady Bug · Seattle, WA · PNSQC 2026 · October 12–14, 2026 · Portland, Oregon

Abstract

An agent driving a browser through a CLI issues hundreds of sequential commands, so per-call overhead — not per-call work — sets the pace. This study measured five candidate optimizations for one such CLI (Vibium, OSS) across two real journeys under fixed semantics.

Four did not survive. One persistent-process technique lost outright. One was faster only because it stopped checking that elements were actionable — time bought by doing less. One vanished under measurement. The winner turned out not to be an optimization at all but a packaging defect: the command on your PATH is a 41-line Node script that spawns the real binary, so every invocation pays a Node runtime boot first.

In five numbers: 5 techniques measured under fixed semantics · 1 shipped · ~108ms saved per CLI call (118 − 10 = 108) · 691ms off a real 7-command journey · 710 reps with 0 output mismatches. Filed upstream as issue #356.

I.Introduction

Motivation. Agent-driven test automation inverts the usual cost model. A human runs one command and waits; an agent runs a hundred, and the fixed cost of each one dominates. That makes per-invocation overhead a first-class performance property of a CLI — and it is the property nobody profiles, because on a single interactive command it is invisible. The question here is narrow and practical: for one widely used browser-automation CLI, where does the time actually go, and which of the obvious fixes survives contact with a real journey?

Problem Definition. The trap is that almost any change makes a benchmark faster if you let the benchmark change too. Skip an actionability check and clicks get quicker; hold a browser open and startup leaves the total. Neither is a faster tool. So every arm here holds semantics fixed, reports one-time setup apart from the journey, equalizes readiness polling, and is compared against a noise floor measured from a control the change cannot have touched. Under those rules, most of the candidates stopped looking like wins.

II.Approach · A. Review — the five candidates

Five techniques, two AUTs, semantics fixed. Absolute milliseconds lead: the same underlying factor measured at two dispatch levels produced ratios differing by 1.85× on identical data, so a ratio without its level is not a result.

technique · verdict · basis flat
6 cmd
navheavy
7 cmd
evidence
T1Call the native binary, not the vibium shim
SHIP IT · semantics unchanged
k=5 x n=15 x 2 profiles, 600/600 correct
+405ms
1.33×
+691ms
1.69×
cli-v2/docs/TECHNIQUES.md:23
T3Direct-attach BiDi socket
worth it in a narrow case · semantics JS dispatch only
k=5 x n=15 x 2 profiles, 225/225 correct each
+80ms
1.09×
+45ms
1.15×
cli-v2/docs/TECHNIQUES.md:24
T5Flat post-navigate wait
no effect — closed · semantics unchanged
640 interleaved reps, drift-checked
nonenonecli-v2/docs/TECHNIQUES.md:27
T4JS-dispatch click/fill
not an optimization · semantics degraded
faster only because it stops checking actionability — it buys time by doing less
+250ms+655mscli-v2/docs/TECHNIQUES.md:25
T2vibium pipe (one persistent process)
LOSES · semantics matched
5 AUT-runs, all negative
-740ms
0.57×
-212ms
0.62×
cli-v2/docs/TECHNIQUES.md:26

Only T1 is free. It changes nothing about what the tool does — same verbs, same arguments, same actionability-checked click and fill, byte-identical output — because it is not a change to the tool at all. It is a change to how the tool is installed.

#356

Filed 2026-08-15 — “Enhancement: vibium CLI pays ~108ms of Node startup on every invocation”, currently merged as #432. Who it does not help: MCP users get no benefit — vibium mcp is one long-lived process, so the cost is paid once at server start, never per tool call. Interactive human use gets no meaningful benefit either. Stated in the issue, because it is the fastest way for a maintainer to close it.
https://github.com/VibiumDev/vibium/issues/356 · repro github.com/lana-20/vibium-cli-startup-repro

II.Approach · B. Experiment — where the time goes

The wrapper is the whole finding. vibium on PATH is not Vibium: it is a Node script that resolves a platform package and execFileSyncs the real native binary. Every vibium <verb> is two process spawns, and the first is a Node runtime boot — paid before any browser work begins.

On a near-empty command the shim costs ~118ms against ~10ms calling the platform binary directly. The gap is constant whatever the command does, which is what a runtime boot looks like and not what work looks like.

That per-call figure does not multiply into a journey total. On the flat profile 6 × 106.19ms predicts 637ms against 405ms measured, and the 232ms shortfall is 10.3× the campaign's own 22.6ms single-difference floor. The tax is partly self-refunding: the shim arm arrives at each post-navigate command ~107ms later, and a later-arriving call is a cheaper call.

Stated before running: if the delta is flat across groups, the refund story is wrong. It did not fire. after_poll is the discriminator — a post-navigate position that pays the FULL tax, which rules out 'post-navigate steps are simply different steps'.

By position (board, Figure 2): clean 106.2ms / 122.6ms (full tax) · after_poll — / 123.2ms (full tax — the poll already spent the decay) · after_nav 23.7ms / 67.3ms (45–78% refunded).

II.Approach · C. Results

The refund shrinks with page weight — but not to zero

A falsifier registered before running required the refund to vanish on a page with nothing to hydrate. It shrank monotonically, then stopped at 19.2ms — excluding zero. Hydration explains ~77%, not all.

AUTpage weightrefundof tax
automation-exerciseheavy JS82.5ms78%
saucedemolighter55.2ms45%
local static pagenone19.2ms17%

The residual is Chrome's BiDi mapper, not the page

Chrome's WebDriver BiDi is a JS bundle in its own renderer, a target named BiDi-CDP Mapper. Probed over CDP, its context decays 9.4ms (95% CI [8.2, 13.8]) in the 150ms after a navigation; the page-context control -0.1ms. cli-v2/docs/TECHNIQUES.md:160-176

13 runs, 710 reps, 0 mismatches, saving 107.4–111.9ms per call at 40×–60× the measured noise floor. Every sweep was also run in reverse, so a direction effect could not hide in the result.

runnsavingratiofloor
n15_fwd15109.4ms11.63×2.5ms
n50_fwd50108.5ms11.61×2.2ms
n100_fwd100108.0ms11.44×2.7ms

Spread 1.7% on the saving, 3.5% on the ratio. The 3 reverse-order runs land at 107.6–109.3ms, so there is no direction effect to explain. repro:README.md:163-172

The wrapper-skipper — packaging enhancement, not a bug — output is byte-identical either way

The fix is not in the CLI. Resolve bin to the platform binary at install time — hard-link the real binary over the shim in postinstall.js, which is what esbuild already does — and the per-invocation boot disappears. It declines out loud rather than failing: on Windows, under Yarn, or on any link error it prints vibium: keeping the Node shim (<reason>) and leaves a working install alone.

— packages/vibium/postinstall.js — abridged; filed patch at repro:patch/postinstall.diff
+function maybeReplaceShimWithBinary(vibiumPath) {
+  if (os.platform() === 'win32') return 'skipped: win32';
+  if (isYarn())                 return 'skipped: yarn';
+  const shim = path.join(__dirname, 'bin', 'cli.js');
+  const tmp  = `${shim}.${process.pid}.tmp`;
+  try   { fs.linkSync(vibiumPath, tmp);
+          fs.renameSync(tmp, shim);   // atomic swap, keeps the shim on failure
+          return 'linked'; } catch { /* clean up tmp */ }
+}

Link then rename, so a failure never leaves a half-written entry point. Every skip path keeps the working shim. Guarded off on win32 · yarn · binary not a file · any link/rename error; on every skip it prints vibium: keeping the Node shim (<reason>) rather than failing the install.

How the numbers were kept honest

Accounting rules, applied to every arm

cli-v2/docs/TECHNIQUES.md:8-18 — AUTs automation-exercise.daisyladybug.com (6 cmd) and saucedemo.com (7 cmd), Vibium 26.5.31.

A ratio depends on its denominator; lead with the absolute difference

~11.6× is a true measurement of a near-empty command. The journey figures are 1.33× and 1.69×. repro:README.md:21-42

A figure true of a microbenchmark can be false of the system

[RETRACTED] The persistent-process arm previously measured 55–180× on a trivial command; on a real journey it LOSES. cli-v2/docs/TECHNIQUES.md:26

Publish the claims that failed to replicate, in place

A published ~2.9ms direction effect re-measured at −0.2ms and is withdrawn, in place. repro:README.md:179-186

Measure run-to-run noise from a control the change cannot have touched

The floor is the same arm timed twice per rep — 2.2ms on the patch sweep, 22.6ms on the campaign. The saving is 40×–60× the sweep's. repro:README.md:127-131

Register the prediction and its falsifier before re-running

The refund's falsifier was registered first. It fired at 19.2ms, and that located the residual. cli-v2/docs/TECHNIQUES.md:90-112

III.Conclusion

The cheapest performance win is often not in the code. It is in the packaging.

  1. Profile the invocation, not just the operation.
  2. Hold semantics fixed or you are not measuring speed.
  3. Measure a noise floor from a control the change cannot have touched.
  4. Lead with the absolute difference.
  5. Register the falsifier before you re-run.
  6. Report the technique that lost, and why.

Filed upstream, at a maintainer's public invitation, as an enhancement in the register the project's own CONTRIBUTING.md asks for — symptom first, patch offered in one line at the end, and an explicit statement of who it does not help: MCP users get no benefit, because that is one long-lived process, and nobody notices 108ms on a single interactive command. It is scripted and agent-driven CLI use where it adds up.