Research

Papers

What we learn building KoraCode: how a browser agent should work, and how we measure it.

1. Oct 2026

   In Review

   Same model, different harness: what makes a browser agent succeed on HealthAdminBench

   The KoraCode team

   Browser agents are usually compared by the model they run. The harness around the model, meaning what the agent sees after it acts, what stays in its context, and how its actions are carried out, is rarely varied or even reported. We hold the model fixed and change only the harness. We run KoraCode, an agent that acts by writing JavaScript against a live browser through the Chrome DevTools Protocol, on the full HealthAdminBench suite of 135 tasks: healthcare administration work in an electronic health record, payer portals, and a fax portal, graded by rubric checks on the final portal state and on the notes the agent writes. The model (Qwen3.5‑27B), the tasks, and each task's step limit are the same in every run. Tracing lost tasks shows failures that come from the harness, not the model. After an action the agent was not shown the resulting page, so about 40% of its turns did not move the task forward. Every earlier page stayed in its context, so input grew to 1.6 million tokens per task. Streamed tool calls appeared twice in the history the model read, and element references broke whenever a page re-rendered. With the page shown after each action, stale observations trimmed, duplicate calls removed, and elements found again by role and name, KoraCode v0.3.16 (public unreleased) raises the share of rubric checks passed from 71% to 97.8%, with 64% fewer input tokens, 45% fewer turns, and 64% lower cost per task. It passes 124 of the 135 tasks (92%) at $0.064 per task. With the same model, the benchmark's own agent, updated and run by us, passes 58 (43%) at $0.091 per task, and the Qwen agent in the original paper passes 18 (13.3%). Because the harness alone moves both accuracy and cost this far, evaluations of browser agents should report and control the harness alongside the model.

   Agent	Tasks passed	Checks passed	Cost per task
   Original paper, as reported	18/135 (13.3%)	not reported	not reported
   HAB's own agent, updated, run by us	58/135 (43%)	86.6%	$0.091
   KoraCode v0.3.16 (public unreleased)	124/135 (92%)	97.8%	$0.064

   The original paper's Qwen agent sees screenshots, with the task description and portal guidance. It passes 10 of 60 prior authorization tasks, 3 of 60 appeals, and 5 of 15 equipment orders (Table 3). The paper gives no overall check score or cost for it.