Simulang Robustness Bench

Grackle Provisions Co., week 2026-W36

Two applications with no API in common, one Simulang script, and the question that decides whether it survives contact: which element on this screen is the thing I mean?

A distributor reconciles carrier invoices against its ledger by hand every Monday. The workflow is nine steps. Writing it is an afternoon. Making it still work on the customer's screen, at their window size, on their workstation, is the job.

So this page ships the script and the bench that argues for it: five ways to find an element, twenty configurations of the same two windows, and two numbers per run that are not the same number.

Runs 20 Locators 5 Steps 9 Executions 100 Grounding calls, shipped path 1 per run

What runs where, before anything claims anything. Simulang drives real desktop applications through the OS accessibility APIs. It runs on a workstation, not in a browser tab, so the two windows below are a faithful model of the customer's applications rather than the applications themselves, and the script in the code panel is real Simulang against the v13 API, runnable on a machine where Kestrel and Ledgerwood exist.

Everything the bench reports is measured here, in your browser, off the real geometry and the real accessibility properties of those two windows. Nothing on this page is a stored number. The accessibility path is modelled by what it can read, which includes a tooltip; the vision paths by what a screen paints, which does not. Where a vision path has to choose between two matching elements it takes the nearer one, and its own recognition error is set to zero, so the vision numbers here are the optimistic bound rather than a fair fight.

The desktop

Kestrel bills here and Ledgerwood accrues there. Change the window and watch what each locator does about it.

The run Nine steps, resolved by the shipped locator

The bench

Twenty configurations of those two windows, five locators, nine steps each. Steps is how far a run got. Outcome is whether it reached the right number, which is a 240.00 short pay on KST-4472.

Results Not run yet

Correct Found, not proven Duplicate missed, and nothing errored Aborted
Every run, one row each

The finding

Two facts decide this week's reconciliation, and they live in two different channels. Neither channel carries both.

Only the tree has it That the 240.00 detention on KST-4472 is a re-bill of the one already invoiced on KST-4471-R is written in the line's tooltip, which surfaces as helpText. Nothing paints it, so no screenshot contains it.
Only the screen has it That the carrier's own rate confirmation authorises one occurrence is inside a scan. The image carries no accessible name, no value and no ref, so no amount of tree walking reaches it.

Run the bench and this line reports what came out of it.

The code

Real Simulang against the v13 API. The bench swaps locators.mts and touches nothing else.

Loading.

Ask the agent

The bench says the first draft is wrong. Saying why, and which change fixes it, is the part that takes an engineer. This one reads the run it is given, proposes named repairs, and then the harness re-runs all twenty and reports what actually happened.

Sonnet 5, capped. The bench, the resize and the run cost nothing.