FFFF browser.click/get_text always fail with ref_not_found right after a fresh browser.snapshot · Issue #121 · amirlehmam/wmux · GitHub
[go: up one dir, main page]
Skip to content

browser.click/get_text always fail with ref_not_found right after a fresh browser.snapshot #121

New issue
New issue
Closed
Closed
browser.click/get_text always fail with ref_not_found right after a fresh browser.snapshot#121

Description

@yatran911
yatran911
opened on Jul 27, 2026

Summary

browser.click and browser.get_text always fail with ref_not_found, even immediately after a fresh browser.snapshot returns a valid ref tree with a nonzero refCount. Confirmed this is not related to the CLI, PowerShell, connection reuse, or timing - reproduced by talking to the named pipe directly.

Repro steps (raw JSON-RPC over the named pipe, bypassing the CLI entirely)

Using a single persistent TCP connection to \\.\pipe\wmux (so it's not a "new connection per request" issue):

  1. Send {"method":"browser.snapshot","params":{"caller":"<browser-surface-id>"},"id":1,"token":"<pipe-token>"}
    -> returns {"tree":"@e1: RootWebArea ...", "refCount": 300} (or similar, tree/refCount vary by page).
  2. Immediately (same connection, no delay) send {"method":"browser.click","params":{"ref":"e2","caller":"<same-surface-id>"},"id":2,"token":"<pipe-token>"}
    -> returns {"error":{"code":-32000,"message":"ref_not_found"}}

Also tried via the CLI (wmux browser snapshot then wmux browser click e2) - same result.

What I ruled out

  • Not a connection-lifetime issue: sent snapshot+click over the SAME socket without closing it between requests - still ref_not_found.
  • Not a timing issue: sent click with zero artificial delay right after snapshot - still fails. Also tried inside one long-lived Node process reusing the connection.
  • Not a caller/surface mismatch: explicitly passed the same caller (browser surface id, matching WMUX_SURFACE_ID / the id returned by list-surfaces for the type: "browser" entry) to both calls.
  • Not a ref format issue: tried "e2", "2", 2, and "@e2" as the ref value. All bare formats fail with ref_not_found; only the @-prefixed form fails differently, with Invalid parameters (consistent with docs saying refs take no @ prefix).
  • browser.get_text with the same ref also fails with ref_not_found, so this isn't click-specific - it looks like every ref-consuming command can't resolve refs that browser.snapshot just generated.

Environment

  • wmux (Windows), CLI at resources/cli/wmux.js
  • open, snapshot, screenshot, eval all work correctly - only ref-based interaction commands (click, get_text, presumably type/fill/wait) are affected.

Expected

browser.click (and other ref-based commands) should successfully resolve a ref returned by the immediately preceding browser.snapshot on the same browser surface.

Workaround

browser.eval with direct DOM JS (document.querySelector(...).click(), manual value + dispatchEvent('input')) works reliably and is what I'm using in the meantime.

Activity

  1. amirlehmam
    closed this as completedin 7a3973eon Jul 28, 2026

  2. amirlehmam commented on Jul 28, 2026

    @amirlehmam
    amirlehmam
    on Jul 28, 2026
    Owner

    Fixed in 7a3973e, shipping in v0.37.0.

    Your investigation was right on every single count, and that's what made this findable. Not a connection lifetime issue, not timing, not a caller mismatch — and once all three are ruled out, what's left is that the map never had a key called e2 in the first place.

    The bug

    buildAccessibilityTree keys its ref map with the display form the snapshot tree prints:

    const ref = `@e${refCounter}`;
refMap.set(ref, { nodeId, backendNodeId });   // key is "@e2"

    resolveRef then looked it up verbatim — so only the exact string @e2 matched. The CLI (wmux browser click e2), the orchestrator reference and every doc example pass the bare e2. Nothing that went through a normal path could ever resolve, which is why it hit click, type, fill, get_text and wait uniformly rather than looking like a per-command bug.

    And @e2 was not a workaround, which you also found. That's PowerShell: a leading @ is the splatting operator, so the argument is mangled by the shell before wmux sees it. Our own orchestrator reference has a "Sharp edges" note telling people to drop the @ for exactly that reason — which, given the map was keyed with the @, meant the docs were confidently steering everyone into the one form that could never work.

    The fix

    Refs are normalized to the canonical key before lookup, so e2, @e2, 2, E2 and e002 all resolve to the same node. The bare form stays the documented one, because it's the one that survives every shell.

    ref_not_found also says something useful now. It used to be a bare token, which is what sent you hunting connection lifetimes; it now distinguishes the two real causes:

    ref_not_found: no snapshot yet for this browser — run browser.snapshot first
ref_not_found: e400 — the last snapshot of this browser has @e1..@e300
ref_not_found: "submit" is not a ref — expected e12 or @e12

    Tests pin both spellings against a map built by a real buildAccessibilityTree run, so the display form and the lookup form can't drift apart again.

    Still true after this

    The staleness caveat is unchanged and unrelated: against client-rendered SPAs, refs can genuinely go stale between the snapshot and the click because the DOM was re-rendered underneath. browser.eval with a DOM query stays the right fallback there — and the user still watches every action live in the panel, which is the point of driving the browser this way at all.

    Thanks for the report. The raw-pipe repro that bypassed the CLI entirely is what turned this from "someone's shell is weird" into a one-line root cause.

  3. amirlehmam
    added a commit that references this issue on Jul 28, 2026

    fix: browser refs, drop marker, in-app update, dead popup links (#123 #…

    7882751
  4. bchap1n
    added a commit that references this issue on Jul 28, 2026

    fix(browser): resolve the bare `eN` refs every caller actually sends

    7a3973e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions
      0