8000 bug(pty): uncaught Napi::Error from conpty.node aborts the main process — 12 crashes, one signature (0.10 → 0.41) · Issue #150 · amirlehmam/wmux · GitHub
[go: up one dir, main page]

Skip to content

bug(pty): uncaught Napi::Error from conpty.node aborts the main process — 12 crashes, one signature (0.10 → 0.41) #150

Closed

Description

@Ray0483

Environment

  • wmux: crashes observed on 0.10.0, 0.15.0, 0.17.0, 0.20.0, 0.25.0, 0.27.2, 0.29.1, 0.30.0, 0.38.0, 0.39.1, 0.41.0 (currently on 0.46.0)
  • Windows 11 Pro 26200 (10.0.26200.2.0.0.256.48), x64
  • Chromium/ANGLE in this build: 2.1.27949 git hash: bbf3d8a47552
  • Bundled node-pty: 1.1.0 (resources/app.asar.unpacked/node_modules/node-pty), ConPTY backend

Symptom

wmux disappears instantly — the entire window, every workspace, every pane, with no error dialog and nothing written to the sidebar. It reads like a process kill, not a UI fault. Roughly once every 2–4 days under normal use.

Windows records it as Application Error / BEX64:

Faulting application name: wmux.exe, version: 0.41.0.0, time stamp: 0x6a3b18dc
Faulting module name: wmux.exe
Exception code: 0xc0000409          (STATUS_STACK_BUFFER_OVERRUN)
Fault offset:   0x0000000003290be5
Additional parameter: 0x7           (FAST_FAIL_FATAL_APP_EXIT)

0xc0000409 here is not memory corruption — parameter 7 means the process deliberately called __fastfail(FAST_FAIL_FATAL_APP_EXIT), i.e. the CRT abort() path.

Evidence: 12 crashes, one signature

The Windows Event Log records 12 crashes between 2026-06-27 and 2026-08-04. I have minidumps from %LOCALAPPDATA%\CrashDumps for the 8 most recent, and every one is identical:

Crash time (local) wmux version Exception Param Fault offset Crashing thread
2026-07-19 15:40:37 0.25.0 0xc0000409 0x7 wmux.exe+0x3290be5 index 0 (main)
2026-07-22 14:04:18 0.27.2 0xc0000409 0x7 wmux.exe+0x3290be5 index 0 (main)
2026-07-22 14:05:48 0.29.1 0xc0000409 0x7 wmux.exe+0x3290be5 index 0 (main)
2026-07-25 11:10:17 0.30.0 0xc0000409 0x7 wmux.exe+0x3290be5 index 0 (main)
2026-07-29 19:41:32 0.30.0 0xc0000409 0x7 wmux.exe+0x3290be5 index 0 (main)
2026-07-30 10:20:32 0.38.0 0xc0000409 0x7 wmux.exe+0x3290be5 index 0 (main)
2026-08-02 12:44:53 0.39.1 0xc0000409 0x7 wmux.exe+0x3290be5 index 0 (main)
2026-08-04 11:52:55 0.41.0 0xc0000409 0x7 wmux.exe+0x3290be5 index 0 (main)

The four older Event Log entries (0.10.0, 0.15.0, 0.17.0, 0.20.0) carry the same exception code and parameter; the two oldest sit at a different offset (+0x45e3515), which is consistent with the same bug before an Electron bump.

Note the two crashes 90 seconds apart on 07-22: restarting wmux hit the same path almost immediately.

Root cause: an unhandled C++ Napi::Error thrown inside conpty.node

The faulting stack carries both the raise path and the abort path, which means the exception was never handled:

KERNELBASE!RaiseException+0x8a                  <- C++ throw
ntdll!RtlVirtualUnwind / ntdll!__C_specific_handler
KERNELBASE!UnhandledExceptionFilter+0x1f3       <- nobody caught it
wmux.exe+0x3294e5a
wmux.exe+0x32abfd2                              <- abort()
wmux.exe+0x3290be5                              <- __fastfail(7)  *** crash ***

The stack also contains 0xE06D7363 (MSVC C++ EH exception code, "msc") and 0x19930520 (C++ EH magic), so it is a C++ throw, not an SEH fault.

Decoding the EXCEPTION_RECORD gives a well-formed C++ throw record (NumberParameters=4, ExceptionInformation[0] == 0x19930520). Walking _ThrowInfo -> _CatchableTypeArray -> TypeDescriptor yields the thrown type:

.?AVError@Napi@@                            <- Napi::Error
.?AVObjectReference@Napi@@
.?AV?$Reference@VObject@Napi@@@Napi@@
.?AVexception@std@@

And critically, ExceptionInformation[3] (the throw's image base) equals the load address of
resources\app.asar.unpacked\node_modules\node-pty\prebuilds\win32-x64\conpty.node
in every dump — the _ThrowInfo lives in that module's .rdata, so the throw statement is compiled into conpty.node.

So: conpty.node throws Napi::Error in a context with no handler. The C++ exception escapes into the main thread's message loop, the CRT unhandled-exception path calls abort(), and the Electron browser process dies — taking every pane with it.

Because a Napi::Error thrown at a normal N-API method boundary is caught by node-addon-api's wrapper and converted into a JS throw, the fact that it reached UnhandledExceptionFilter means the throw happened outside such a boundary.

Where it likely comes from (inferred, not proven)

conpty.node contains these strings:

SetupExitCallback_resource
ThreadSafeFunction.BlockingCall() failed
ThreadSafeFunction.Release() failed

node-pty's ConPTY path registers a process-exit callback via a ThreadSafeFunction (windowsPtyAgent.js:89):

const connect = this._ptyNative.connect(this._pty, commandLine, cwd, env, this._useConptyDll,
                                        c => this._$onProcessExit(c));

A Napi::Error thrown from a TSFN/libuv callback runs on the main thread with no node-addon-api try/catch above it — which matches all observed facts (main thread, shallow ~24 KB stack, no handler). That would make the trigger a pane's shell process exiting, which fits the irregular every-few-days cadence.

To be explicit: this last section is inference from the module's string table and the crashing thread's identity, not something I proved from the dump. Everything in the previous section is proven.

For completeness, the JS call sites into the native module are unguarded (windowsPtyAgent.js:123/130/150/156), but wrapping them in try/catch would not fix this — the exception never becomes a JS exception.

Suggested next steps

  1. You can symbolize this quickly: the crash is at wmux.exe+0x3290be5 in the 0.41.0 build (time stamp 0x6a3b18dc), and conpty.node ships with a PDB path of D:\a\_work\1\s\build\Release\conpty.pdb. I'm happy to upload the minidumps (~6 MB each) if that would help — just say where.
  2. A cheap hardening regardless of the exact throw site: build/ship conpty.node with NAPI_DISABLE_CPP_EXCEPTIONS, so N-API failures surface as JS errors instead of C++ throws that can abort the process.
  3. Related but distinct, already fixed: bug(pty): MSVC assertion crash on close — remove_pty_baton(baton->id) #50 (remove_pty_baton MSVC assertion on close). This one is a different mechanism — a 0xE06D7363 C++ throw, not an assertion — and still reproduced on 0.41.0.

How to capture better data

The current dumps are minidumps, so the Napi::Error message is unrecoverable: Napi::Error::_message is lazily computed and still empty at throw time, and the text lives in the V8 heap behind a napi_ref. Setting

HKLM\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\wmux.exe
  DumpType = 2   (full dump)

on an affected machine will capture the actual error message on the next occurrence.

Activity

  1. amirlehmam commented on Aug 10, 2026

    @amirlehmam
    Owner

    This is the best bug report this project has received. Decoding _ThrowInfo -> _CatchableTypeArray -> TypeDescriptor to get .?AVError@Napi@@, and then matching ExceptionInformation[3] against conpty.node's load address across all eight dumps, is what turned "wmux randomly disappears" into something actionable. Thank you.

    Mitigation shipped in v0.47.0. Details, including what it does not prove.

    Your inference was right, and it's checkable from the source

    You flagged the last section as inference. It holds up. In node-pty's src/win/conpty.cc, every other throw Napi::Error::New(...) — PtyStartProcess, PtyConnect, PtyResize, PtyClear, PtyKill — sits inside a function registered as an N-API method, so node-addon-api's details::WrapCallback try/catch converts it into a JS throw. The one place that isn't wrapped is the ThreadSafeFunction callback:

    auto callback = [](Napi::Env env, Napi::Function cb, ExitEvent *exit_event) {
      cb.Call({Napi::Number::New(env, exit_event->exit_code)});
      delete exit_event;
    };

    That runs on the main thread from the TSFN dispatcher with no node-addon-api wrapper above it — exactly the "outside such a boundary" condition you derived from the fact that it reached UnhandledExceptionFilter at all. It's the only such site in the file.

    Why suggestion (2) wasn't available

    NAPI_DISABLE_CPP_EXCEPTIONS was the right instinct, but wmux can't get there: we ship node-pty's prebuilt binaries and never compile them (npmRebuild: false), and node-pty's own source uses throw Napi::Error::New(...) at ~10 sites, which doesn't compile with C++ exceptions disabled. So it would need to happen upstream, in node-pty.

    What shipped instead

    Napi::Function::Call throws Napi::Error when napi_call_function comes back napi_pending_exception — i.e. the C++ throw is reachable through a JS throw. The JS end of that call is WindowsPtyAgent._$onProcessExit, so making that function total removes the pending exception, and there is nothing left for cb.Call to convert.

    New file src/main/pty-crash-guard.ts wraps it at load time, before any PTY can exist (node-pty registers the callback inside pty.spawn, so installing later would leave already-spawned panes on the unguarded path).

    You correctly noted that wrapping the JS call sites at windowsPtyAgent.js:123/130/150/156 would not help. Right — those are calls into the native module. This wraps the callback the native module calls out to, which is the other direction and the one that matters.

    Worth noting how thin that function is under useConptyDll: true: it's one assignment. The fallback branch is the interesting one — wmux retries with useConptyDll: false when the bundled conpty.dll fails to load, and that branch touches this._outSocket, which is precisely the sort of thing that's null after a half-failed spawn.

    A second, unrelated crash found while in there

    lib/windowsTerminal.js re-throws socket errors it doesn't recognise:

    this._socket.on('error', err => {
      this._close();
      if (err.code) { if (~err.code.indexOf('errno 5') || ~err.code.indexOf('EIO')) return; }
      if (this.listeners('error').length < 2) { throw err; }
    });

    Terminal._forwardEvents() registers only data and exit, so that count is 0 for every PTY wmux spawns and the guard never holds. That throw lands in a libuv callback → unhandled exception in main → same blast radius, different signature (a JS crash, not 0xc0000409). wmux now claims the error event on each terminal. This is not your crash, but it was the same trapdoor.

    What this does not prove

    If the Napi::Error is coming from cb.Call / Napi::Number::New failing for a reason other than a JS-thrown exception — environment teardown during quit being the obvious candidate — this guard will not stop it, and the crash will keep the same signature. I can't distinguish those two from a minidump either, for the reason you identified: _message is lazily computed and empty at throw time.

    So: yes please to the dumps. Attaching them here works if GitHub takes the size, otherwise anywhere you like and I'll fetch them. Full dumps (DumpType = 2) from a machine that reproduces would be worth more than the existing eight, since those would carry the actual Napi::Error message and settle it — but send what you have either way, the fault offsets alone are useful against the 0.47.0 build.

    I'm leaving this issue open rather than closing it on a mitigation. If it recurs on 0.47.0, post the new fault offset here. A crash that survives this guard tells us something specific: that the throw never went through JS at all, which points upstream at node-pty and makes a much sharper issue to file there.

    Two related fixes rode along in the same release: #149 and #148.

  2. Ray0483 commented on Aug 10, 2026

    @Ray0483
    ContributorAuthor

    Thanks — and the conpty.cc walkthrough is the part I couldn't reach from the outside. Confirming that the TSFN callback is the only unwrapped site in the file is exactly the thing a minidump can't tell you.

    On the dumps: I can't send these particular eight

    They carry the process environment block, and this machine keeps live exchange API credentials in user-scope environment variables — so every dump has them in cleartext. The worst one has 11 of 11, including a live-trading key/secret/passphrase set and a Telegram bot token. That's my problem, not wmux's: user-scope variables are inherited by every process, so anything that dumps this machine's memory picks them up.

    I'd rather not hand-scrub minidumps either — rewriting byte ranges inside MemoryList risks producing something that reads fine but analyses wrong, which is worse than sending nothing.

    I'm moving those credentials out of user-scope env vars. Once that's done, dumps from this machine are clean and I'll send them without the caveat.

    Three things I can give you now instead

    1. The useConptyDll: false fallback is ruled out for these crashes. You flagged that branch as the interesting one because it touches this._outSocket. I checked the loaded-module list in all eight dumps:

    Dump conpty.node bundled conpty\conpty.dll winpty
    all 8 loaded loaded absent

    The bundled conpty.dll was resolved and loaded in every single crash, and winpty.dll / winpty-agent.exe never appear. So all eight happened on the useConptyDll: true path — the thin one-assignment version of _$onProcessExit — not the fallback. Whatever throws, it isn't a half-failed spawn leaving _outSocket null on the retry path.

    2. A cadence baseline, so "did it stop?" is measurable. Crash intervals over the eight dumps:

    07-19 15:40  →  07-22 14:04   2d 22h
    07-22 14:04  →  07-22 14:05   90 seconds
    07-22 14:05  →  07-25 11:10   2d 21h
    07-25 11:10  →  07-29 19:41   4d 08h    <- longest observed gap
    07-29 19:41  →  07-30 10:20   14h
    07-30 10:20  →  08-02 12:44   3d 02h
    08-02 12:44  →  08-04 11:52   1d 23h
    

    As of 2026-08-10 12:00 local, it has been 6d 0h with no crash — already longer than the longest gap in that window. Worth flagging honestly: that clean streak is on 0.43.0 → 0.46.0, i.e. entirely before your guard. I moved off 0.41.0 within hours of the last crash. So it's either normal variance in my usage, or something between 0.42 and 0.46 already reduced the trigger rate. I don't think 6 days is enough to call it either way — I'm reporting it so it doesn't get mistaken later for evidence that 0.47.0 fixed it.

    3. I upgraded to 0.47.0 today and will keep watching. The clean-streak clock above therefore resets as a test of your guard from this point, not before it. If it recurs I'll post the new fault offset, the exception parameter, and whether conpty.dll was loaded, same format as above. Given the historical cadence, I'd treat ~3 weeks of silence as the point where the numbers start meaning something.

    On the second bug

    The windowsTerminal.js listeners('error').length < 2 guard never holding is a nasty one — the condition reads like a safety check but is dead in every wmux-spawned PTY. Glad it fell out of this.

    Happy to run anything specific on my end if it would help discriminate the two cases in your "what this does not prove" section — I have the tooling to parse these dumps and can answer targeted questions about them without publishing the files.

  3. amirlehmam commented on Aug 13, 2026

    @amirlehmam
    Owner

    Status update, since 0.50.0 is out today: the crash guard from 0.47.0 is unchanged and still installed at module load in pty-crash-guard.ts — before any pty.spawn, since the exit callback it wraps is registered inside spawn() and a later install would leave every already-spawned pane on the unguarded path. tests/unit/pty-crash-guard.test.ts pins both halves (the wrapped _$onProcessExit, and the claimed error listener that keeps node-pty's under-2-listeners re-throw from firing).

    The question this issue is still open on is yours to answer, and only time can: has it crashed since 0.47.0? Your baseline was roughly one every 2–4 days across a dozen releases, so a clean stretch on 0.47.0 or later is the only evidence that matters. If it has crashed again, the Event Log line alone (Faulting module name, exception code, and the Additional parameter) is enough to tell whether it is the same signature or a new one — no dump needed, and given what you found in those dumps, no dump wanted.

    On that: you were right to withhold them, and right about whose problem it is. A minidump carrying the process environment block is a credential leak in a file people hand to strangers, and "wmux crashed, here's a dump" is a request maintainers make casually. Worth its own issue on the wmux side about what a crash report should and shouldn't include.

    Leaving open pending your soak result.

  4. amirlehmam commented on Aug 14, 2026

    @amirlehmam
    Owner

    Status update for 1.0.0, since this is still open on your soak result.

    Nothing in the crash guard changed. pty-crash-guard.ts is unchanged since 0.47.0, still installed at module load — before any pty.spawn, since the exit callback it wraps is registered inside spawn() and a later install would leave every already-spawned pane on the unguarded path. tests/unit/pty-crash-guard.test.ts still pins both halves.

    Your clock therefore keeps running from 0.47.0 with no reset. Given the baseline you established — one crash every 2–4 days across a dozen releases — you're now four weeks past the last one on a guarded build, which is well beyond the ~3 weeks you called the point where the numbers start meaning something. I'm not going to declare it fixed on your behalf; that's your data. But it's worth saying out loud that the silence has passed the threshold you set.

    One thing in 1.0.0 touches PTY-adjacent code, so flagging it rather than have you wonder: #164 changed how the renderer tracks mouse modes across a terminal remount, and added a synthetic mouseup before terminal.dispose(). That's all renderer-side — it does not touch pty-manager.ts, pty-crash-guard.ts, or the node-pty load path. If a crash does recur, the signature is what tells us, and this shouldn't be able to change it.

    On the crash-report question you raised, which I said deserved its own issue: it still does, and I still haven't filed it. You were right on both counts — that withholding those dumps was correct, and that it's your machine's problem rather than wmux's. But "wmux crashed, here's a dump" is a request maintainers make casually, and a minidump carrying the process environment block is a credential leak in a file people hand to strangers. Eleven live keys including a trading secret set is a good argument that the casual request needs a warning attached to it. I'll file it.

    Your offer to answer targeted questions about the dumps without publishing them stands open on my side too — the discriminator I named in the 0.47.0 comment is still the one I can't settle: whether the Napi::Error came from a JS-thrown exception (which the guard catches) or from cb.Call / Napi::Number::New failing during environment teardown (which it doesn't, and which would point upstream at node-pty). If it never crashes again, that question stays open and harmless. If it does, that's the first thing to ask.

    So: leaving open, pending your call. The Event Log line alone — faulting module, exception code, Additional parameter — is enough to tell same-signature from new, and given what you found in those dumps, still no dump wanted.

  5. Ray0483 commented on Aug 14, 2026

    @Ray0483
    ContributorAuthor

    It crashed. 2026-08-14, on 0.50.0.

    Answering your question directly: yes, it has crashed since 0.47.0. Once, today.

    2026-08-14 13:15:11 local (UTC+8)
    Faulting application name: wmux.exe, version: 0.50.0.0, time stamp: 0x6a3b18dc
    Faulting module name:      wmux.exe, version: 0.50.0.0
    Exception code:            0xc0000409
    Fault offset:              0x0000000003290be5
    Additional parameter:      0x7            (FAST_FAIL_FATAL_APP_EXIT)
    

    Same signature, not a new one. Identical exception code, identical fault offset, identical fastfail parameter to all eight previous dumps. From the minidump's module list (file not leaving this machine): conpty.node loaded, bundled conpty\conpty.dll loaded, winpty absent — the same useConptyDll: true path as before.

    I also walked the C++ EH structures on the new dump. The throw record on the stack has NumberParameters=4, ExceptionInformation[0] == 0x19930520, and ExceptionInformation[3] equal to conpty.node's load address. Resolving _ThrowInfo → _CatchableTypeArray → TypeDescriptor against the on-disk module gives:

    .?AVError@Napi@@                            ← Napi::Error
    .?AVObjectReference@Napi@@
    .?AV?$Reference@VObject@Napi@@@Napi@@
    .?AVexception@std@@
    

    Byte-for-byte the same four types as the original report. Nothing about the bug has changed shape.

    First, a correction on the count — the soak never got near three weeks

    Your 1.0.0 comment says "four weeks past the last one." That looks like it anchored on the first row of my crash table (07-19) rather than the last. The last crash before today was 2026-08-04, which my 08-10 comment also stated ("6d 0h with no crash" as of 08-10).

    The real arithmetic:

    Span Length Build
    08-04 → 08-10 11:44 6 days 0.43.0–0.46.0, unguarded
    08-10 11:44 → 08-14 13:15 4 days 0.47.0 → 0.50.0, guarded

    So the guarded-build streak was four days, not four weeks, and then it crashed. I'd rather flag this than let a "soak passed" data point end up in the issue history — the ~3-week threshold I set was never approached.

    Two things about this crash that differ from the previous eight

    1. The process lived five seconds. Windows records the faulting process start time as 13:15:06; it died at 13:15:11. Every one of the previous eight crashes happened mid-session, in a window that had been up for hours.

    2. A planned OS restart began 57 seconds later. MoNotificationUx.exe initiated a Windows Update service-pack reboot at 13:16:08. So this crash happened inside the window where the machine was tearing things down, not during normal use.

    I'm reporting the circumstance rather than a conclusion, but it is worth noticing that both facts land squarely in the "environment teardown" half of the discriminator you named.

    On the discriminator — the evidence now leans one way

    You wrote that the open question is whether the Napi::Error came from a JS-thrown exception (which the guard catches) or from cb.Call / Napi::Number::New failing during environment teardown (which it doesn't, and which points upstream at node-pty).

    Three things now point at the second branch:

    The guard is present in the build that crashed. I didn't take this on faith — resources/app.asar in my 0.50.0 install contains pty-crash-guard, installPtyCrashGuard, and guardExitCallback, with the Symbol.for('wmux.ptyExitCallbackGuarded') marker and the attachErrorSink listener-count logic. It's there.

    But it is entirely JS-side, and conpty.node was never rebuilt. The shipped conpty.node is the stock node-pty prebuild (node-pty 1.1.0) — 312,320 bytes, MD5 CBEA9B78B3393F83DEEF5DED76632FB5 — and there's no conpty.cc in the package to compile. That file is byte-identical across every build I have on disk: 0.47.0, the 0.50.0 that crashed, and wmux 1.1.0, which I hashed a few minutes ago after upgrading. So guardExitCallback wrapping _$onProcessExit can only neutralise the case where the JS callback body throws and cb.Call converts a pending JS exception into a C++ Napi::Error. If the Napi::Error is raised by cb.Call or Napi::Number::New themselves, the throw happens before control ever reaches the wrapped function — the guard is upstream of nothing.

    The signature did not change at all. Not a different offset, not a different parameter, not a different thrown type. A guarded build hitting a byte-identical signature is what you'd expect if the guard sits on a path the throw never takes.

    The caveat, stated plainly: I can't prove the guard was actually installed in that particular five-second process. defaultOnError logs via console.warn, and I have no main-process log from that run to check, so "installed at module load" is your design claim, not something I verified at runtime for this crash. If wmux wrote that warning anywhere I can retrieve, tell me where and I'll look.

    Scope notes

    • I was on 0.50.0, not 1.0.0 — the install dates to 08-13 17:48 local. So bug(terminal): SGR mouse encoding is lost after terminal remount #164 and the renderer-side mouse-mode changes are not implicated in this one.
    • I've now upgraded to 1.1.0 (installed tonight, 08-14 23:25 local). The clock restarts there, honestly labelled: anything I report from here is a 1.1.0 data point, and the 0.50.0 result above stands on its own.
    • My offer from the 0.47.0 round still stands: I can run targeted analysis against these dumps and answer specific questions without publishing the files. If there's a particular address, structure, or module state that would separate cb.Call failing from Napi::Number::New failing, name it and I'll go read it.

    On the crash-report issue

    You said it deserves its own issue and you hadn't filed it yet. I'm happy to write it up if you'd rather not — I have the concrete case, and I'd keep it generic: what a minidump carries, why "here's a dump" is a bigger ask than it looks, and what a crash-report request should warn about. Say the word and I'll open it; if you'd rather own it, that's fine too and I'll leave it with you.

  6. amirlehmam commented on Aug 14, 2026

    @amirlehmam
    Owner

    First, the correction — you're right, and it was a bad one

    I wrote "four weeks past the last one." The last crash was 2026-08-04, which you had already stated plainly in your 08-10 comment. I anchored on the first row of your table (07-19) instead of the last. The guarded streak was four days, not four weeks, and the ~3-week threshold you set was never approached.

    That is worse than an arithmetic slip, because I used it to imply the soak had told us something. It hadn't. Thank you for catching it before it became the version of events the issue history remembers — that is exactly the kind of thing that ossifies into "we tried that, it was fine."

    Your analysis of why the guard is upstream of nothing is correct

    I've verified the parts I can check locally.

    The binary is the one I have. conpty.node, win32-x64, from the node-pty 1.1.0 prebuild wmux ships:

    md5  cbea9b78b3393f83deef5ded76632fb5      312,320 bytes
    

    Byte-identical to yours. So we are reading the same code, and no wmux release has ever changed it — npmRebuild: false, prebuilds only.

    One small correction, in your favour rather than against. conpty.cc does ship, at node_modules/node-pty/src/win/conpty.cc — but the conclusion you drew from its absence holds anyway, because wmux never compiles it. The source being present actually helps: it means a rebuild is available as an option if it ever comes to that.

    And the source settles the ordering. The callback is:

    auto callback = [](Napi::Env env, Napi::Function cb, ExitEvent *exit_event) {
      cb.Call({Napi::Number::New(env, exit_event->exit_code)});
      delete exit_event;
    };

    Napi::Number::New is an argument — it is evaluated before cb.Call is entered. So the two hypotheses are not symmetric:

    • JS callback body throws (the case the guard covers): Number::New succeeds, cb.Call returns napi_pending_exception, Napi::Error is constructed from that pending JS exception.
    • Environment dying: Number::New fails first, and the throw happens before control reaches the wrapped _$onProcessExit at all. The guard is upstream of nothing, precisely as you said.

    A guarded build reproducing a byte-identical signature is exactly what the second branch predicts.

    Answering your question about the log: there isn't one, and that was the real gap

    You asked where to look for defaultOnError's console.warn. The honest answer is nowhere. wmux had no log file at all — no electron-log, no app.getPath('logs'), no write stream anywhere in src/main/. In a packaged build that console.warn goes to a console nobody is attached to.

    So "installed at module load" stayed a design claim, and you were right to label it as one. That is a bad place for an investigation to be stuck, and it is on me rather than on you.

    Fixed in 1.1.1. %APPDATA%\wmux\logs\main.log now carries a line at start and one at each teardown path:

    2026-08-14T21:15:06.312Z pid=24312 start version=1.1.1 electron=43.0.0 guard=true
    2026-08-14T21:15:11.004Z pid=24312 session-end ptys=6 guard=true
    2026-08-14T21:15:11.118Z pid=24312 will-quit ptys=0 guard=true
    

    guard= is the runtime answer to the question you couldn't verify. ptys= at each stage is how wide the teardown race was — and session-end ptys=6 followed by will-quit ptys=0 is the observation that the early kill actually ran.

    Process lifecycle only: no pane contents, no cwds, no command lines, no environment. Deliberately — for the reason you demonstrated with the dumps, that a file people hand to strangers must not carry anything else. Capped at 256 KB, inside wmux's own data directory, and it cannot throw.

    What 1.1.1 actually changes

    Your two new circumstances are the most useful thing in this report, and they point somewhere specific: session-end was not handled at all.

    Windows fires it on logoff, shutdown, and a Windows Update restart — your 13:16:08 — and it is not cancellable, so it is the earliest honest notice that the process is going away. wmux was killing PTYs at will-quit, which on an OS-driven teardown is late and may not run at all before the process is force-terminated.

    That matters because every live PTY is one ThreadSafeFunction whose exit callback runs on the main thread. When Windows kills the children as part of ending the session, they all fire at once, against an environment that is itself being destroyed. Killing on session-end makes those callbacks fire while the environment is still healthy instead of racing its teardown.

    I want to be exact about what that is: it narrows a window, it does not fix the bug. The throw is in Napi::Number::New / cb.Call inside node-pty, outside any handler wmux owns, and nothing in JS can catch it. If the callbacks fire late enough anyway, you will see the identical signature again. Please report it if so — that result is informative rather than disappointing.

    The five-second process

    Worth noting what it rules out on this side: a losing second instance returns from whenReady before creating any window or restoring any session (index.ts:456), so it never spawns a PTY and has no TSFN to fire. A five-second process that crashed in conpty.node therefore had PTYs — it was a first instance that restored a session, spawned its panes, and was hit by the teardown almost immediately.

    Which fits the rest: session restore spawns every pane at once, so that is also the moment with the most outstanding exit callbacks in flight. If the machine was already tearing down when it launched, session-end is exactly the event that should have fired — and until 1.1.1 nothing was listening.

    The discriminator you asked me to name

    Two things, in order of how cheaply they separate the branches:

    1. Does the Napi::Error wrap a real JS Error object? This is the decisive one, and it needs no symbols. Napi::Error derives from ObjectReference; follow its napi_ref to the referenced value.

    • cb.Call path — node-addon-api built the error from napi_get_and_clear_last_exception, so the reference points at the actual JS Error _$onProcessExit threw, with a message and a stack string naming windowsPtyAgent.js.
    • Number::New path — there is no pending JS exception to adopt, and no such object exists.

    A live JS Error with a windowsPtyAgent.js stack means the guard's premise was right and something else is wrong. Its absence is the teardown branch.

    2. Was the environment already stopping? From the napi_env passed to the callback: node_napi_env__ wraps a node::Environment, and is_stopping() / the can_call_into_js flag being false at throw time says the answer directly. Harder to walk without symbols, so try (1) first.

    If you get either, that closes the question — and if it is (2), the next step is an upstream node-pty issue, not another wmux release.

    On upstream

    This is now sharp enough to file against node-pty: a TSFN callback that calls Napi::Number::New and Function::Call with no try/catch, dispatched during environment teardown, where any failure is an uncaught C++ throw from a frame with nothing above it. The fix upstream is small — wrap the callback body, or check the env before calling.

    I'd rather not open that on your evidence without asking. Would you like to file it, or shall I? You have the nine-dump history and the EH walk; I have the reproduction of the ordering argument from source. Either works, and I'm happy to co-sign whichever way round.

    On the crash-report issue

    Yes please — write it up. I've said twice I'd file it and twice haven't, and you have the concrete case that makes it more than a policy note: eleven live credentials, including a trading key set, in a file I was casually asking you to send. That is the version people will actually act on.

    If you'd rather not, say so and I'll do it this week rather than promise a third time.

    Scope

    Noted and agreed: 0.50.0, so #164 and the renderer mouse-mode work are not implicated. 1.1.1 is out now — https://github.com/amirlehmam/wmux/releases/tag/v1.1.1 — so if you take it, the clock restarts there with the log available from the first launch.

    Leaving this open. Nothing about it is resolved, and one four-day streak told us nothing.

  7. Ray0483 commented on Aug 15, 2026

    @Ray0483
    ContributorAuthor

    The privacy issue is filed

    #174. Kept generic as promised — no vendor names, no variable names, no mention of what the credentials were for. Thanks for the go-ahead.

    Upstream: yours, please

    You should file it. You have the ordering argument from source, which is the part that actually makes the case, and you're the downstream maintainer — an upstream issue tends to move faster when it comes from a package that ships the thing. I'll add the dump history underneath and answer anything they want checked. Link it here and I'll follow.

    A small correction back, on conpty.cc

    You wrote that conpty.cc does ship, at node_modules/node-pty/src/win/conpty.cc. That's true of your tree, but not of the shipped package. I parsed the asar header of my 1.1.0 install and listed every source-file entry in it:

    • conpty.h — present
    • conpty.cc — absent
    • in fact zero .cc or .cpp entries anywhere in the archive; 50-odd .h files, no translation units

    So the packaged app carries node-pty's headers but none of its C++ sources. Nothing in your conclusion changes — you're right that wmux never compiles it either way, and npmRebuild: false settles that more cleanly than the file's absence ever did. Flagging it only because "the source ships, so a rebuild is available as an option" isn't true for anyone working from an installed copy, which is the position every reporter here will be in.

    The discriminator: I ran it. Your test returns "yes" — and the test is wrong.

    I switched WER to DumpType=2 for wmux.exe only, and got a full-memory dump the same day. 664 MB, heap included. So test (1) is finally a straight pointer walk.

    First, scope, because this matters for how you read the rest: this dump is a 1.1.0 crash, not 1.1.1. Version read from the dump's own module list, not from my notes:

    wmux.exe   1.1.0.0
    

    I installed 1.1.1 eleven seconds after this crash, so 1.1.1 has no soak time at all yet. Everything below describes 1.1.0, which carries the same unchanged guard.

    Test (1), run as you specified it. The thrown Napi::Error's napi_ref is a live v8impl::Reference (vtable resolves into the image), and its v8::Global slot resolves to a real V8 heap object. I confirmed it's a genuine heap object without relying on any version-specific constant, by walking the compressed map chain three levels and checking the meta-map self-reference (map(map(x)) == map(map(map(x)))):

    napi_ref   0x52a402269060
      +0x00  vtable  -> wmux.exe+0xb451b58     (live C++ object)
      +0x18  slot    -> 0x52a400cd8260
              tagged -> 0x36c016a3929
              map=0x36c01142c60  meta-map=0x36c010649bc   ✔ self-referential
              instance_type=2133
    

    So by the criterion you named — does napi_ref point to a real JS object — the answer is yes, in both throw records.

    But that criterion can't discriminate, and here's the proof. I read the error object's own in-object fields rather than stopping at "it's an object":

    Error @ 0x36c016a3928   map=0x36c01142c60  instance_type=2133
      properties  = <empty fixed array>
      in-object +0x10 = String[23] 'An exception is pending'
      in-object +0x14 = String[23] 'An exception is pending'
    

    An exception is pending is not a JS-authored message. It is the napi_extended_error_info.error_message text for status napi_pending_exception. That's node-addon-api's synthesized fallback in Error::New(napi_env): it grabs the last error info, checks napi_is_exception_pending, and only adopts a real JS exception if one comes back from napi_get_and_clear_last_exception. When nothing comes back, it constructs its own Error carrying the N-API status string — which is exactly what's in this dump.

    That path always produces a real JS object. Both of your branches end with napi_ref pointing at one, which is why the pointer test returns "yes" either way. The discriminating fact isn't the pointer, it's the message:

    adopted JS exception synthesized fallback
    napi_ref → real JS object yes yes
    message the callback's own error N-API status text
    observed here — 'An exception is pending'

    So a napi call returned napi_pending_exception, and by the time Error::New looked, there was no retrievable pending exception to adopt. That combination is what a terminating isolate looks like: the status says an exception is pending, but it isn't one you can get and clear. It is not what a JS callback body throwing a TypeError looks like — that one hands you the TypeError.

    That is your second branch, the one where the throw happens before control reaches the wrapped _$onProcessExit, and the guard is upstream of nothing.

    Confidence, stated honestly: the dump facts above are direct reads and I'll stand on them. The step from "message is the N-API status text" to "node-addon-api took the synthesized path" is me reading Error::New(napi_env)'s known implementation, not wmux's bundled copy of it. If you want that nailed down, the version of node-addon-api in your tree will settle it in about a minute and you're better placed to check than I am.

    The two throw records, re-confirmed on the full dump — and a correction to my own last comment

    Both live throw records are still there, on the faulting thread, at byte-identical offsets to the 0.50.0 dump:

                        0.50.0 (08-14)    1.1.0 (08-15, full)
    throw record #1     RSP+0x11a0        RSP+0x11a0
    throw record #2     RSP+0x35d0        RSP+0x35d0
    

    The full dump does contain two additional throw-magic records, but they're in heap and other regions, not on any thread stack — so it's still two live ones, not four.

    The correction: when I told you last time that I'd checked both records were above RSP and therefore live, the check itself was broken. My parser was reading MINIDUMP_THREAD.Stack at offset +16 instead of +24, so it was treating the TEB pointer as the stack base and the low half of StartOfMemoryRange as the length. Every thread's "range" came out large enough to contain every address, so the above-RSP test was passing on nothing. I found it because the full dump made the absurdity visible — one address "belonged to" all 57 threads.

    Fixed the offset, re-ran on both dumps, and the conclusion happens to survive intact: both records are on the faulting thread, both above RSP, at the offsets above. But it survived by luck, not by the check, and I'd rather say so than let a claim I made on a broken measurement stand unqualified.

    Why you're still not getting the file — and a warning for #174

    The dump doesn't leave this machine, for two reasons, and the second one is worth your attention for the crash-report issue.

    It's 664 MB. Over any issue attachment limit, before anything else.

    And it still has every credential in it. This is the part I got wrong last time. I told you a full-memory dump was safe now, because I'd moved all eleven keys out of my environment variables into an encrypted store earlier that same day. I scanned the dump to check. All eleven are in it, in cleartext, plus the bot token:

    OKX_API_KEY / _SECRET / _PASSPHRASE            present, non-empty
    OKX_N1_LIVE_API_KEY / _SECRET / _PASSPHRASE    present, non-empty
    OKX_N1_DEMO_API_KEY / _SECRET / _PASSPHRASE    present, non-empty
    TELEGRAM_BOT_TOKEN                             present, non-empty
    

    Deleting the variables doesn't clean a process that's already running. A process gets its environment block copied in at launch and keeps that copy for life. This wmux had been up since the previous evening, so it was still carrying the pre-migration block hours after the variables were gone from the registry. The dump captured what the process held, not what the system held.

    For #174 that's the sharper version of the warning: "I've rotated/removed my secrets, so the dump is safe" is false for any process that was already running when you did it. It only becomes true after a restart. I'd have uploaded this file if I hadn't scanned it, and I'm the person in this thread who already knew about the problem.

    The processes running now started after the migration and are clean — I verified the variables are absent from their inherited environment. So the next dump should genuinely be publishable, and I'll scan it again before saying so rather than reasoning about it.

    Scope

    • Now on 1.1.1, installed 2026-08-15 11:22 local. No crash on it yet — zero soak time, no claim either way.
    • WER is on full-memory dumps for wmux.exe from here on.
    • The parser is a few hundred lines of stdlib Python against the dump format, offsets now audited. Name a structure and an offset and I'll read it out — that arrangement still works, and still doesn't require the files to leave this machine.
  8. amirlehmam commented on Aug 16, 2026

    @amirlehmam
    Owner

    You asked me to check Error::New(napi_env) against the copy in my tree. I did, and it does more than confirm your reading — it names the fix.

    Filed upstream: microsoft/node-pty#951.

    Short version: the throw is unhandleable by design, node-addon-api ships the guard for exactly this case, and node-pty does not compile it in. The fix is one line in binding.gyp.

    First, your correction stands

    conpty.cc is absent from the packaged asar. You're right and I was describing my tree as though it were the shipped article. It matters for precisely the reason you gave — "the source ships, so a rebuild is an option" is false for anyone working from an installed copy, which is every reporter here. npmRebuild: false settles it more cleanly anyway.

    Your Error::New reading, verified

    node-addon-api 7.1.1, which is what node-pty@1.1.0 resolves. napi-inl.h:2822:

    status = napi_is_exception_pending(env, &is_exception_pending);
    if (is_exception_pending) {
      status = napi_get_and_clear_last_exception(env, &error);   // adopt the real JS error
    } else {
      const char* error_message = last_error_info_copy.error_message ...;  // <- your string
      ...
    }

    One refinement, which strengthens rather than weakens it. You wrote that it "only adopts a real JS exception if one comes back from napi_get_and_clear_last_exception". The branch is actually decided earlier, by napi_is_exception_pending alone — once that returns true, whatever get_and_clear returns is adopted unconditionally.

    That makes your evidence stronger, because it pins a more specific state. Observing 'An exception is pending' proves napi_is_exception_pending returned false, while the immediately preceding napi call had failed with status napi_pending_exception.

    And only one thing produces that pair. In Node's js_native_api_v8.cc, NAPI_PREAMBLE returns napi_pending_exception when !(env->last_exception.IsEmpty() && env->can_call_into_js()), whereas napi_is_exception_pending reports only !last_exception.IsEmpty(). False from the second with napi_pending_exception from the first means can_call_into_js() == false.

    The environment is stopping. Your branch 2, from the message text, with no version-specific constants involved.

    (Provenance, held to the standard you've been holding this thread to: Error::New and everything below I read in my node_modules. The NAPI_PREAMBLE step is from Node's source, not from a header in my tree.)

    The part neither of us had

    I kept going into ThrowAsJavaScriptException (napi-inl.h:3039), and this is where it turns actionable:

    #ifdef NODE_API_SWALLOW_UNTHROWABLE_EXCEPTIONS
        ...
        if (status == napi_pending_exception) {
          // The environment must be terminating as we checked earlier and there
          // was no pending exception. In this case continuing will result
          // in a fatal error and there is nothing the author has done incorrectly
          // in their code that is worth flagging through a fatal error
          return;                                    // <- the guard
        }
    #else
        napi_status status = napi_throw(_env, Value());
    #endif
    
    #ifdef NAPI_CPP_EXCEPTIONS
        if (status != napi_ok) {
          throw Error::New(_env);                    // <- uncaught
        }
    #endif

    node-addon-api has a guard for this exact scenario, and its comment describes our crash in its own words — "the environment must be terminating", "continuing will result in a fatal error", "nothing the author has done incorrectly."

    node-pty does not define the macro. binding.gyp depends on node_addon_api_except; its except.gypi defines NAPI_CPP_EXCEPTIONS and nothing else. NODE_API_SWALLOW_UNTHROWABLE_EXCEPTIONS appears nowhere in the package. So the #else compiles in, and the shipped prebuilds carry the throwing path.

    The full sequence — and it explains your two throw records

    1. A PTY child exits; the TSFN dispatcher runs the conpty.cc callback on the main thread.
    2. node-addon-api wraps it in WrapVoidCallback (napi-inl.h:95), which catches Napi::Error and calls ThrowAsJavaScriptException().
    3. Environment terminating, so Napi::Number::New / cb.Call fail and throw a Napi::Error carrying the synthesized 'An exception is pending'. ← throw Korean IME (한글) input issues on Windows 11 #1
    4. WrapVoidCallback catches it, calls ThrowAsJavaScriptException().
    5. napi_throw cannot throw into a dying isolate. No swallow guard → throw Error::New(_env). ← throw feat: improve terminal and markdown link behavior #2, uncaught
    6. Nothing above that frame → UnhandledExceptionFilter → abort() → __fastfail(7) → 0xc0000409.

    Both live throw records are still there, on the faulting thread, at byte-identical offsets to the 0.50.0 dump

    That's steps 3 and 5. Two live records, byte-identical across dumps taken months and five releases apart, was an observation sitting in this thread with no explanation attached. It has one now, and it is a prediction of the mechanism rather than a retrofit — the count falls out of the code path whether or not anyone had counted.

    It also explains why WrapVoidCallback being in the picture is consistent with your finding that the throw's image base is conpty.node: it is a header-inlined template, so its throw compiles into conpty.node's .rdata like any other.

    What this means for wmux, plainly

    Nothing further I ship can fix it. I want to be exact rather than encouraging:

    • The 0.47.0 guard is upstream of nothing, as you said. Napi::Number::New is an argument, evaluated before cb.Call is entered, so the failure begins before control reaches the JS callback. Confirmed by source ordering, and consistent with a guarded build reproducing a byte-identical signature.
    • The 1.1.1 session-end kill narrows the window. It does not close it. If a callback fires late enough anyway, you get the same signature.
    • I cannot define the macro without compiling node-pty myself, and wmux ships prebuilds (npmRebuild: false) deliberately.

    So this issue's remaining life is upstream. I'm leaving it open and linked, not closing it on someone else's fix.

    What I'd ask of you, and it is less than before

    The soak question is retired — you were right that one four-day streak told us nothing, and I'm not restarting that clock. If it recurs on 1.2.0, the Event Log line alone is enough, and no dump is wanted.

    wmux crash-report (new in 1.2.0, from your #174) prints exactly that line, plus guard= and ptys= from the lifecycle log so the runtime question you couldn't answer for the 0.50.0 crash is answerable for the next one. It reads the events' positional properties rather than the rendered message, so it works on a non-English Windows — I only found that because it silently returned (not found) for everything on the machine I tested it on.

    If node-pty takes the one-liner, the thing that would actually settle it is a wmux build against a patched conpty.node on your machine. I'll do that build if it comes to it; I'm not asking you to compile anything.

    Two smaller things

    On the count I got wrong. Noted properly this time — four days, not four weeks, and I'd used it to imply the soak had told us something. That is in the record now above my own correction, which is where it belongs.

    On the dumps. Still not wanted, and #174 is now shipped rather than promised. Your finding that a full dump of an already-running process still carries variables you deleted hours earlier is the sharpest thing in that document. I would have got it wrong.


    Thirteen crashes, thirteen months, and the thing that broke it open was a string in a heap object. Thank you for not stopping at "it's an object" — I named a test that returns "yes" on both branches, and you noticed and said so instead of reporting the answer I was expecting.

  9. amirlehmam commented on Aug 17, 2026

    @amirlehmam
    Owner

    The one-liner is now a PR: microsoft/node-pty#954

    Filing #951 and then waiting was the wrong shape for something whose fix is nine lines of binding.gyp, so I opened it.

    'defines': [ 'NODE_API_SWALLOW_UNTHROWABLE_EXCEPTIONS' ],
    

    in target_defaults, which is where it reaches every target rather than only conpty.

    Two things I found preparing it that were not in #951

    There are two guarded sites in node-addon-api 7.1.1, not one. We had been talking about ThrowAsJavaScriptException (napi-inl.h:3042). The Error(napi_env, napi_value) constructor guards its napi_define_properties call the same way at napi-inl.h:2924:

          status = napi_define_properties(env, wrappedErrorObj, 1, &wrapObjFlag);
    #ifdef NODE_API_SWALLOW_UNTHROWABLE_EXCEPTIONS
          if (status == napi_pending_exception) {
            // ... the environment is shutting down ...

    Unguarded, that one ends at NAPI_FATAL_IF_FAILED rather than at an uncaught throw — a different abort with the same cause. One define closes both.

    It builds, and I built it. Windows, VS 2022 BuildTools, Node 24.13.0, node-gyp 11.4.2: gyp info ok, conpty.node and conpty_console_list.node produced, no new warnings — the only ones on conpty.cc are the pre-existing C6387s at lines 414 and 448. The define lands next to NAPI_CPP_EXCEPTIONS in the generated project, which is the pairing that matters, since NAPI_CPP_EXCEPTIONS is what makes throw Error::New(_env) compile in the first place:

    PreprocessorDefinitions>NODE_API_SWALLOW_UNTHROWABLE_EXCEPTIONS;NODE_GYP_MODULE_NAME=conpty;…;NAPI_CPP_EXCEPTIONS;_HAS_EXCEPTIONS=1;…
    

    Held to the standard you have been holding this thread to, two caveats about that build rather than a clean claim: I disabled SpectreMitigation locally because this machine lacks the Spectre-mitigated libraries, and unmodified main fails identically on it (MSB8040) — I checked, rather than assuming my diff was innocent. And deps/winpty was not checked out, so the pty target never generated. The target that matters here is conpty, and it built.

    Nothing in wmux 1.3.0 touches this

    pty-crash-guard.ts is unchanged since 0.47.0, still installed at module load, and tests/unit/pty-crash-guard.test.ts still pins both halves. The 1.1.1 session-end kill is unchanged. I am not shipping another wmux-side mitigation, for the reason we established: Napi::Number::New is an argument, evaluated before cb.Call is entered, so the failure begins before control reaches anything wmux can wrap. A guarded build reproducing a byte-identical signature is what proved that, and it is your evidence, not mine.

    Two unrelated fixes shipped in 1.3.0 (#175, #167). Neither goes near the PTY layer, and neither should be able to change the signature if it recurs.

    What I would ask, still less than before

    Nothing, unless it recurs — and if it does, the Event Log line alone is enough. wmux crash-report prints it, plus guard= and ptys=, so the runtime question you could not answer for the 0.50.0 crash is answerable for the next one. No dump wanted.

    If node-pty takes #954, the thing that settles it is a wmux build against a patched conpty.node on your machine. I will do that build; I am still not asking you to compile anything.

    I have linked this issue from the PR and pointed at your dump analysis there, since thirteen crashes with a byte-identical signature and a stdlib parser that can read the throw records is a stronger argument for merging than anything I can write about my own machine. If you would rather not be named there, say so and I will edit it out.


    1.3.0 is out — https://github.com/amirlehmam/wmux/releases/tag/v1.3.0. Leaving this open and linked upstream, not closing it on someone else's fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0