Unofficial TypeScript SDK for chatting with Muse, based on the Muse Gadget SDK protocol. Supports streaming chat, image messages, VM lookup, token refresh, and macOS Bluetooth pairing.
Requires Node.js 22.18 or later. Pairing supports macOS; chat also runs on Linux.
npm install muse-clientPairing requires Xcode Command Line Tools (xcode-select --install) and a personal
SDK token.
From the project directory:
bun install
bun run cli pair- Paste your SDK token when prompted and allow Bluetooth access.
- In the Muse phone app, enable Settings > Devices > Developer mode, then choose Add Device.
- Select the printed
MuseGadgetXXXXXXname, confirm, and choose Use current connection for Wi-Fi. - Wait for
Paired successfully.
The phone must be able to reach hatch-api.meta.ai to obtain device credentials.
If pairing fails after selecting a network, check the phone's network or proxy rules.
For pairing from your own application:
import { pairMacOS } from 'muse-client/pairing';
const credentials = await pairMacOS({
sdkToken: process.env.MUSE_SDK_TOKEN,
onProgress: console.log,
});Pairing saves credentials locally. The default directory is
~/Library/Application Support/MuseGadgetPair/ on macOS and /var/lib/musegadget/
on Linux. Use --credentials <directory> in the CLI or directory in pairMacOS
to choose another location. Existing pairings are preserved.
The SDK token is associated with a device when it pairs. After changing that
token, or revoking the token used for pairing, run bun run cli unpair followed by
bun run cli pair with the new token. Refreshing device credentials does not
change this association. New pairings save sdk_token_hint (the first 12
characters); the CLI warns when the configured token differs. Older pairing files
without a hint remain usable, but their original SDK token cannot be determined.
The CLI runs from the repository:
bun run cli chat # Interactive chat; /quit or Ctrl+C to exit
bun run cli vms # List Muse VMs
bun run cli check # Check the connection without sending a message
bun run cli send "Hello Muse" # Send a message and print its acknowledgement
bun run cli watch --json # Stream chat events
bun run cli unpair # Remove local pairing credentialsUse --vm <id> to choose a VM, --session <id> for an existing side chat, and
--help for all options.
Interactive chat waits for each request's replies before sending the next input.
chat --json prints typed turn events; watch --json prints raw subscription
events.
Before running unpair, stop any chat or pairing process. It preserves the device
identity and SDK token. To remove the device association from Muse, use
Settings > Devices in the phone app.
After pairing:
import { MuseClient } from 'muse-client';
import { loadCredentials, saveCredentials } from 'muse-client/credentials';
const client = await MuseClient.connect({
credentials: await loadCredentials(),
onCredentials: saveCredentials,
onWarning: console.warn,
});
try {
for await (const event of client.chat('Hello!')) {
if (event.type === 'reply') {
// text is the current snapshot, not an append-only delta.
if (event.done) console.log(event.text);
} else if (event.type === 'agent-status') {
console.log(event.activityText ?? event.activityCode);
}
}
} finally {
client.close();
}chat()opens a subscription before sending, associates replies with the acknowledgement, and closes its subscription on completion, early loop exit, cancellation, or error. Setsignalto cancel, ortimeoutMsto change the 120-second overall deadline.- Completion waits for all bound replies to finish and the associated task to
finish, or for three seconds without activity when no busy status remains.
This quiet-period fallback is a heuristic. Terminal tasks with empty replies
get a short grace period for reordered text.
task-finishedis a server task status event; the iterator ending indicates that the whole turn has settled. - Shared-chat replies with no provable parent link are omitted by
chat(). For an existing side chat used exclusively by your application, pass{ sessionId, ownsChat: true }to accept parentless replies. The acknowledgement must confirm the side chat. Sequential owned turns share a bounded replay history on the client; overlapping owned turns in the same chat are rejected. - A new side chat must be created by sending its first message with
sendMessage(message, { sessionId })before subscribing.chat()does not create a new side chat. - Persist refreshed credentials with
onCredentials. Avoid concurrent processes sharing the same credentials. The SDK does not automatically reconnect or resend.
For raw events, use subscribe() and sendMessage() directly; subscribe before
sending and pass the same sessionId to both. Raw subscriptions may include other
clients' activity. A slow consumer exceeding its 4 MiB response buffer fails and
resets only that request, leaving other streams on the connection usable.
Both chat() and sendMessage() accept images. Text and all images are sent in
one request and belong to the same turn. Pass an empty string to send only images.
Existing text-only calls are unchanged.
Pass File objects directly; the SDK reads their names, MIME types and contents:
await client.sendMessage('What is in this picture?', { images: [file] });
// Also supported: client.chat('What is in this picture?', { images: [file] })images has type readonly ImageInput[], where ImageInput is
File | ImageAttachment. You can mix both types in the same request. Supported
formats are JPEG, PNG, GIF and WebP, with a maximum of 2 MiB per image.
If a File's type is empty, the SDK infers it from .jpg, .jpeg, .png, .gif
or .webp. For other extensions, set one of those formats' MIME types when
creating the File. Other formats, including HEIC and AVIF, must be converted first.
For raw bytes, use ImageAttachment:
import { readFile } from 'node:fs/promises';
import type { ImageAttachment } from 'muse-client';
const images: ImageAttachment[] = [{
data: await readFile('photo.jpg'),
mimeType: 'image/jpeg',
filename: 'photo.jpg',
}];
for await (const event of client.chat('What is in this picture?', { images })) {
if (event.type === 'reply' && event.done) console.log(event.text);
}
// Alternatively, send and receive only the acknowledgement:
const acknowledgement = await client.sendMessage('', { images });data contains raw Uint8Array or Buffer bytes; the SDK base64-encodes them.
It does not fetch URLs, read paths, or resize images. Supply the correct mimeType
and a non-empty filename. Empty images and unsupported MIME types are rejected.
The SDK limits the total JSON request (including base64, text, and metadata) to
4 MiB. Base64 adds roughly one third to the image size, so multiple images must
also fit that aggregate limit. Requests larger than 64 KiB are uploaded in
16 KiB body chunks; the transport's per-message limit does not cap the whole image.
This follows /chat/stream image and text items in the merged upstream
PR #197. The text
is also included as an item because Muse ignores the top-level message when
images are attached.
For custom event loops, MuseTurn exposes the tracking state separately:
import { MuseTurn } from 'muse-client';
const turn = new MuseTurn({ sessionId });
// Start feeding events before sending. feed() buffers until acknowledge().
// In the subscription reader: handle(turn.feed(event)).
const updates = turn.acknowledge(await client.sendMessage('Hello!', { sessionId }));
// Handle updates, then keep feeding events and checking turn.complete().
// turn.completionDelay() gives the next timer delay; Infinity means wait for events.feed() and acknowledge() return TurnEvent[]. Low-level Reply events include
bound, so callers can distinguish unlinked shared-chat messages; only bound
replies participate in completion. replies is a snapshot map keyed by message
ID. Create a fresh MuseTurn for each request. To filter replays in an owned chat,
share a ReplayScope between turns and call turn.retire() when each turn ends.
Times passed to complete(now) use performance.now() milliseconds.
| Import | Exports |
|---|---|
muse-client |
MuseClient, MuseAccount, MuseTurn, ReplayScope, transcriptText |
muse-client/credentials |
loadCredentials, saveCredentials, unpair |
muse-client/pairing |
pairMacOS, buildPairingHelper, validateSDKToken |
MuseClient provides connect, chat, sendMessage, subscribe, and close.
MuseAccount provides listVMs and refresh.
Both accept onWarning to report an SDK token change since pairing. The warning
does not block chat or trigger a token refresh.
Audio, non-image attachments, history listing, and independent account login are not supported. The SDK does not expose local shell commands or device controls to Muse.
bun run typecheck
bun run test
bun run buildThe optional upstream interoperability test requires MUSE_GADGET_SDK and
MUSE_TEST_PYTHON; it is skipped when these are unset.
Apache-2.0; see LICENSE and NOTICE. This project is unofficial and is not
affiliated with Meta. Muse service access is subject to the
Gadget SDK Terms.