8000 GitHub - dxer/onyx-sync: Self-hosted zero-knowledge E2EE sync for Obsidian — runs on Cloudflare Workers free tier or any VPS with Docker. Multi-user, device tokens, 3-way merge, all platforms. · GitHub
[go: up one dir, main page]

Skip to content
dxerPublic

About

Self-hosted zero-knowledge E2EE sync for Obsidian — runs on Cloudflare Workers free tier or any VPS with Docker. Multi-user, device tokens, 3-way merge, all platforms.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Onyx

Onyx

Self-hosted, Zero-Knowledge End-to-End Encrypted Sync for Obsidian

简体中文 · English

License: MIT Platform Node PRs Welcome

Onyx Server runs on a free Cloudflare Workers tier — or any cheap VPS with Docker. Onyx Sync is the Obsidian plugin that talks to it.


Why Onyx?

The server never sees your plaintext. It stores only AES-256-GCM ciphertext blobs, encrypted paths, and HMAC content hashes. Lose the server, lose nothing — your passphrase is the key.

Onyx ships as a complete product: a lightweight multi-user server with a management console, plus a polished cross-platform client plugin — not just a sync script.


Comparison with mainstream Obsidian sync solutions

Onyx Obsidian Sync (official) Remotely Save Self-hosted LiveSync
Cost ✅ Free (Workers tier) or ~$1/mo VPS 💰 $4–8 / month ✅ Free (cloud free tiers) ✅ Free (self-hosted CouchDB)
Data ownership 🏠 100% your server ☁️ Obsidian's cloud 🏠 your cloud bucket 🏠 your CouchDB
Server software included ✅ Purpose-built lightweight server (Hono, single binary/container) ❌ closed service ⚠️ none needed — client talks to cloud storage directly ⚠️ generic CouchDB (heavier, ops on you)
Free Cloudflare deploy ✅ Workers + D1 + R2 ❌ ❌ ❌
Zero-knowledge E2EE ✅ always on (AES-256-GCM + PBKDF2/HKDF) ✅ optional per vault ⚠️ optional, S3/WebDAV only ⚠️ optional
Encrypted file paths ✅ server never sees file names ❌ paths in metadata ❌ paths are object keys ❌
Multi-user & teams ✅ admin console, per-user isolation, closed registration ⚠️ per-account seats ❌ single-user per setup ❌ per-database, no user mgmt UI
Device credential lifecycle ✅ generate / rename / rotate / revoke per device ⚠️ unlink device ❌ shared key per cloud ❌ DB credentials
At-rest credential sealing ✅ DPAPI / Keychain / mobile sandbox key — stolen data.json is useless ✅ ❌ plaintext in plugin data ⚠️ DB password in settings
Physical storage isolation ✅ /blobs/<vault>/… per-vault partition ❌ ⚠️ per bucket/prefix config ⚠️ per database
Conflict resolution ✅ 3-way merge (Markdown) + conflict copies; scans never infer deletions ✅ versioning ⚠️ last-write-wins ✅ Conflict resolution via CouchDB replication
Realtime push ✅ WebSocket notify + smart background sync ✅ ❌ manual/interval ✅ near-live replication
Management console ✅ Vercel-style dashboard: users, vaults, devices, storage, 365-day activity heatmap ⚠️ basic web settings ❌ ❌
Setup complexity 🟡 run one server + paste 3 fields in plugin 🟢 easiest 🟡 create cloud bucket + keys 🔴 configure CouchDB properly
Maturity 🌱 young project 🏆 official, battle-tested 🏆 mature, popular 🏆 mature, popular

Choose Onyx if you want official-Sync-grade UX (multi-device pairing, token management, activity insights) on infrastructure you own, at zero or near-zero cost, with strict zero-knowledge guarantees — including for file names.

Consider alternatives if you want zero server at all (Remotely Save), real-time collaborative editing streams (LiveSync), or fully managed convenience (official Sync).


Architecture

Onyx architecture — clients hold keys, server stores only ciphertext

Zero-knowledge encryption pipeline

Onyx encryption pipeline — PBKDF2 to HKDF to AES-GCM and HMAC keys

Conflict resolution

Markdown notes use three-way merge (base vs. local vs. remote via diff-match-patch); binary files fall back to last-write-wins with conflict copies. Deletions are only propagated from explicit filesystem events — a fresh install never wipes your vault.


Feature Highlights

  • 🔐 Zero-knowledge E2EE — AES-256-GCM content, encrypted paths, HMAC-based content addressing; the server is a dumb encrypted blob store
  • 👥 Multi-user & multi-vault — physical storage isolation per vault (/data/blobs/<vault_id>/…), admin-managed accounts, no public registration
  • 🪙 Device tokens — generate / rename / rotate / revoke per-device credentials from the web console; tokens are bound to a single vault
  • 🛡️ At-rest secret protection — plugin credentials sealed with Windows DPAPI / macOS Keychain on desktop, sandbox keys on mobile; a stolen data.json is useless
  • 📊 Activity heatmap — GitHub-style 365-day sync activity per vault
  • 🌐 Bilingual console — Vercel-style dashboard, one-click 中文 / English
  • ⚡ Realtime push — WebSocket notify + background interval sync + silent change detection
  • 📱 Cross-platform — one codebase for Windows, macOS, Linux, iOS, Android

Quick Start

1. Run the server

Option A — VPS / local machine (Node.js ≥ 18):

git clone https://github.com/dxer/onyx-sync.git
cd onyx-sync
cp .env.example .env          # edit ADMIN_USERNAME / ADMIN_PASSWORD
pnpm install
pnpm build
pnpm start

Console is now live at http://localhost:8080.

Option B — Docker (prebuilt Alpine image, auto-built by CI):

# uses ghcr.io/dxer/onyx-sync:latest (linux/amd64)
docker compose up -d

Option C — Cloudflare Workers (100% free tier):

# create D1 database + R2 bucket, fill wrangler.toml
wrangler d1 create onyx-db
wrangler deploy

2. Create a vault & device token

  1. Open the console, sign in with your admin account
  2. New Vault → name it
  3. Authorize Device → copy the generated ost_… token

3. Install the Obsidian plugin

Install the official client plugin from onyx-sync-obsidian:

  • Via Community Plugins: Search for Onyx Sync in Obsidian settings (or install via BRAT).
  • Manual Install: Download onyx-sync-<version>.zip from Plugin Releases, extract it into <vault>/.obsidian/plugins/onyx-sync/.

Enable Onyx Sync in Obsidian settings and fill in:

Field Value
Server URL http://your-server:8080
Device Token ost_… from the console
Passphrase your E2EE master password — must match on every device

Done. Edit a note and watch it appear on your other devices.


Configuration

Server config lives in .env (see .env.example):

Variable Default Description
PORT 8080 HTTP port
ADMIN_USERNAME admin Auto-provisioned super administrator (first boot only)
ADMIN_PASSWORD — Consumed once on first boot to create the admin; ignored afterwards
DB_PATH ./data/sync.db SQLite metadata file
STORAGE_TYPE local local or s3 (S3 / MinIO / R2)
STORAGE_LOCAL_DIR ./data/blobs Blob root (one subfolder per vault)
S3_* — Endpoint / bucket / keys when STORAGE_TYPE=s3
WS_TICKET_SECRET ephemeral Signs short-lived WebSocket tickets; set it to keep tickets valid across restarts
CORS_ORIGINS — (allow all) Comma-separated browser origins allowed to call the API (e.g. a separately hosted dashboard)
LOGIN_RATE_LIMIT_MAX_ATTEMPTS 10 Failed logins allowed per IP per window
LOGIN_RATE_LIMIT_WINDOW_SECONDS 600 Rate-limit window for failed logins
TRUST_PROXY 0 Set 1 behind a trusted reverse proxy that overwrites X-Forwarded-For (otherwise rate limits key on the TCP peer)
MAX_VAULT_BYTES 10737418240 (10 GiB) Live-record byte quota per vault; over-quota commits get 413 vault-quota-exceeded
LOG_LEVEL info Log verbosity: debug / info / warn / error
LOG_FORMAT text text for humans, json (one object per line) for Loki/ELK
SHUTDOWN_TIMEOUT_MS 10000 Drain grace period on SIGTERM/SIGINT before forced exit

Production Checklist

The server speaks plain HTTP itself — put it behind an HTTPS reverse proxy (Caddy, Nginx, …) before exposing it to the internet.

  • HTTPS: terminate TLS at the proxy; never expose PORT directly.
  • CORS_ORIGINS: set it to the exact origins of any separately hosted dashboard. The bundled dashboard is same-origin and needs no entry; an empty value accepts every browser origin.
  • Trusted proxy headers: login rate limiting keys on X-Forwarded-For, so the proxy must overwrite (not append to) that header — otherwise a client can spoof its IP and dodge the limit.
  • WS_TICKET_SECRET: set a stable value, otherwise all WebSocket sessions drop on every restart.
  • ADMIN_PASSWORD: consumed exactly once, on first boot, to create the admin account. Later edits are ignored — rotate passwords in the console (Users → Reset password) instead.
  • Single-node only: rate limiting, WebSocket fan-out and the WS ticket replay cache live in process memory. Never run more than 1 replica; the server logs this at every boot.
  • Backups: follow docs/backup-restore.md (SQLite VACUUM INTO + blob rsync, verified with integrity_check). An unverified backup is not a backup — rehearse a restore before you need one.
  • Audit trail: admin creation, user creation/deletion and every password reset are logged as [AUDIT] action=… lines. Ship server logs to persistent storage; they are your proof of who changed what.

Account Recovery (forgotten passwords)

Normal path — any other admin resets it in the console (Users → Reset password, backed by POST /api/v1/admin/users/:id/password). The user logs in with the new password; nothing else needs to change.

Last-admin-locked-out path — requires shell access to the server host, which already implies full control over the data (that is what makes this safe while .env-driven resets are not):

# Node / Docker (SQLite). Password via env keeps it out of shell history:
ONYX_RESET_PASSWORD=<new-secret> pnpm admin:reset-password -- --username admin

# ...or inside the running container:
docker exec onyx-sync-server node dist/node/entry-node.js admin:reset-password \
  --username admin --password <new-secret>

# Cloudflare Worker (D1, no shell): one command (needs wrangler auth for the account)
pnpm admin:reset-password:d1 -- --username admin --password <new-secret>
# ...or raw SQL if you prefer (get salt+hash from the line above without --username):
pnpm admin:hash-password -- --password <new-secret>
wrangler d1 execute onyx-db --command \
  "UPDATE users SET password_hash = '<hash>', salt = '<salt>' WHERE username = 'admin';"

API Surface (v1)

GET    /api/v1/session                    # token handshake: vault, salt, device
GET    /api/v1/sync/status                # latest version clock
GET    /api/v1/sync/changes?since=N&limit=M  # incremental change log (paginated, default 200, max 500)
POST   /api/v1/sync/commit                # push encrypted changes (requestId => idempotent replay)
POST   /api/v1/sync/blobs/check           # CAS dedup check
PUT    /api/v1/sync/blobs/:hash           # upload ciphertext blob
GET    /api/v1/sync/blobs/:hash           # download ciphertext blob
POST   /api/v1/ws/ticket                  # exchange device token for a 60s single-use WS ticket
GET    /api/v1/user/vaults/:id/activity   # 365-day heatmap data
POST   /api/v1/user/vaults/:id/gc         # delete unreferenced blobs (optional { "graceDays": 7 })
DELETE /api/v1/user/vaults/:id            # async deletion job (metadata first, blobs second)
GET    /api/v1/user/deletion-jobs/:jobId  # deletion job status
POST   /api/v1/user/deletion-jobs/:jobId/retry
PATCH  /api/v1/user/tokens/:tokenId       # rename device
POST   /api/v1/user/tokens/:tokenId/rotate  # rotate credential (old secret dies immediately)
DELETE /api/v1/user/tokens/:tokenId       # revoke credential

All sync endpoints are automatically scoped to the token's bound vault — cross-vault access is structurally impossible.

Concurrency contract: the version clock never loses increments, but there is no lost-update protection on file content — two devices committing different edits to the same file both succeed as successive versions (last-writer-wins per file). The version-conflict code is D1-internal (CAS exhaustion, "please retry") and never means "your edit overwrote someone's". Clients resolve concurrent edits with three-way merge on pull, never by expecting a 409 on push.

Realtime, per deployment: the Node/Docker server supports WebSocket push — clients fetch a 60-second single-use ticket via POST /api/v1/ws/ticket and connect to /api/v1/ws?ticket=…; revoking a token closes its live sockets. The Cloudflare Worker deployment is REST + polling only (no cross-instance WebSocket fan-out); the ticket endpoint answers 501 there and the plugin automatically falls back to interval polling.


Security Model

  • Server knows: ciphertext, opaque path ciphertext, HMAC hashes, vault version clock, device names
  • Server never knows: passphrase, encryption keys, plaintext content, plaintext file names
  • Credentials at rest: token secrets are stored server-side as SHA-256 hashes only (auth_tokens); the plaintext ost_… value is shown exactly once when created or rotated, and revocation/expiry (master 30d, device 180d) is enforced on every request
  • Content integrity: content hashes are client-side HMACs over plaintext (zero-knowledge), so the server cannot recompute them — clients verify decrypt(blob) hashes correctly before upload and after every download
  • At rest on clients: token + passphrase sealed via DPAPI / Keychain (desktop) or app-sandbox keys (mobile)
  • Registration: closed by default — the first account becomes admin; all later accounts are provisioned by an admin

Project Layout

onyx-sync/
├── src/           # Onyx Server — Hono app, storage drivers, console
├── shared/        # @onyx/shared — crypto, protocol types
├── docs/          # logos, diagrams & social cards
├── Dockerfile     # Multi-arch Alpine image
├── docker-compose.yml
└── wrangler.toml  # Cloudflare Workers deployment

Looking for the Obsidian plugin? The client plugin is maintained in the dedicated repository: onyx-sync-obsidian.

Development

pnpm install
pnpm dev            # server with hot reload
pnpm build          # shared + server
pnpm test           # unit tests

License

MIT


Onyx — your notes, your server, your keys.

Onyx

About

Self-hosted zero-knowledge E2EE sync for Obsidian — runs on Cloudflare Workers free tier or any VPS with Docker. Multi-user, device tokens, 3-way merge, all platforms.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

0