8000 feat(secrets): read secret settings from Vaultwarden by reference by adeelahmad · Pull Request #550 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,16 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
a time (for the same sender, say), when one came a number of times, when one didn't come in time after another (no
reply within two days), or once for a batch collected over a time. What it waits on outlasts a restart, and switching
the automation off drops it ([Joins](docs/automations.md#joins)).
- **Secrets from Vaultwarden.** A secret field in Settings (a model's API key, a chat app's token, a mail password)
can refer to an item in a Vaultwarden or Bitwarden vault, like `vaultwarden://OpenAI/password`, instead of holding
the secret. Lens reads the value, keeps it in memory for `secrets.cache_minutes` (10 by default) and reads it again
in the background, so a key changed in the vault reaches Lens by itself. Only the reference is stored, and it's
checked against the vault before it's saved. Lens signs in as an account of its own and reads one folder of it
(`secrets.vaultwarden_folder`, "Lens"). Settings, the API (`GET /api/v1/secrets`, `GET /api/v1/secrets/items`,
`POST /api/v1/secrets/test`), the MCP tools `list_secrets`, `list_vault_items` and `set_secret_reference`, and the
assistant's `list_secrets` and `vault_items` show references and whether they could be read, never a value. The
client (python-vaultwarden, Apache-2.0) is fetched once a Vaultwarden address is set. Secrets saved in Lens stay
the default ([Secrets](docs/secrets.md)).
- **A decide node for judgement calls.** An automation's workflow can ask the decision model a question with a few
options ("Does this need me today?") and go on along the option it picked, or along `unsure` when it isn't sure
enough, none is set up or the budget is spent. The email it reads is marked as outside text, the answer is kept per
Expand Down
16 changes: 16 additions & 0 deletions DEVLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,22 @@

Plans and progress for work in flight. Newest first.

## 2026-10-18 · Secrets from Vaultwarden

Adeel: a secrets manager, Vaultwarden first, with others to come. A secret field in Settings can hold a reference
(`vaultwarden://OpenAI/password`) instead of the secret; `secret_providers.py` reads it when the settings are built,
keeps it in memory for `secrets.cache_minutes` and reads it again in the background, and anything that shows settings
shows the reference, never the value. The built-in sealed store stays the default. Vaultwarden is read with
python-vaultwarden (Apache-2.0), fetched as a component once an address is set: the Bitwarden CLI is GPL and needs
Node, and Bitwarden's SDK isn't permissively licensed and reads only its Secrets Manager product. Lens reads one
folder of its own vault account. Tested against a fake Vaultwarden that encrypts with the library's own crypto.

Next: Settings › Secrets in the web app, with a picker for the vault's items beside every secret field.

Refine later: references in sources' credentials, MCP servers' environments and sign-in providers; checking a value
read from the vault the way a typed one is checked; a custom CA for a self-hosted vault; an environment variable for
the master password; more providers (1Password, HashiCorp Vault, Infisical).

## 2026-10-17 · Read any chat message aloud

Adeel: play any message in the chat through the TTS model. A speaker button on every question and answer reads it by
Expand Down
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,9 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
up in Settings, with certificates from Let's Encrypt or ZeroSSL that renew on their own, answering only for its own
domains and, if you choose, only to devices holding a client certificate; or reach it from anywhere through a
Cloudflare tunnel with nothing opened on your router. Either way, the S3 and WebDAV shares and the MQTT hub answer at
that same address once you turn them on. ([Encryption](docs/encryption.md), [access](docs/access.md),
that same address once you turn them on. Keep API keys and tokens in a Vaultwarden or Bitwarden vault: a secret
field in Settings can refer to an item there, and Lens reads it without anyone, agents included, seeing it.
([Encryption](docs/encryption.md), [secrets](docs/secrets.md), [access](docs/access.md),
[prompt injection](docs/assistant.md#third-party-content-and-prompt-injection), [HTTPS](docs/https.md),
[remote access](docs/remote-access.md); [checks](docs/requirements.md#stay-private-and-safe))
- **Use open standards.** RDF and Dublin Core for every record, OAuth for apps and OpenID Connect sign-in, MCP, MQTT,
Expand Down Expand Up @@ -261,6 +263,8 @@ Next, and not in a pull request yet; each links to where it is tracked:
activity ledger ([speech providers](docs/speech-providers.md)).
- **Storage:** uploads and assets' media in the file store, and a storage key to download
([storage](docs/storage.md)).
- **Secrets:** Settings › Secrets in the web app, with a picker for the vault's items, and references in sources'
credentials and MCP servers ([secrets](docs/secrets.md)).
- **Logging:** sending straight to Loki or another HTTP endpoint, and client certificates for TLS
([logging](docs/logging.md)).
- **Proof:** each release tested by upgrading an install of the last one, holding data
Expand Down
5 changes: 3 additions & 2 deletions 8BD0 docs/assistant.md
Original file line number Diff line number Diff line change
Expand Up @@ -904,15 +904,16 @@ switch shows the line to run on the machine instead. Agents use the same switche

Admins also get the server tools: `server_status` (what's set up and what's missing), `find_model_servers`,
`read_settings` and `change_settings` (the processing and AI sections, not the server's hosts, cookies or tokens),
`create_namespace`, the news feed tools: `news_feeds`, `subscribe_feed` (follow a feed into a namespace in one step,
`create_namespace`, `list_secrets` and `vault_items` (which secret fields refer to a [password manager](secrets.md),
and the items they can refer to, never a value), the news feed tools: `news_feeds`, `subscribe_feed` (follow a feed into a namespace in one step,
[News feeds](processing.md#news-feeds-rss-and-atom)) and `unsubscribe_feed`, and the people tools: `list_people`, `invite_person` (an account, optionally with a role in one
namespace; Lens emails the sign-in link when mail is set up) and `change_person` (a role in a namespace, or disabling
and enabling an account). Removing an account for good stays in the web app and the API. In an ordinary conversation
their changes are approval cards. In a **setup conversation**
(`POST /api/v1/chats` with `"kind": "setup"`, or **Finish with the assistant** in the setup wizard once a model is
connected) the admin has asked the assistant to set the server up, so it makes the changes itself and says what it
changed. Every change is audited with `assistant` in its detail, and all of them stay changeable in Settings.
Turning telemetry on or off, inviting people and changing their access always ask first, and so does every change once the answer has read text from outside
Turning telemetry on or off, changing the Secrets section, inviting people and changing their access always ask first, and so does every change once the answer has read text from outside
Lens ([third-party content](#third-party-content-and-prompt-injection)).

### Routine choices
Expand Down
1 change: 1 addition & 0 deletions docs/components.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ never fetch a speech or voice model.
| yt-dlp | `documents.video_urls` is on | the yt-dlp program, release 2026.08.19 (Unlicense), to download videos and podcasts imported by address |
| Chat model, embedding model | the LLM provider or embeddings server is Ollama | pulled on that server (`/api/pull`) |
| Code sandbox client | the [code sandbox](assistant.md#code-sandbox) is on (`sandbox.enabled`) | OpenShell's Python SDK (Apache-2.0) and gRPC, about 10 MB, to reach its gateway |
| Vaultwarden client | a Vaultwarden address is set (`secrets.vaultwarden_url`) | python-vaultwarden (Apache-2.0, about 10 MB with its crypto libraries), to read [secrets](secrets.md) from a Vaultwarden or Bitwarden vault |
| Outlook .msg emails | listed in `components.also` | extract-msg (GPL-3.0, so only when asked for) |

- Python packages go into `data_dir/python`, at the versions in `uv.lock`. Packages the image already has win.
Expand Down
3 changes: 2 additions & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,8 @@ search and search by meaning), reports and graph,
video, OCR, faces and objects, [NSFW classification](nsfw.md), workers, sign-in, components (and the edition, full or
lean), access and embedding (whether other devices on the network reach Lens, embed frame ancestors, text and transcript
upload limit, allowed hosts, trusted proxies, session length), remote access, email, chat rooms, notifications,
automations, telemetry, the Fedora repository, sensors, file shares, uploads, storage, documents, API keys, and IIIF and metadata. The API
automations, telemetry, the Fedora repository, sensors, file shares, uploads, storage, documents, API keys,
[secrets](secrets.md) (a password manager that secret fields can refer to), and IIIF and metadata. The API
refuses an allowed-host list that leaves out the address you are using.

The sections are in groups: AI and search, Speech and voices, Media and documents, Chat and email, Security and
Expand Down
5 changes: 4 additions & 1 deletion docs/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,9 @@ For an app that speaks OpenAI's API rather than MCP, the namespace can also answ
| `set_file_storage` | keep new files on this machine or a storage connection (folder, rclone crypt); checked before it's saved. Admins only, with a write-scope token |
| `services` | Lens's optional containers ([Optional containers](assistant.md#optional-containers)): the MCP runner, the browser and monitoring, whether each is on and answering, and how the machine's last change went. Admins only |
| `set_service` | start or stop one of them (`mcp`, `browser` or `monitoring`): the machine's upgrade agent does it. Admins only, with a write-scope token |
| `list_secrets` | every secret field in Settings: whether it's set, and for one that refers to a password manager ([Secrets](secrets.md)) its reference and whether it could be read; and the password managers there are. Never a value. Admins only |
| `list_vault_items` | the items of a password manager Lens may refer to (those in its folder): name, type, the fields that hold something and the reference to each; never their values. Admins only |
| `set_secret_reference` | point a secret field (`llm.api_key`, `slack.bot_token`, ...) at a password manager's item, like `vaultwarden://OpenAI/password`; checked against the vault before it's saved. Admins only, with a write-scope token |
| `picture_descriptions` | what describes the pages and shots of resources ([Descriptions](configuration.md#descriptions)): the LLM provider's model that can see images, or Florence-2 or Moondream on this server, its model, whether it's ready and why not, and the engines there are. Admins only |
| `set_picture_descriptions` | choose what describes pictures (`engine` `llm`, `florence-2` or `moondream`), the provider's `vision_model`, Florence-2's size (`florence_model`), which Moondream (`moondream_model`) and `describe_max`. Admins only, with a write-scope token |
| `mcp_servers` | the MCP servers the assistant calls ([Adding an MCP server](assistant.md#adding-an-mcp-server)): each one's address or command, the names of its headers and env (never their values), whether it's on, its tools and those switched off, and its page; and the ready-made ones (browser, github), whether each is added and what it needs. Admins only |
Expand Down Expand Up @@ -161,7 +164,7 @@ which records an undoable change (proposed by default), `undo_graph_change`, `ed
app (merges and deletes can be undone), `write_note`, which adds a note, `create_podcast`, which
queues an episode, `create_feed` and `publish_to_feed`, which make and fill a [published feed](feeds.md),
`write_spec`, which queues a spec, `set_nsfw` and `mark_nsfw`, which change [NSFW classification](nsfw.md) (owners or admins, and
editors), `set_namespace_integrations`, which turns a namespace's own MCP server or chat completions on or off (owners), `set_picture_descriptions`, which changes what describes pictures (admins), `set_file_storage`, which changes where new files are kept (admins), `add_mcp_server`, which adds an
editors), `set_namespace_integrations`, which turns a namespace's own MCP server or chat completions on or off (owners), `set_picture_descriptions`, which changes what describes pictures (admins), `set_file_storage`, which changes where new files are kept (admins), `set_secret_reference`, which points a secret field at a password manager's item (admins), `add_mcp_server`, which adds an
MCP server (admins), `subscribe_feed` and `unsubscribe_feed`, which follow and stop following news feeds (admins), and `set_monitoring`, `set_service`,
`set_sandbox_gateway` (the [code sandbox](assistant.md#code-sandbox)'s gateway) and `set_install_settings`, which have the machine change how Lens runs (admins).
Importing stays in the web app and the API.
Expand Down
1 change: 1 addition & 0 deletions docs/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,7 @@ request, to another check that proves the same thing; loosening a requirement or
| Passkeys instead of passwords | [test_passkeys.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_passkeys.py) `test_first_admin_with_a_passkey_and_no_password`, `test_sign_in_with_a_passkey` |
| A passkey at another of Lens's addresses (remote access, HTTPS) added from a signed-in one: Open signed in, then one fingerprint | [test_passkeys.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_passkeys.py) `test_taking_your_sign_in_to_another_address`, `test_the_addresses_lens_opens_at`; [passkeys-panel.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/passkeys-panel.test.tsx) `opens an address you have no passkey for signed in, to add one there`; [webauthn.test.ts](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/webauthn.test.ts) `points a failed sign-in to the address the passkey was made at` |
| Encryption at rest, with a key per namespace | [test_keyring.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_keyring.py) `test_each_namespace_gets_its_own_key_kept_only_wrapped`; [test_encryption.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_encryption.py) `test_a_new_archive_encrypts_and_still_plays_seeks_and_processes` |
| Secret fields that refer to a Vaultwarden or Bitwarden vault, read by Lens and never shown to anyone, agents included | [test_secrets.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_secrets.py) `test_a_secret_field_reads_its_value_from_the_vault`, `test_the_vault_is_read_again_and_what_vanishes_stops_being_used`, `test_the_vault_is_set_up_and_checked_in_the_app`, `test_agents_refer_to_secrets_and_never_read_them` |
| Roles per namespace: viewer, editor, owner, or custom roles made of permissions | [test_roles.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_roles.py) `test_builtin_roles_keep_their_meaning`, `test_custom_roles_allow_exactly_their_permissions`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `person-add` |
| Email invitations, and removing someone for good | [test_users.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_users.py) `test_invites_are_emailed_when_mail_is_set_up`, `test_admins_remove_people_for_good` |
| An audit log | [test_settings.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_settings.py) `test_settings_are_editable_and_secret`; [test_files.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_files.py) `test_change_a_file_and_the_audit_log` |
Expand Down
Loading
Loading
0