8000 feat(settings): cloud model presets in Settings and the setup wizard by adeelahmad · Pull Request #334 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,12 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
essentials: the folder, port and network under **Advanced**, the optional extras under **More options** on the
review.

- **Cloud model presets.** Settings → LLM provider and the setup wizard offer OpenAI, Anthropic, Google Gemini,
OpenRouter, Groq, Mistral and DeepSeek next to the model servers found nearby: one click fills in the address and a
model and puts the cursor in the API key field, with a link to where the provider hands out keys, so switching to a
cloud model is a click and a paste. Agents get the same list from `GET /api/v1/setup/llm/providers` and the
assistant's `find_model_servers` tool ([Cloud model providers](docs/configuration.md#cloud-model-providers)).

- **The graph as files.** Graph nodes are folders, nested along its links: collections inside collections, topics
inside their broader topics, entities by type. Each folder holds its assets' files, with a `.json` sidecar of
metadata and a `.txt` of the text; everyone sees what the web app shows them. Browse and read it through the API
Expand Down
2 changes: 2 additions & 0 deletions ci/clicks/journeys.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,8 @@ export const JOURNEYS = [
await meter.click(page.getByRole("main").getByRole("button", { name: "New note" }).first(), "New note");
await page.waitForURL(/\/notes\/\d+/);
const title = page.getByRole("textbox", { name: "Title" });
// A person starts typing once the note has loaded and its title has the cursor (a click is counted if it never does)
await page.waitForFunction((el) => el === document.activeElement, await title.elementHandle(), { timeout: 5000 }).catch(() => {});
await meter.replace(title, "Pi setup", "title");
const editor = page.locator("main affine-page-root .inline-editor").first();
await editor.waitFor(); // the page has loaded its editor
Expand Down
27 changes: 25 additions & 2 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ A fresh install (no accounts when the API first starts) walks its first admin th
2. **Namespace**: the first one, with its knowledge graph shared or isolated. Already done when archive.yaml or
`LENS_NAMESPACE` names namespaces, or an install script made one.
3. **Model provider**: an OpenAI-compatible server (OpenAI, Ollama, llama.cpp, LM Studio, vLLM), its model and key,
with a test.
with a test. Servers running nearby and [cloud providers](#cloud-model-providers) are a click.
4. **Storage**: where data lives (shown; set in archive.yaml and `SURREAL_URL`), where Lens keeps its own files (this
machine or a storage connection, [Storage](storage.md)), the largest upload, and optionally a folder inside
`sources.local_roots` to watch.
Expand All @@ -62,7 +62,8 @@ A fresh install (no accounts when the API first starts) walks its first admin th
Every step can be skipped, and the whole wizard too; all of it stays in Settings. Values from the environment win and
show locked. Installs that already had accounts never see the wizard. The API side is `GET /api/v1/setup`,
`POST /api/v1/setup/namespace`, `PUT /api/v1/setup/llm`, `GET /api/v1/setup/llm/detect` (model servers running on
this machine or the Docker host, with their models), `PUT /api/v1/setup/storage`, `PUT /api/v1/setup/oauth`,
this machine or the Docker host, with their models), `GET /api/v1/setup/llm/providers` (the
[cloud providers](#cloud-model-providers)), `PUT /api/v1/setup/storage`, `PUT /api/v1/setup/oauth`,
`PUT /api/v1/setup/telemetry` and `POST /api/v1/setup/finish`
(admins); `GET /api/v1/auth/status` says whether it is still pending (`wizard_pending`).

Expand Down Expand Up @@ -145,6 +146,28 @@ Changing the model drops the stored vectors (another model's can't be compared)
doesn't have the model), indexing jobs skip for ten minutes rather than each waiting on it, and the hourly routine
waits too; **Test** in Settings → Search, **Index now**, or `lens embed` try again at once.

## Cloud model providers

The setup wizard's model provider step and Settings → LLM provider offer the model servers running nearby (Ollama, LM
Studio, llama.cpp, vLLM, LocalAI) and these cloud providers, each through its OpenAI-compatible API. Picking one fills
in the address and a model, and a cloud provider's API key field takes the focus, with a link to where the provider
hands out keys: paste the key and save. The model stays editable, with the provider's other models suggested.

| Provider | Address | Model filled in |
|---|---|---|
| OpenAI | `https://api.openai.com/v1` | `gpt-5-mini` |
| Anthropic | `https://api.anthropic.com/v1` | `claude-opus-5-5` |
| Google Gemini | `https://generativelanguage.googleapis.com/v1beta/openai` | `gemini-flash-latest` |
| OpenRouter | `https://openrouter.ai/api/v1` | `openrouter/auto` |
| Groq | `https://api.groq.com/openai/v1` | `llama-3.3-70b-versatile` |
| Mistral | `https://api.mistral.ai/v1` | `mistral-small-latest` |
| DeepSeek | `https://api.deepseek.com/v1` | `deepseek-chat` |

The list is `GET /api/v1/setup/llm/providers` (admins), and the assistant's `find_model_servers` tool returns it with
the servers it finds, so an agent setting Lens up uses the same addresses. Any other OpenAI-compatible service works
by typing its address. Picking a provider changes nothing until you save; from then on, what Lens asks its model
(summaries, chat, descriptions, with the text they need) goes to that provider.

## Chat models

People can choose the model a conversation uses, and ask a question again with another (Retry with another model).
Expand Down
8 changes: 8 additions & 0 deletions fastapi_backend/app/api/v1/routes/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
from app.api.deps import AdminReader, AdminWriter, Db
from app.domain import auth, setup
from app.schemas.setup import (
CloudModelProvider,
LocalModelServer,
SetupBudget,
SetupFinish,
Expand Down Expand Up @@ -69,6 +70,13 @@ def detect_llm(user: AdminWriter) -> list[LocalModelServer]:
return [LocalModelServer(**s) for s in setup.detect_llm()]


@router.get("/llm/providers")
def llm_providers(user: AdminReader) -> list[CloudModelProvider]:
"""Cloud providers with an OpenAI-compatible API (OpenAI, Anthropic, Gemini, ...), each with its address and a
model, so picking one leaves only the API key to paste."""
return [CloudModelProvider(**p) for p in setup.CLOUD_PROVIDERS]


@router.put("/storage")
def save_storage(body: SetupStorage, user: AdminWriter, request: Request, db: Db) -> SetupSaved:
"""The upload limit, and optionally a folder on this machine (inside sources.local_roots) to watch."""
Expand Down
6 changes: 4 additions & 2 deletions fastapi_backend/app/domain/ops_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,9 @@
),
(
"find_model_servers",
"Model servers running on this machine or the Docker host (Ollama, LM Studio, llama.cpp, vLLM, LocalAI), with their models.",
"Model servers running on this machine or the Docker host (Ollama, LM Studio, llama.cpp, vLLM, LocalAI), with their "
"models, and the cloud providers Lens knows (OpenAI, Anthropic, Gemini, ...) with their address and a model: "
"a cloud one also needs the admin's API key.",
{},
[],
False,
Expand Down Expand Up @@ -244,7 +246,7 @@ def t_server_status(self):

def t_find_model_servers(self):
found = setup.detect_llm()
return {"servers": found}, f"Looked for model servers nearby: {len(found)} found"
return {"servers": found, "cloud": list(setup.CLOUD_PROVIDERS)}, f"Looked for model servers nearby: {len(found)} found"

def t_read_settings(self, section):
if section not in SECTIONS:
Expand Down
55 changes: 55 additions & 0 deletions fastapi_backend/app/domain/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,61 @@ def save_storage(db, cfg, max_upload_mb=None, folder=None, namespace=None, user=
return sources.create_watch(db, cfg, src, folder, sid, user)


# Cloud providers with an OpenAI-compatible API: picking one fills in its address and a model, so only the key is
# pasted. `models` are suggestions (the model stays editable); `key_url` is where the provider hands out keys.
CLOUD_PROVIDERS = (
{
"name": "OpenAI",
"base_url": "https://api.openai.com/v1",
"model": "gpt-5-mini",
"models": ["gpt-5-mini", "gpt-5", "gpt-5-nano", "gpt-4.1-mini"],
"key_url": "https://platform.openai.com/api-keys",
},
{
"name": "Anthropic",
"base_url": "https://api.anthropic.com/v1",
"model": "claude-opus-5-5",
"models": ["claude-opus-5-5", "claude-sonnet-5-5", "claude-haiku-5-5"],
"key_url": "https://console.anthropic.com/settings/keys",
},
{
"name": "Google Gemini",
"base_url": "https://generativelanguage.googleapis.com/v1beta/openai",
"model": "gemini-flash-latest",
"models": ["gemini-flash-latest", "gemini-2.5-pro", "gemini-2.5-flash-lite"],
"key_url": "https://aistudio.google.com/apikey",
},
{
"name": "OpenRouter",
"base_url": "https://openrouter.ai/api/v1",
"model": "openrouter/auto",
"models": ["openrouter/auto", "openai/gpt-5-mini", "google/gemini-2.5-flash", "meta-llama/llama-3.3-70b-instruct"],
"key_url": "https://openrouter.ai/keys",
},
{
"name": "Groq",
"base_url": "https://api.groq.com/openai/v1",
"model": "llama-3.3-70b-versatile",
"models": ["llama-3.3-70b-versatile", "llama-3.1-8b-instant", "openai/gpt-oss-120b"],
"key_url": "https://console.groq.com/keys",
},
{
"name": "Mistral",
"base_url": "https://api.mistral.ai/v1",
"model": "mistral-small-latest",
"models": ["mistral-small-latest", "mistral-medium-latest", "mistral-large-latest"],
"key_url": "https://console.mistral.ai/api-keys",
},
{
"name": "DeepSeek",
"base_url": "https://api.deepseek.com/v1",
"model": "deepseek-chat",
"models": ["deepseek-chat", "deepseek-reasoner"],
"key_url": "https://platform.deepseek.com/api_keys",
},
)


# Model servers people run themselves, on the ports they listen on by default. Looked for on this machine, on the
# Docker host (host.docker.internal; the compose files map it on Linux too) and as a compose service named ollama.
LOCAL_SERVERS = (("Ollama", 11434), ("LM Studio", 1234), ("llama.cpp", 8080), ("vLLM", 8000), ("LocalAI", 8081))
Expand Down
10 changes: 10 additions & 0 deletions fastapi_backend/app/schemas/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,16 @@ class LocalModelServer(ResponseModel):
suggested: str


class CloudModelProvider(ResponseModel):
name: str
base_url: str
# the model picking it fills in, then other suggestions
model: str
models: list[str]
# where the provider hands out API keys
key_url: str


class SetupStorage(RequestModel):
max_upload_mb: int | None = None
# a folder inside sources.local_roots to watch into `namespace`
Expand Down
17 changes: 17 additions & 0 deletions fastapi_backend/tests/api/test_setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -286,6 +286,23 @@ def test_finds_model_servers_running_nearby(fresh, monkeypatch):
srv.shutdown()


def test_offers_cloud_model_providers(fresh):
"""Picking a cloud provider fills in its address and a model; only the API key is left to paste."""
app, client = fresh()
make_user(app.state.db, "ada@x.io", "admin password 1", admin=True)
make_user(app.state.db, "ed@x.io", "editor password 1")
app_settings.save(app.state.db, app.state.archive.base, "auth", {"passwords": True}) # made after startup
got = client.get("/api/v1/setup/llm/providers", headers=login(client, "ada@x.io", "admin password 1")).json()
names = [p["name"] for p in got]
assert names[:2] == ["OpenAI", "Anthropic"] and len(names) == len(set(names))
openai = got[0]
assert openai["base_url"] == "https://api.openai.com/v1" and openai["model"] == openai["models"][0]
for p in got:
assert p["base_url"].startswith("https://") and not p["base_url"].endswith("/")
assert p["model"] in p["models"] and p["key_url"].startswith("https://")
assert client.get("/api/v1/setup/llm/providers", headers=login(client, "ed@x.io", "editor password 1")).status_code == 403


def test_log_prints_a_setup_link(fresh, caplog, monkeypatch):
"""The setup code comes with a link that fills it in, so nobody has to copy the code across."""
monkeypatch.setenv("LENS_SETUP_CODE", "a b&c")
Expand Down
2 changes: 2 additions & 0 deletions fastapi_backend/tests/api/test_setup_assistant.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ def test_a_setup_conversation_sets_the_server_up(app, db, cfg, folder, new_clien
]
status = json.loads(llm.seen[-1]["messages"][3]["content"]) # what server_status told the model
assert status["namespaces"] == ["calls", "pods"] and "missing" in status
found = json.loads(llm.seen[-1]["messages"][4]["content"]) # and find_model_servers: nearby and cloud providers
assert found["servers"][0]["kind"] == "Ollama" and found["cloud"][0]["base_url"] == "https://api.openai.com/v1"
asked = llm.seen[-6]["messages"]
assert "setting up this Lens server" in asked[0]["content"]
assert f"attachment id {up['id']}" in asked[-1]["content"] # the model hears about the file
Expand Down
108 changes: 108 additions & 0 deletions nextjs-frontend/__tests__/settings-llm-presets.test.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { fireEvent, render, screen, waitFor } from "@testing-library/react";
import "@testing-library/jest-dom";

import { Admin, Setup } from "@/app/openapi-client";
import { SettingsSection } from "@/components/settings/section";
import type { SettingsView } from "@/components/settings/model";
import { TooltipProvider } from "@/components/ui/tooltip";

jest.mock("@/app/openapi-client", () => ({
Admin: { updateSettings: jest.fn(), testLlm: jest.fn() },
Setup: { detectLlm: jest.fn(), llmProviders: jest.fn() },
Metadata: {},
}));
jest.mock("next-auth/react", () => ({ useSession: () => ({ data: { accessToken: "t" } }) }));
jest.mock("next/navigation", () => ({ useRouter: () => ({ push: jest.fn() }) }));
jest.mock("@/lib/hooks/session", () => ({ useArchive: () => ({ namespaces: [] }) }));
jest.mock("@/components/ui/toast", () => ({ useToast: () => jest.fn() }));

const ok = (data: unknown) => Promise.resolve({ data, response: { ok: true, status: 200 } });
const m = (f: unknown) => f as jest.Mock;

const view = (keySet: boolean, locked: string[] = []) =>
({
llm: {
values: {
base_url: "http://localhost:11434/v1",
model: "llama3.1",
chat_models: [],
vision_model: null,
describe_max: 20,
api_key: { secret: true, set: keySet },
api_key_env: null,
max_chars: 24000,
timeout: 120,
},
locked,
},
}) as unknown as SettingsView;

function show(v: SettingsView) {
const qc = new QueryClient({ defaultOptions: { queries: { retry: false }, mutations: { retry: false } } });
return render(
<QueryClientProvider client={qc}>
<TooltipProvider>
<SettingsSection id="llm" view={v} =&g D945 t; undefined} />
</TooltipProvider>
</QueryClientProvider>,
);
}

describe("Settings › LLM provider presets", () => {
beforeEach(() => {
m(Admin.updateSettings)
.mockReset()
.mockImplementation(() => ok({ ok: true }));
m(Setup.detectLlm)
.mockReset()
.mockImplementation(() =>
ok([{ kind: "Ollama", base_url: "http://localhost:11434/v1", models: ["llama3.1"], suggested: "llama3.1" }]),
);
m(Setup.llmProviders)
.mockReset()
.mockImplementation(() =>
ok([
{
name: "OpenAI",
base_url: "https://api.openai.com/v1",
model: "gpt-5-mini",
models: ["gpt-5-mini", "gpt-5"],
key_url: "https://platform.openai.com/api-keys",
},
]),
);
});

it("switches to a cloud provider with one click and a pasted key", async () => {
show(view(false));
expect(await screen.findByRole("button", { name: /Ollama/ })).toHaveAttribute("aria-pressed", "true");
fireEvent.click(await screen.findByRole("button", { name: "OpenAI" }));
expect(screen.getByLabelText("Base URL (OpenAI-compatible)")).toHaveValue("https://api.openai.com/v1");
expect(screen.getByLabelText("Model")).toHaveValue("gpt-5-mini");
const key = screen.getByLabelText("API key");
await waitFor(() => expect(key).toHaveFocus()); // ready for the paste
expect(screen.getByRole("link", { name: "Get a key from OpenAI" })).toBeInTheDocument();
fireEvent.change(key, { target: { value: "sk-test" } });
fireEvent.click(screen.getByRole("button", { name: "Save" }));
await waitFor(() => expect(Admin.updateSettings).toHaveBeenCalledTimes(1));
expect(m(Admin.updateSettings).mock.calls[0][0]).toMatchObject({
path: { section: "llm" },
body: { base_url: "https://api.openai.com/v1", model: "gpt-5-mini", api_key: "sk-test" },
});
});

it("opens the saved key for replacing when the provider changes", async () => {
show(view(true));
expect(screen.getByText(/^Set/)).toBeInTheDocument();
fireEvent.click(await screen.findByRole("button", { name: "OpenAI" }));
await waitFor(() => expect(screen.getByPlaceholderText("Paste the new key")).toHaveFocus());
});

it("offers no presets when .env sets the address", async () => {
show(view(false, ["base_url"]));
expect(screen.getByLabelText("Base URL (OpenAI-compatible)")).toHaveAttribute("readonly");
expect(screen.queryByRole("button", { name: "OpenAI" })).not.toBeInTheDocument();
expect(Setup.llmProviders).not.toHaveBeenCalled();
});
});
Loading
Loading
0