8000 feat(telemetry): opt-in OpenTelemetry traces and metrics by adeelahmad · Pull Request #40 · adeelahmad/lense · GitHub
[go: up one dir, main page]
Skip to content

feat(telemetry): opt-in OpenTelemetry traces and metrics - #40

Merged
adeelahmad merged 3 commits into
mainfrom
claude/telemetry-mekabp
Oct 2, 2026
Merged

feat(telemetry): opt-in OpenTelemetry traces and metrics#40
adeelahmad merged 3 commits into
mainfrom
claude/telemetry-mekabp

Conversation

@adeelahmad
@adeelahmad adeelahmad commented Oct 2, 2026
Copy link
Copy Markdown
Owner

Requested by Adeel · project thread

Description

Before: Lens had no telemetry at all, so there was no way to see slow requests, step timings, job failures, or what model calls cost.

After: an admin can opt in under Settings → Telemetry, in the setup wizard's new last step, or with LENS_TELEMETRY=on plus LENS_TELEMETRY_ENDPOINT in .env. Lens then sends OpenTelemetry traces and metrics over OTLP/HTTP to that endpoint only. It is off by default, has no built-in destination, and LENS_TELEMETRY=off keeps it off whatever Settings say.

  • Traces: API requests (named by route template), jobs and each step, routines and their actions, workflow runs, and model calls (GenAI conventions: model, server, tokens, estimated cost).
  • Metrics: request, step and model-call durations, job, routine and workflow outcomes, token usage, and lens.llm.cost in USD. Cost comes from per-model prices set in Settings; the AI assistant's prices count for the configured model.
  • Nothing personal is sent: no paths, query strings, prompts, answers, transcript text, file names, titles, namespace or speaker names, emails, client IPs, host names or error messages. Only ids, step types, model names, counts and timings go out. A test checks the raw export bytes for this.
  • The Settings section says plainly whether it is on and where it sends, shows the last export results, and has a Send a test span button. Headers (for collector auth) are stored sealed. The API and every lens worker / lens watch pick up changes without a restart, and turning it off never blocks a request on a slow collector.

How: a new app/domain/telemetry.py imports the OpenTelemetry SDK only while telemetry is on and uses its own providers, not the global ones. It is applied whenever settings.Settings.current() rebuilds. A pure ASGI middleware sits innermost so it can read the matched route. The llm.py and analyze._llm calls are wrapped. Docs are in docs/telemetry.md, including a sample collector setup.

Motivation and Context

Telemetry backlog item. The feature plan requires telemetry to be opt-in only.

Screenshots (if appropriate):

Checked in a browser against a local fake collector: the wizard's Telemetry step (off by default, endpoint and test span when on) and Settings → Telemetry showing "On: sending to …".

Steps to reproduce (if appropriate):

  1. Run a collector, for example the config in docs/telemetry.md.
  2. Go to Settings → Telemetry, set the endpoint, click Send a test span, then turn on Send telemetry.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change; ! in the title)

Checklist:

  • The title is a Conventional Commit.
  • My code follows the code style of this project (make hooks checks it on commit).
  • I added or updated tests. tests/api/test_telemetry.py runs against an in-process OTLP collector; the setup wizard and settings model have Jest tests. The full backend suite passes on embedded SurrealDB and on a SurrealDB 3.2.4 server, except test_office_and_text_files_become_documents, which also fails on main in this sandbox because of its LibreOffice.
  • My change requires documentation updates, and I have updated docs/ accordingly.
  • Users need more than the title to understand it, so I wrote it under ## Unreleased in CHANGELOG.md.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CFYD76r8LUcnfTYKBwRfPn

Generated by Claude Code

@claude
feat(telemetry): opt-in OpenTelemetry traces and metrics
25ca07a 
Off by default and only on when an admin turns it on (Settings, the setup
wizard's new last step, or LENS_TELEMETRY=on in .env; LENS_TELEMETRY=off
keeps it off). Sent only to the OTLP/HTTP endpoint that is set; there is
no built-in destination.

- Traces: API requests by route template, jobs and each step, routines
  and their actions, workflow runs, and model calls (GenAI conventions)
- Metrics: request, step and model-call durations, job/routine/workflow
  outcomes, tokens, and estimated cost from per-model prices
- No personal content: no paths, queries, prompts, answers, transcript
  text, file names, titles, people, hosts or error messages
- Settings shows on/off, where it sends and the last exports, and can
  send a test span; every process follows changes without a restart

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CFYD76r8LUcnfTYKBwRfPn
@adeelahmad adeelahmad self-assigned this Oct 2, 2026
@adeelahmad Claude
adeelahmad commented Oct 2, 2026
Copy link
Copy Markdown
Owner Author

CI is red here, but none of the jobs ran. FastAPI (embedded SurrealDB), Next.js, pre-commit and Conventional Commit title all failed within about 5 seconds, with no runner assigned and no logs (job logs return 404).

Every other branch shows the same thing right now, including dependabot PR #36. So this is the account's GitHub-hosted Actions minutes, not this PR's code.

The fix in progress is #27, which moves the jobs to the self-hosted runner. It is still a draft, another thread owns it, and it depends on that runner being online. I haven't ported it into this PR because that would tie this PR to an unmerged runner setup. Once #27 merges, I'll bring main in and let CI run here.

Checked locally on this commit:

  • backend: ruff, ruff format, mypy and the OpenAPI schema check are clean
  • backend tests: the full suite passes on embedded SurrealDB and on a SurrealDB 3.2.4 server, except test_office_and_text_files_become_documents, which also fails on main in this sandbox because of its LibreOffice
  • frontend: tsc, eslint, prettier and all 56 Jest suites (537 tests) pass

Generated by Claude Code

claude added 2 commits October 2, 2026 13:59
@claude
Merge origin/main into telemetry-mekabp
929eb3b 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SNbRrpHTCqLewRKdkmd45X
@claude
style(openapi): format the generated client with prettier 3.9
fb510c7 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SNbRrpHTCqLewRKdkmd45X
@adeelahmad
adeelahmad merged commit 8547add into main Oct 2, 2026
0 of 9 checks passed
@adeelahmad adeelahmad mentioned this pull request Oct 2, 2026
Merged
4 of 8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@adeelahmad adeelahmad

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@adeelahmad @claude
0