8000 GitHub - adeelahmad/lense at v0.15.0 · GitHub
[go: up one dir, main page]

Skip to content
adeelahmadPublic

About

Lens: intelligence for your agent. Safe, secure and cost-effective. Your live data as a cited knowledge graph for agents over MCP and REST, on your own hardware.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

667 Commits

Folders and files

 
NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Lens

Lens: intelligence for your agent

Safe, secure and cost-effective.

Connect your live data streams; Lens turns them into a cited knowledge graph your agent can reason over, through MCP, a REST API or its own assistant. It runs on your hardware with local models or any cloud model, private by default and cost-aware.

Install

curl -fsSL https://raw.githubusercontent.com/adeelahmad/lense/main/install.sh | sh

That one line is all: it opens an installer in your browser that checks the computer, then either installs with the defaults in one click (with a model server it finds on the computer) or asks how Lens should run (in Docker or a microsandbox microVM), the AI model and its key and any extras. Either way it installs what's missing (Docker included), starts Lens and opens its setup page (details). Run it again to update. On Windows, run irm https://raw.githubusercontent.com/adeelahmad/lense/main/install.ps1 | iex in PowerShell; or skip the terminal and double-click the launcher for your system from the latest release. On a Mac with Apple silicon, Lens.app from the same release installs Lens, runs it without Docker and keeps it in the menu bar (built, early: not signed yet). Deployment covers servers and Cloudron; there are also packages for Synology, QNAP and Proxmox. The checks say which of these CI proves.

To work on the code instead, make run builds and runs everything from a checkout, make setup-code prints the first-admin setup code, and make dev runs it with hot reload; see Get started.

Why Lens

AI doesn't lack intelligence. It lacks structure. Lens gives your agent eyes. Without structure, every prompt is a guess about who you are and what happened. Lens supplies the people and things involved, how they relate, what happened when, and the source of each fact, so your agent stops guessing.

Lens is one private AI hub for your whole life. Every source is a sensor: recordings, documents, email, calendars, web pages, chat rooms, devices and webhooks all flow in, and everything becomes memory you can trust, because every answer cites where it came from, down to the second of a recording or the line of a document. It runs on your own hardware, from a Raspberry Pi to the cloud, and you own every byte.

Lens is built first for agents and AI assistants. People talk to their agent in chat, by voice, or in Slack, WhatsApp and other chat rooms; the web app is for setup, settings, pipelines and workflows, review, debugging and the charts that show what is in the archive and what it cost.

Lens is also infrastructure to build on. An app built on Lens, such as an email responder or a customer manager, starts with real, structured context (who is involved, what was said, what was decided, with sources) instead of a blank prompt. Apps reach it the same way agents do: through the MCP server, the REST API and OAuth sign-in, limited to what their person may read.

What you can do with it

Everything below is built and merged, and each item links to the checks that prove it: tests, clicks journeys and measured numbers (requirements and their checks). Work that is still in a pull request is listed under In progress, and what comes next under Planned.

  • Connect your agents. Lens is an MCP server (/mcp): agents search, find assets by face, voice, entity, topic, type, content type or date in one call, find what was said about a day, classify an asset or a text with the decision model, read assets' text and notes, cite exact moments, find similar content, explore the graph with Cypher or SPARQL or browse it as folders and files, suggest topics and graph changes, edit and merge entities and topics (with undo), write notes and specs, and make podcasts. They sign in with OAuth and see only what their person may read; the account menu copies the MCP address in one click. A namespace can also have an MCP server of its own (/mcp/<namespace>) and answer OpenAI's chat completions API for any app, cited from that namespace alone, each off until its owner turns it on, with API keys that open that one namespace and nothing else. The same folders and files can be mounted over S3, WebDAV, NFS or SMB (S3 and WebDAV also at the web app's address, through remote access), so file managers and scripts read Lens too, and a file copied into any of them is imported where it was put. The web app runs on the same REST API, so an agent can do what a person can. (MCP, one namespace's MCP server, chat completions, API, the graph as files, authentication; checks)
  • Ask, and get cited answers. A chat assistant on every page, and by voice, searches your archive by words (and by meaning, once turned on), reads it, cites it and acts on it with tools: changes wait for your approval. Once an admin turns them on, it also searches the web and reads pages, cited like the archive, and runs Python and bash in a sandbox of each conversation's own. An admin's assistant searches and reads the mail, calendars and files connected to Lens (Gmail, Google Calendar, Google Drive, S3 and the rest), cites messages and events, and brings them in once you approve; where an admin allows it, it sends mail, adds events and writes files, each only after you read all of it and say yes. Regenerating an answer or editing a question makes a new version beside the old one, and nothing is lost. Each answer keeps a record of what produced it (the model, its tools, how long it took and what it cost), opened from its header. It reads answers aloud with a speech provider, macOS say or StyleTTS2 on this machine (opt-in), any message in one click. Long conversations are summarised before they fill the model's context, and /clear and /compact start afresh or summarise now. Conversations title themselves after their first answer, unless you name them. Each namespace can have its own assistant with its own name, instructions and memory, answering in Telegram and Slack (built in, with voice notes and files), in Discord, Matrix, WhatsApp and other chat rooms through Matterbridge, and by email at an address of its own that you email or forward things to. (Chat, web search, mail, calendars and files, code sandbox, answer records, chat rooms, email, speech providers; checks)
  • Extend the assistant. Add tools (a prompt, a web request, a graph drawn on the canvas, or sandboxed Python), skills, hooks and MCP servers, by code, on the canvas or just by asking. Remote MCP servers connect by address, and ones that run as a program (npx, uvx) run in a sandboxed container of their own, off until you turn it on. Add one with a form or by pasting another app's config, its tokens sealed and each of its tools switched on or off. Install skills and plugins from marketplaces in the open plugin format, or from a repository or an archive. (Extending the assistant, adding an MCP server, the MCP runner; checks)
  • Bring in everything. Watched folders on S3, Dropbox, Google Drive, OneDrive, SFTP, SMB, WebDAV or local disks (a local folder in four clicks); IMAP mail, Gmail and Google Workspace, iCal calendars, Google Calendar and RSS and Atom news feeds; a built-in MQTT broker, syslog and webhook streams; import webhooks for scanners and phone shortcuts. Google accounts connect with Connect with Google once Lens has a Google client, and Dropbox, OneDrive and Google Drive can sign in through rclone on the Lens server, with no app of your own to register. Upload audio, video, PDFs, Office files, photos, JSON and zip or tar archives, and, with an Apache Tika server (an opt-in extra), documents nothing else here can read; capture web pages one at a time, from a link list or a bookmarks file, by API or by asking the assistant, from private networks you allow in Settings too; and import videos and podcasts by their address (off by default). (Sensors, processing, Apache Tika, import webhooks; checks)
  • Clip from the browser. The Lens web clipper for Chrome, Edge, Firefox and Safari turns a page, a selection or a part you pick into Markdown, lets you look it over, and sends it to a namespace. New releases reach installed copies through the browser's store or a self-hosted update file. In Chrome and Edge, its side panel is a chat with the assistant, which can also use that browser on the sites you allow (open, read, click, type), asking before anything that sends, deletes, buys or posts. (Web clipper, the assistant in your browser; checks)
  • Understand it. Transcription, people recognised by voice (long recordings go to a speech provider in pieces that fit its limits), text on screen and in scans, faces (off by default), NSFW pictures flagged by a model on your own server (off by default, per namespace), entities and topics in a knowledge graph you can explore, query with Cypher or SPARQL, and see as it was at any version. Every entity has its own page at a wiki address (/wiki/<namespace>/<Name>), and entities, topics and people are edited and merged from their lists, the graph or an asset, with Undo instead of a confirm. Names get their real types (people, places, organisations, events, works) by rules, without a model, and a decision model types the ones the rules can't. Find what is most like a passage, an asset, a topic or an entity, in a Similar tab on every asset or by selecting text anywhere, and, with anytopdf, photos by what they show. Pages and shots are described in words by the model provider's vision model or, off by default, by Florence-2 or Moondream on your own server. Transcripts can be cleaned up before analysis (stutters, "um"s, spacing and, with a local model, punctuation put back), entities found by spaCy instead of the rules, and the dates people say read as days on the calendar, each off by default. (Graph, entity pages, similar content, graph history, topics, video, NSFW classification, long recordings, cleaning up transcripts, dates; checks)
  • Learn from it. Make a two-host podcast episode about what you pick, every claim cited and checked against its source, read aloud in two voices. Turn a voice session or a meeting into a technical spec, in a note linked to it, that keeps every decision, every no and every constraint in the speaker's own words, linked to when they said them, flags what was left to decide later and marks any quote the transcript doesn't hold. Keep notes beside every asset, entity and topic, written by you or the assistant, in a tree where notes are renamed, moved, copied and pinned (pins are your own) and each note shows the notes it's inside and the ones inside it, with today's note one click away, templates to start from, a search by their words, a download as Markdown, and Undo and Recently deleted instead of a confirm. Export an asset or a collection as an evidence PDF with everything Lens found about it, or the parts you pick. Publish a namespace's resources, episodes, timeline activities or text as a private RSS feed or a podcast feed that any feed reader or podcast app follows, and let a routine fill it on a schedule with what's new, or make an episode about the week's new resources. (Podcasts, specs from transcripts, notes, evidence PDFs, published feeds; checks)
  • Automate it. Pipelines and workflows on a canvas, routines on a schedule, batch runs over a collection, and notifications to webhooks, Slack, Discord and Matterbridge. Everything that happens in Lens (new assets, emails, calendar events, files, feed posts, what people say to Lens, sensor readings, notes, graph changes, runs, budget warnings) is kept as a stream of activities in W3C Activity Streams form, which agents read through the API, MCP and the assistant, and people review on the Timeline page (and Latest on Home), marking what they've seen. Automations act on that stream, on a schedule, at a time or when their own webhook is called, alone or with their triggers joined (all of them, a number of times, one that doesn't come in time, or a batch): they test conditions (text, patterns, numbers, lists, times, days, whether a value changed and whether something like it happened before in their namespaces) and let a decision model pick the branch, then tell you in your own chat app, ask the model or the agent, use a tool, call a web address (its credentials kept sealed), run a workflow, pipeline or routine, or change assets and notes. A Classify node in a workflow sorts an asset into labels, content types or topics, acting only on a sure answer, and an Entity filter node keeps the entities you want. Anything that reaches outside Lens or changes more than what set it off asks first, until you let it run on its own: on the Approvals page or, for a change inside Lens, with a yes in the chat it told you in; agents can decline an approval but never give one. Each automation has limits and budgets, and agents make, try and run them over MCP and in the assistant, or draft one from a sentence; the studio to draw them in the web app is planned. (Processing, notifications, automations, approvals, timeline; checks)
  • See what it costs. An activity ledger of every model call, with reports, charts and budgets that cap spending. Setup asks for one overall budget after a cloud model, with a soft limit that warns and a hard limit that stops paid calls. A meter in the top bar shows spend against the budget, and the Costs page breaks it down by day, model and what cost most. Routine choices go to small decision models, and the LLM only when it adds value. (Activity and costs, budgets, the meter; checks)
  • Scale out. Scaling groups add workers when work waits and let them go when it doesn't, by target, step and schedule rules, within min and max and a monthly budget, with every launch, join and hour of cost in the activity ledger. A group launches nothing until you approve its terms in Settings › Auto scaling; agents propose and manage groups over the API, MCP and the assistant, and never approve one. It runs more workers on this machine today; cloud, Proxmox, Linux and model hosts are planned. (Auto scaling; checks)
  • Keep it up to date and backed up. Lens checks for a new release once a day and tells admins by email or a notification. One click in Settings › Updates (or one API call) upgrades it through a small agent on the machine, showing each step, and rolls back when the new version doesn't start; nothing installs unless an admin says so. The same agent changes whether other devices reach Lens and which edition runs, from Settings. Settings › Backups takes, lists, downloads and restores backups of the database and Lens's files (a file that hasn't changed takes no new room), takes them on any schedule, and can copy each one, encrypted, to a storage connection. Before a version is released, CI upgrades the one before it, holding data, to it, and releases nothing when any of that data is lost. (Updates and backups, backups, the upgrade check; checks)
  • Watch it run. Structured logs from every process, read in Settings › Logs and optionally sent to a syslog server. OpenTelemetry traces and metrics, and a Prometheus /metrics endpoint, both off until you turn them on, and sending nothing anywhere else. On a Docker install, one switch in the web app (or one API call from an agent) starts Prometheus, node-exporter and Grafana next to Lens, already connected, with the Lens dashboard as Grafana's home. The Analytics page shows admins active people, churn, retention and which features get used, worked out on this machine; the same daily events can also go to a Google Analytics 4 property of your own (off unless you turn it on). (Logging, telemetry, Prometheus, usage analytics; checks)
  • Stay private and safe. Passkeys instead of passwords (one at Lens's remote address added from a signed-in one with Open signed in), encryption at rest with a key per namespace, roles per namespace (viewer, editor, owner, or custom roles made of permissions), email invitations, removing someone for good, and an audit log. Prompt injection meets several layers: what the assistant reads (recordings, documents, web pages, tool results) reaches the model marked as data, not instructions, and text that reads like instructions to an AI is flagged (by free patterns, or an optional classifier model that runs on your machine and is fetched only when you pick it). Once an answer has read such text, sending anything out, or the browser opening or typing what the assistant wrote, waits for your yes. Chat-room answers have links they built themselves broken up, agents over MCP get the archive as data, and what Lens caught goes into the audit log. Serve Lens over HTTPS, set up in Settings, with certificates from Let's Encrypt or ZeroSSL that renew on their own, answering only for its own domains and, if you choose, only to devices holding a client certificate; or reach it from anywhere through a Cloudflare tunnel with nothing opened on your router. Either way, the S3 and WebDAV shares and the MQTT hub answer at that same address once you turn them on. (Encryption, access, prompt injection, HTTPS, remote access; checks)
  • Use open standards. RDF and Dublin Core for every record, OAuth for apps and OpenID Connect sign-in, MCP, MQTT, IIIF for publishing, OpenTelemetry and Prometheus for monitoring, and an optional Fedora mirror. (Linked data, IIIF, Fedora; checks)

Light by default

Lens runs on a small machine. Heavy parts (local speech models, a local LLM, document conversion, an Apache Tika server, the NSFW model, the MCP runner, Fedora) stay off or download only when you turn them on, and any of them can be swapped for a cloud provider instead. Setup asks only what has no default: the model, and a budget for a cloud model. Picking a cloud model provider fills in its address and a model, so only its key is left to paste. Every setting lives in the web app, which works on a phone too: settings are in named groups, a search finds any one by name, and the options most people leave as they are stay folded. A thin bar along the top of the web app shows a page or its data is still loading. (Components, speech providers, local models, cloud model providers; checks)

In progress

These are open pull requests, not yet part of Lens:

  • The assistant answering questions about Lens itself, from its docs and this server (#400).
  • Chat: a conversation exported as Markdown, PDF or JSON (#474).
  • Pictures: low-quality images enhanced before they're read (#479).
  • Web clipper: signing in, or using your API token, not only an import webhook's (#406).
  • Automations: more operators for conditions (#434).
  • Nodes: Lens's own private network between the machines it runs on, a Nebula mesh it issues the certificates for (#546).

Planned

Next, and not in a pull request yet; each links to where it is tracked:

  • The automations studio: drawing automations in the web app (automations).
  • Timeline: what each item caused, and Approve and Decline on waiting approvals (timeline).
  • Notes: nesting a note under its project or area without a click, pages in the graph, and wiki addresses for topics, recordings and the other things that have pages (notes).
  • Local models: GPU builds of llama.cpp fetched by Lens, and local embedding models (local models).
  • Auto scaling: providers for AWS, DigitalOcean, Hetzner, Proxmox and Linux hosts over SSH, joining over the Nebula mesh, model hosts (RunPod, Lambda, Hugging Face, SageMaker, Bedrock), and making providers and groups in Settings (auto scaling).
  • Speech providers: their summaries, chapters and entities as optional imports, and per-minute prices in the activity ledger (speech providers).
  • Storage: uploads and assets' media in the file store, and a storage key to download (storage).
  • Logging: sending straight to Loki or another HTTP endpoint, and client certificates for TLS (logging).
  • Proof: each release tested by upgrading an install of the last one, holding data (#482), and the scorecard's install-to-first-answer and Raspberry Pi numbers, not measured yet (#481).

Stack

fastapi_backend/ FastAPI API (/api/v1), MCP server, the processing engine, and the lens CLI and workers
nextjs-frontend/ Next.js web app with a typed client generated from the API's OpenAPI schema
extensions/browser/ the web clipper browser extension
installer/ the web installer and the double-click launchers
SurrealDB documents, graph edges, the job queue and graph history; word search in a small SQLite index beside it

Documentation

The guides are on the docs site and in the docs folder:

Security, threat model, changelog.

License

Lens is open source under the MIT License. Optional extras that you install yourself may carry their own licenses (for example extract-msg, GPL-3.0, for Outlook .msg files); none is part of the default install.

Built on the Next.js FastAPI Template (MIT).

About

Lens: intelligence for your agent. Safe, secure and cost-effective. Your live data as a cited knowledge graph for agents over MCP and REST, on your own hardware.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages

0