8000 Releases · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content

Releases

v0.10.0

Choose a tag to compare

@github-actions github-actions released this 15 Oct 11:07
  • The assistant always answers. When a question used up its tool steps (ai.max_steps), the assistant used to stop
    with "I ran out of steps before finishing". Now it's asked to answer from what it found, with the tools off, and says
    what it didn't get to check; a reply with nothing in it is asked again the same way. It also knows its step budget up
    front, so it looks things up in fewer rounds, and the same lookup twice in one answer points it at the result it
    already has instead of running it again (The assistant's tools).
  • Backups hold Lens's files too. Each backup now also keeps the files under data_dir (uploads, objects, media,
    frames, renditions, web pages, reports, exports, podcasts, certificates), and restoring one puts them back. Files
    that haven't changed since the previous backup are hard links, so a backup takes only the room of what changed; the
    first one takes as much room as the files. Downloads and copies on a storage connection carry them as well. Older
    backups show database only (Backups).
  • Analytics. A new Analytics page (admins) shows active people per day, week and month, stickiness, churn rate,
    retention by sign-up week and feature use, worked out on the server from the activity ledger and sign-ins. Each day
    Lens also keeps a few counted events (install, daily active, feature used, upgrade, inactive, return). They can
    optionally go to your own Google Analytics 4 property too; that is off by default and is set in Settings → Usage
    analytics or the setup wizard (Usage analytics).
  • Merge entities in fewer clicks, and undo instead of confirm. On the Entities page, tick rows and the bar at the
    bottom merges them into one (the one on the fixed list, else the most mentioned) or gives them one type. An entity's
    side panel has Merge, a find box where Enter merges the entity picked into it, and Maybe the same, its likely
    doubles with Merge and Not the same. Renaming an entity to another's name offers to merge them. Merges, type
    changes and hiding say what happened with Undo (Entities).
  • Edit and merge from the graph. In the graph explorer, an entity's or topic's panel has Edit, which opens the
    same editor as its own page, and an entity's has Merge and Maybe the same. The node menu has Edit… and
    Merge into another…, which merges the node into the one clicked next (entities and speakers of one namespace;
    entities of two namespaces are linked as the same thing), each with Undo (The graph).
  • Undo topic merges and deletes. Merging or deleting a topic happens at once, with Undo on its notice instead
    of a confirm step: the topics come back with their labels, recordings, broader and related topics and pages. A
    topic's drawer has a find box to merge another topic in, and topics merge on the graph canvas too. The API answers
    a merge or delete with an undo id (POST /api/v1/topics/changes/{id}/undo) and lists recent ones
    (Topics).
  • Edit entities from a recording. In a recording's Entities tab, the pencil next to an entity opens its side
    panel, so editors rename, retype, hide or merge it without leaving the recording
    (Entities).
  • Agents edit and merge entities and topics. The MCP server has edit_entity, edit_topic, merge_topics,
    delete_topic, undo_topic_change and undo_graph_change (with a write-scope token and editor access), and the
    assistant proposes topic merges, renames, definitions and deletes for the person to approve
    (propose_topic_change) (MCP, Topics).

Features

  • settings: group the sections and find a setting by name (#390)
  • settings: fold the options most people leave as they are (#392)
  • nav: name the navigation's groups (#394)

v0.9.0

Choose a tag to compare

@github-actions github-actions released this 15 Oct 06:15
  • Evidence PDFs hold everything Lens found. After each resource's own file come pages of what Lens knows about
    it: its details, metadata and custom fields, tags, topics, the summary and analysis models wrote, its sections,
    speakers, emotion and sounds (overall, by speaker, and marked on each transcript line, such as "(Happy, Laughter)"),
    entities, keywords, the faces, objects, descriptions and text on screen seen in it, and the places and dates found
    in it. Notes, highlights and comments people shared go in when picked. Export › Choose what goes in it… (and the
    button beside a collection's Evidence PDF) picks the parts, and include does the same in the API
    (Evidence PDFs).

  • Roles with permissions. Viewer, editor and owner are now sets of permissions (read, edit, import, process,
    curate, share, publish, remove, members, manage), and admins make custom roles from them in Admin → Roles, given per
    namespace like the built-in ones. Existing access is unchanged. Someone who manages members but isn't an admin gives
    only roles they hold all of (Access).

  • Invite and remove people. Making an account in Admin → People now emails the sign-in link as an invitation when
    mail is set up, and the link is still shown to share yourself. Remove for good ends every way someone signs in,
    takes their roles and their private chats, notes and views, and keeps what they shared under their name; their email
    can be invited again (Authentication). The assistant gets list_people,
    invite_person and change_person, which always wait for an admin's approval.

  • Reachable from the network, and the edition, changed in Settings. Settings › Access & embedding has Reachable
    from other devices on this network
    , and Settings › Components the edition: Full (LibreOffice and Chromium, to read
    Office files, web pages and emails) or Lean. Changing either says what will happen and applies with one button: the
    machine's upgrade agent runs the install again for the version running, and Lens restarts. Before, the install line
    had to run again with LENS_PUBLIC or LENS_TARGET; without an agent, Settings shows that line. Agents use
    GET/PUT /api/v1/admin/install and the MCP tools install_settings and set_install_settings
    (Updates).

  • NSFW classification, with a model on your server. Turn it on in Settings → NSFW for every namespace, or on a
    namespace's page for that one (off by default). The new nsfw step scores a video's sampled frames, a document's
    pages or an image with an Apache-2.0 model (fetched on first use, about 105 MB with ONNX Runtime) and flags the
    resource when a picture scores at least the threshold. Flagged resources are blurred until someone chooses to see
    them, left out of the Library, or shown with a label; editors can mark one NSFW or not by hand, and Classify
    existing
    classifies what was there before. Agents use it through the API and MCP (nsfw_settings, set_nsfw,
    mark_nsfw) (NSFW classification).

  • Apache Tika. Lens can use an Apache Tika server (--profile tika in the compose files, or one of your own, set
    in Settings → Documents). Documents nothing here can make into a PDF, such as Word files on a lean server or a
    Raspberry Pi, are read by Tika as text, and every document's own title, authors, language and dates are read as
    Dublin Core terms. Off until a Tika server is set (Configuration).

  • New pipelines, workflows, templates and API tokens come with a name. The web app guesses one from what you're
    making ("Transcribe to report", "Recording workflow", "Prompt template 2", "Read only token"), as it already did for
    routines, so making one takes no typing; type a name to choose your own. The pencil next to the name renames it
    later, and PATCH /api/v1/pipelines/{id}, /templates/{id} and /tokens/{id} rename them through the API.

  • Long recordings go to a speech provider in pieces that fit it. A recording longer or larger than a provider takes
    in one request is cut at pauses into pieces within its limits, and their transcripts are joined back into one, where
    only OpenAI-compatible servers had ten-minute parts before. Each provider starts with its published limits (10 hours
    for ElevenLabs and AssemblyAI, 2 GB for Deepgram, 25 MB for OpenAI), editable in Settings → Speech providers
    (docs).

  • Running the install line again keeps .env readable only by you. Each update rewrote .env with the shell's
    default permissions, so after an update its secrets could be read by other users of the machine. It is now written
    for its owner alone, and an install an earlier update left open is closed again the next time the line runs.

  • A microsandbox install whose keys don't open its database comes up when the line runs again. A reinstal 8000 l made
    before 0.8.1, after ~/lens was deleted, wrote new keys but kept the earlier install's lens-data volume, and every
    update after it kept both: its API stopped at every start with SURREAL_USER/SURREAL_PASS don't match. The sandbox
    now checks that the keys in .env open the database before it starts the API, and when they don't, the install line
    moves Lens to a volume of its own (lens-data-2, kept in .env as LENS_VOLUME) and starts it there. The earlier
    volume is left as it is (microsandbox).

  • Lens on a phone. Everything works at phone width now: the namespace switcher is at the top of the menu, Import
    is in the top bar, Settings opens its sections as a list instead of a row to swipe through, the graph's filters close
    after a pick, Copy no longer covers webhook addresses, sensors and connections read as cards, and Save and the other
    form buttons stay in sight above the Ask button. A recording's pipeline steps no longer make its page wider than the
    screen. On a phone the clicks survey goes from 2 features that couldn't be done to none, and from 15 swipes to 2.

  • Similar content. Every resource has a Similar tab, and selecting text anywhere (a transcript, a page, a
    title, a graph label) shows the resources most like it, each citing the passages, words, topics and entities they
    share, with Find more for the full list. Topics, entities and graph nodes list similar content too, and the
    assistant and the MCP server can ask for it (find_similar). It works without search by meaning, by key words,
    topics and entities (Processing).

  • Watch a folder in four clicks. Watch a local folder on the Sensors page opens this machine's allowed
    folders, and each has a Watch button, so it no longer has to be opened first. Adding This machine as a
    connection skips the test-and-name step unless its test fails (Sensors).

  • MCP servers that run as a program. Most MCP servers in marketplaces run as a program (npx or uvx), not on
    the network. Admins can now add one as an MCP server extension with its command, args and env, as in Claude's
    .mcp.json, and a plugin's such servers come along when an admin installs it. They run in the MCP runner, an opt-in
    container with none of Lens's data or keys (COMPOSE_PROFILES=mcp), start on first use and stop after 15 idle
    minutes (Assistant).

Features

  • updates: upgrade page with live steps, and roll back when the new version doesn't start (#262)
  • setup: ask only what has no default in the first-run wizard (#376)
  • settings: private networks for web pages and feeds in Settings → Documents (#378)
  • settings: a Hugging Face token in Settings → Components (#379)

Fixes

  • install: install on a computer without Docker or KVM access, and run the line again there (#368)

Dependencies

  • move to lucide-react 1.x (#369)
  • move to Tailwind CSS 4 (#377)

v0.8.1

Choose a tag to compare

@github-actions github-actions released this 12 Oct 02:36
  • The installer stops waiting when Lens can't start, and shows why. A microsandbox install that couldn't start Lens
    waited 30 minutes at "Waiting for the web app..." and then named only a command to run. It now lists the sandbox's own
    steps as they happen, and stops as soon as a step fails, the sandbox stops or a part of Lens keeps stopping, with the
    last lines Lens logged; the web installer shows them under the reason. Running the line again stops the running
    sandbox first and waits for its port (microsandbox).

  • A microsandbox install after the install folder was deleted comes up. The data volume (lens-data) outlives
    ~/lens, but its database opens only with the keys in that folder's .env. A first install made new keys, mounted
    the earlier volume anyway, and waited 30 minutes for a web app whose API couldn't open the database. It now leaves
    that volume as it is and starts on one of its own (lens-data-2, kept in .env as LENS_VOLUME); put the earlier
    .env back to use the earlier data, or remove it with msb volume rm lens-data
    (microsandbox).

v0.8.0

Choose a tag to compare

@github-actions github-actions released this 11 Oct 10:00
  • A microsandbox's full edition installs LibreOffice and Chromium the first time they're needed. They were
    installed at every start, so a first start downloaded them before it built Lens, whether or not anything used them.
    A sandbox now starts without them; the first Office file, email or web page that needs one waits while it's
    installed, and later starts install it again in the background from the packages kept in the volume. An install that
    already had them keeps them (microsandbox).

  • A microsandbox install gets past name lookups that fail. The first build in a microsandbox could stop at the
    backend's packages with dns error ... No address associated with hostname when this machine's resolvers didn't
    answer inside the sandbox. The sandbox now waits for the names it downloads from, looks them up with public
    resolvers (1.1.1.1 and 9.9.9.9; LENS_DNS_FALLBACK changes or turns that off) when this machine's don't answer
    there, and runs each download step up to three times (microsandbox).

  • Running the install line again keeps the lean edition. An install made lean (LENS_TARGET=lean, or the web
    installer's pick on a computer with less than 4 GB of memory) became full again when the line ran without
    LENS_TARGET, so the next start installed LibreOffice and Chromium. It now keeps the edition the install has, as it
    does the port and the network setting (Get started).

  • A Mac app. Lens.app (Lens-macOS-arm64.dmg, Apple silicon) installs Lens in the web installer shown in its
    window, with microsandbox inside it so no Docker is needed, then lives in the menu bar: open, start, stop and update
    Lens, and show its logs. Lens itself opens in the browser, where passkeys work
    (Get started). It updates itself from each release, checking the update's
    signature, once the repository has an update key (desktop/README.md).

  • Slack, built in. Settings › Slack creates a Slack app from Lens's manifest in one click; paste its two tokens
    and the assistant answers direct messages, what's said to it in channels and /lens …, over Socket Mode: no
    Matterbridge, no public address. Voice clips are heard, files are held for the assistant to import, and everything
    Matterbridge rooms do works there (Chat rooms).

  • The file tree over NFS. Settings → NFS share serves the archive's folders and files (namespaces, collections,
    topics and entities as folders, assets as files) read-only over NFS, to the networks you allow, showing what one
    chosen account sees. Off until turned on; published on port 12049 of the host (File shares).

  • The graph as files over SMB. Settings → SMB share serves the file tree read-only to Finder, Linux and (on port
    445) Windows: each person signs in with a file access key and sees what the web app shows them, from the networks
    an admin allows. Off by default; the SMB server (impacket) is fetched when it's turned on (SMB share).

  • A spending limit from the start. The setup wizard has a Budget step, and the web installer asks for one with a
    cloud provider: a scale for a hard limit on everything Lens spends on AI, with a soft limit at 80% that warns. At
    the hard limit runs wait (or are skipped) and paid model calls stop until it goes up; models on your own computer
    keep working. The model's price comes with it, so the spending is counted. It's the new all budget, changeable in
    Settings › Budgets (Budgets). The web installer now asks only the
    essentials: the folder, port and network under Advanced, the optional extras under More options on the
    review.

  • Cloud model presets. Settings → LLM provider and the setup wizard offer OpenAI, Anthropic, Google Gemini,
    OpenRouter, Groq, Mistral and DeepSeek next to the model servers found nearby: one click fills in the address and a
    model and puts the cursor in the API key field, with a link to where the provider hands out keys, so switching to a
    cloud model is a click and a paste. Agents get the same list from GET /api/v1/setup/llm/providers and the
    assistant's find_model_servers tool (Cloud model providers).

  • The graph as files. Graph nodes are folders, nested along its links: collections inside collections, topics
    inside their broader topics, entities by type. Each folder holds its assets' files, with a .json sidecar of
    metadata and a .txt of the text; everyone sees what the web app shows them. Browse and read it through the API
    (/api/v1/tree) and the MCP tools list_files and read_file (The graph as files).
    An NFS share is in progress.

  • HTTPS with automatic certificates. Lens can serve itself at https:// your domain with a certificate it gets
    and renews on its own from Let's Encrypt or ZeroSSL (or any ACME authority), proving the domain over HTTP, TLS-ALPN
    or DNS, with Cloudflare, Route 53 and the other caddy-dns providers, so wildcards and machines the internet can't
    reach work too. Run the install line with LENS_HTTPS=1 to open ports 443 and 80, then set it up with
    PUT /api/v1/settings/https; the Settings page comes next. Off by default (HTTPS).

  • Budget meter and Costs page. The top bar shows what the namespace picked spent against its budget, used and
    left, highlighted when near or over, and follows the namespace switcher; it opens the details for a day, week or
    month. Costs in the left menu shows spend by day, model and resource, and sets the namespace's budget
    (Budgets).

  • The graph over S3 and WebDAV. Turn on Settings, File shares, and any S3 client (rclone, Cyberduck, the AWS CLI) or
    WebDAV client (Finder, Windows Explorer, file managers) lists and reads the graph as folders and files,
    read-only, with access keys each person makes on their API tokens page (S3,
    WebDAV). Docker publishes S3 on port 9555 (LENS_S3_PORT) and WebDAV on 9556
    (LENS_WEBDAV_PORT), where the web app is.

  • Ask Lens from your browser. The web clipper extension has the Lens assistant in a side panel: Ask Lens in
    its popup or on the right-click menu. It signs in with one click through Lens's own sign-in page (OAuth, nothing to
    copy), and the conversations are the same ones Lens's chat shows, with cited answers, the steps taken and approvals.
    While the panel is open, the assistant can use that browser: open, read, click, type and download files into the
    conversation, only on sites you allow, asking before anything that sends, deletes, buys or posts, never typing
    passwords or card numbers, and showing each action as it happens. Once it has read a page, opening an address or
    typing text you didn't write asks you first too (The assistant in your browser).

  • Telegram, built in. Paste a bot token from @Botfather in Settings › Telegram and the assistant answers there:
    no Matterbridge, no public address. It talks only to people you let in (one click on the settings page), hears
    voice notes, takes files for the assistant to import, and does everything it does in Matterbridge rooms
    (Chat rooms). Chat apps now share one channel layer, so the next one is an
    adapter.

  • The assistant by email. Give it an address of its own in Settings › Assistant email (the address and an app
    password; the servers are guessed) and the people you let in can email or forward things to it: it answers by
    email in the same thread, lists the excerpts it cites with links, and imports what's attached, a forwarded email
    whole. It answers only mail their provider vouched for (DMARC, DKIM or SPF), never automatic mail, and holds a
    stranger's mail until you let them in. lens+pods@… or a routing rule sends mail to a namespace's assistant
    (The assistant by email).

Features

  • assistant: mark third-party text as data in the model's prompts (#302)
  • backups: restore, download, any schedule and copies to an rclone connection from the web app (#263)
  • assistant: let the assistant use a browser connected through the Lens side panel (#311)
  • clipper: build the web clipper for Firefox (#282)
  • assistant: ask before sending data out once an answer read third-party text (#309)
  • monitoring: Prometheus, node-exporter and Grafana as an opt-in compose profile (#297)
  • install: install with the defaults in one click; the API token form opens itself (#331)
  • assistant: flag third-party text that reads like instructions to an AI (#329)
  • ui: new pipelines and routines are named for you and Enter creates them; Enter in a note's title goes to its first line (#336)
  • bridge: break up links a chat-room answer built from what it read (#338)
  • mcp: hand agents what the archive says as data (#343)
  • audit: log what Lens caught of prompt injection (#344)
  • assistant: an optional local classifier for prompt injection ([#347](https://github.co...
Read more

v0.7.0

Choose a tag to compare

@github-actions github-actions released this 10 Oct 21:12
  • Prometheus metrics. Lens can serve /metrics for Prometheus and Grafana: jobs by status and pipeline, imports,
    model calls, tokens, cost and failures by model, budgets, workers and disk. Off by default; turned on in Settings →
    Telemetry (or LENS_PROMETHEUS=on), it answers only a scraper with the scrape token, and follows LENS_BIND. A
    sample Grafana dashboard comes with it (Prometheus).

  • Install on Windows, or by double-clicking. install.ps1 installs Lens on Windows: it gets Python 3 if needed and
    opens the web installer, which installs Docker Desktop (and WSL) and runs Lens in it. Each release now carries
    launchers that need no terminal at all: Install Lens.app for macOS, Install Lens.cmd for Windows and
    Install Lens.desktop for Linux, each opening the web installer
    (Get started).

  • Install in the browser. On a first install at a terminal, the one-line install opens a web installer: it shows
    what the computer has and what Lens will install, asks how Lens runs (Docker or microsandbox, full or lean, folder,
    port, network), the AI model (finding Ollama, LM Studio and other servers on the computer, or a provider and its key,
    with a test), and optional extras (search by meaning, the decision model, cloud transcription, email, remote access,
    anytopdf, telemetry), takes the administrator password when installing needs it, then installs with live progress
    and opens Lens's setup page. Its answers become ordinary settings in Lens, editable there
    (Get started). LENS_WIZARD=0 keeps the install in the terminal.

Features

  • updates: release check, backups and one-click upgrade through a machine agent (#260)
  • documents: scroll through every page of a document in one view (#267)
  • updates: tell admins and notification targets when a new release is out (#270)
  • chat: /clear and /compact commands, and early compaction near the context limit (#269)
  • voice: read aloud with Gemini TTS or macOS say, and pick the voice in Settings (#268)
  • clipper: Lens web clipper browser extension, Chrome first (#277)
  • import: capture one URL, several or a list from the API and the assistant (#272)
  • extensions: install skills, plugins and MCP servers from Anthropic-format marketplaces (#276)
  • clipper: publish browser extension releases for auto-updates (#294)
  • voice: read aloud with StyleTTS2 on this machine, as an opt-in service (#291)
  • clipper: Edge build, Edge Add-ons package and Edge smoke test (#285)
  • search: find content similar to a text, a resource, a topic or an entity (#296)
  • clipper: Safari build of the web clipper (#283)

Fixes

  • jobs: read documents and images instead of transcribing them (#265)
  • sources: let Google Drive sign in with a service account key file (#266)
  • migrations: read documents already waiting to be transcribed (#278)
  • sources: keep working when Google Drive rate limits (#288)
  • sensors: keep a retained MQTT message before acknowledging it (#299)

Dependencies

  • bump @tanstack/react-query (#249)
  • bump dotenv from 16.4.5 to 18.0.5 in /nextjs-frontend (#252)
  • close the frontend's open security alerts (#300)

v0.6.0

Choose a tag to compare

@github-actions github-actions released this 10 Oct 17:31

Features

  • install: list the prerequisites and ask before installing them (#258)

Fixes

  • install: pull the microsandbox base image from mirrors that need no sign-in (#259)

v0.5.0

Choose a tag to compare

@github-actions github-actions released this 10 Oct 16:27
  • Podcasts from what you pick. Lens makes a two-host learning episode about resources and passages: one host
    explains, the other asks what a learner would ask, every claim is cited to the passage it comes from and checked
    against it before it's kept, and the script is read aloud in two voices by your text-to-speech provider (a local
    OpenAI-compatible server works). Episodes are recordings in a podcasts namespace, so they're searchable and on the
    graph. Start one from the Library's selection bar or a resource's menu, with POST /api/v1/podcasts, or with the MCP
    tool create_podcast; each episode's page plays it beside its cited script (Podcasts).

Features

  • install: run Lens in a microsandbox microVM as an alternative to Docker (#255)

Fixes

  • store: say which setting to fix when SurrealDB refuses the password (#254)

v0.4.2

Choose a tag to compare

@github-actions github-actions released this 09 Oct 17:51

Dependencies

  • bump jest to 30 and keep BlockSuite on 0.19 (#241)

v0.4.1

Choose a tag to compare

@github-actions github-actions released this 09 Oct 13:55

Fixes

  • release: cut release notes over GitHub's 125,000 character limit (#236)

Dependencies

  • bump backend and frontend dependencies, TypeScript 6 included (#239)

v0.4.0

Choose a tag to compare

@github-actions github-actions released this 09 Oct 12:02
  • Import videos and podcasts by their address. Turn on Settings › Documents › Import videos and podcasts by their
    address (off by default), then Import › Web page › It's a video or a podcast. Lens downloads its media with yt-dlp,
    fetched on first use, through the same public-addresses-only proxy as web pages, and transcribes it
    (Videos and podcasts).

  • Zip and tar archives. Upload a .zip, .tar or .tgz and Lens lists what's in it, keeps its files, and reads the
    ones it can as resources of their own (a WhatsApp export with media becomes the chat and its photos). Encrypted
    entries, links and nested archives aren't opened, and unpacking stops at 100 files or 512 MB
    (Archives).

  • Camera RAW and HEIC photos. DNG, CR2, CR3, NEF, ARW, ORF, RAF, RW2, HEIC and HEIF can be uploaded and scanned
    when anytopdf is here or a conversion node is set: anytopdf reads them, and RAW photos get the new Camera RAW photo
    content type. Without anytopdf, Lens says what reading them needs (anytopdf).

  • Capture many web pages at once. Import › Web page › Many pages at once takes pasted links, a browser's
    bookmark export or Chrome's Bookmarks file. Each link is captured like a single page, and its bookmark folders
    become its tags. Links already captured in the namespace are left as they are
    (Web pages).

  • JSON and JSON Lines as records. A JSON file that isn't a transcript (orders, contacts, sensor logs, an API
    dump) is read one record at a time, every field searchable, each line pointing back at its record; it gets the new
    Records content type. JSON files can now be uploaded with the other files (Records).

  • Export an evidence PDF. A resource's Export menu, and a collection's page, download it as one PDF made by
    anytopdf: each file's pages (audio and video with their transcript), searchable, then a provenance page with every
    file's SHA-256, so it can be handed on and checked (Evidence PDFs).

  • anytopdf can find faces. Settings › Video, OCR, faces and objects › Face engine › anytopdf: its plugins find
    the faces in frames and pages and describe them with the same SFace model, so a server needs no OpenCV, and the
    people a namespace already knows still match (Video). Who someone is stays in Lens.

  • Search photos by what they show. Turn on Settings › Search › Search photos by what they show (off by default)
    and Lens downloads anytopdf's CLIP model (about 600 MB). Search then also shows the photos most like what you typed,
    such as "a bus on a city street", whatever words are on them (anytopdf).

  • anytopdf can find objects. Settings › Video › Object detector › anytopdf runs Lens's YOLOX model with
    anytopdf's objects plugin (sandboxed, no network), so a lean server or a Raspberry Pi finds objects without ONNX
    Runtime. Lens fetches anytopdf and the model on first use. YOLOX on ONNX Runtime stays the default
    (Objects).

  • Places and dates from anytopdf. Turn on Settings › Analysis › Find places and dates with anytopdf (off by
    default) and analysis also asks anytopdf, offline, for the places a text names, the dates it gives (with their ISO
    value) and where a photo was taken (from its GPS fix, with exiftool installed). They become places and dates in the
    graph, and the resource's Details tab lists them for the namespace's members
    (anytopdf).

  • WhatsApp, Telegram, Slack and iMessage chats. Import an exported chat (or upload a WhatsApp .txt) and Lens
    reads it message by message, with who sent each one as its speaker and when it was sent, so the assistant can
    answer from your real conversations. It gets the new Chat export content type
    (Chat exports).

  • anytopdf can make Lens's PDFs. Settings › Documents › Make PDFs with anytopdf. On auto (the default) nothing
    changes unless anytopdf is here: then it reads the emails, web pages and text a server without Chromium couldn't.
    Choose "For every document and image" and Lens downloads anytopdf 0.4.0 (checked against its release checksum);
    it then also reads images, flattening a photographed page before OCR. Or set a conversion node, another machine
    running anytopdf queue serve, and a small server reads Word, PowerPoint and spreadsheet files too
    (anytopdf).

  • Where anytopdf made a document's PDF, Lens uses the words it read. anytopdf 0.4.0 lists every word it found on
    a picture page with where it is, so a photographed or scanned page's text comes from anytopdf's OCR, block by block
    with its place on the page, without Lens reading the page again. Pages without those words are read as before.

  • A conversion node gets the token and nothing more. Lens refuses a node that redirects (so its token never
    goes on to another server), caps what the node answers and refuses a PDF that isn't one. Settings › Documents ›
    Check it says whether the node answers and takes the token (anytopdf).

  • Web pages and emails keep their look when anytopdf makes the PDF. Where Chromium and pdftoppm are installed,
    anytopdf draws the page as page images, offline, with its text as the search layer. Without them it reads the text,
    as before.

  • Connect Slack, WhatsApp, Telegram, Discord and Matrix from the web app. Start the optional matterbridge
    service, then turn on Settings › Chat rooms › Run Matterbridge here and add a bot token and rooms. To link
    WhatsApp, scan the QR code the page shows. Lens writes Matterbridge's config, a gateway per room, and restarts it
    when the settings change.

  • Visitors can no longer choose the address IP groups and sign-in throttles see. The web app drops an
    X-Forwarded-For a browser sends and tells the API the address it was reached from. Behind a reverse proxy, set
    TRUST_PROXY_HEADERS=true on the web app, or every visitor counts as the proxy's address
    (Trusted proxies). The Cloudflare tunnel keeps working as before.

  • The sensor hub's ports follow the web app's. The production compose file publishes MQTT (1883) and syslog
    (5514) where the web app is: on this machine only unless LENS_BIND=0.0.0.0 (which install.sh sets on a server).
    If devices on your network send to a Lens whose web app is local only, set LENS_SENSOR_BIND=0.0.0.0
    (Sensors).

  • Approve the assistant's reversible changes from the chat room. When Lens proposes merging entities or a new
    namespace in Slack, WhatsApp or another bridged room, the people listed in Settings › Chat rooms can reply "yes" or
    "no" there instead of opening the web app. Settings, imports, runs and extensions still wait for the web app.

  • Chat rooms route each question to its namespace. In a room no namespace is given to, Lens picks the namespace
    each question is about (or the one whose assistant it names), says where it looked, and names the likeliest when
    it isn't sure. "use pods" keeps the conversation in one namespace until "use everything".

  • Long conversations are summarised. The assistant reads the latest messages word for word and a running summary
    of the older ones, so long chats and chat-room conversations keep what was said instead of forgetting all but the
    last six messages. Settings › AI assistant › Summarise long conversations turns it off.

  • Encryption at rest has its own page in Settings. Settings › Encryption turns encrypting files on disk on or
    off, and the files already kept follow in the background, with how far it has got on the page. Turning it off
    leaves vaults' files encrypted (also for lens encrypt --off). It also sets how long working copies and unlocked
    vaults last.

  • A moved recording's files move to the new namespace's key. Moving a recording re-encrypts its files with the
    key of the namespace it goes to, so they open only with that namespace's key, and a vault it leaves no longer holds
    them. Moving out of or into a locked vault waits until it's unlocked (423).

  • The copies Lens keeps of files on storage sources are encrypted too. With encryption at rest on (or in a
    vault), the cached copy of a recording that lives on S3, Drive, a mailbox or another connection is stored with its
    namespace's key, and still plays, seeks and is processed as before. lens encrypt converts the ones already there.

  • Frames, page images and face crops are encrypted too. With encryption at rest on (or in a vault), the stills
    Lens draws from videos, documents and images are stored encrypted and still show as before; visitors still get faces
    pixelated where the namespace asks. lens encrypt converts the ones already there.

  • Word search no longer weighs down the database. Searching words in transcripts, text on screen, files, objects
    and descriptions now uses a small SQLite index next to the database instead of SurrealDB's own full-text index. On
    10,000 transcript lines the database used 64 MB of memory instead of 380 MB, and writing them took half the time.
    Nothing changes in how search, the assistant or IIIF search behave. Existing installs build the new index on their
    first start and give back the old one's memory. Settings › Search › Word search switches back to SurrealDB's
    index, or to an OpenSearch cluster on another machine for a big archive (then Reindex now); the opensearch
    compose profile runs one next to Lens.

  • A much smaller web app image. The production image runs Next.js's standalone server with only the packages it
    uses: 465 MB instead of 2.56 GB. Cloudron and pnpm build keep `next star...

Read more
0