8000 Releases · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content

Releases

v0.20.0

Choose a tag to compare

@adeelahmad adeelahmad released this 20 Oct 12:55
  • Photos and videos on a map. Library → Map (beside Table and List, on a phone too) shows each asset whose own
    file said where it was taken as a dot, with the Library's filters, framed to show them all; a dot links to its
    asset. Besides a photo's EXIF GPS fix, a video's location is now read as it arrives (the ISO 6709 tag iPhones and
    Android phones write). Agents get the same through GET /api/v1/resources?located=true&bbox=west,south,east,north,
    which also gives every row its place, and the MCP tool list_recordings (located, bbox); only members see
    where an asset was taken. The tiles under the map come from Settings → Maps (OpenStreetMap's by default); each
    viewer's browser fetches them, so the tile server sees the area looked at: point it at your own tile server, or leave
    it empty for none (Where an asset was taken,
    Maps). Assets that arrived before keep what they had: nothing needs upgrading.

  • Blur faces for sharing. A video's People tab has Blur faces for sharing: every face found is ticked, and
    Enter makes a copy of the video with those faces blurred, by ffmpeg over the faces' tracks, in the background as a
    heavy step (quiet processing hours apply), with its progress shown. The copy is kept with the video's files, the
    original stays as it is, and an owner can have visitors (public pages, embeds, share links, IIIF, feeds) play the
    copy instead. Nothing changes for a video until someone asks. Agents use GET and
    POST /api/v1/recordings/{id}/blur-faces (all, unnamed or chosen faces), PUT .../blur-faces/visitors, and the
    MCP tools blur_faces, get_face_blur and set_face_blur_for_visitors
    (Blurring faces for sharing).

  • Line up a transcript with its audio. A transcript imported without times (plain text, Word, PDF, Markdown) now
    says its times are estimated, and once its audio is attached its page offers Line up with the audio: one click
    transcribes the audio in the background with the engine transcription already uses (activity ledger, budgets and
    quiet processing hours apply), matches the transcript's words to the words heard, and gives each line their times;
    lines none of whose words were heard sit between their neighbours. The text and speakers stay exactly as they were.
    Nothing is lined up unless someone asks. Agents use POST /api/v1/resources/{id}/align and the MCP tool
    align_transcript; GET /api/v1/resources/{id} says times (estimated or aligned). Upgrading marks transcripts
    imported before with estimated times (the estimated-times step)
    (Lining up a transcript with its audio).

  • Thumbs up or down on answers. Under a finished answer in chat, a thumbs up marks it good and a thumbs down bad;
    pressing the one you gave takes it back. The rating is saved on the answer, shown when the conversation opens and
    carried in its JSON export (lens.rating), the label training data is built from; it makes no model call. Agents
    rate answers with PUT /api/v1/chats/{cid}/messages/{mid}/rating or the MCP tool rate_answer, and read it back on
    each message (Rating answers). Nothing needs upgrading: older answers have no
    rating.

  • An empty chat names the next step. A new conversation with nothing in its scope yet (nothing imported, or none
    in the namespaces picked) says chat has nothing to answer from and offers Import assets, plus Change scope when
    namespaces are picked, instead of suggesting questions nothing can answer. Someone who can't import is told an
    editor or admin can (Chat). Nothing needs upgrading.

  • Google service accounts that act as someone sign in with the scope the admin delegated. Google matches a
    domain-wide delegation's scopes exactly, so a client delegated all of Gmail (https://mail.google.com/, as backup
    tools ask for) was refused gmail.readonly and gmail.compose, and the source said only "domain-wide delegation"
    with the fix cut off. Lens now signs in with the broader scope the admin gave (all of Gmail, Calendar or Drive) and
    still calls only what the narrower scope allows. When Google refuses, the source's error names the scopes the
    delegation lacks, the ones it allows and the exact list to paste in the Admin console, in full
    (Gmail).

Dependencies

  • bump aiohttp from 3.14.3 to 3.14.4 in /fastapi_backend (#698)

v0.19.0

Choose a tag to compare

@github-actions github-actions released this 19 Oct 22:30
  • Circles in the web app. Your circles are under Profile: a suggested name makes one in one click, people are
    added from a list and taken out with ×, and deleting one is undone from its message. Share on a timeline item,
    or Share with a circle… in a resource's menu, picks your only circle, takes a note, and is undone from its
    message; shares show under Shared in the timeline (Circles of trust).

Hovering a reaction on the timeline says who gave it ("You, Sam and 3 others"), and the API, MCP and the page
carry up to five names of the others who gave each one.

  • Lambda GPU instances for auto scaling. A Lambda provider launches GPU instances in your Lambda Cloud account
    that join Lens over the mesh as worker nodes and transcribe and tell voices apart, priced from Lambda's list, found
    by Lens's tag and terminated when the group scales in (Machines elsewhere).

  • A search result opens the asset with its words found. Results link with ?q=, the words searched for (not
    filters or left-out words), so the asset opens with Find in its text showing them, on the match at the moment the
    result was, or on the longest of the words when the text doesn't have them together
    (Search).

Comments on the timeline can be edited in place (Enter saves, Escape cancels), and a thread can be resolved or
reopened there, as on an asset's page.

  • Amazon Bedrock as the LLM provider. Set the address to https://bedrock-runtime.<region>.amazonaws.com, the
    model to a Bedrock model id or inference profile, and the key to an AWS access key: Lens's chat, tool calls, images
    and summaries go to Bedrock's Converse API, signed with the key (LLM provider).

The timeline has saved views: a name for a set of filters and an order, picked from a Views menu on the page, saved
from what the page shows with a name guessed from the filters, and listed, saved and deleted by agents over the API
and MCP (timeline_views, save_timeline_view, delete_timeline_view). Each person's views are their own.

  • Amazon Bedrock provisioned throughput for auto scaling. A Bedrock group holds provisioned throughput of a model
    in your AWS account, with no commitment and at the price you give, and Lens's model calls go to it while it runs,
    signed with the provider's AWS key (Model hosts).

A file a watched folder finds changed in its source now shows on the timeline as "Office share updated “Minutes”"
(file.changed, an Update that links the asset it was before) rather than as a second "added". Automations on new
assets fire on it as before.

  • Share with your circles on their own. The share_with_circle automation action shares what comes in (the
    school's emails, a calendar's events, a sensor's alerts) with a circle and a note that can say what came in. It
    asks you first, unless you let its node run on its own; it shares as you, only what you can see in the
    automation's namespaces, never from a vault (Rules).

The mute menu on a timeline item now snoozes a source, person or kind for an hour or a week as well as a day.

  • When they're free. Let a circle see when you're free (never what): its people and their agents ask
    POST /api/v1/circles/when-free, MCP when_free or the assistant for a window and get each person's busy times
    and when everyone is free, within the hours of the day asked for. Your events are those you organise or are
    invited to, and your own calendar's; each ask shows in your timeline as agent.asked (When they're
    free
    ).

Review marks and reactions left on timeline activities that are gone (an event log entry past events.keep_days, a
deleted asset) are now dropped by an hourly tidy, a batch at a time. Comments on them stay.

  • Not them here. When a voice in one asset was matched to the wrong person, Not them here on their card in the
    asset's Speakers tab (or POST /api/v1/speakers/{sid}/detach) gives its lines there to a new person, works out the
    first person's voice again without it, and never suggests the two as the same; Undo merges them back
    (People and namespaces).

  • Ask their agent. Let a circle ask your agent questions about the namespaces you pick: it answers only from
    them, citing what it used, and you see each question and answer in your timeline. Through
    POST /api/v1/circles/ask, MCP ask_their_agent, the assistant, and Profile → Circles
    (Ask their agent).

For you on the timeline now counts what you open: following a line to what it is about, or opening an item by its
address, lifts its source, kind and entities a little for you alone (up to four opens of one item).

  • Auto scaling finds units Lens lost track of. Every ten minutes each provider is asked for the units it runs with
    Lens's tag; one no unit of Lens's stands for (a launch whose answer never came back, say) shows on the provider in
    Settings › Auto scaling and the API, and goes in the activity ledger once. Lens lets none of them go itself: you do,
    with Let go (and Undo for a few seconds), and the assistant can only ask you to
    (Units Lens lost track of).

The mute menu on a timeline item can now snooze its source or kind until a time you pick (Mute Work Gmail
until…
), starting at nine tomorrow morning; Enter mutes.

  • A scaling unit that goes leaves the mesh. When auto scaling lets a unit go, or one fails or never comes up, its
    unspent join token is withdrawn and the machine that joined with it is taken off the mesh, as removing it on the
    Nodes page does: its certificates go on the blocklist and its database user is removed
    (Machines elsewhere).

  • Search finds moments by the sounds in them. Laughter, applause, music and the other sounds a speech provider
    hears are a search filter like emotion: type event:laughter (any case; music is the BGM tag), pick one under
    Sounds in the filters, or pass event to GET /api/v1/search, a saved search or the MCP search tool. Each hit
    says its sound, and the facets count them (events), leaving plain speech out (API,
    search).

For you on the timeline reuses each person's affinity for a minute while their reactions, comments, marks and opens
stay the same, so paging through it looks up fewer activities.

  • SageMaker and Bedrock answers stream. With an Amazon SageMaker endpoint or Amazon Bedrock as the LLM provider, a
    chat's answer now comes as it is written (InvokeEndpointWithResponseStream and ConverseStream) instead of all at the
    end, and an error the model hits mid-answer says so. Keys also allowed sagemaker:InvokeEndpointWithResponseStream
    or bedrock:InvokeModelWithResponseStream stream; keys without them get the answer whole, as before (LLM provider).

  • Model calls spread across a group's units. While a model group serves Lens's model calls, every running unit of
    it takes them in turn, not only the one Settings › LLM provider shows, so scaling a group out adds capacity. A
    group's units now share one key, sealed on the group, and the scaler keeps the new llm.spread setting at their
    addresses; a unit that can't be reached hands its call to the next (Auto scaling).

  • Photos and videos are dated by when they were taken. An asset uploaded, scanned or imported is now dated by its
    own file when its name gives no date: a photo by its EXIF (DateTimeOriginal, else DateTime), a video or a
    recording by its creation time (QuickTime's creationdate, else creation_time), before the file's last change. A
    photo's GPS fix is kept as its place (lat, lon), which GET /recordings/{rid} and MCP get_recording give its
    namespace's members. A date in its name, or one its source gives, still wins, and a clock never set (1970, 1904) dates
    nothing (When an asset was made).

  • Lambda as a model host. A lambda-models provider runs a model with vLLM on Lambda GPU instances, which serve
    Lens's model calls while they run. Each instance serves HTTPS with a certificate signed by its group's own
    certificate authority, which Lens makes and keeps sealed, and Lens trusts only that authority for it (the new
    llm.tls_ca setting); a firewall ruleset Lens makes once per region opens the port (Auto scaling).

  • Circles with other servers. Once an admin turns it on (Settings → Circles elsewhere) and you make yourself
    findable under Profile, you get a handle like adeel@lens.example, and your circles can hold people on other Lens
    or ActivityPub servers by theirs. What you share with those circles reaches them as a signed ActivityPub Note, and
    what they send you lands in your timeline, taken only signed, recent and from someone in a circle. Through the API,
    MCP circles_elsewhere and save_circle, and Profile → Circles
    (Circles with other servers).

  • Access that ends. Giving someone a role in a namespace, a role on a collection or permission on an asset now
    takes an optional end date: an "Until" day in the asset's Access dialog, a collection's People and a namespace's
    members, expires_at in the API, and until on the assistant's change_person. After it, the grant opens nothing
    anywhere access is checked, and lists show it as expired until it's given again or taken away. The field starts
    empty, so grants without an end work as before (Grants that end).

  • A downloads switch for each asset. Its owners can turn downloads off in its Access dialog or with `PUT
    /api/v1...

Read more

v0.18.0

Choose a tag to compare

@github-actions github-actions released this 18 Oct 19:56
  • Client certificates for log servers. Settings → Logs takes this machine's certificate and key (PEM, stored
    encrypted) for syslog over TLS and https log endpoints that ask for one (mutual TLS), and a private CA's certificate
    to check the log server against (Logging).

The Mute button on a timeline item now has a small menu beside it: mute the source or person for a day (a snooze that
runs out by itself), or mute the item's kind, for good or for a day, without leaving the page. The Muted row shows
when a snooze ends.

  • Voice recognition off, per namespace. An owner can turn off People › Recognise voices across recordings: voices
    are still told apart in each recording, but no voiceprint is kept and the namespace's voiceprints are deleted
    (People and namespaces).

  • Circles of trust. Put the people close to you in circles of your own (Immediate family, Close family, Friends,
    Colleagues are suggested) and share an activity from your timeline or a resource with a circle and a note: each of
    them gets it in their own timeline as share.received, which their automations can follow, and you get
    share.sent. With the item also gives them permission on that one resource (it needs publish); undo takes the
    share back everywhere, with the permissions it gave. Nothing in a vault can be shared. Works through
    /api/v1/circles, MCP (list_circles, save_circle, share_with_circle, undo_share) and the assistant, which
    asks first (Circles of trust).

  • Ask where budgets stand. The assistant answers "where do my budgets stand?" with a new budgets tool, and agents
    call it over MCP: what each namespace you can read spent this period against its budget, what's left and whether
    it's ok, near or over, and for admins all of Lens against the overall budget and every other budget
    (Budgets).

The timeline opens one item by its id (/timeline?item=<id>), above the rest with its details and actions and a way
back. A comment or a reaction on an item, and an approval's "because", now link to that item rather than to the
timeline as a whole.

Reinstalling in a microsandbox no longer stops at "Getting uv". A sandbox whose volume an earlier install had
built in asked the backend's virtualenv, which has no pip, for uv and failed with "No module named pip". It now asks
the image's own Python. Rerun the install line to pick it up.

v0.17.0

Choose a tag to compare

@github-actions github-actions released this 18 Oct 17:31

Features

  • microsandbox: the MCP runner runs inside the sandbox, turned on and off from the web app like Docker's; it runs as a user of its own with the database and the archive closed to it

  • A new conversation's first answer no longer goes missing. A quick answer in a new conversation sometimes showed
    "No answer was saved for this question" though it was saved: the conversation's first read, made before the answer
    was saved, was used in place of reading it again. The web app (Chat, the page assistant and a recording's Chat tab)
    now drops a read under way and reads the conversation afresh once the answer is saved. And an answer is now written
    on a thread of its own, so closing the tab, losing the connection or moving to another page no longer drops it: it's
    finished and saved all the same, and only Stop ends it early (Chat).

  • The code sandbox can run on compute nodes. Once the mesh registers the install's compute nodes, run_python
    and run_bash take a node to run on, sandbox_nodes lists where code can run and why a node can't now, and a
    conversation's sandbox stays where it last ran (moving starts a fresh one). Each node's sandboxes are stopped,
    swept and deleted through its own gateway, and each run's ledger row names its node. Until nodes are registered
    nothing changes (Compute nodes).

  • Choose the content type as it comes in. Import has a What it is pick beside each file, pasted text and web
    page, offering the content types of its base type (Recognised automatically until you pick one). POST /uploads,
    POST /import, POST /import/web and /import/web/links take a content_type, and the assistant's import_files
    and import_urls take one when you say what the files or pages are: the asset is that content type from the start,
    so its pipeline runs the first time, not after a change and a rerun. One of another base type (a podcast for a
    photo) is refused before anything is saved, naming the ones that fit. Without one, nothing changes: it's recognised
    from the file (Content types).

  • lens ext: extensions as files. lens ext pull writes your extensions' manifests into a folder (skills and
    prompt tools as Markdown, the rest as YAML), lens ext push saves the changed ones as new versions and the new ones
    as extensions, lens ext check checks them without saving, and lens ext list lists them (--json for agents). It
    works on this install's database, or on any Lens server with --url and an API token. A pulled file says its
    version, so a push won't save over changes made in Lens since, and a pull won't overwrite changes made in the file,
    without --force (Manifests as files).

  • Transcription expects your own names. With Settings → Transcription → Expect your own names on (off by
    default), each recording is transcribed knowing up to 60 words its namespace already knows: the custom vocabulary,
    its named people, its defined entities and the names people fixed in its transcripts. Whisper and OpenAI-compatible
    servers get them as the prompt, Deepgram as key terms or keywords, AssemblyAI as key terms or boosted words; SenseVoice
    and ElevenLabs take no hints (Your own names in transcription).

  • Nebula on the primary (the private network between Lens's machines, in progress). On the primary Nebula runs in
    the optional mesh container, which the upgrade agent starts when a node is added: it makes its own key pair, checks
    Nebula's download against the kept checksums and relays the database and the API on the mesh address only. The
    install line that joins a machine and the Settings pages follow in their own pull requests
    (Nodes and the mesh).

  • Projects. A project is a goal you work toward with the assistant: a name, the goal in your words, instructions
    and the namespaces it draws on. It comes with a coordinator conversation about the project as a whole, which is told
    about every thread, and its threads are conversations scoped to its namespaces, marked done when finished; your own
    conversations can be brought in or taken out, and deleting a project keeps them. Every answer in a project reads its
    goal and instructions. The coordinator starts threads with a brief (start_thread), reads them and marks them done;
    a brief, from it, you or an agent, is answered in the background as you, with your tools, approvals and budgets.
    The Projects page, at the top of Chat's conversation list, starts a project with just a name, starts threads with a
    brief on Enter, shows which are working and marks them done in one click.
    Schedules ask a prompt on a schedule, in a new thread each time or the same one, and wait while switched off or the
    project is archived; agents set them with the MCP tool schedule_project.
    /api/v1/projects and the MCP tools list_projects, read_project, create_project and
    update_project do it all for agents (Projects).

  • Feeds in the web app. The Feeds page, opened from Podcasts and under Explore once there is a feed, lists your
    published feeds; New feed asks for a title, kind and namespace (Enter makes it); a feed's page copies the address in
    one click, gets a new one, edits what readers show, takes items out and deletes the feed with Undo instead of a
    confirm. The publishing nodes are on the canvas, with the feed picked from a list (In the web
    app
    ).

  • Grammar fixed by LanguageTool, when you run one. With transcripts cleaned up and Settings → Analysis →
    LanguageTool server set (none by default; the compose files' new languagetool profile starts one at
    http://languagetool:8010), the clean step makes its grammar and punctuation fixes too, leaving spelling, style and
    names alone. Each change is in Change history with Revert; without the server the rules run alone
    (Cleaning up transcripts).

  • Routine run alerts. A routine's run that fails (all its actions, some of them, or one that stops without
    finishing) or leaves graph changes to review can now be sent through Matterbridge, to a signed webhook, or to Slack
    and Discord: pick the new routine.failed and routine.proposed events for a namespace's notification target. Each
    namespace the routine runs over gets its own message, linking the routine or the run's changes; new targets' default
    events are unchanged (Notifications).

  • Talk to the assistant from a timeline item. An opened item on the Timeline has Ask about this, and a on
    a line does the same: the chat beside the page opens with the item (its id, line, time, where it opens and its
    fields) as what your question is about, read as data, so you type the question and press Enter. Nothing changes
    until you use it (timeline).

  • Joining a node (in progress). Another machine joins with the install line and LENS_JOIN and LENS_JOIN_TOKEN:
    it asks nothing, publishes no port, and runs Nebula and lens node, which joins once and then follows the primary's
    config, renews its certificate and reports over the mesh. Adding a node token now returns that install line
    (On a node).

  • Clear the job queue, reorder it and retry failed runs in bulk. Activity has Clear queue (it asks, then can
    be undone) and Retry failed, and under Queued the runs show in the order workers take them and can be dragged
    into another order (or moved with Alt+↑ and Alt+↓, or Move to top and Move to bottom). The API has the same:
    POST /jobs/cancel cancels every queued and held run you may process (or one namespace's, or chosen ids),
    POST /jobs/{jid}/move moves a queued run, and POST /jobs/retry retries every failed run, the dead letter queue.
    Agents use the MCP tools runs, clear_queue, move_run and retry_runs
    (Background work, the API). A retried run now joins the
    back of the queue.

  • Enhance low quality images before they're read. The new enhance step, first in the standard pipeline,
    measures each image (size, sharpness, noise, contrast, and face size where the namespace finds faces) and keeps an
    enhanced copy of a low quality one, which OCR, faces, objects and descriptions then read; the original stays as it
    is. Off by default: Settings → Enhance. The light classic engine (Pillow) is the default; Real-ESRGAN general
    x4v3 (BSD-3-Clause, 5 MB) is fetched the first time it's chosen. The Details tab says what was found, saves the
    copy, and enhances an image on request; agents use POST /api/v1/resources/{rid}/enhance, the MCP tool enhance
    or the enhance automation action (Enhancing images).

  • Whisper keeps the language it heard. With no language set in Settings › Transcription, recordings Whisper
    transcribes now keep the language it detected, as other engines' do, for search filters and the text cleanup.

  • Worker nodes run jobs (in progress). On a worker node, lens node also runs lens worker, against this
    install's database through the mesh, signed in as a user of that one database made for the node (removed with it;
    SURREAL_AUTH=database). Its jobs fetch their files from the primary over the mesh, plain, never a vault's or
    another namespace's, and it never gets the archive's secret, so it runs transcribe and diarize with the models
    on that machine, skipping any other step LENS_WORKER_STEPS names. A worker that serves some namespaces takes only
    their jobs, and an auto scaling group on machines elsewhere counts only the jobs a worker node takes, so i...

Read more

v0.16.0

Choose a tag to compare

@github-actions github-actions released this 18 Oct 09:38

Features

  • web: SchemaForm, a form generated from a JSON Schema with a variable picker (#566)
  • chat: recover when the model's context is full, and show what a summary covers (#558)
  • pipelines: convert steps that make a video's soundtrack an audio asset and a document's pages image assets (#568)
  • scaling: DigitalOcean and Hetzner Cloud providers (#573)
  • workflows: draw an automation's workflow on the canvas, with action and decide nodes (#576)

v0.15.0

Choose a tag to compare

@github-actions github-actions released this 18 Oct 07:27
  • The assistant sends mail, adds events and writes files, after your yes. New tools send_mail (send, or save a
    draft, from Gmail; reply_to keeps a message's thread), add_event (an event on Google Calendar, inviting people
    if asked) and write_file (a text file on a storage connection, replacing one only when asked). Each always waits
    for your approval, which now shows all of it first: the whole message with who it goes to, the event, the file's
    text. A Gmail or Google Calendar connection only reads until an admin picks "Also sends mail and saves drafts" or
    "Also adds events" under The assistant in its settings and connects the account again, since Google is then asked
    for gmail.compose or calendar.events; Google Drive connected read only is never written. Every change is in the
    audit log (Mail, calendars and files).
  • The upgrade agent starts the code sandbox's gateway. Settings → Code sandbox → Run the gateway on this machine
    now works on installs with an upgrade agent: the agent knows the sandbox profile and starts or stops its three
    containers, as it does the MCP runner, the browser and monitoring. An older agent still says to run the line by hand
    until it has updated itself (The gateway).
  • Deep research. Give the assistant a goal (/research <goal>, research: true on a question, POST /api/v1/chats/<id>/research, its start_research tool, or MCP) and it plans the research: up to three follow-up
    questions, the sub-questions, where it looks (the archive, the graph, notes, the web, the code sandbox), the tool
    calls it may make, the report's outline, and a labelled cost estimate. Approve the plan as it is or edited, revise
    it, or say "yes" in a chat room; an API key or an app only declines it unless research.agents_skip_review is on.
    It runs in the background, one run per person at a time, within its tool calls, research.max_minutes and your
    budgets, survives a restart, and stops when asked; the cited report comes back in the conversation, its claims
    checked against their sources, and saves as a note or makes a podcast episode. In the web app, the composer's Deep
    research switch asks for it; the plan is a card to edit and approve with Enter, then a progress card that survives a
    reload; and the report saves as a note, reads aloud, makes a podcast or exports as Markdown. Settings → AI assistant
    → Deep research holds its limits (Deep research).
  • Read any message aloud. Every question and answer in a conversation has a speaker button: one click reads it
    aloud by the speech model and voice picked in Settings → AI assistant → Voice (else the browser's own speech),
    another stops it. One message plays at a time, long answers go in pieces under every provider's limit, and each
    piece's audio is kept in the page, so playing it again doesn't call the model again. Each call is logged in the
    activity ledger (Read a message aloud).
  • Conversations title themselves. After a conversation's first answer, the model gives it a short title from the
    question and the answer, with the model for summaries (ai.compact_model) when one is set, in one call counted for
    the conversation in the activity ledger; until then, and when there's no model, it's titled with the first
    question's words. A title someone gives it is kept. title_by says who titled it, POST /api/v1/chats/<id>/title
    and the MCP tool rename_chat (without a title) have Lens title it again, and rename_chat with a title renames it.
    Settings → AI assistant → Title conversations (on) turns it off (Titles).
  • More tests for automations' conditions. An If, a trigger's where and a condition node can now test that text
    starts or ends with something or matches a pattern, that a value is in a list, that a number or a time is between
    two, before or after a time or within a duration, falls on certain days or times of day, and whether a value changed
    since the last activity. Each value is checked when it's saved, and patterns that could tie up the worker are refused
    (Operators).
  • Automations look back. An automation's If can ask whether something like this happened before: activities of
    some kinds within a time, from the same sender or about the same thing, at least or at most so many times, looked for
    only in the automation's namespaces and only as its person may see them. changed in a trigger's where or an If
    compares each activity with the last one, and trying an automation keeps nothing
    (Looking back).
  • Automations wait for their triggers together. An automation's join runs it when all of its triggers came within
    a time (for the same sender, say), when one came a number of times, when one didn't come in time after another (no
    reply within two days), or once for a batch collected over a time. What it waits on outlasts a restart, and switching
    the automation off drops it (Joins).
  • A decide node for judgement calls. An automation's workflow can ask the decision model a question with a few
    options ("Does this need me today?") and go on along the option it picked, or along unsure when it isn't sure
    enough, none is set up or the budget is spent. The email it reads is marked as outside text, the answer is kept per
    activity so a retry doesn't pay again, and the call shows in the activity ledger under the run
    (The decide node).
  • Specs from transcripts. Turn a voice session or a meeting into a technical spec, saved as a note linked to it:
    the model writes the spec, then lists every named decision, every "no" and "not that, this", every constraint and
    every gap the speaker left to decide later, each with its quote and a link to the moment it was said; a quote the
    transcript doesn't hold word for word is marked. The prompt is the "Spec from transcript" template, editable on
    the Templates page. In the web app it's Write a spec, then Open spec, in an asset's More actions menu, and a Spec
    step in the pipeline editor. Agents write one with POST /api/v1/recordings/{id}/spec, the MCP tools write_spec and
    get_spec, and the assistant's write_spec (which asks first); pipelines and batches run it as a spec step
    (Specs from transcripts).
  • MQTT through Remote access. Devices away from home can publish to the sensor hub, and subscribe to it, at the
    web app's own address: MQTT over WebSockets at /mqtt (wss:// under HTTPS), through the Cloudflare tunnel and
    HTTPS too. It's off until an admin turns on MQTT at the web app's address too in Settings › Sensors, and there
    every client signs in with a hub login, with too many wrong passwords from one address turned away for a while. It's
    the same broker as on port 1883. SMB, NFS and syslog stay on the local network, and Remote access now says why
    (Sensors, Remote access).
  • Passkeys at your remote address in one step. A passkey works only at the address it was made at, so one made at
    localhost or on the network didn't work at the Remote access or HTTPS address, and getting one there took a
    sign-in link from email or an admin. Now Profile and sign-in › Passkeys lists Lens's other addresses with Open
    signed in
    , which opens one with a 15-minute sign-in link of your own, where one fingerprint adds a passkey for it
    (Authentication).
  • Files added over NFS are imported. A file copied into a collection's folder on the NFS share is imported a moment
    after it's closed, as the account the share shows the tree as (nfs.account), with the same checks as an upload.
    rclone keeps files being written in a cache on Lens's disk and puts them to Lens once they're closed; a file Lens
    can't take shows in the share's log. Import files people add in Settings, NFS share (nfs.add_files, on) turns it
    off and serves the share read-only again (Adding files).
  • A private network between Lens's machines (in progress). Lens is the certificate authority of a Nebula mesh
    between the machines it runs on: an admin adds a node with a join token (POST /api/v1/mesh/tokens), the machine
    joins without ever sending the token (POST /api/v1/mesh/join: an HMAC proves it, the answer comes back sealed with
    a key derived from it) and gets an address, a certificate naming its role and namespaces, and its Nebula config,
    whose firewall lets worker nodes reach the database and every node the API, and nothing else; a node renews, reports
    and refreshes its config with its own node token, and removing one blocklists its certificates on every machine.
    Off until a node is added; settings in mesh. Nebula running on the primary, the install line that joins a machine
    and the Settings pages follow in their own pull requests (Nodes and the mesh).
  • Export a conversation. GET /api/v1/chats/<id>/export.md, .pdf or .json gives the version showing, one
    message (?message=) or, as JSON, every version (?all=true). The Markdown has each question and answer, its
    citations as footnotes linking back to the moment in Lens and the tools it used; the PDF is that Markdown printed by
    Chromium, else anytopdf, else LibreOffice (409 when none can run); the JSON is OpenAI's chat message format in a tree
    with Dublin Core terms, or ?format=messages for one {"messages": [...]} line, checked against a published schema.
    Agents use list_chats, read_chat and export_chat over MCP. In the web app, an answer's cop...
Read more

v0.14.0

Choose a tag to compare

@github-actions github-actions released this 17 Oct 07:21
  • Dates read as days, for agents. With Settings → Analysis → Read dates on (off by default), analysis reads the
    dates each line gives as days on the calendar: "next Tuesday" in a recording made on a Wednesday is the Tuesday
    after, "last week" Monday to Sunday before, "March 2024" all of March, worked out from the asset's own date. Agents
    ask what was said about a day or a range with GET /api/v1/dates, the MCP tool list_dates and the assistant's
    find_dates, each date with its line and a link to that moment. English rules, no model; a weekday alone or
    "10/11/2026" is left out rather than guessed (Dates).
  • Choose the tools, a limit and the namespaces of an answer. A conversation, or one question, can pick the tool
    groups or tools its answers may use (tools), cap their tool calls (max_tool_calls, up to the new Settings → AI
    assistant → Most tool calls a question may ask for, 50 by default) and narrow one answer to its own namespaces
    (scope). GET /api/v1/chats/tools lists the groups a person has, what each tool does (reads, asks first, sends data
    out) and why a group is missing. An admin's switches still win, and new conversations start with the person's last
    choice. In the web app, a Tools chip beside the scope picks them, and "All my namespaces" tops the scope's list
    (Choosing tools, namespaces and a limit).
  • Names get their real types, not Term. The built-in extractor now tells people, places, organisations, events,
    works and dates apart without a model: by a title or role before a name, a given name, the word a name ends or
    starts with (Corp, University of, Summit, Act, River), lists of countries, regions and large cities and of
    well-known companies and products, what a name is called ("a company called Lumen") and the namespace's named
    speakers. Only names nothing tells stay Terms, and "and"
    no longer joins two names into one. Existing entities analysis left as Terms are typed once on upgrade, keeping
    every type people set; the change is in the graph's history to roll back
    (How a name gets its type).
  • Entities found by spaCy, fetched when picked. Settings → Analysis → Entity extraction → spaCy now fetches spaCy
    (MIT) and its small model the first time an asset needs it (about 60 MB): English, German, or one for many
    languages. The defined entities still come first, a lone word like "Okay" isn't taken for a name, and until spaCy is
    there the rules find them and the job log says so. Workflows' Extract entities (rules) node uses it too
    (Entities and the graph explorer).
  • Punctuation put back in transcripts. With transcripts cleaned up, Settings → Analysis → Put punctuation back
    adds full stops, commas and question marks with a local model (kredor/punctuate-all, MIT, 12 languages), to
    transcripts that have hardly any (imported captions) or to every one, keeping the marks they have. The model and
    PyTorch (about 2 GB) are fetched the first time it's needed; off by default
    (Cleaning up transcripts).
  • Clean up transcripts before analysis. With Settings → Analysis → Clean up transcripts before analysis on (off
    by default), a new Clean up step runs before Analyze: a word said three times in a row is kept once ("I I I looked"
    → "I looked"), doubled pronouns and articles once, "um" and "uh" go, and spacing, "i" and sentence capitals are
    fixed, by rules, on English lines. Every changed line is in Change history with Revert, and lines someone corrected
    are left alone. Reprocess, pipelines, POST /api/v1/recordings/<id>/reprocess and lens clean run it
    (Cleaning up transcripts).
  • Describe pictures on this machine with Florence-2. Settings → LLM provider → Describing pictures can now have
    Microsoft's Florence-2 (MIT) describe each page and shot instead of the provider's model that can see images
    (llm.describe_engine florence-2). It runs on the CPU through ONNX Runtime, so no picture leaves the machine and
    no provider is needed; its 8-bit export is fetched the first time a job waits to describe something, pinned and
    checked against its SHA-256: base (about 280 MB, llm.florence_model) or large (about 830 MB). Each picture is a
    model.describe row in the activity ledger. Agents read and change it with the MCP tools picture_descriptions and
    set_picture_descriptions. The default stays the provider's model (Descriptions).
  • Automations run. An automation saved over the API (/api/v1/automations) now reacts to new assets in its
    namespaces: when its If passes, it runs its workflow as the person who made it, within its limits and budgets, and
    each run shows what its actions did. Try one on an activity without keeping or sending anything, or run it now;
    Settings → Automations switches them all off and sets how far they may go
    (Running automations).
  • Each answer says what produced it. The i in an answer's header shows the model and the host it ran on, the
    tools it was offered and used, the scope, its limits and what it used, tokens, cost (an estimate from your prices),
    how long it took, how it ended and which tool results read like instructions to an AI, with the calls made for it.
    Agents read the same record over the API (GET /api/v1/chats/<id>/messages/<mid>), and each tool step shows how long
    it took. Answers from before have no record (What each answer records).
  • Download notes as Markdown. A note's ⋯ menu in the tree downloads it as a Markdown file, and the Notes page's ⋯
    menu downloads all of a namespace's notes as a zip in folders that follow the tree, for Obsidian and other notes apps
    (Downloading as Markdown).
  • Templates. Notes inside a namespace's Templates note are templates: New note from a template… on the Notes
    page and in Jump to starts a note from one, with its text, drawings, files and filing. Any note's menu in the tree can
    also Make a copy (Templates).
  • A note says where it is. Above a note's title, Notes › namespace › each note it's inside, every one a link;
    under its text, Inside this note lists the notes inside it with Add a note inside. The text now starts in line
    with the title, the word count shows next to Saved, and Pin, History, Download and Delete are in the note's ⋯
    menu. Costs and activity is folded at the bottom (On a note).
  • Automations for agents. Agents list, save, try and run automations through the MCP tools automations,
    automation_actions, save_automation, test_automation and run_automation, with problems in plain words. The
    assistant saves them switched off and asks before switching one on or running it
    (Agents).
  • File shares through Remote access. S3 and WebDAV can now be reached at the web app's own address, /s3 and
    /dav, so they work through the Cloudflare tunnel and HTTPS as well as on the network, and on Cloudron, which
    publishes only the web app. It's off until an admin turns on At the web app's address too in Settings › File
    shares; then a new access key's dialog gives those addresses. Signing in is the same: a person's own access key,
    read-only. MQTT, syslog, SMB and NFS aren't HTTP and stay on their own ports
    (The graph as files, Remote access).
  • Files added over S3 and WebDAV are imported. Copy a file into a collection's folder (rclone copy, Finder,
    Windows Explorer, any S3 client) and it's imported there as an upload is, with the same checks and the namespace's
    pipeline; a new folder under Collections is a new collection, and a file anywhere else goes to the namespace's
    default collection. It takes the import permission (or an editor's role on the collection), files the system writes
    for itself (.DS_Store, Thumbs.db, half-written files) are never imported, and a re-sync copies nothing twice.
    The S3 server now takes streamed and multipart uploads; WebDAV takes PUT, MKCOL and Finder's and Explorer's ways of
    copying. This is on whenever the shares are; Import files people add in Settings, File shares
    (shares.add_files) turns it off and makes them read-only again (Adding files).
  • The web clipper takes your API token, or just signs in. It used to take only an import webhook's token, so an API
    token was refused with "Lens didn't recognise this token: make a new import webhook and paste its token". Now its
    options have Sign in: Lens asks you to allow the clipper, nothing to copy, and clips go into the namespace you
    pick (the popup offers each one you can add to). Use a token instead takes an API token with write access or a
    hook's token, and when Lens refuses one the clipper says why: unknown, revoked, read-only, or a Lens too old for
    API tokens. Anything acting as you (an agent, a script) can push to POST /api/v1/hooks/import?namespace= with
    your own token too, and GET there says where a token's pushes go (Web clipper,
    API).
  • Versions in conversations. A conversation is now a tree: editing a question makes a new version beside it
    instead of deleting what followed, regenerating an answer (regenerate, with the same model or another) writes one
    beside it, and a question can go on from any message (parent). GET /api/v1/chats/<id> shows each me...
Read more

v0.13.0

Choose a tag to compare

@github-actions github-actions released this 16 Oct 02:18
  • Updating Lens in a microsandbox on a Mac gets past "Building Lens". An update after a build that stopped
    partway stopped there with rm: cannot remove ... Directory not empty, then tar: /data/.lens/backend: Cannot open:
    the Mac's file sharing can list a folder as empty while a file is still in it, so removing the earlier build's
    node_modules didn't always get through. The sandbox now tries again and moves what's left aside, and builds the
    web app on its own disk instead of in the volume. Run the install line again to update
    (microsandbox).

Features

  • automations: match trees for conditions (#425)
  • automations: stream of activities with kinds, an event log and following (#426)
  • automations: checks for automation shapes and an action registry (#427)
  • automations: read the stream of activities over the API, MCP and the assistant (#428)
  • settings: an Automations section for the stream of activities (#429)
  • timeline: mark activities reviewed, important or hidden (#432)
  • automations: tell a person in their own chat app, the notify action (#433)
  • automations: ask the model, use an assistant tool, ask the agent (#438)
  • automations: new assets as emails, calendar events, files, uploads, web pages and imports (#435)
  • timeline: a Timeline page to review what came in, and Latest on Home (#437)
  • automations: call a web address, with credentials sealed in headers (#439)
  • automations: finished runs, deletions and calendar moves as activities (#440)
  • automations: run a workflow, a pipeline or a routine (#441)
  • automations: comments, notes and graph changes as activities (#443)
  • automations: set a field, move to a collection, write a note (#444)
  • automations: a webhook address per automation trigger (#442)
  • automations: sensor readings, alert patterns and new devices as activities (#445)
  • automations: timers that outlast a restart and fire once (#446)
  • automations: what people say to Lens as chat.message activities (#448)
  • workflows: an event scope for automations, with action and decide nodes (#436)
  • automations: routine runs, budget warnings, backups and releases as activities (#449)
  • automations: schedule, at and relative triggers as timers (#450)
  • automations: actions that wait, carried out at their time (#452)
  • automations: an automation's action that asks first waits as an approval (#447)
  • automations: decide an automation's approval with a yes in your chat app (#455)
  • automations: approvals as activities and an approval.waiting notification (#457)
  • automations: agents read and decline approvals, and never approve (#458)
  • automations: templates in params and prompts, with outside text kept marked (#454)
  • automations: saved variables, kept between runs and set by the set_variable action (#460)

v0.12.0

Choose a tag to compare

@github-actions github-actions released this 15 Oct 15:46
  • microsandbox runs on Debian 13 too. Its sandbox now starts from python:3.12-slim-trixie, like the backend
    image, so a local model runs there on a Mac with Apple silicon. The packages kept from the earlier Debian are
    cleared on the next start (microsandbox).
  • A local model's server asks for its key again. llama.cpp's current builds read the server's API key from a
    differently named variable, so the server Lens started for a local model answered without one (to other containers
    on the same Docker network, not from outside the machine). Lens now sets both names, and stops the server, saying
    why, if it answers a request without the key (Local models).
  • A local model runs on arm64 in Docker (a Raspberry Pi, Docker on a Mac). The backend image is now built on
    Debian 13 (trixie) instead of Debian 12, as llama.cpp's Linux arm64 build needs its newer system library (glibc
    2.38). Upgrading rebuilds the image; nothing else changes (Local models).
  • A local model starts again. llama.cpp's versioned releases (v0.6.0 on) carry only the source, so Lens found no
    llama.cpp build to fetch and a model picked under Settings → Local model didn't run (llama.cpp v0.6.0 has no build
    for linux arm64
    ). Lens now takes the build that release names, and says why when the build it fetched can't start
    on this machine (Local models).
  • Today's note. Today on the Notes page, or Today's note in Jump to, opens the note for the day, made the first
    time it's asked for inside a Daily notes note (Today's note).
  • Pin notes. A note's ⋯ menu in the tree has Pin: pinned notes show at the top of the tree and first on the
    Notes page, for the person who pinned them only (Pinned notes).
  • Notes tree menu. Each note in the tree in the left navigation has a ⋯ menu to add a note inside, rename it in
    place, move it (find the note to put it in by name, or move it to the top) and delete it with Undo, so nothing needs
    dragging and it works on a phone. The navigation links and the tree now scroll together, so a long tree has room
    (The tree).
  • Delete a note with Undo. Deleting a note no longer asks first: the notice has Undo, which puts it back where it
    was. Deleted notes wait in Recently deleted, at the bottom of the Notes page, for 30 days with their files and
    history, to put back with one click or delete for good; then they go on their own
    (Deleting and putting back).
  • Connect Gmail with Google. Add connection › Gmail now has the Google account and Service account choice and
    Connect with Google, as Google Calendar does: Google asks to read mail (read only; all of Gmail to read over
    IMAP), the account's address becomes the connection's, and its labels open to pick one to watch. A service account
    takes its key and the address it reads, and says which client ID and scope a Workspace admin allows. A Gmail
    connection whose sign-in stopped working has Connect Google account on its row
    (Gmail and Google Workspace).
  • Connect with Google. A Google Calendar connection signs in to a Google account from a button now: Google's consent
    screen asks for read-only calendar access and sends you back to Sources with the account connected and its calendars
    open. It uses the Google sign-in provider's OAuth client, an earlier connection's, or one entered in the form, which
    also shows the redirect URI to register. A connection whose sign-in stopped working has Connect Google account
    on its row; pasting the authorized user JSON still works
    (Google Calendar).
  • Google Drive connects with Google too. When Lens has a Google client (Google sign-in's, or an earlier
    connection's), a Drive connection signs in with Connect with Google in one click and shows the account it is
    connected as; Read only asks Google for read-only Drive access. With no client it signs in through rclone, as before,
    or through a Google app of your own (Google Drive).
  • Every entity has its own page. An entity's page is named by its name, as a wiki names pages:
    /wiki/<namespace>/<Name> (spaces as underscores), so Ada Lovelace in work is /wiki/work/Ada_Lovelace. It shows
    what the entity is, the notes people and the assistant keep on it in the BlockSuite editor, every line that says it,
    and the assets, people and entities it comes up with. Its name on the Entities page now opens it (the rest of
    the row still opens the side panel), as do @ links in notes, the graph panel and the entity's RDF URI. Any case, a
    name it had before a rename, or the name of one merged into it opens it too (Entity pages).
  • The update notice no longer pushes pages off the screen. While Lens vX is out shows above them, Chat, the
    graph and a recording's page are that much shorter, so a phone's question box and a recording's player stay in
    sight.
  • An MCP server for each namespace. A namespace's owner can turn on its own MCP server under Admin → Namespaces →
    the namespace → Connect apps, which copies its address (/mcp/<namespace>). An agent connected there has the same
    tools but sees that namespace alone, whatever else the person who signs it in can read, and a token an app gets for
    it opens nothing else in Lens. Off by default (One namespace's MCP server).
  • Chat completions for any app, from one namespace. A namespace can answer OpenAI's chat completions API at
    /api/v1/namespaces/<namespace>, so any app that speaks it (with a Lens API key and the model lens) gets answers
    from the configured model, grounded in that namespace and citing it: [n] with a Sources list, citations and
    lens.sources, streamed or not. Its owners pick the read tools the model may use there (none by def 92AA ault: one search,
    one model call). Calls count for the namespace's budget. Off by default; turned on under Admin → Namespaces → the
    namespace → Connect apps (Chat completions for any app).
  • API keys for one namespace. A new API key can be made for one namespace (Opens, on the API tokens page, which
    Connect apps fills in). It works on that namespace's own MCP server and chat completions, with your role there, and is
    refused everywhere else in Lens, so an app given it sees nothing more (API tokens).

v0.11.0

Choose a tag to compare

@github-actions github-actions released this 15 Oct 12:26
  • Installing an update from the web app works again in a microsandbox. Back up and install stopped at the
    backup with Permission denied: '/data/.lens/apt/lock', and so did every backup there, because backups had started
    keeping everything under /data, including the Lens the microsandbox runs (.lens). Backups now keep only Lens's
    own files: hidden folders at the top of data_dir, and SurrealDB's own folder (the backup's export already holds the
    database), are left out, and a file Lens can't read is left out with a warning instead of stopping the backup. When
    backing up stops an upgrade, Settings now marks Back up the database as the step that failed
    (Backups).
  • The Ask button no longer covers buttons. The floating Ask button at the bottom right used to sit on whatever was
    there, such as Import on the Import page. Now, when a button or field is under it and scrolling can't move it clear
    (a row of buttons pinned to the bottom of the screen, or the last button on a page scrolled to its end), the Ask
    button rises above it, and steps out of sight when there's no room to rise. Dialogs, drawers and toasts drawn over
    it don't move it. The clicks survey now fails a journey whose control the Ask button covers, and checks every page
    for it (Chat on any page).
  • Google Calendar as a sensor. Add a Google Calendar connection on the Sources page (or POST /api/v1/sources
    with type gcal): each calendar is a folder and each event becomes text, as an iCal feed's do, read-only through
    Google's Calendar API. It signs in as a Google account (an OAuth client's authorized user JSON) or as a service
    account, which reads the calendars shared with it or, in Google Workspace, a user's calendars through domain-wide
    delegation. Calls to Google show in the activity ledger
    (Google Calendar).
  • Find a note. The Notes page has a search box that finds notes by the words in their title, summary or text (the
    pages of recordings and entities too), with the words around each match marked, and Enter opens the first. A filter
    shows only Projects, Areas, Resources or Archive, and each note says which notes it's inside. Jump to (⌘K) finds notes
    as well. GET /api/v1/notes/search; find_notes gives agents the words around each match
    (Finding notes).
0