8000 feat(automations): finished runs, deletions and calendar moves as activities by adeelahmad · Pull Request #440 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion docs/automations.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ merged is called built.
| Activities, the kinds catalog, the event log, following the stream, timeline reads (`app/domain/events.py`) | built |
| New assets as `asset.added` (the `recordings` source) | built |
| New assets by kind: emails received, calendar events, files from storage, uploads, pages saved from the web, imports | built |
| More kinds: jobs, deletions, calendar moves, comments, notes, graph changes, sensors, chat | planned |
| Runs that finished or failed, deletions and calendar events that moved | built |
| More kinds: comments, notes, graph changes, sensors, chat | planned |
| The timeline API (`GET /api/v1/activities`), the MCP tools `timeline` and `event_kinds`, the assistant's `timeline` | built |
| Review marks on the timeline (reviewed, important, hidden), their filters and `mark_activities` ([Timeline](timeline.md)) | built |
| The shapes of triggers, joins, an If, policy and intents, checked (`app/domain/automation_spec.py`) | built |
Expand Down Expand Up @@ -87,6 +88,10 @@ and the paths holding outside text are in `GET /api/v1/activities/kinds` (`event
| `upload.added` | `asset.added` | `recordings` | `path` (the file's name), `mediaType`, `size`, `duration` | Sam uploaded “Call with the school” |
| `web_page.saved` | `asset.added` | `recordings` | `href` (the page's address), `site` | Sam saved “Museum opening hours” from museum.example.org |
| `import.received` | `asset.added` | `recordings` | `via`: hook (an import hook), paste ( 8000 pasted text) or file (a file read as text) | Phone shortcut imported “Shopping list” |
| `asset.processed` | | `jobs` | the asset's fields, `job` (`lens:job/<id>`), `steps`, `batch`; tagged with its entities | “School trip on Friday” finished processing (pages, analyze, embed) |
| `asset.failed` | | `jobs` | as `asset.processed`, with `step` (where it failed) and `error` | Processing “Call with the school” failed at transcribe |
| `asset.deleted` | | `log` | `type`, `id`, `name`, `media` | Sam deleted “Old minutes” |
| `calendar_event.changed` | | `log` | `startTime`, `endTime`, `location`, `calendar`, and `before` (`startTime`, `endTime`) | Family calendar moved “Parents' evening” to 2026-10-21 18:00 |

A new asset is one activity, of the most specific kind its row tells, decided by its content type and where it came
from, never by reading its file: an email (content type email, or from an IMAP or Gmail mailbox), a calendar event
Expand All @@ -95,6 +100,13 @@ from, never by reading its file: an email (content type email, or from an IMAP o
imported), else `asset.added`. A file an email or archive held, made a resource of its own, stays `asset.added`: the
email is what arrived. A trigger on `asset.added` matches every one of them.

A run of an asset's pipeline that finishes is `asset.processed` (it succeeded) or `asset.failed`, one activity per
run, batch runs included. Its actor is whoever queued it, and its tags are the asset's entities (the ten most
mentioned), which exist only once it has been processed: `asset.processed` is the kind for "when an email about St
Mary's is understood". A run tried again finishes again, as another activity. A deletion is logged as
`asset.deleted` where the asset is deleted (once per asset); a calendar event read again that starts at another time
is logged as `calendar_event.changed` (once per event and new start), with the times it had before.

Fields come from what the row keeps, so reading them costs nothing on the ingest path. A message's addressing is
kept as it is listed (`recording.mail`: who sent it to whom, its subject, list, mailbox, and Gmail's snippet), an
event's times, place and people as it is imported (`recording.event`), and an email's attachments and the start of
Expand Down Expand Up @@ -125,6 +137,9 @@ A kind belongs to the source named in its `source`, and a source may list more i
`lens:sensor/<id>`), or Lens (a folder scan). `events_content.actors(db, labels)` names them the way Lens writes
them down (an account id, an email, `watch:<id>`, `routine:<id>`, `import_hook:<id>`, `automation:<id>`, ...),
in a few queries for a whole page, and keeps names for 30 seconds.
- `jobs` (`app/domain/events_content.py`) reads `job` by `finished_at`, the id breaking ties, as `asset.processed`
and `asset.failed`; their ids carry the time they finished (`processed-job-51-1792081450`), so a run tried again
is a new activity.
- `log` reads `event_log`: activities no row of their own records (a deletion, a webhook call), written by
`events.emit(db, kind, *, actor, obj, space, summary, target=None, tag=None, key=None, published=None, spaces=None,
audience=None, owner=None, untrusted=None)`, which gives the id (`log-<n>`). Emitting a `key` again changes
Expand Down Expand Up @@ -337,6 +352,8 @@ written. `ask_agent` passes the activity it is about the same way.
- Name an automation actor by its automation's name once automations are stored.
- A timeline page filtered to one child of `asset.added` (only emails, say) reads every new asset and keeps those of
that kind, up to five source pages; narrow it in the query (path, `remote`, `web`) if that proves slow.
- Moves of Google Calendar and iCal events only: events in a calendar file on storage are read as new files. An
event that was cancelled or deleted from its calendar isn't an activity yet.
- Folder scans of a namespace's own folders as `file.added`; uploaded `.ics` files with their event's fields; a
snippet for IMAP messages before they are read (it needs the message body).
- Per-recipient threading of email conversations; entity tags on new assets once extraction runs at ingest.
Expand Down
10 changes: 9 additions & 1 deletion fastapi_backend/app/domain/deletion.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,14 @@

from __future__ import annotations

import logging
import pathlib
import shutil

from . import access as acc, files as filemod, notebook, render, store, video

R = store.R
log = logging.getLogger("lens")
# rows that belong to the recording
OWN = (
"mentions",
Expand Down Expand Up @@ -149,7 +151,7 @@ def orphans(db, speakers, faces):
def delete(db, cfg, rid, by=None):
"""Delete a recording (see the module docstring). Returns what the audit log keeps of it. Raises KeyError when there
is no such recording and Running when a job is working on it."""
rec = db.one("SELECT title, space, path, fingerprint, remote FROM $r", r=R("recording", rid))
rec = db.one("SELECT title, space, path, fingerprint, remote, source, media FROM $r", r=R("recording", rid))
if not rec:
raise KeyError(rid)
stop_jobs(db, rid)
Expand Down Expand Up @@ -187,6 +189,12 @@ def delete(db, cfg, rid, by=None):
orphans(db, speakers, faces)
if acc.published(a):
acc.activity(db, rid, "Delete") # harvesters drop it
try:
from . import events_content

events_content.deleted(db, rid, rec, by) # asset.deleted, for automations and the timeline
except Exception: # noqa: BLE001 - the asset is gone either way; a missed activity mustn't undo that
log.exception("couldn't log the deletion of recording %s", rid)
return store.clean({"title": rec.get("title"), "namespace": ns, "path": rec.get("path"), "access": a["access"]})


Expand Down
Loading
Loading
0