8000 feat(automations): more operators for conditions by adeelahmad · Pull Request #434 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
question's words. A title someone gives it is kept. `title_by` says who titled it, `POST /api/v1/chats/<id>/title`
and the MCP tool `rename_chat` (without a title) have Lens title it again, and `rename_chat` with a title renames it.
Settings → AI assistant → Title conversations (on) turns it off ([Titles](docs/assistant.md#titles)).
- **More tests for automations' conditions.** An If, a trigger's where and a condition node can now test that text
starts or ends with something or matches a pattern, that a value is in a list, that a number or a time is between
two, before or after a time or within a duration, falls on certain days or times of day, and whether a value changed
since the last activity. Each value is checked when it's saved, and patterns that could tie up the worker are refused
([Operators](docs/automations.md#operators)).
- **Specs from transcripts.** Turn a voice session or a meeting into a technical spec, saved as a note linked to it:
the model writes the spec, then lists every named decision, every "no" and "not that, this", every constraint and
every gap the speaker left to decide later, each with its quote and a link to the moment it was said; a quote the
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,8 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
calendar events, files, feed posts, what people say to Lens, sensor readings, notes, graph changes, runs, budget
warnings) is kept as a stream of activities in W3C Activity Streams form, which agents read through the API, MCP and
the assistant, and people review on the Timeline page (and Latest on Home), marking what they've seen. Automations act
on that stream, on a schedule, at a time or when their own webhook is called: they test conditions, then tell you in
on that stream, on a schedule, at a time or when their own webhook is called: they test conditions (text, patterns,
numbers, lists, times, days and whether a value changed), then tell you in
your own chat app, ask the model or the agent, use a tool, call a web address (its credentials kept sealed), run a
workflow, pipeline or routine, or change assets and notes. A Classify node in a workflow sorts an asset into labels,
content types or topics, acting only on a sure answer. Anything that reaches outside Lens or changes more than what
Expand Down
54 changes: 49 additions & 5 deletions docs/automations.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ merged is called built.
| Part | Status |
| --- | --- |
| Match trees: one condition language for triggers, an automation's If and workflow nodes | built |
| Operators for times, lists, text patterns and changes (`in`, `matches`, `within`, `changed`...) | built |
| Activities, the kinds catalog, the event log, following the stream, timeline reads (`app/domain/events.py`) | built |
| New assets as `asset.added` (the `recordings` source) | built |
| New assets by kind: emails received, calendar events, files from storage, uploads, pages saved from the web, imports | built |
Expand Down Expand Up @@ -328,15 +329,57 @@ M := {"all": [M, …]} | {"any": [M, …]} | {"not": M} | {"path": "a.b.0", "op"
```

- A test (`{path, op, value}`) reads the part of the value at a dotted path, list items by number
(`object.from.0`); no path tests the value itself. The tests are the workflow engine's: `exists`, `empty`,
`equals`, `not_equals`, `contains` (text in any case, or an item of a list), `gt` and `lt` (numbers).
(`object.from.0`); no path tests the value itself. The tests are in the table below.
- `all` holds when every condition in it does (an empty `all` always holds), `any` when one does (it needs at least
one), `not` when its condition doesn't.
- Values are literals, never templates, so matching stays cheap and safe.
- A tree nests at most 6 deep and holds at most 50 tests. A problem is named with where it is: `the condition,
all.1.not: compare with a number`.
- `match.describe(tree)` reads a tree in words: `object.from contains “@stmarys.org” and (object.subject contains
“trip” or object.subject contains “outing”)`.
- `match.describe(tree, kind=None)` reads a tree in words: `object.from ends with “@stmarys.org” and
object.subject matches “trip” or “outing”`. With a kind, fields are named by the titles in its schema.

#### Operators

The first seven are the workflow engine's tests and work exactly as a condition node's always have. The others are
built in `app/domain/match.py` (`match.OPS` lists them all). When the value tested is a list, those pass when one of
its items does (as 8000 `contains` does), and `not_in` passes when none does. A value of the wrong kind, such as text for
`between` on numbers, doesn't pass; a test's own value is checked when it is saved, with the problem in words.

| Test | Value | Passes when the value tested |
| --- | --- | --- |
| `exists`, `empty` | none | is set; is missing, empty text, `[]` or `{}` |
| `equals`, `not_equals` | anything | is (or isn't) exactly the value |
| `contains` | anything | is text holding it in any case, or a list or object holding it |
| `gt`, `lt` | a number | is a number more (less) than it |
| `in`, `not_in` | a list of up to 500 values | is (or isn't) one of them; text in any case |
| `starts_with`, `ends_with` | text (a number is read as text) | is text, or a number, starting (ending) with it, in any case |
| `matches` | a pattern of up to 200 characters, such as `trip` or `^re:` | is text holding a match, in any case |
| `between` | two numbers, or two times: `[1, 5]`, `["-1d", "now"]` | is a number, or a time, from the first to the second (both included) |
| `before`, `after` | a time: `2026-10-16T07:30`, `now`, or from now: `2h`, `-30m` | is a time before (after) it |
| `within` | a duration: `30m`, `2h`, `1d` | is a time no further than that from now, either way |
| `weekday` | days: `["sat", "sun"]`, `["monday"]` or ISO numbers (1 is Monday) | is a time on one of those days |
| `time_between` | two times of day: `["22:00", "06:00"]` | is a time of day from the first up to the second, across midnight when the second is earlier |
| `changed` | none, or a path to keep values apart by: `actor.id` | differs from the value this test saw last time (the first one seen is no change) |

- **Times** are ISO 8601 times or dates. A time written without a zone (in the value or in what is tested) is in the
automation's time zone, and so are `weekday` and `time_between`. Durations are a whole number and `s`, `m`, `h` or
`d`; a minus points back (`-2h` is two hours ago).
- **Patterns** are regular expressions in Python's syntax, read in any case. To keep one pattern from tying up the
worker on a crafted email, a pattern that repeats a part that repeats itself (`(a+)+`), repeats alternatives that
may start alike (`(a|aa)+`) or refers back to a group (`\1`) is refused when it is saved, a pattern reads at most
the first 20,000 characters of a value, and a match that takes over 50 ms counts as no match. Compiled patterns are
cached.
- **`env`**: `match.test(tree, value, env)` reads time and state from `env`, never from globals, so tests and dry
runs can fix them: `now` (the clock when not given), `timezone` (UTC when not given) and `state`.
- **`changed`** compares with the last value the same test saw for the same automation and trigger, kept in
`automation_state` (kind `last`) by a `match.State(db, automation, alias, dry=False)` in `env["state"]`. Make one
State per activity tested, so the If and a node testing the same thing agree. With `dry` it reads and keeps nothing.
Without a State nothing has changed. With a path in `value` (`actor.id`), each sensor or sender keeps its own last
value. Like every test, it is only reached when the tests before it in an `all` pass, so `changed` compares with
the last value seen when it was reached.
- **`automation_state`** holds what automations keep between activities. Each automation keeps at most 1000 rows; past
that the oldest go and are counted (`match.dropped(db, automation)`). A `last` value is kept 30 days after it was
last seen; `match.tidy` drops expired rows once an hour, from the routine schedule.

In a workflow, a condition, filter or switch case takes either one test, as before, or a tree in `match`; with a
tree, `path` (when given) is the part of the value the tree tests. Graphs saved before keep running unchanged.
Expand Down Expand Up @@ -1074,7 +1117,8 @@ held run until you pick.

## Refine later

- More operators: `in`, `matches` (bounded regular expressions), `between`, times against now, `changed`.
- Variables inside match values (`{"var": "vars.threshold"}`), once saved variables are built.
- Statistical conditions, such as a reading above its 7-day average.
- The canvas shows and edits match trees on condition, filter and switch nodes (today it edits one test; a tree
saved through the API is kept).
- A `?format=as2` JSON-LD view and per-actor outboxes for ActivityPub; SurrealDB `DEFINE EVENT` for Update verbs that
Expand Down
1 change: 1 addition & 0 deletions docs/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,7 @@ request, to another check that proves the same thing; loosening a requirement or
| Automations act on the stream, on a schedule, at a time or when their own webhook is called | [test_automation_runs.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_runs.py) `test_each_new_activity_runs_each_automation_it_matches_once`, `test_a_schedule_runs_its_automation_when_its_timer_fires`, `test_a_call_to_a_webhook_runs_its_automation`; [test_timer_triggers.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_timer_triggers.py) `test_an_at_trigger_fires_once_then_reads_done` |
| They test conditions, then tell you, ask the model or the agent, use a tool, call a web address with sealed credentials, run Lens work, or change assets and notes | [test_automation_runs.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_runs.py) `test_the_if_limits_and_dry_runs`; [test_actions_messages.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_messages.py) `test_notify_tells_me_in_my_chat_app`; [test_actions_ai.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_ai.py) `test_ask_model_answers_in_text_or_as_the_fields_a_schema_names`, `test_ask_agent_works_on_what_happened_and_saves_the_conversation`, `test_call_tool_reads_as_the_person_and_changes_only_by_asking_them`; [test_actions_web.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_web.py) `test_credentials_in_headers_are_sealed_to_the_automation_and_the_host_and_never_shown`; [test_actions_run.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_run.py) `test_run_workflow_queues_it_on_the_asset_as_the_automation`, `test_run_pipeline_processes_the_asset_again`; [test_actions_lens.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_lens.py) `test_set_field_is_kept_in_the_assets_history_by_the_automation`, `test_write_note_writes_a_page_adds_to_a_things_page_or_notes_an_asset` |
| What reaches outside Lens, or changes more than what set it off, asks first: on the Approvals page or with a yes in chat; agents decline but never approve | [test_automation_spec.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_spec.py) `test_risk_follows_the_effect`; [test_approvals.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_approvals.py) `test_approve_carries_the_action_out_and_the_run_records_it`, `test_agents_over_mcp_list_and_decline_but_never_approve`; [test_approvals_chat.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_approvals_chat.py) `test_a_yes_in_my_chat_decides_an_action_inside_lens`; [approvals-page.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/approvals-page.test.tsx) `groups what waits, puts the first in focus, and Enter approves it` |
| Conditions test text, patterns, numbers, lists, times, days and whether a value changed, checked as they're saved and read in words | [test_match_ops.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_match_ops.py) `test_each_test_passes_what_it_should`, `test_checks_refuse_values_a_test_cant_use`, `test_a_pattern_reads_a_long_value_only_so_far_and_gives_up_when_slow`, `test_changed_compares_with_the_last_value_seen`, `test_the_added_tests_read_in_words` |
| Each automation has limits and budgets | [test_automation_runs.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_runs.py) `test_the_if_limits_and_dry_runs`, `test_a_run_over_budget_is_held_for_the_persons_pick` |
| Agents make, try and run automations over MCP and in the assistant, or draft one from a sentence | [test_automation_tools.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_automation_tools.py) `test_agents_save_try_and_run_automations_over_mcp`, `test_the_assistant_saves_at_once_and_asks_to_switch_on_or_run`; [test_automations.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_automations.py) `test_an_automation_is_drafted_from_a_sentence` |

Expand Down
4 changes: 3 additions & 1 deletion fastapi_backend/app/api/mcp_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -2650,7 +2650,9 @@ def unsubscribe_feed(ctx: Context, feed: int) -> dict[str, Any]:
Arg(
"where",
"string",
'a match tree on the activity as JSON, like {"all": [{"path": "object.name", "op": "contains", "value": "trip"}]}',
'a match tree on the activity as JSON, like {"all": [{"path": "object.name", "op": "matches", "value": "trip|outing"}]}; '
"op is exists, empty, equals, not_equals, contains, gt, lt, in, not_in, starts_with, ends_with, matches, between, "
"before, after, within, weekday or time_between (docs/automations.md, Operators)",
max_length=20000,
),
Arg("unreviewed", "boolean", "only those you haven't marked reviewed"),
Expand Down
Loading
Loading
0