8000 feat(automations): decide an automation's approval with a yes in your chat app by adeelahmad · Pull Request #455 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 23 additions & 8 deletions docs/automations.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,8 @@ merged is called built.
| Actions that wait (`delay`, `when`): kept as timers, carried out at their time, asking first when they should then ([Time](#time)) | built |
| The Timeline page with filters, grouping, review keys and new items, and Home's Latest card ([Timeline](timeline.md)) | built |
| Approvals: an automation's action that asks first waits for your pick (approve, decline, edit, allow from now on), told once in your chat app, capped and expiring (`app/domain/approvals.py`) | built |
| The Approvals page and its badge, a yes in your chat app, approval activities and notifications, agents' approval tools | planned |
| Deciding an automation's action with a yes in the chat it told you in | built |
| The Approvals page and its badge, approval activities and notifications, agents' approval tools | planned |
| Storing and running automations, joins, variables, the studio | planned |

## Contract
Expand Down Expand Up @@ -601,10 +602,23 @@ Deciding (`POST /api/v1/approvals/{aid}`):
- `always` approves it, then lets that action node run without asking from now on (the automation's policy, `auto`).
Only an action that reaches outside Lens or other people asks each time, so only those can be allowed.

Approving, editing and `always` are yours: signed in to Lens, on the approval's card. An API key, an app, MCP and the
assistant can list and decline approvals, never approve them, since approving is the person's pick; each approval's
`can` says what the caller may do with it. This is the one place where an agent can't do what you can. The
assistant's own approvals in a conversation are decided as before.
Approving, editing and `always` are yours: signed in to Lens, on the approval's card, or approving with a yes in your
own chat app (below). An API key, an app, MCP and the assistant can list and decline approvals, never approve them,
since approving is the person's pick; each approval's `can` says what the caller may do with it. This is the one
place where an agent can't do what you can. The assistant's own approvals in a conversation are decided as before.

**In your chat app.** A reply of "yes" in the private chat you were told in approves an automation's action, under
the rules a chat room's approvals follow ([Chat rooms](chat-rooms.md#what-it-does)), and only when all of these hold:

- the action changes something inside Lens (effect `lens`); one that reaches outside Lens or other people is
approved on its card in Lens, and the reply says where;
- the channel lets you approve there (approving in the room is on, and you are on its list of people who may talk to
Lens);
- it is the only thing waiting in that chat: with several, a bare "yes" lists them and "yes 12" picks one ("yes all"
leaves an automation's actions waiting, to be picked one at a time).

"No" declines it, whatever it does. The message that told you says "Reply “yes” to do it or “no” not to" only when a
yes there would count; otherwise it gives the link.

Undecided approvals expire after `automations.approval_days` (Settings → Automations, 1–365, default 7): a sweep
called from the scheduler once an hour marks them `expired`, nothing runs, and the run says so. The assistant's own
Expand Down Expand Up @@ -768,6 +782,7 @@ the timer was cancelled, or Lens was off past the catch-up window.
source pages, so narrow it by account if that proves slow.
- Lens itself: a run held over budget, waiting for someone's pick, as an activity of its own; backups taken before
upgrades or with `lens backup` (only those from Settings, the API and the schedule are activities today).
- Approvals: a cost estimate on actions that ask (a run on another asset could say what it would cost); buttons in
Telegram and Slack; approvals that need two people; one card for many alike ("approve all 12"); trying a failed one
again; expiring an automation's approvals when it is removed.
- Approvals: a cost estimate on actions that ask (a run on another asset could say what it would cost); approve
buttons in Telegram and Slack (today a yes decides only actions inside Lens); approvals that need two people; one
card for many alike ("approve all 12"); trying a failed one again; expiring an automation's approvals when it is
removed.
5 changes: 4 additions & 1 deletion docs/chat-rooms.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,10 @@ That's all. Without an account given, it answers as the admin who turned it on.
several waiting, "yes all" or "yes 12". Only the chat usernames listed may message Lens and approve, so it needs
that list; **Approving in the room: only in the web app** turns it off. Settings, imports, batch runs, extensions
and forgetting are always approved in the web app. Each approval is in the audit log with "via chat room" and who
said yes. Without a list, anyone in the room may be someone the account doesn't vouch for, so their messages count
said yes. An [automation's action](automations.md#approvals) that changes something inside Lens is approved the same
way, in the private chat it told you about it in, when it's the only thing waiting there (or by its number); one
that reaches outside Lens or other people is approved on its card in Lens, and "no" declines either.
Without a list, anyone in the room may be someone the account doesn't vouch for, so their messages count
as text from outside Lens: web tools and other servers' tools wait for approval in the web app too
([third-party content](assistant.md#third-party-content-and-prompt-injection)).
- **Links it can't be made to send.** Slack, Telegram, WhatsApp and most chat apps open the links in a message to
Expand Down
35 changes: 28 additions & 7 deletions fastapi_backend/app/domain/approvals.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,12 +122,26 @@ def link(cfg, db, aid):
return f"{app_url(cfg, db)}/approvals?id={aid}"


def message(name, what, url):
"""What the person is told when an automation's action waits for them."""
def message(name, what, url, yes=False):
"""What the person is told when an automation's action waits for them; `yes`: a reply there decides it."""
said = what[:1].lower() + what[1:] if what else "do something"
if yes:
return f"{name} wants to {said}. Reply “yes” to do it or “no” not to, or open {url} to decide."
return f"{name} wants to {said}. Open {url} to decide."


def yes_counts(db, v, r, aid, effect):
"""Whether a yes in the chat of chat_route `r` (channel view `v`) would decide approval `aid`: only for an action
that changes something inside Lens (one that reaches outside is decided on its card), when the channel lets the
person in that chat approve (bridge.room_approver: approving on, and they're on its list of people), and when
nothing else waits in that chat (a bare yes would be ambiguous)."""
from . import bridge

if effect != "lens" or r.get("conversation") is None or not bridge.room_approver(v, r.get("target") or {}):
return False
return not [a for a in bridge.pending(db, r["conversation"], r.get("account")) if a["id"] != aid]


def ask(db, cfg, intent, event=None):
"""An approval for an intent that asks first (the engine's hand-off): the row, for the automation's person, with
what it would do, the activity that caused it (`event`, else the run's last) and when it expires; then the person
Expand Down Expand Up @@ -174,17 +188,22 @@ def ask(db, cfg, intent, event=None):
}
),
)
told = tell(db, cfg, person, aid, name, what)
told = tell(db, cfg, person, aid, name, what, i["effect"])
if told:
db.q("UPDATE $r SET told = $t", r=R("approval", aid), t=told)
return aid


def tell(db, cfg, person, aid, name, what):
"""Tell the person once, in their own chat apps, that something waits: the chats it went to."""
def tell(db, cfg, person, aid, name, what, effect=None):
"""Tell the person once, in their own chat apps, that something waits, saying a reply decides it where it would
(yes_counts): the chats it went to."""
url = link(cfg, db, aid)

def text(ch, v, r):
return message(name, what, url, yes_counts(db, v, r, aid, effect))

try:
got = channels.tell(db, cfg, person, message(name, what, url), links=(url,))
got = channels.tell(db, cfg, person, text, links=(url,))
except Exception: # noqa: BLE001 - the approval waits in the inbox either way
log.warning("approval %s: couldn't tell %s", aid, person.get("email"), exc_info=True)
return []
Expand Down Expand Up @@ -319,7 +338,9 @@ def decide(db, cfg, row, user, decision="approve", params=None, via="web"):
with (
activity.scope(db, f"automation:{intent['automation']}", f"automation_run:{intent['run']}", cfg=cfg),
events.caused_by(cause),
graph_history.acting(actor=who["email"], via=who["id"], why=f"approved by {user.get('email')}", approval=row["id"]),
graph_history.acting(
actor=who["email"], via=who["id"], why=None if via == "chat" else f"approved by {user.get('email')}", approval=row["id"]
),
):
result = actions.perform(db, cfg, {**intent, "status": "waiting"}, person)
except Exception as e: # noqa: BLE001 - recorded on the approval and the run
Expand Down
51 changes: 41 additions & 10 deletions fastapi_backend/app/domain/bridge.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@

The assistant's reversible changes (entity merges and edits, new namespaces: ROOM_SAFE) can be approved by replying
"yes" (or "no") in the room, when bridge.approve is "low_risk" and the sender is one of bridge.users; everything else
waits for approval in the web app, linked from the answer.
waits for approval in the web app, linked from the answer. So can an automation's action that changes something inside
Lens, in the private chat its person was told about it in (approvals.ask), when it's the only thing waiting there or
named by its number; one that reaches outside Lens is decided on its card in Lens.

What happens in a room is the same in every chat channel (channels.py): Matterbridge is one adapter, Telegram's own
bot (telegram.py) another. handle() takes a message from any of them, with their voice notes (transcribed) and files
Expand Down Expand Up @@ -398,54 +400,83 @@ def room_approver(cfg, m=None):
return (m.get("username") or "").strip("<> ").lower() in allowed or (m.get("userid") or "").lower() in allowed


def pending(db, cid):
return db.rows("SELECT record::id(id) AS id, tool, summary FROM approval WHERE chat = $c AND status = 'pending' ORDER BY id", c=cid)
def pending(db, cid, account=None):
"""What waits in a conversation: what the assistant proposed there, and the automations' actions the account's
person was told about there (approvals.ask), oldest first."""
return db.rows(
"SELECT record::id(id) AS id, tool, summary, intent FROM approval WHERE status = 'pending'"
+ (" AND account = $a" if account is not None else "")
+ " AND (chat = $c OR told.*.conversation CONTAINS $c) ORDER BY id",
c=cid,
a=account,
)


def decide_in_room(db, cfg, base, account, cid, m, q):
"""A "yes" or "no" to what this conversation is waiting on: carried out (the ROOM_SAFE ones) or declined, and the
reply to post. None when q isn't one, or nothing waits."""
"""A "yes" or "no" to what this conversation is waiting on: carried out (the ROOM_SAFE ones, and an automation's
action that changes something inside Lens) or declined, and the reply to post. None when q isn't one, or nothing
waits. An automation's action is decided by a reply only when it is the one thing waiting here or named by its
number; one that reaches outside Lens is decided on its card in Lens."""
yes, no = YES_RX.match(q.strip()), NO_RX.match(q.strip())
if not (yes or no) or not room_approver(cfg, m):
return None
waiting = pending(db, cid)
waiting = pending(db, cid, account["id"])
if not waiting:
return None
pick = (yes or no).group(1)
== 1
if pick and pick.lower() != "all":
waiting = [a for a in waiting if a["id"] == int(pick)]
if not waiting:
return f"Nothing is waiting as #{pick}."
>
elif not pick and len(waiting) > 1 and yes: # "yes" to several things is ambiguous: say which
listed = "; ".join(f"#{a['id']} {a['summary']}" for a in waiting)
return f"Several things are waiting: {listed}. Say “yes all” or “yes <number>”."
roles = auth.roles(db, account)
readable, editable = set(roles), {s for s in roles if auth.allows(roles, s, "editor")}
who = (m.get("username") or "").strip("<> ")
user = {"id": account["id"], "email": account["email"]}
lines, web = [], []
lines, web, held = [], [], []
for a in waiting:
if yes and a["tool"] not in ROOM_SAFE:
if a.get("intent") and not one: # "no" or "all" to several: an automation's action is picked on its own
held.append(a)
continue
if yes and (a["intent"].get("effect") != "lens" if a.get("intent") else a["tool"] not in ROOM_SAFE):
web.append(a)
continue
decision = "approve" if yes else "decline"
try:
with graph_history.acting(why=f"approved in a chat room by {who}" if yes else None):
out = ai_tools.approve(db, cfg, a["id"], user, editable, decision, base, account["admin"], readable)
out = ai_tools.approve(db, cfg, a["id"], user, editable, decision, base, account["admin"], readable, via="chat")
except (ValueError, PermissionError, KeyError) as e:
lines.append(f"Couldn't do “{a['summary']}”: {e}")
continue
auth.audit(db, user, "assistant.approval", f"approval:{a['id']}", {"decision": decision, "via": "chat room", "by": who, **out})
if out.get("status") == "failed":
lines.append(f"Couldn't do “{a['summary']}”: {out.get('error')}")
continue
lines.append(("Done: " if yes else "Declined: ") + a["summary"])
if held:
listed = "; ".join(f"#{a['id']} {a['summary']}" for a in held)
lines.append(f"Still waiting, one at a time: {listed}. Say “yes <number>” or “no <number>”.")
if web:
lines.append(_web_line(cfg, db, cid, web))
return "\n".join(lines)


def _web_line(cfg, db, cid, approvals):
"""Where to decide what a reply here can't: the conversation in Lens, or an automation's action on its card."""
from app.email import app_url

return "Waiting for approval in Lens: " + "; ".join(a["summary"] for a in approvals) + f"\n{app_url(cfg, db)}/chat/{cid}"
base, out = app_url(cfg, db), []
asked = [a for a in approvals if not a.get("intent")]
if asked:
out.append("Waiting for approval in Lens: " + "; ".join(a["summary"] for a in asked) + f"\n{base}/chat/{cid}")
for a in approvals:
if a.get("intent"):
out.append(f"Decide in Lens: {a['summary']}\n{base}/approvals?id={a['id']}")
return "\n".join(out)


def reply_text(cfg, text, approvals, cid, db=None):
Expand Down
8 changes: 5 additions & 3 deletions fastapi_backend/app/domain/channels.py
Original file line number Diff line number Diff line change
Expand Up @@ -334,8 +334,9 @@ def where(ch, r):

def tell(db, cfg, account, text, links=()):
"""Tell `account`'s own person `text` in their chat apps: through each channel that answers as that account, in the
private chat it last had with someone it vouches for (routes()). Web addresses in it are broken up as a room's
answers' are (bridge.safe_links) unless they are in `links` (what the person or Lens wrote) or Lens's own.
private chat it last had with someone it vouches for (routes()). `text` may be fn(channel, its view, its
chat_route) giving the text for that chat. Web addresses in it are broken up as a room's answers' are
(bridge.safe_links) unless they are in `links` (what the person or Lens wrote) or Lens's own.
Returns {sent: [{channel, label, chat, conversation, target}], failed: [{channel, error}], error}, `error` saying
why nothing was sent (NOT_LINKED when no channel can reach them)."""
from . import bridge
Expand All @@ -345,7 +346,8 @@ def tell(db, cfg, account, text, links=()):
for ch, v, r in found:
error = None
try:
ch.send(v, r["target"], bridge.safe_links(v, db, text, links))
said = text(ch, v, r) if callable(text) else text
ch.send(v, r["target"], bridge.safe_links(v, db, said, links))
except (ChannelError, RuntimeError, OSError, ValueError) as e:
error = str(e) or type(e).__name__
failed.append({"channel": ch.key, "error": error})
Expand Down
Loading
Loading
0