8000 feat(automations): agents read and decline approvals, and never approve by adeelahmad · Pull Request #458 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/assistant.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,11 @@ When the configured model supports function calling, chat becomes an agent.
- **The person's review marks:** `mark_activities` marks activities of the timeline reviewed, important or hidden for
the person ([Timeline](timeline.md#review-marks)). After it has read text from outside Lens, hiding or marking
reviewed waits for their yes; marking important or bringing things back acts at once.
- **The person's approvals:** `approvals` lists what waits for their pick ([Approvals](automations.md#approvals)):
automations' actions about the namespaces the conversation reads, and its own proposals in this conversation.
`decline_approval` declines one; after it has read text from outside Lens (the approvals' own summaries count),
declining waits for the person's yes. It can't approve: that is the person's pick. An automation's agent doesn't
get `decline_approval`.
- **Limits:** everything is limited to what the person can read and to the conversation's scope, and capped by
`ai.max_steps` (Settings → AI assistant, Tool steps per answer; 6 by default) and `ai.max_transcript_reads`.
- **Always an answer:** the model is told its step budget. When the steps run out, or it replies with nothing, it is
Expand Down
7 changes: 5 additions & 2 deletions docs/automations.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@ merged is called built.
| Approvals: an automation's action that asks first waits for your pick (approve, decline, edit, allow from now on), told once in your chat app, capped and expiring (`app/domain/approvals.py`) | built |
| Deciding an automation's action with a yes in the chat it told you in | built |
| Approvals as activities (`approval.waiting`, `approval.decided`) and the `approval.waiting` notification | built |
| The Approvals page and its badge, agents' approval tools | planned |
| Agents' approval tools: `approvals` and `decline_approval` over MCP and in the assistant, and no way for an agent to approve | built |
| The Approvals page and its badge | planned |
| Storing and running automations, joins, variables, the studio | planned |

## Contract
Expand Down Expand Up @@ -622,7 +623,9 @@ Deciding (`POST /api/v1/approvals/{aid}`):
Approving, editing and `always` are yours: signed in to Lens, on the approval's card, or approving with a yes in your
own chat app (below). An API key, an app, MCP and the assistant can list and decline approvals, never approve them,
since approving is the person's pick; each approval's `can` says what the caller may do with it. This is the one
place where an agent can't do what you can. The assistant's own approvals in a conversation are decided as before.
place where an agent can't do what you can: agents read them with `approvals` and decline with `decline_approval`
(over [MCP](mcp.md#tools) and in the [assistant](assistant.md#the-assistants-tools)), and there is no tool to approve.
The assistant's own approvals in a conversation are decided as before.

**In your chat app.** A reply of "yes" in the private chat you were told in approves an automation's action, under
the rules a chat room's approvals follow ([Chat rooms](chat-rooms.md#what-it-does)), and only when all of these hold:
Expand Down
2 changes: 2 additions & 0 deletions docs/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,8 @@ For an app that speaks OpenAI's API rather than MCP, the namespace can also answ
| `read_file` | a text file of that tree (an asset's `.txt` text or `.json` metadata), a page at a time; other files' download link |
| `timeline` | what happened in Lens, newest first ([Automations](automations.md#timeline-reads)): each activity's kind, who did it, what it is about, when, and a one-line summary; by kinds, actor, namespace, entity, time, words or a match tree, a page at a time; `unreviewed` for what the person hasn't marked reviewed, with their marks on the page in `marks`. Fields holding outside text are listed in `lens.untrusted` |
| `mark_activities` | mark activities reviewed, important or hidden, or clear those marks, for the person alone ([Timeline](timeline.md#review-marks)); up to 500 by id, `unknown` lists those that can't be seen. Needs a write-scope token |
| `approvals` | what waits for your pick, newest first ([Approvals](automations.md#approvals)): an automation's actions and the assistant's proposals, each with what it would do, why, when it expires, its link and what an agent may do (`can`); one by `id`, by `status` (pending by default, or `any`), `origin` or `automation`, a page at a time. On a namespace's own server, that namespace's alone. What it would do holds text the automation read from outside Lens |
| `decline_approval` | decline one of your approvals by id: nothing is done, and the run records it. There is no tool to approve: that is your pick, in Lens or your chat app. Needs a write-scope token |
| `event_kinds` | the kinds of activity there are: label, source, parent and children, a schema with each field's title, the fields holding outside text, and a sample |
| `file_storage` | where Lens keeps its own files (notes' images and attachments) and the connections it could use; `check` tests it. Admins only |
| `set_file_storage` | keep new files on this machine or a storage connection (folder, rclone crypt); checked before it's saved. Admins only, with a write-scope token |
Expand Down
69 changes: 69 additions & 0 deletions fastapi_backend/app/api/mcp_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,9 @@
from app.api.v1.routes import search as search_routes
from app.api.v1.routes import topics as topic_routes
from app.domain import (
ai_tools,
analyze,
approvals,
auth,
blobs,
events,
Expand Down Expand Up @@ -2097,6 +2099,73 @@ def mark_activities(
return activity_routes.mark(ctx.db, ctx.cfg, who, ids, {"reviewed": reviewed, "important": important, "hidden": hidden})


def _approval_space(ctx: Context) -> int | None:
"""On a namespace's own server, the approvals about that namespace alone."""
return store.ns_id(ctx.db, ctx.ns) if ctx.ns else None


def _my_approval(ctx: Context, aid: int) -> dict[str, Any]:
a = approvals.get(ctx.db, aid)
sid = _approval_space(ctx)
if not a or a.get("account") != ctx.user.id or (sid is not None and a.get("space") != sid):
raise ToolError(f"no approval {aid} of yours")
return a


@tool(
"approvals",
"Approvals",
"What waits for the person's pick, newest first: automations' actions that ask first and the assistant's proposals, "
"each with what it would do (summary, which holds text the automation read from outside Lens), why (the activity "
"that caused it), when it expires, where the person decides it (url) and what you may do (can). Give id for one; "
"filter by status (pending, the default, or any), origin (automation or chat) or automation; give next as before "
"for more. Only the person approves, signed in to Lens or in their own chat app: an agent can list and decline.",
Arg("id", "integer", "one approval, by its id", minimum=1),
Arg(
"status",
"string",
"pending (waiting, the default), done, failed, declined, expired or any",
enum=("pending", "done", "failed", "declined", "expired", "any"),
),
Arg("origin", "string", "automation or chat (the assistant's proposals)", enum=("automation", "chat")),
Arg("automation", "integer", "one automation's, by its id", minimum=1),
Arg("before", "integer", "the next of the page before", minimum=1),
Arg("limit", "integer", "how many", default=20, minimum=1, maximum=100),
)
def approvals_list(
ctx: Context, id: int | None, status: str | None, origin: str | None, automation: int | None, before: int | None, limit: int
) -> dict[str, Any]:
if id is not None:
return {"approval": approvals.brief(_my_approval(ctx, id), ctx.web)}
st = None if status == "any" else status or "pending"
rows = approvals.listing(ctx.db, ctx.user.id, st, origin, automation, None, before, limit, _approval_space(ctx))
out: dict[str, Any] = {"approvals": [approvals.brief(a, ctx.web) for a in rows], "next": rows[-1]["id"] if len(rows) >= limit else None}
if not ctx.ns:
out["waiting"] = approvals.count(ctx.db, ctx.user.id)
return out


@tool(
"decline_approval",
"Decline an approval",
"Decline something waiting for the person's pick, by its id from approvals: nothing is done, and an automation's "
"run records that it was declined. Only the person's own; approving stays theirs. Needs a token with the write scope.",
Arg("id", "integer", "the approval's id", required=True, minimum=1),
writes=True,
)
def decline_approval(ctx: Context, id: int) -> dict[str, Any]:
if not ctx.user.can_write:
raise ToolError("this token is read-only: ask for a token with the write scope to decline approvals")
a = _my_approval(ctx, id)
if a["status"] != "pending":
raise ToolError(f"approval {id} is {a['status']}, not waiting")
try:
out = ai_tools.approve(ctx.db, ctx.cfg, id, ctx.user.as_audit(), set(ctx.acl.editable()), "decline", via="agent")
except ValueError as e:
raise ToolError(str(e)) from None
return {"approval": id, "status": out.get("status")}


@tool(
"event_kinds",
"Kinds of activity",
Expand Down
2 changes: 1 addition & 1 deletion fastapi_backend/app/domain/actions_ai.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
ABOUT_CHARS = 12000
OUTSIDE = "the activity that started this automation"
KEPT = ("id", "type", "kind", "published", "actor", "object", "target", "tag", "summary", "space")
LEFT_OUT = {"remember"} # a namespace assistant's memory is for conversations with it
LEFT_OUT = {"remember", "decline_approval"} # a namespace assistant's memory is for conversations with it; the person's picks are theirs


def _short(text, n=80):
Expand Down
91 changes: 88 additions & 3 deletions fastapi_backend/app/domain/ai_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,32 @@
["ids"],
False,
),
(
"approvals",
"What waits for the person's pick, newest first: automations' actions that ask first and your own proposals in "
"this conversation, each with what it would do, why (the activity that caused it), when it expires and where "
"the person decides it. One by id, or by status (pending, the default, or any), origin (automation or chat) or "
"automation; give next as before for more. Only the person approves, in Lens or their chat app: you can list "
"and decline.",
{
"id": _I,
"status": {"type": "string", "enum": ["pending", "done", "failed", "declined", "expired", "any"]},
"origin": {"type": "string", "enum": ["automation", "chat"]},
"automation": _I,
"before": _I,
"limit": _I,
},
[],
False,
),
(
"decline_approval",
"Decline something waiting for the person's pick, by its id from approvals: nothing is done, and an "
"automation's run records that it was declined.",
{"id": _I},
["id"],
False,
),
(
"read_transcript",
"Read part of one asset's text (its transcript, for audio and video), by seconds.",
Expand Down Expand Up @@ -386,9 +412,16 @@ def builtin_names():
# tools whose results the model may take as Lens's own word: the server's state and settings, the skills the person
# chose, what the graph holds, and what the built-in tools that change things report (ids, what waits for approval).
# What every other tool gives back is third-party text, marked as data (untrusted.py).
OWN_RESULTS = {"server_status", "find_model_servers", "read_settings", "file_storage", "graph_schema", "use_skill", "mark_activities"} | {
t[0] for t in TOOLS + ops_tools.ADMIN_TOOLS + ops_tools.FILE_TOOLS + AUTHOR_TOOLS + MEMORY_TOOLS if t[4]
}
OWN_RESULTS = {
"server_status",
"find_model_servers",
"read_settings",
"file_storage",
"graph_schema",
"use_skill",
"mark_activities",
"decline_approval",
} | {t[0] for t in TOOLS + ops_tools.ADMIN_TOOLS + ops_tools.FILE_TOOLS + AUTHOR_TOOLS + MEMORY_TOOLS if t[4]}


def read_before(history, summary=None):
Expand Down Expand Up @@ -970,6 +1003,46 @@ def t_mark_activities(self, ids, reviewed=None, important=None, hidden=None):
marks = event_marks.mark(self.db, self.account, known, flags)
return {"marks": marks, "unknown": unknown}, f"Marked {what}"

# ---- approvals (approvals.py): the person's own, read and declined; approving is theirs ----
def _approval_seen(self, a):
"""An approval of the person's this conversation may see: an automation's about a namespace it reads (or none),
or one the assistant asked for here."""
if not a or a.get("account") != self.account:
return False
if a.get("origin") or a.get("intent"):
return a.get("space") is None or a["space"] in self.readable
return a.get("chat") == self.chat

def t_approvals(self, id=None, status=None, origin=None, automation=None, before=None, limit=20):
from . import approvals

web = approvals.link(self.cfg, self.db, 0).removesuffix("/approvals?id=0")
if id is not None:
a = approvals.get(self.db, int(id))
if not self._approval_seen(a):
raise ValueError(f"no approval {id} of the person's here")
return {"approval": approvals.brief(a, web)}, f"Read the approval “{a.get('summary')}”"
n = max(1, min(int(limit or 20), 50))
st = None if status == "any" else status or "pending"
rows = approvals.listing(self.db, self.account, st, origin, automation, None, before, n)
out = [approvals.brief(a, web) for a in rows if self._approval_seen(a)]
nxt = rows[-1]["id"] if len(rows) >= n else None
return {"approvals": out, "next": nxt}, f"Read {len(out)} approval{'' if len(out) == 1 else 's'}"

def t_decline_approval(self, id):
from . import approvals

a = approvals.get(self.db, int(id))
if not self._approval_seen(a):
raise ValueError(f"no approval {id} of the person's here")
if a["status"] != "pending":
raise ValueError(f"approval {id} is {a['status']}, not waiting")
# what it read from outside Lens may have asked it to throw away what the person would pick: it asks first then
if self.wary():
return self._approval("decline_approval", {"approval": a["id"]}, f"Decline “{a.get('summary')}”" + self.why())
out = approve(self.db, self.cfg, a["id"], self.user, self.editable, "decline", via="agent")
return {"approval": a["id"], "status": out.get("status")}, f"Declined “{a.get('summary')}”"

def t_read_transcript(self, recording_id, start_seconds=0, end_seconds=None, max_lines=60):
cap = self.cfg["ai"].get("max_transcript_reads") or 20
if self.reads >= cap:
Expand Down Expand Up @@ -1447,6 +1520,8 @@ def approve(db, cfg, aid, user, editable, decision="approve", base=None, admin=F
a = approvals.get(db, aid)
if not a or a["status"] != "pending":
raise ValueError("nothing to approve")
if via == "agent" and decision != "decline":
raise PermissionError("approving is the person's own pick: an agent can only decline")
if a.get("intent"):
return approvals.decide(db, cfg, a, user, decision, params, via)
if decision in ("edit", "always"):
Expand Down Expand Up @@ -1477,6 +1552,16 @@ def _carry_out(db, cfg, a, user, editable, decision, base, admin, readable):
result = author_extension(db, a["tool"], args, user, admin, readable if readable is not None else editable, editable)
elif a["tool"] in ("change_settings", "create_namespace", "import_files", "import_urls", "invite_person", "change_person"):
result = ops_tools.apply(db, cfg, base or cfg, a["tool"], args, user, editable, admin)
elif a["tool"] == "decline_approval": # the person's yes to declining another of their approvals
from . import approvals

b = approvals.get(db, int(args["approval"]))
if not b or b.get("account") != a.get("account"):
raise ValueError("that approval is gone")
if b["status"] != "pending":
result = {"approval": b["id"], "status": b["status"]} # decided meanwhile: nothing to decline
else:
result = {"approval": b["id"], **approve(db, cfg, b["id"], user, editable, "decline", via="agent")}
elif a["tool"] == "mark_activities": # the person's own marks, on activities they could see when it was asked
flags = {k: v for k, v in args.items() if k in event_marks.FLAGS}
result = {"marked": len(event_marks.mark(db, user["id"], args["ids"], flags))}
Expand Down
41 changes: 37 additions & 4 deletions fastapi_backend/app/domain/approvals.py
Original file line number Diff line number Diff line change
Expand Up @@ -261,11 +261,11 @@ def edited(intent, new):

def can(row, via="web"):
"""What the person can do with a waiting approval from where they are (`via`: web for a signed-in session, chat for
their own chat app, token or oauth for an API key or an app, agent for MCP and the assistant)."""
their own chat app, token or oauth for an API key or an app, agent for MCP and the assistant, which only decline)."""
pending = row.get("status") == "pending"
intent = row.get("intent")
if not intent:
return {"approve": pending, "decline": pending, "edit": False, "always": False}
return {"approve": pending and via != "agent", "decline": pending, "edit": False, "always": False}
person = via in ("web", "chat")
return {
"approve": pending and person,
Expand Down Expand Up @@ -368,10 +368,13 @@ def decide(db, cfg, row, user, decision="approve", params=None, via="web"):


# ---------- the inbox ----------
def listing(db, account, status=None, origin=None, automation=None, chat=None, before=None, limit=50):
def listing(db, account, status=None, origin=None, automation=None, chat=None, before=None, limit=50, space=None):
"""The person's approvals, newest first, `limit` at a time before the id `before`: those with `status`, from
automations or chats (`origin`), from one automation, or asked in one chat."""
automations or chats (`origin`), from one automation, asked in one chat, or about one namespace (`space`)."""
where, v = ["account = $a"], {"a": account, "limit": max(1, min(int(limit or 50), 200))}
if space is not None:
where.append("space = $sp")
v["sp"] = int(space)
if status:
where.append("status = $s")
v["s"] = status
Expand All @@ -391,6 +394,36 @@ def listing(db, account, status=None, origin=None, automation=None, chat=None, b
return db.rows(f"SELECT {FIELDS} FROM approval WHERE {' AND '.join(where)} ORDER BY id DESC LIMIT $limit", **v)


def brief(a, web=""):
"""An approval as an agent reads it (MCP's and the assistant's `approvals`): what it would do and why, where the
person decides it (`url`, on the web app's address `web`), and what the agent may do (decline it). The assistant's
own proposals keep their args out; an intent's credential headers show as ••••."""
intent, why, o = shown(a.get("intent")) or {}, a.get("why") or {}, a.get("origin") or {}
out = {
"id": a["id"],
"status": a.get("status"),
"summary": a.get("summary"),
"from": "automation" if o else "chat",
"automation": o.get("automation"),
"automation_name": a.get("automation"),
"run": o.get("run"),
"node": o.get("node"),
"chat": a.get("chat"),
"action": intent.get("action"),
"effect": intent.get("effect"),
"params": intent.get("params"),
"why": {k: why[k] for k in ("activity", "kind", "summary") if why.get(k)} or None,
"created_at": a.get("created_at"),
"expires_at": a.get("expires_at"),
"decided_by": a.get("decided_by"),
"decided_at": a.get("decided_at"),
"error": a.get("error"),
"url": f"{web}/approvals?id={a['id']}",
"can": can(a, "agent"),
}
return {k: v for k, v in out.items() if v is not None}


def count(db, account):
"""How many approvals wait for the person: all of them, and those from automations."""
rows = db.rows(
Expand Down
Loading
Loading
0