8000 feat(activity): graph changes in each entity's, namespace's and recording's activity history by adeelahmad · Pull Request #134 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version

## Unreleased

- **Graph changes in activity history.** An entity's, a namespace's and a recording's activity history now list
the graph changes that touched them, with who, how and why, and the version to find in Routines › History.
- **The entity graph keeps its history.** Every change to entities, their other names, links across namespaces and
"not the same" pairs is recorded as a numbered version: what changed, who (a person, a routine, the assistant,
analysis), through what (the web app, a token, an app, MCP) and why. `GET /api/v1/graph/history` lists versions
Expand Down
2 changes: 1 addition & 1 deletion DEVLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Todo:
- [x] The graph API and Cypher as of a version (`as_of`); the graph cache is keyed on the graph's version
- [ ] An as-of picker in the explorer (with the graph explorer thread)
- [x] Web app: a history timeline with diffs and rollback (Routines › History; an entity's History button)
- [ ] Graph events in each resource's activity history (with the activity history thread)
- [x] Graph events in each resource's activity history (an entity's, a namespace's, a recording's analysis)

Refine later: speakers (same-person links, names) in the history; a mention-level history.

Expand Down
4 changes: 3 additions & 1 deletion docs/activity.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,9 @@ A run row (a job or a routine run ending) repeats what its calls cost in all, an

## Reading it

A resource's history merges its ledger rows with its audit log entries (kind `change`), newest first:
A resource's history merges its ledger rows with its audit log entries and its changes to the entity graph (both kind
`change`; a graph change's action is `graph.<op>`, with its version, how it was made and why in `detail`; see
graph-history.md), newest first:

```http
GET /api/v1/activity?resource=routine:3&limit=100
Expand Down
4 changes: 4 additions & 0 deletions docs/graph-history.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,10 @@ version or a version's name: entities and their cross-namespace links are as the
speakers are today's, so an entity merged away since shows only if it is still mentioned. The explorer's graph is
cached per graph version, so any recorded change shows at once.

Graph changes also show in each resource's activity history (docs/activity.md, `GET /api/v1/activity`): an entity's
changes under `entity:<id>` (anyone who can read its namespace), a namespace's curation under `space:<id>` (what
analysis found is left out there), and what analysing a recording changed under `recording:<id>`.

## Refine later

- Speakers (same-person links, names) in the history.
Expand Down
9 changes: 8 additions & 1 deletion fastapi_backend/app/api/v1/routes/activity.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,12 @@ def _may_see(db: Db, user: CurrentUser, acl: Acl, resource: str) -> None:
if table == "space":
acl.need(int(key))
return
if table == "entity":
e = db.one("SELECT space FROM $r", r=activity.R("entity", int(key)))
if not e:
raise HTTPException(404, "not found")
acl.need(e["space"])
return
if table == "chat" and db.one("SELECT id FROM $r WHERE account = $a", r=activity.R("chat", int(key)), a=user.id):
return
raise HTTPException(403, "only admins see this resource's activity")
Expand All @@ -50,7 +56,8 @@ def resource_history(
limit: int = Query(100, ge=1, le=1000),
) -> list[ActivityEntry]:
"""A resource's h 8000 istory, newest first: API requests that changed it, calls made for it (models, embeddings, the
decision model, webhooks, web tools) with tokens and cost, runs that ended, and its audit log entries."""
decision model, webhooks, web tools) with tokens and cost, runs that ended, its audit log entries, and its graph
changes (an entity's, a namespace's, or what analysing a recording changed; docs/graph-history.md)."""
_may_see(db, user, acl, resource)
return activity.history(db, resource, limit, before, kind)

Expand Down
5 changes: 4 additions & 1 deletion fastapi_backend/app/domain/activity.py
Original file line number Diff line number Diff line change
Expand Up @@ -358,7 +358,7 @@ def _when(at):


def history(db, resource, limit=100, before=None, kind=None):
"""A resource's activity, newest first: its ledger rows and its audit log entries, merged."""
"""A resource's activity, newest first: its ledger rows, its audit log entries and its graph changes, merged."""
where, p = ["resources CONTAINS $res"], {"res": resource, "n": limit}
if before:
where.append("at < $before")
Expand Down Expand Up @@ -388,6 +388,9 @@ def history(db, resource, limit=100, before=None, kind=None):
}
for a in audit
]
from . import graph_history # the graph's own history (docs/graph-history.md)

rows += graph_history.activity(db, resource, before, limit)
rows.sort(key=lambda r: _when(r.get("at")), reverse=True)
rows = rows[:limit]
return rows
Expand Down
32 changes: 32 additions & 0 deletions fastapi_backend/app/domain/graph_history.py
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,38 @@ def versions(db, spaces=None, entity=None, before=None, limit=50):
return rows


def activity(db, resource, before=None, limit=100):
"""The graph changes in a resource's activity history (activity.history), newest first: an entity's
(`entity:<id>`), a namespace's other than what analysis found (`space:<id>`), and what analysing a recording
changed (`recording:<id>`). None for other resources."""
table, _, key = str(resource).partition(":")
if not key.isdigit():
return []
where, args = (
{"entity": "$k INSIDE entities", "space": "$k INSIDE spaces AND op != 'analysis'", "recording": "origin.recording = $k"}.get(table),
{"k": int(key), "n": int(limit)},
)
if where is None:
return []
if before:
where += " AND at < $b"
args["b"] = str(before)
rows = db.rows(f"SELECT {SUMMARY} FROM graph_event WHERE {where} ORDER BY version DESC LIMIT $n", **args)
return [
{
"id": None,
"at": r["at"],
"kind": "change",
"action": f"graph.{r['op']}",
"resources": [resource],
"email": r.get("actor"),
"ok": True,
"detail": store.clean({"version": r["version"], "via": r.get("via"), "why": r.get("why"), "changes": r.get("changes")}),
}
for r in rows
]


def event(db, version, spaces=None):
ev = db.one("SELECT *, record::id(id) AS version FROM $r", r=R("graph_event", int(version)))
if not ev or not _visible(ev, spaces):
Expand Down
2 changes: 1 addition & 1 deletion fastapi_backend/app/schemas/activity.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ class ActivityEntry(ResponseModel):
at: str
kind: Literal["in", "out", "run", "change"] = Field(
description="in: an API request; out: a call Lens made (a model, embeddings, the decision model, a webhook, a "
"web tool); run: a job or routine run ended; change: an audit log entry"
"web tool); run: a job or routine run ended; change: an audit log entry, or a change to the entity graph (action `graph.<op>`)"
)
action: str = Field(description="what it was: `POST /api/v1/routines/{rid}/run`, `model.chat`, `job.succeeded`, `routine.update`")
resources: list[str] = Field(default_factory=list, description="every resource it counts for, as table:id")
Expand Down
19 changes: 19 additions & 0 deletions fastapi_backend/tests/api/test_graph_history.py
Original file line number Diff line number Diff line change
Expand Up @@ -110,3 +110,22 @@ def names(**extra):
assert schema.status_code == 200
assert c.get("/api/v1/graph/schema", headers=vi, params={"as_of": "nope"}).status_code == 404
assert c.get("/api/v1/graph/schema", headers=vi, params={"as_of": 10**6}).status_code == 400


def test_graph_changes_in_activity_history(env):
c, ed, vi = env.c, env.h["editor"], env.h["viewer"]
dyno = env.eid("Dyno Therapeutics")
c.post(f"/api/v1/entities/{dyno}/rename", headers=ed, json={"name": "Dyno"})
got = c.get("/api/v1/activity", headers=vi, params={"resource": f"entity:{dyno}"})
assert got.status_code == 200, got.text
mine = [e for e in got.json() if e["action"].startswith("graph.")]
assert mine[0]["action"] == "graph.entity.rename" and mine[0]["kind"] == "change"
assert mine[0]["email"] == "ed@x.io" and mine[0]["detail"]["via"] == "web" and mine[0]["detail"]["version"]
# a namespace's history has its curation, and a recording's what analysing it changed
sid = env.db.one(f"SELECT space FROM entity:{dyno}")["space"]
acts = c.get("/api/v1/activity", headers=vi, params={"resource": f"space:{sid}", "kind": "change"}).json()
assert "graph.entity.rename" in {e["action"] for e in acts} and "graph.analysis" not in {e["action"] for e in acts}
rid = env.db.values("SELECT VALUE origin.recording FROM graph_event WHERE op = 'analysis' LIMIT 1")[0]
acts = c.get("/api/v1/activity", headers=ed, params={"resource": f"recording:{rid}"}).json()
assert "graph.analysis" in {e["action"] for e in acts}
assert c.get("/api/v1/activity", headers=vi, params={"resource": "entity:999999"}).status_code == 404
3 changes: 2 additions & 1 deletion nextjs-frontend/app/openapi-client/sdk.gen.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2502,7 +2502,8 @@ export class Activity {
* Resource History
*
* A resource's history, newest first: API requests that changed it, calls made for it (models, embeddings, the
* decision model, webhooks, web tools) with tokens and cost, runs that ended, and its audit log entries.
* decision model, webhooks, web tools) with tokens and cost, runs that ended, its audit log entries, and its graph
* changes (an entity's, a namespace's, or what analysing a recording changed; docs/graph-history.md).
*/
public static resourceHistory<ThrowOnError extends boolean = false>(
options: Options<ResourceHistoryData, ThrowOnError>,
Expand Down
2 changes: 1 addition & 1 deletion nextjs-frontend/app/openapi-client/types.gen.ts
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ export type ActivityEntry = {
/**
* Kind
*
* in: an API request; out: a call Lens made (a model, embeddings, the decision model, a webhook, a web tool); run: a job or routine run ended; change: an audit log entry
* in: an API request; out: a call Lens made (a model, embeddings, the decision model, a webhook, a web tool); run: a job or routine run ended; change: an audit log entry, or a change to the entity graph (action `graph.<op>`)
*/
kind: "in" | "out" | "run" | "change";
/**
Expand Down
5 changes: 5 additions & 0 deletions nextjs-frontend/components/costs/costs.tsx
< E571 /tr>
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { useState } from "react";

import { Activity, Budgets } from "@/app/openapi-client";
import type { ActivityEntry, BudgetSet, BudgetStatus, ResourceSpend } from "@/app/openapi-client/types.gen";
import { OPS } from "@/components/routines/history-model";
import { Badge, type Tone } from "@/components/ui/badge";
import { Banner } from "@/components/ui/banner";
import { Button } from "@/components/ui/button";
Expand Down Expand Up @@ -132,6 +133,10 @@ function actionText(e: ActivityEntry): string {
if (a.startsWith("notify.")) return `Notification · ${a.slice(7)}`;
if (a === "tool.http") return "Web tool";
if (a.startsWith("budget.")) return a === "budget.over" ? "Budget used up" : "Budget nearly used";
if (a.startsWith("graph.")) {
const d = (e.detail ?? {}) as { version?: number; why?: string | null };
return `Graph · ${OPS[a.slice(6)] ?? a.slice(6)}${d.version != null ? ` · v${d.version}` : ""}${d.why ? ` · ${d.why}` : ""}`;
}
return a;
}

Expand Down
4 changes: 2 additions & 2 deletions nextjs-frontend/openapi.json
Original file line number Diff line number Diff line change
Expand Up @@ -2720,7 +2720,7 @@
"activity"
],
"summary": "Resource History",
"description": "A resource's history, newest first: API requests that changed it, calls made for it (models, embeddings, the\ndecision model, webhooks, web tools) with tokens and cost, runs that ended, and its audit log entries.",
"description": "A resource's history, newest first: API requests that changed it, calls made for it (models, embeddings, the\ndecision model, webhooks, web tools) with tokens and cost, runs that ended, its audit log entries, and its graph\nchanges (an entity's, a namespace's, or what analysing a recording changed; docs/graph-history.md).",
"operationId": "resource_history",
"parameters": [
{
Expand Down Expand Up @@ -22161,7 +22161,7 @@
"change"
],
"title": "Kind",
"description": "in: an API request; out: a call Lens made (a model, embeddings, the decision model, a webhook, a web tool); run: a job or routine run ended; change: an audit log entry"
"description": "in: an API request; out: a call Lens made (a model, embeddings, the decision model, a webhook, a web tool); run: a job or routine run ended; change: an audit log entry, or a change to the entity graph (action `graph.<op>`)"
},
"action": {
"type": "string",
Expand Down
Loading
0