8000 feat(sandbox): run the assistant's code on compute nodes by adeelahmad · Pull Request #572 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions changelog.d/572.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
- **The code sandbox can run on compute nodes.** Once the mesh registers the install's compute nodes, `run_python`
and `run_bash` take a `node` to run on, `sandbox_nodes` lists where code can run and why a node can't now, and a
conversation's sandbox stays where it last ran (moving starts a fresh one). Each node's sandboxes are stopped,
swept and deleted through its own gateway, and each run's ledger row names its node. Until nodes are registered
nothing changes ([Compute nodes](docs/assistant.md#compute-nodes)).
20 changes: 20 additions & 0 deletions docs/assistant.md
Original file line number Diff line number Diff line change
Expand Up @@ -536,6 +536,26 @@ people it's turned on for: those who can edit a namespace, and admins (`sandbox.
- **Its client.** Lens's own client for the gateway (the `sandbox` Python extra, OpenShell's SDK, Apache-2.0) is
fetched once the sandbox is on ([Components](components.md)).

### Compute nodes

In progress: the seam is built, and the mesh that joins the nodes is being built in its own pull request. Once the
install's mesh registers compute nodes (its other machines, each with its own sandbox gateway, reached only over the
mesh's own network), the assistant can run a conversation's code on one of them:

- **`sandbox_nodes`** lists where code can run: this machine (`here`) and each compute node, with whether it can run
code now and, if not, why (it hasn't reported in the last 3 minutes, it isn't on the mesh, its sandbox gateway isn't
running), its machine, its groups, and which one has this conversation's sandbox.
- **`node`** on `run_python` and `run_bash` (a node's id or name, or `here`) runs there. Left out, code runs where the
conversation's sandbox already is: this machine until a run asks for a node. A sandbox that moves starts fresh where
it moves to, without earlier files, and the one it left is deleted. The model is told to pick a node only when the
person asks for a machine or for more power than this one has.
- **The same rules everywhere.** A node's sandbox has the same image, limits and network setting as this machine's,
is stopped when idle and deleted when unused or with its conversation, by its own gateway. Each run's ledger row
names its node (`sandbox.run`, `node`). Turning off this machine's gateway leaves the nodes' sandboxes alone, and a
node removed from the mesh takes its sandboxes with it.
- **Nothing changes without nodes:** until the mesh registers some, there is no `sandbox_nodes` and no `node`, and
every sandbox runs on this machine's gateway.

### The gateway

Built: the Docker Compose install has a `sandbox` profile that runs OpenShell's gateway (release 0.1.4) next to Lens.
Expand Down
33 changes: 22 additions & 11 deletions fastapi_backend/app/domain/ai_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -713,7 +713,7 @@ def catalog(box):
if browsermod.enabled(box.cfg):
offered |= {t for t in GROUPS["browser"]["tools"] if t not in off}
if sandboxmod.offered(box.cfg, box.admin, bool(box.editable)): # every conversation of theirs has its sandbox
offered |= {t for t in GROUPS["code"]["tools"] if t not in off}
offered |= {t[0] for t in sandboxmod.tools() if t[0] not in off}
if researchmod.available(box.cfg): # every conversation of theirs can start research
offered |= {t for t in GROUPS["research"]["tools"] if t not in off}
# code that can reach another host can carry out what it's given
Expand Down Expand Up @@ -841,6 +841,8 @@ def system_note(self):
out = self._home_note() + (browsermod.NOTE if self.browser else "")
if self.sandbox:
out += sandboxmod.NOTE.format(network=sandboxmod.network_note(self.cfg))
if sandboxmod.nodes_on():
out += sandboxmod.NODES_NOTE
given = {s["function"]["name"] for s in self.specs()}
if self.web and webmod.NAMES & given:
out += webmod.NOTE
Expand Down Expand Up @@ -918,7 +920,7 @@ def specs(self):
if self.browser:
tools += browsermod.TOOLS
if self.sandbox:
tools += sandboxmod.TOOLS
tools += sandboxmod.tools()
if self.web:
tools += webmod.TOOLS
if self.chat is not None and researchmod.available(self.cfg):
Expand Down Expand Up @@ -1156,26 +1158,33 @@ def _sandbox(self, fn, *args):
except sandboxmod.SandboxError as e:
raise ValueError(str(e)) from None

def _sandbox_run(self, kind, source, files):
def _sandbox_run(self, kind, source, files, node=None):
source = str(source or "")
files = [str(f) for f in ([files] if isinstance(files, str) else files or [])]
node = str(node).strip() if node is not None and str(node).strip() else None
if sandboxmod.sends_out(self.cfg) and self.wary() and f"run_{kind}" not in self.allowed_out:
# code that reaches another host can carry out what the answer read: after text from outside Lens, which
# may have put it there, a run asks first
first = next((x.strip() for x in source.splitlines() if x.strip()), "")[:80]
args = store.clean({"code" if kind == "python" else "command": source, "files": files or None})
args = store.clean({"code" if kind == "python" else "command": source, "files": files or None, "node": node})
lang = "Python" if kind == "python" else "bash"
where = sandboxmod.reaches(self.cfg)
return self._approval(f"run_{kind}", args, f"Run {lang} in the code sandbox, which can reach {where}: {first}" + self.why())
out = self._sandbox(sandboxmod.run, self.account, kind, source, files)
on = f" on {node}" if node else ""
return self._approval(f"run_{kind}", args, f"Run {lang} in the code sandbox{on}, which can reach {where}: {first}" + self.why())
out = self._sandbox(sandboxmod.run, self.account, kind, source, files, None, node)
self.detail = sandboxmod.shown(out) # the output under the code, on the step the person sees
return out, sandboxmod.summary(kind, source, out)

def t_run_python(self, code, files=None):
return self._sandbox_run("python", code, files)
def t_run_python(self, code, files=None, node=None):
return self._sandbox_run("python", code, files, node)

def t_run_bash(self, command, files=None):
return self._sandbox_run("bash", command, files)
def t_run_bash(self, command, files=None, node=None):
return self._sandbox_run("bash", command, files, node)

def t_sandbox_nodes(self):
out = sandboxmod.node_list(self.db, self.chat)
n = sum(1 for x in out["nodes"] if x["id"] != sandboxmod.HERE)
return out, f"Looked at where code can run ({n} compute node{'s' if n != 1 else ''})"

def t_sandbox_files(self):
out = self._sandbox(sandboxmod.listing, self.account)
Expand Down Expand Up @@ -2290,7 +2299,9 @@ def _carry_out(db, cfg, a, user, editable, decision, base, admin, readable, via=
kind = a["tool"].removeprefix("run_")
source = args.get("code") if kind == "python" else args.get("command")
try:
result = {"result": sandboxmod.run(db, cfg, a["chat"], a.get("account") or user["id"], kind, source, args.get("files") or [])}
account = a.get("account") or user["id"]
got = sandboxmod.run(db, cfg, a["chat"], account, kind, source, args.get("files") or [], None, args.get("node"))
result = {"result": got}
except sandboxmod.SandboxError as e:
raise ValueError(str(e)) from None
elif a["tool"] in (
Expand Down
Loading
Loading
0