8000 GitHub - simon-inkie/inkie-auto-mode: Don't let your agent delete the database. Hybrid static + LLM permission classifier for Claude Code, OpenClaw and Cursor. · GitHub
[go: up one dir, main page]

Skip to content

About

Don't let your agent delete the database. Hybrid static + LLM permission classifier for Claude Code, OpenClaw and Cursor.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

io-auto-mode

A hybrid static + LLM exec security classifier for AI coding agents. Stops prompt injection and accidental destruction without making automation impossible.

Status: early (v0.1.0). OpenClaw, Claude Code, Cursor, Antigravity (agy), and Codex adapters all shipping. Claude Code adapter has been in production use for ~2 weeks; Cursor, agy and Codex adapters are fresh. Tests cover static patterns, file zones, runtime schema mappings, MCP policy/redaction and adapter decisions (340 tests). Looking for feedback from people running agentic dev workflows.


⚠️ Important — please read

This is a security product. Read the source before you trust it with real systems. It runs in your tool-call hot path. Bugs, misconfigurations, prompt-injection of the classifier itself, model outages — any of these can let through commands that delete data, ship code, or move money.

By using io-auto-mode you accept that:

  • It is provided as-is, no warranty, no fitness-for-purpose guarantee (per the MIT licence)
  • You are responsible for understanding what it does and reviewing the classifier rules + zone config for your environment before relying on it
  • The author(s) accept no liability for damage arising from bugs, misconfiguration, model failures, or any other cause
  • This is not a substitute for backups, version control, real least-privilege OS sandboxing, code review, or a human in the loop on high-stakes actions

Treat it as a layer in defence-in-depth, not the only layer.


The problem

Permissioning for AI coding agents is currently binary:

  • Prompt for everything — annoying, breaks any kind of automation
  • Allow everything — yolo mode; one prompt-injection or one misread instruction and you've lost a database

There's no middle path. Static allow / deny lists can't capture the thing that actually matters: a git commit during a coding task is expected; the same command while summarising a PDF is suspicious. The risk is contextual.

io-auto-mode is a permission classifier that evaluates every shell-tool / file-tool call with the conversation context before it runs.


How it works

Three layers, fail-closed:

Agent requests tool call
        │
        ▼
┌─── Static Pattern Layer (sub-ms) ───────────────┐
│  Hard-coded dangerous regex → BLOCK             │
│  Hard-coded safe regex      → ALLOW             │
│  Otherwise                  → fall through      │
└────────────────────┬────────────────────────────┘
                     │
                     ▼
┌─── Stage 1 — Fast LLM (~1s p50) ────────────────┐
│  Gemini Flash, plain text output                │
│  Biased toward blocking                         │
│  ALLOW → proceed                                │
│  BLOCK → escalate to Stage 2                    │
└────────────────────┬────────────────────────────┘
                     │ (only on BLOCK)
                     ▼
┌─── Stage 2 — Thinking LLM (~5s p50, blocks only)┐
│  Gemini w/ thinking, JSON output                │
│  Chain-of-thought reasoning                     │
│  Final decision + reason                        │
└─────────────────────────────────────────────────┘

Plus a separate path-based classifier for Read / Write / Edit tool calls — pure regex, no LLM, sub-millisecond.

Design principles

  • Fail closed. Every error case (API failure, parse failure, timeout) returns BLOCK. Never fail open.
  • Context-aware. The classifier sees the conversation history. Same command, different risk depending on source and task.
  • Prompt-injection hardened. Assistant text is excluded from classifier input — only user turns and tool_use blocks. The model can't craft text that influences its own classification.
  • Symlink-resolved. Paths are resolved through realpathSync() before pattern-matching, so an attacker can't escape an allowWrite zone via a symlink to a credential file.
  • Fast by default. Static layer at sub-millisecond, Stage 2 only runs on blocks. The happy path is cheap.
  • Tamper-resistant. Dangerous-pattern lists are compiled into code, not loaded from editable files at runtime.
  • Transparent. Blocked actions surface a clear reason; allowed actions are silent.
  • Zero runtime dependencies. Core has none. Both shipped adapters bundle into single self-contained ESM files via esbuild. The OpenClaw adapter pulls in openclaw as an optional peer — install only if you use that runtime. npm install -g io-auto-mode adds one package, ~400KB on disk. Nothing else can deprecate, break, or get supply-chain-attacked under us.

Live performance

Pulled from ~/.io-auto-mode/auto-mode-log.jsonl over ~3,000 real classifications across our agents:

Stage n p50 p90 avg
Static 792 0ms 0ms 0ms
Stage 1 (Gemini Flash) 951 799ms 1216ms 934ms
Stage 2 (Gemini Flash w/ thinking, blocks only) 1018 4533ms 6583ms 4725ms
Fallback (one-shot retry) 235 1509ms 2355ms 1734ms

Caveat: these were measured over a flaky home wifi connection — your numbers on a stable link will be lower, especially Stage 2.

The headline takeaway is the shape, not the absolute numbers: ~80% of decisions resolve at the static layer at 0ms; everything else pays a sub-second LLM cost; Stage 2 only fires on blocks. The happy path is cheap.


Cost

At Gemini 2.5 Flash pricing (~$0.30/M input tokens, ~$2.50/M output tokens), a casual session of ~50 tool calls/day costs single-digit pence. A heavy agentic-dev day (300+ tool calls) sits around 20-50p. Most of that is Stage 2 thinking output, which only fires on blocks — so the bill scales with how often the classifier escalates, not how often the agent runs commands.

Static-layer hits (~30% of all calls in our usage) are free. Stage 1 is one short LLM call per agent action; Stage 2 is rarer + slightly chunkier. You can swap any stage to a local LLM via config if you want zero cost — see specs/ai-sdk-migration.md for the planned Ollama / LM Studio path.


Benchmark evidence

The INK-923 benchmark compares Jev with pinned Gemini variants on fixture data; the raw evidence bundle includes the methodology and results. It is benchmark-only evidence, not a live provider switch.


What's in the repo

core/                      Shared classifier logic — no platform deps
  classifier.ts            Hybrid static + LLM pipeline
  static-patterns.ts       Hard-coded regex layer
  transcript.ts            Conversation context extraction
adapters/
  openclaw/                OpenClaw plugin (reference impl)
  claude-code/             Claude Code PreToolUse hooks (Bash + MCP + file)
  cursor/                  Cursor hooks (prompt + shell + MCP + file + write/edit)
  antigravity/             Antigravity (agy) PreToolUse classifier hook
  codex/                   Codex PreToolUse classifier hook (Bash + apply_patch)
specs/                     Future-work design specs (AI SDK migration, ...)
docs/                      Contributor docs — adapter guide etc.
tests/                     Tier 1 tests — static patterns + file-hook zones + cursor/agy/codex mappings
INSTALL.md                 Install + config guide for all adapters

Small, focused codebase. Core has no runtime deps. The OpenClaw adapter pulls in openclaw; the Claude Code adapter is dep-free.


Quick start (OpenClaw)

See INSTALL.md for the full guide. TL;DR:

# 1. Register a Gemini key (used by all stages by default)
openclaw models auth login --provider google --method gemini-api-key

# 2. Add to openclaw.json
openclaw config patch '{"plugins":{"load":{"paths":["/path/to/io-auto-mode"]},"entries":{"io-auto-mode":{"enabled":true,"config":{"mode":"classify"}}}}}'

# 3. Restart
openclaw gateway restart

# 4. Watch decisions land
tail -f ~/.openclaw/workspace/memory/auto-mode-log.jsonl

Modes

Mode Behaviour
classify (default) Run the three-layer pipeline
yolo Allow everything (development / debugging)
strict Block everything not on the static allowlist

Quick start (Claude Code)

See INSTALL.md for the full guide. TL;DR:

# 1. Clone + install + build the adapter
git clone https://github.com/simon-inkie/inkie-auto-mode.git io-auto-mode
cd io-auto-mode
pnpm install
node scripts/build.mjs

# 2. Drop your Gemini key where the hooks can read it
mkdir -p ~/.io-auto-mode
echo 'GOOGLE_GENERATIVE_AI_API_KEY=your-key-here' >> ~/.io-auto-mode/.env

# 3. Wire the three PreToolUse matchers into ~/.claude/settings.json
#    (full snippet in INSTALL.md — Bash + mcp__.* + Read|Write|Edit)

# 4. Restart your Claude Code session, then watch decisions land
tail -f ~/.io-auto-mode/auto-mode-log.jsonl

Three matchers register: Bash and MCP calls use the LLM-backed fail-closed classifier; file operations use the path classifier.


Quick start (Cursor)

See INSTALL.md for the full guide. TL;DR:

# 1. Clone + install + build the adapter
git clone https://github.com/simon-inkie/inkie-auto-mode.git io-auto-mode
cd io-auto-mode
pnpm install
node scripts/build.mjs

# 2. Drop your Gemini key where the hooks can read it
mkdir -p ~/.io-auto-mode
echo 'GOOGLE_GENERATIVE_AI_API_KEY=your-key-here' >> ~/.io-auto-mode/.env

# 3. Wire five hooks into ~/.cursor/hooks.json
#    (full snippet in INSTALL.md — beforeSubmitPrompt, beforeShellExecution,
#     beforeMCPExecution, beforeReadFile, preToolUse with matcher Edit|Write)

# 4. Restart Cursor, then watch decisions land
tail -f ~/.io-auto-mode/auto-mode-log.jsonl

Five hooks total: shell and MCP classifiers, a file classifier, and prompt capture that gives Stage 2 conversation context for prompt-injection hardening.


Quick start (Antigravity / agy)

Antigravity (CLI: agy) is the non-Anthropic generalist runtime on the team. Its PreToolUse hook fires before every tool call; the adapter gates run_command and canonical mcp__server__tool calls through the shared classifier.

Contract: the agy hook adapter reads camelCase JSON on stdin and emits {"allowTool": bool} on stdout, exiting 0 on every path. Shell infrastructure failures remain fail-open; MCP infrastructure failures fail closed. Read-only tools are allowed without classification.

# 1. Clone + install + build the adapter
git clone https://github.com/simon-inkie/inkie-auto-mode.git io-auto-mode
cd io-auto-mode
pnpm install
node scripts/build.mjs

# 2. Drop your Gemini key where the hook can read it
mkdir -p ~/.io-auto-mode
echo 'GOOGLE_GENERATIVE_AI_API_KEY=your-key-here' >> ~/.io-auto-mode/.env

# 3. Wire the PreToolUse hook into your agent's .agents/hooks.json
#    Replace <repo-path> with the absolute path you cloned to.
{
  "your-agent-name": {
    "PreToolUse": [
      {
        "matcher": "*",
        "hooks": [
          {
            "type": "command",
            "command": "<repo-path>/adapters/antigravity/bin/pretooluse-classify.sh"
          }
        ]
      }
    ]
  }
}

A ready-to-edit template lives at adapters/antigravity/hooks/hooks.json.

Shares the same core classifier, ~/.io-auto-mode/config.json config, and ~/.io-auto-mode/.env API keys as the Claude Code and Cursor adapters -- one config to maintain across all runtimes.

What it catches (illustrative):

  • rm -rf ~/project, dropping a prod table, piping an untrusted curl into a shell -> blocked (destructive / irreversible).
  • git push --force to a shared branch -> blocked or flagged, depending on your config.
  • cat README.md, npm test, ls -la, scoped builds -> allowed.

Exact verdicts come from your config.json plus the LLM stage, so they adapt to context rather than a fixed denylist.

Scope (v1): the agy adapter gates run_command and canonical MCP calls. File read/write zone classification remains on the roadmap.


Quick start (Codex)

Codex fires a PreToolUse hook before every tool call. The adapter gates Bash, apply_patch and canonical mcp__server__tool calls through the same three-layer classifier as the other runtimes.

Contract: the hook reads snake_case request JSON on stdin and emits a response JSON on stdout, always exiting 0. Allow is an empty body ({}); deny is hookSpecificOutput.permissionDecision: "deny" plus a non-empty permissionDecisionReason. Unknown non-MCP tools pass through with a stderr warning.

apply_patch is classified alongside Bash on purpose: a patch can write a malicious script, overwrite the hook itself, or append to ~/.bashrc, laundering a payload past a shell-only gate.

# 1. Clone + install + build the adapter
git clone https://github.com/simon-inkie/inkie-auto-mode.git io-auto-mode
cd io-auto-mode
pnpm install
node scripts/build.mjs

# 2. Drop your Gemini key where the hook can read it
mkdir -p ~/.io-auto-mode
echo 'GOOGLE_GENERATIVE_AI_API_KEY=your-key-here' >> ~/.io-auto-mode/.env

# 3. Wire the PreToolUse hook into .codex/hooks.json
#    Replace <repo-path> with the absolute path you cloned to.
{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": ".*",
        "hooks": [
          {
            "type": "command",
            "command": "<repo-path>/adapters/codex/bin/pretooluse-hook.sh"
          }
        ]
      }
    ]
  }
}

A ready-to-edit template lives at adapters/codex/hooks/hooks.json.

Shares the same core classifier, ~/.io-auto-mode/config.json config, ~/.io-auto-mode/.env API keys, and ~/.io-auto-mode/auto-mode-log.jsonl ledger as every other adapter -- decisions from this runtime are tagged "adapter": "codex".

Scope: Bash, apply_patch and canonical MCP calls are classified. Codex has no hook-level "ask" state, so an ask decision collapses to deny (conservative). Unknown non-MCP tools pass through with a stderr warning that records only the tool name and input keys.

MCP arguments use a dedicated prompt rather than shell-command rules. Credential-shaped keys and values are redacted before model calls and ledger writes. Optional mcpAllowPatterns and mcpBlockPatterns regexes match the full canonical name; block wins when both match.

Codex also has a native MCP approval layer. For a server protected by this trusted PreToolUse hook, set its native mode to approve so Codex does not show a second prompt before the hook's decision:

[mcp_servers.apify]
default_tools_approval_mode = "approve"

Use a per-tool override for a narrower rollout:

[mcp_servers.apify.tools."call-actor"]
approval_mode = "approve"

Configuration

All options under plugins.entries.io-auto-mode.config:

Option Default Description
mode classify classify / yolo / strict
stage1Model google/gemini-3.8-flash Fast LLM for Stage 1
stage1Fallback google/gemini-3.8-flash Stage 1 fallback
stage2Model google/gemini-3.8-flash Thinking LLM for Stage 2
stage2Fallback google/gemini-3.8-flash Stage 2 fallback
userAllowPatterns [] Extra shell regex patterns always allowed
userBlockPatterns [] Extra shell regex patterns always blocked
mcpAllowPatterns [] MCP canonical-name regexes allowed before the LLM
mcpBlockPatterns [] MCP canonical-name regexes blocked before allow or the LLM

Defaults are all-Gemini for cost + latency. Any provider OpenClaw supports (Anthropic, OpenAI, etc.) can be swapped in by changing the model strings — see specs/ai-sdk-migration.md for the planned migration to AI SDK that makes this even smoother (Ollama / LM Studio / self-hosted included).

For the Read / Write / Edit file-tool classifier, configure fileZones (allowRead / allowWrite / deny) in either user-global (~/.io-auto-mode/config.json) or per-project (<project>/.io-auto-mode.json). Layers merge.

Optional: Mission Control consent receipts (off by default)

The Claude Code adapter can accept a signed consent receipt — evidence that a human tapped a decision card — and upgrade an ask or block verdict to allow. It is upgrade-only (an allow is never downgraded) and fails closed on any error.

It is an authorisation bypass by design, so it is disabled unless BOTH a consent directory and a signing key are configured — the directory from either the IO_AUTO_MODE_CONSENT_DIR env var or consentDir in ~/.io-auto-mode/config.json, and the key from the MC_CONSENT_HMAC_KEY env var (env-only; never read from the config file). A receipt only counts if its HMAC verifies, it is inside its TTL, and its scope explicitly names the tool and matches the command — an empty scope authorises nothing. ask upgrades are multi-use within the TTL; block upgrades additionally require a confirmed: true two-tap and are consumed single-use via atomic rename before the allow is emitted.

The consent directory must be write-denied to the agents being gated (put it on your fileZones deny list) — an agent that can write its own receipts can authorise itself. The HMAC is symmetric, so this defends against sloppy self-authorisation and prompt-injected forgery, not against an adversary already running as the same uid.

Full setup, the receipt schema, and the threat boundary: INSTALL.md.


Decision log

Every classification is logged with timestamp, command, stage (static / stage1 / stage2 / error), decision, duration, model used, and chain-of-thought reasoning where applicable:

~/.openclaw/workspace/memory/auto-mode-log.jsonl

Useful both for debugging surprising blocks and for reviewing what your agent has been up to.


Status & roadmap

  • OpenClaw adapter (exec + bundle MCP classifier)
  • Per-project config overlays
  • Static-layer hardening (top-level critical-dir rule, mid-path glob matching)
  • Claude Code adapter (PreToolUse hooks; in production ~2 weeks)
  • Cursor adapter (beforeSubmitPrompt + shell + MCP + file hooks; prompt-injection-hardening parity with Claude Code)
  • Antigravity (agy) adapter (PreToolUse matcher:* -- run_command + canonical MCP classifier)
  • Codex adapter (PreToolUse matcher:.* -- Bash + apply_patch + MCP classifier, empty-body-allow contract, fail-open)
  • Tier 1 tests -- static patterns + file zones + runtime schema mappings + MCP policy/redaction (tsx --test)
  • CI -- GitHub Actions running typecheck + tests on every push / PR
  • Tier 2 tests — full classifier pipeline (mocked LLM) + transcript prompt-injection coverage
  • MCP tool classifier — shared canonical policy and redaction across all five adapters
  • AI SDK migration — provider-agnostic model calls (spec)

See BACKLOG.md for more.


Why I built it

I started this project because I wanted to give my OpenClaw agent Io full autonomy without sleepless nights wondering if it was halfway through rm -rf on my home directory, force-pushing to main, or quietly trashing a production database because some PDF it was summarising contained an instruction it took too literally.

The risk isn't theoretical. Recently a Cursor coding agent — running on Claude — wiped a company's database in nine seconds. The backups went with it. I'd been designing for exactly that failure mode for months — the news just confirms why a permission classifier with teeth has to exist before agents are trusted with real systems.

The "ask before everything" mode kills automation. "Allow everything" mode puts your data one prompt-injection away from gone. There was no middle path that understood context — that the same git push is fine during a coding task and suspicious during a PDF summary. So I built one.

Anthropic shipped their own auto mode shortly after I started this. Theirs is a great default for individual Claude Code users on Sonnet. Where io-auto-mode is shaped differently:

  • Rules over prompts. Patterns and file zones are declarative JSON + regex. Diffable, code-reviewable, version-controlled — same shape as any other ops config. We ship our rules through CI alongside the code they protect, which is how the team and our agents share write access to the same systems without sleepless nights.
  • Multi-platform. Runs in front of OpenClaw (chat-platform agents) and Claude Code, not just one runtime.
  • Provider-flexible. Pick your model per stage — Gemini Flash for the hot path, Anthropic / OpenAI / a local LLM for thinking. AI SDK migration (spec) makes Ollama / LM Studio first-class, which matters for cost and privacy.
  • File-op classifier. Separate path-based allow / deny / write zones for Read / Write / Edit tool calls — useful for stopping an agent reading ~/.aws/credentials or writing outside its project root, without burning an LLM call per file touch.

If you're running agents with hands on real systems and you've been white-knuckling through --dangerously-skip-permissions, this is for you.


Credits

Built by Simon Dixon (@inkie) and Io, his AI coordinator, starting April 2026. Platform-side implementation by Doctor Two, a Claude Code agent specialising in classifier internals and the OpenClaw runtime.


Contributing

Early days. Issues + discussion welcome on GitHub. If you're running an agentic dev workflow and have an opinion about classifier behaviour, drop a note — failure modes from the wild are the most useful input.

Writing an adapter for a new runtime? See docs/adapter-guide.md — five contracts, ~150 lines of glue, three shipped adapters as worked examples.


License

MIT — see LICENSE.

About

Don't let your agent delete the database. Hybrid static + LLM permission classifier for Claude Code, OpenClaw and Cursor.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

0