The most dangerous dependency in a repository isn't in requirements.txt. It's the one that exists only on the maintainer's laptop. Phantom discovers undocumented setup assumptions that stop a new contributor's first run.
Why this matters (2025 research):
- Atlassian found 50% of developers lose 10+ hours a week to inefficiencies; finding information and switching contexts are among the largest sources of friction. State of DevEx.
- Stack Overflow found 66% are frustrated by AI answers that are almost correct, and more respondents distrust AI accuracy than trust it. Developer Survey.
- DORA found about 90% of technology professionals use AI at work, while stressing engineering rigor and fast feedback for sustainable gains. DORA's discussion of its 2025 findings.
These figures describe broader development friction, not time spent setting up a repository. We do not claim a measured setup-time saving.
RUN → NAME → TRACE → FIX → LOCK
| Stage | Command | What happens |
|---|---|---|
| RUN | phantom haunt |
Runs the committed README's shell steps in a fresh Docker container |
| NAME | phantom haunt |
Deterministic rules name the missing requirement; a local model may explain it |
| TRACE | phantom bisect |
Binary search over first-parent history finds the commit that broke setup |
| FIX | phantom exorcise |
Patches the README, then proves the fix: it must fail before and pass after on the same commit |
| LOCK | phantom exorcise, phantom doctor |
Records verified requirements in assumptions.lock and checks a machine against them |
AI explains; rules decide. The model never runs commands, never writes shell, and
can never mark anything verified. Only the before/after container run assigns
verified_fix or verified_together, and only verified requirements enter the lock.
Requires Python 3.11+, Git, and Docker Desktop running Linux containers.
python -m venv .venv
source .venv/bin/activate
pip install -e '.[dev]'
phantom --helpAI explanations are optional. Install Ollama and pull the model:
ollama pull gemma3:4bThese steps were run on macOS (Apple Silicon, linux/arm64 containers). The
generator builds demo/haunted-todo, an eight-commit repository whose fourth commit
starts reading DATABASE_URL without documenting it. The SHAs are identical on
every run.
python scripts/make_haunted_todo.py
cd demo/haunted-todo1. Haunt: run setup in a blank container (exit 1, because a requirement is missing)
phantom haunt --no-aiPhantom
❌ step 1 python app.py init
╭──────────────────────────────── Setup finding ─────────────────────────────────╮
│ Status hypothesis │
│ Type / Name env_var / DATABASE_URL │
│ Evidence KeyError: 'DATABASE_URL' │
│ Cause rule_matched │
│ Explanation AI disabled with --no-ai │
│ Suggested fix (not yet verified) export DATABASE_URL=sqlite:///./todo.db │
╰────────────────────────────────────────────────────────────────────────────────╯
2. Bisect: find the commit that broke setup (exit 0 when a culprit is found)
phantom bisect --no-ai✅ 2e797c7 pass (cached)
❌ 54490e6 fail (cached)
❌ a124636 fail (cached)
✅ 7ae2de7 pass (cached)
✅ 1d15a65 pass (cached)
╭─────────────────────────────── First bad commit ───────────────────────────────╮
│ Commit a124636 │
│ Author Haunted Maintainer <maintainer@example.com> │
│ Date 2026-10-04T10:00:00+05:30 │
│ Subject refactor: make database location configurable │
│ Requirement env_var / DATABASE_URL │
│ Evidence KeyError: 'DATABASE_URL' │
│ Explanation AI disabled with --no-ai │
│ │
│ .env.example | 1 + │
│ app.py | 2 +- │
│ 2 files changed, 2 insertions(+), 1 deletion(-) │
│ diff --git a/app.py b/app.py │
│ @@ -12,7 +12,7 @@ def main() -> None: │
│ add.add_argument("text") │
│ commands.add_parser("list") │
│ args = parser.parse_args() │
│ - db_path = "todo.db" │
│ + db_path = os.environ["DATABASE_URL"].removeprefix("sqlite:///") │
│ with sqlite3.connect(db_path) as db: │
│ if args.command == "init": │
│ db.execute("CREATE TABLE IF NOT EXISTS todos (id INTEGER PRIMARY │
╰────────────────────────────────────────────────────────────────────────────────╯
This is the cached rerun (0.3 s); the first run probed the same five commits in about 3 s here.
3. Exorcise: verify a README fix and write the lock
phantom exorcise --no-aiPhantom exorcise 54490e6
❌ before: step 1 `python app.py init` fails
✅ after: every README step passes with the patch (same commit, fresh container)
Requirement Evidence Cause Fix verified Introduced in
env_var DATABASE_URL KeyError: 'DATABASE_URL' rule_matched ✅ alone a124636
+# required: DATABASE_URL (added by phantom)
+export DATABASE_URL=sqlite:///./todo.db
Apply these changes? [y/N]:
Answering N writes nothing. Answering y (or passing --apply) writes
README.md and assumptions.lock to the working tree. Phantom never commits:
requirements:
- id: env.DATABASE_URL
type: env_var
name: DATABASE_URL
evidence: 'step 1 `python app.py init`: KeyError: ''DATABASE_URL'''
cause: rule_matched
introduced_in: a124636
fix: export DATABASE_URL=sqlite:///./todo.db
verification: {command_hash: 079f969a…, before: fail, after: pass}
status: verified_fix
check: {kind: env_set}4. Doctor: check your own machine against the lock (no Docker, no AI)
phantom doctorPhantom doctor
❌ env.DATABASE_URL DATABASE_URL is not set
hint: export DATABASE_URL=sqlite:///./todo.db
1 of 1 checks failed.
DATABASE_URL=sqlite:///./todo.db phantom doctorThat run exits 0 with ✅ env.DATABASE_URL. To reset the demo:
git checkout README.md && rm assumptions.lockDrop --no-ai from haunt or bisect to add a local model's explanation. With
gemma3:4b pulled, haunt on the demo shows a validated "Gemma explanation" and took
about 2.5 s here, or about 5 s while the model loads on the first call.
| Command | Exit 0 | Exit 1 | Exit 2 |
|---|---|---|---|
phantom haunt [PATH] [--rev HEAD] [--no-ai] [--json] |
setup passed | requirement found | infrastructure or inconclusive |
phantom bisect [PATH] [--good SHA] [--bad HEAD] [--no-ai] |
culprit found | bad boundary or unreadable history | skips hide the culprit, or infrastructure |
phantom exorcise [PATH] [--rev HEAD] [--apply] [--no-ai] [--copilot] |
applied, declined, or nothing to fix | no fix could be verified | dirty README, invalid lock, or inconclusive |
phantom doctor [--lock assumptions.lock] |
all checks pass | a check failed | lock missing or invalid |
Errors are one-line messages, never tracebacks. Examples: Docker not running, a
bad --rev, a missing README, an unreadable lock, Ollama down, or malformed model
output. These failure modes are covered by tests and were run live.
Reads the README and source as committed at --rev, so uncommitted changes are
excluded. It runs fenced bash/sh/shell/console/zsh (or untagged) blocks in
one bash session in a fresh python:3.12-slim container. A block whose first command
starts with a $ or % prompt is read as a transcript, so only prompted lines
run and example output is ignored. Clone boilerplate is skipped because the
repository is already copied to /repo; a skipped git clone ... && cd repo returns
the next step to /repo, and anything after it on the same line still runs.
Servers and dev watchers (npm start, flask run, uvicorn, manage.py runserver,
rails s and similar, from a fixed list) are started in the background when they
are simple standalone commands. Commands containing shell operators, substitutions,
or redirections stay in the foreground, so their remaining work is not skipped. If one is
still running after 10 s it passes as "started" and stays up for later steps, such
as a curl against it; if it exits sooner, its real exit code and output are used.
The Copilot workflow uses the same template for standalone servers.
The container is never given host mounts or host environment variables. It is removed afterwards, even after a timeout. The resolved image digest and platform are recorded.
--json prints a single JSON object (run, finding, ai_note, risk_flags)
with no spinner or prose.
Haunt also prints risk flags for risky README commands. They are warnings only: they never block a run or change the exit code. The flagged patterns are:
curl/wgetpiped into a shellsudochmod 777- unpinned
pip install npm i -g--insecureorcurl -k- exporting a
*TOKEN,*SECRET,*KEYor*PASSWORDvariable with a non-placeholder value. The flag names the variable but never prints the value.
Missing-command diagnosis also works with --no-ai: rules recognize shell errors
and generic messages such as Could not find executable 'cmake' or
no 'java' command could be found. Explicit missing-environment-variable messages
are recognized too. If an error names both a missing executable and a related
variable (such as JAVA_HOME), the missing executable takes priority. These are
hypotheses based on captured output; missing tools do not get automatic install fixes.
Without --no-ai, Phantom asks Ollama for schema-constrained JSON at temperature 0,
with a 20 s timeout. It uses gemma3:4b and falls back to qwen2.5-coder:14b if
Gemma is not pulled. The output label names whichever model actually answered.
An explanation is kept only if all of these hold:
- its evidence line appears verbatim in the step output;
- it names the same requirement as the rule;
- any version claim matches the repository's version files.
Anything else is dropped, and the output says why. If Ollama is unreachable, the rule finding is still shown.
Parses HEAD's README once and runs those same commands against older commits, using
a first-parent binary search. --good must pass, --bad must fail, and --good
must be on --bad's first-parent history.
Infrastructure failures count as skips, never as broken code. If skips hide the
culprit, the possible range is printed. Pass and fail results are cached in
~/.cache/phantom/ by commit, image digest, platform and command hash; skips are
never cached. The model is asked once, after the search, to explain the culprit's
diff.
Refuses to run if the working-tree README differs from the committed one. It runs
the original README (which must fail). It then inserts a fixed-template command
directly before the failing line: export NAME='value' with shell quoting, or
cp NAME.example NAME. Finally it re-parses the patched README and runs it on the
same commit in a new container (which must pass).
Up to three rounds handle a later step failing next. Each fix is also re-tested
alone; fixes that only pass together are marked verified_together. Requirements
no rerun proved are listed as "needs a human" and never enter the lock. Existing
lock entries are merged by id. introduced_in is filled from the last
phantom bisect of the same requirement, and only when that bisect used the same
README steps and its commits are ancestors of the revision being verified.
--copilot also proposes .github/workflows/copilot-setup-steps.yml for the GitHub
Copilot coding agent. The workflow is built only from the patched README commands
that just passed in the container, and they run in one bash step to match the
single shell session they were verified in: no errexit or pipefail, and the job
stops at the first step that exits non-zero, as the verifier does. It appears in the
same diff and the same confirmation, and an existing workflow is never overwritten.
Checks each lock entry locally:
env_set: the variable is set. The value is never printed or validated.file_exists: the file exists.which: the tool is on PATH.version: comparestool --versionagainst the spec.tcp: something accepts a connection on that localhost port.
Each failure prints the entry's fix as a hint.
| Variable | Default |
|---|---|
PHANTOM_IMAGE |
python:3.12-slim |
PHANTOM_MODEL |
gemma3:4b (fallback qwen2.5-coder:14b) |
OLLAMA_HOST |
http://localhost:11434 |
pytest -q
pytest -q -m dockerThe default run uses fake runners and fake models and needs neither Docker nor Ollama. Docker tests skip when the daemon is unavailable. With Docker they cover:
- the broken HEAD and the passing third commit;
- persistent shell exports across steps;
- that the host environment is not forwarded;
- timeout cleanup and no leftover containers;
- live bisect and before/after verification.
python scripts/make_haunted_todo.py --force regenerates the demo with identical SHAs.
- Only fenced shell blocks in the root README are treated as setup. Other docs, Makefiles and scripts are not read.
- Only environment variables and
.examplefiles get executable fixes. Missing tools, services and versions are reported but never auto-fixed. doctorchecks that an env var is set, not that its value works.tcpchecks that a port is open, not which service is listening.cp NAME.example NAMEfixes look for the example file at the repository root, even when the failing step ran in a subdirectory.- A step that never exits and is not on the server list (an endless demo, for example) still times out after 120 s and is reported as inconclusive.
- README commands run with network access inside containers. Containers are not a security boundary for hostile repositories.
- The Copilot workflow is verified in the
python:3.12-slimcontainer, not on a GitHub runner, and is only emitted when exorcise verified a fix (not when the README already passes). - The lock records the image by digest (
python@sha256:…) without its tag. - Tested only on macOS with Apple Silicon and the generated demo. No arbitrary repository or Windows host support is promised.
- No terminal recording is included; the transcripts above are captured output, captured at 82 columns (the exorcise table and lock excerpt are condensed).
Not built yet:
phantom ward: a CI check that blocks PRs adding new undocumented assumptions.- Devcontainer and AGENTS.md output from verified steps.
- Contributor personas, such as Windows or Mac setups.
- Verified fixes for missing tools, services and versions.
- Support for setup instructions outside the README.
- OpenAI Codex: phase one implementation and tests (scaffold, models, parser, Docker runner, rules, haunt CLI).
- Claude Code: phase two implementation and tests (AI diagnosis, bisect, verify, exorcise, lock file, doctor, risk flags, hardening, docs).
- Gemma (
gemma3:4b) via Ollama: optional local explanations at runtime. It is never trusted to execute commands or decide verification. Qwen (qwen2.5-coder:14b) is the fallback model.
Runtime libraries: Typer (CLI),
Rich (terminal rendering) and
PyYAML (lock files). Ollama is called with the standard
library's urllib. Development uses pytest; packaging uses setuptools. Git, Docker
and the official python:3.12-slim image provide the execution infrastructure.
MIT; see LICENSE. Install from source; this package is not published.