A Solana VM compiled into transformer weights.
Reduced sBPF programs run on hand-built attention, FFN and DeltaNet blocks: exact results, exact compute units, zero training.
Live demo: svmllm.rexstjohn.com | github.com/Verafyai/svmLLM | Sibling project: EVMLLM (evmllm.rexstjohn.com)
Solana programs compile to sBPF bytecode. SVMLLM executes that bytecode inside a neural network whose weights were constructed by hand rather than learned. Instruction fetch is a one-hot attention-style lookup, the ALU is a stack of exact lookup-table feed-forward blocks, and registers and memory live in Gated DeltaNet state configured as exact RAM, the trick introduced by Qwenputer. Every run is checked against a reference interpreter, and the reference interpreter is checked against Anza's solana-sbpf.
On top of the VM sits a judgement filter: after the program has run, a separate review step looks at the amount, the destination and the history, and returns allow, hold or block with a reason. The examples wrap that result in a prompt. The reply in those examples is simulated. No language model is called unless you pass one in.
Some of the costliest smart-contract incidents involve code that ran exactly as written, with an outcome a sensible human operator would have stopped. Many others are ordinary bugs or key compromises; this project targets only the first kind. That first kind is what motivated this project: fuse the best of smart contracts (trustless, deterministic and decentralized execution) with the best of human intelligence (judgement based on context).
A transformer that can execute contract bytecode exactly is a step toward a single model that both runs the program deterministically and reasons about whether the result makes sense.
One sBPF instruction is one call to Machine.step() in svllm/machine.py:
sBPF program ──► fetch + decode ──► ALU (FFN) ──► registers / memory (DeltaNet RAM) ──► exit code + CU
8-byte insns one-hot PC query LUT blocks delta-rule writes, beta 0/1 24-bit counter
Representation. A 64-bit value is 16 little-endian 4-bit limbs, each a one-hot vector of 16, so a register is a 256-dim activation. Every activation is exactly 0 or 1 in float32, which is why the computation is exact.
Exact RAM (DeltaNet). A Gated DeltaNet head with gate = 1 and beta in {0, 1} updates S <- S + beta * k (v - S^T k)^T. With a one-hot key k this overwrites exactly one row; reading is S^T q. Three of these hold the program (256 slots x 481 dims: present bit, opcode one-hot, dst, src, offset limbs, imm limbs), the register file (16 x 256) and data memory (256 bytes, two limbs each).
class DeltaNetRAM:
def read(self, q):
return q @ self.S
def write(self, k, v, beta):
self.S = self.S + beta * torch.outer(k, v - k @ self.S)Exact ALU (FFN). LUT turns any function of one-hot inputs into a 2-layer ReLU FFN with one hidden unit per input combination (weights 1, bias -(k-1)). From these: a 16-stage ripple-carry adder over (a, b, carry), nibble-times-word multipliers with shifted adds, 64-stage restoring division for div and mod, nibble permutations plus bit-shift tables for lsh, rsh and arsh, and comparators (signed ones flip the top bit).
L_ADD = LUT([16, 16, 2], [16, 2], lambda a, b, c: ((a + b + c) & 15, (a + b + c) >> 4))
def add(A, B, cin):
out, c = [], cin
for i in range(A.shape[0]):
s, c = L_ADD(A[i], B[i], c)
out.append(s)
return torch.stack(out), cDecode and control. Decode is a single constant linear layer from the opcode one-hot to control gates. Muxes are ReLU gating (relu(v + g - 1)). Branches choose PC + 1 + off or PC + 1; address checks match the high limbs against the input and stack windows; the CU meter is a 6-limb counter compared to the limit. Errors (ExceededMaxInstructions, ExecutionOverrun, InvalidInstruction, DivideByZero, AccessViolation) are flags that also gate every write off.
| In the weights (constant tensors, about 1.13M parameters, none trained) | Not in the weights |
|---|---|
| Fetch, decode, ALU, branch logic, address checks, CU meter, register and memory reads and writes | The driver loop: Python calls step() until the halt flag is 1 |
| Fixed-depth circuits are Python loops over unrolled blocks rather than one fused layer stack | |
| Lazy mode skips ALU branches whose gate is exactly 0 (full mode computes all of them and is tested equal) | |
| The judgement filter (rules plus a simulated LLM reply, unless you plug in a real model) |
This is a standalone PyTorch machine in the style of Qwenputer, not yet embedded in a pretrained language model. See next steps.
Reduced sBPF (SBPF v0 encoding): r0 to r10; all 64-bit ALU ops in imm and reg forms (add, sub, mul, div, mod, or, and, xor, lsh, rsh, arsh, neg, mov); ldx/stx/st in b, h, w and dw; all conditional and unconditional jumps; lddw; call (stub); exit; 1 CU per instruction. Memory is two 128-byte windows, mirroring Solana's layout: input at 0x400000000 (pointed to by r1) and stack below r10 = 0x200001000.
All ten are in svllm/programs.py, each with three presets, and run on the site with a step trace and CU counts.
- Spend cap: exit 1 if the amount is over a $1B limit.
- Counter: memory read, increment, write back, N times.
- Fibonacci: iterative, up to fib(93).
- Factorial with overflow check: exit 2 when a multiply wraps.
- GCD: Euclid with mod64.
- Prime check: trial division to sqrt(n).
- Balance transfer: between two memory slots; exit 1 if short, 2 on overflow.
- Vault with withdrawal limit: exit 1 over balance, 2 over the daily limit.
- Voting tally: 64-bit bitmap rejects double votes with exit 3.
- Escrow: release when delivered, refund after the deadline, otherwise exit 4.
The handoff is five steps:
- A transaction arrives.
- The VM circuits run the program exactly.
- The result is wrapped into the prompt, together with the sender and receiver history and the run rules.
- The reviewer answers allow, hold or block, with a reason.
- The decision can only get stricter. It never gets looser than the rule verdict, and it never overrides a code-level rejection.
The VM answers "is this valid by the code?". The review step answers "would a sensible operator let this through?". The rule review is a heuristic outside the weights. It lives in svllm/judge.py. Rules, in order:
- Program error or nonzero exit code: block (never overrides a code-level rejection).
- More than 3 withdrawals in 15 minutes moving at least 50% of the balance: block (drain pattern).
- At least 50x the median past amount to a never-seen destination: block.
- At least 10x the median past amount: hold.
- New destination and at least 3x the median: hold.
- Otherwise allow.
LLMJudge(complete=fn) builds the prompt below and expects JSON back: {"decision": "allow"|"hold"|"block", "reason": "..."}. complete is the plug-in for a real model: it takes the prompt and returns that JSON string. The default backend is SimulatedLLM, which returns a canned reply and does not call a model. The reply may move allow to hold or block, but it cannot make the verdict looser, and a code-level rejection stays blocked.
The VM numbers in the table are from the hand-set weights machine (results/judge_examples.json). The prompts are built from those results. The JSON replies are simulated. One model that both runs the program and writes the review is future work, and it has not been tested.
Runnable examples (python -m svllm.judge; the VM step is run on the weights machine in run_tests.py):
| Example | Program | VM result (weights) | Rule verdict | Reason |
|---|---|---|---|---|
| Valid by code, 100x usual, never-seen address | vault |
ok, exit 0, 13 CU | BLOCK | 100x the usual amount to a destination never seen before. Valid by code, but a sensible operator would stop it. |
| Normal transfer to a known address | transfer |
ok, exit 0, 12 CU | ALLOW | Valid by code and consistent with history. |
| Rapid repeated drain | vault |
ok, exit 0, 13 CU | BLOCK | Drain pattern: 4 withdrawals in 15 minutes moving 96% of the balance. Each one passed the code checks. |
| Spend cap: over $1B | spend-cap |
ok, exit 1, 5 CU | BLOCK | The program itself rejected this (exit code 1). The judge never overrides a code-level rejection. |
| Moderate amount, new address | transfer |
ok, exit 0, 12 CU | HOLD | New destination and 4.9x the usual amount. Hold for confirmation. |
Wallet X asks to call the balance-transfer program. The history is the usual pattern for this wallet: amounts around 200 to addresses it already pays. Addresses below are placeholders.
SYSTEM
You are the transaction reviewer for a wallet. The sBPF virtual machine has already run the program exactly. Decide allow, hold or block based on everything you know about the sender and the receiver and the run rules. You may only make the decision stricter, never looser than the rule verdict, and you never override a code-level rejection.
TRANSACTION REQUEST
Wallet X has requested calling this Solana program "Balance transfer" (program id: transfer).
Wallet X has requested calling this smart contract. Based on everything you know about the context and the run rules, is this allowed?
Sender (wallet): Wallet X
Receiver (destination): addr:alice-savings
Amount: 210
Time (seconds): 864000
Balance before this request: 1000000
VM RESULT
The sBPF virtual machine has already run the program exactly. These numbers are that result.
status: ok
exit code: 0
compute units: 12
return data: 0000000000000000
SENDER AND RECEIVER HISTORY
Past requests from this sender, oldest first:
- t=0 amount=200 to=addr:alice-savings balance_before=1000000
- t=86400 amount=150 to=addr:landlord balance_before=999800
- t=172800 amount=250 to=addr:alice-savings balance_before=999650
- t=259200 amount=180 to=addr:grocer balance_before=999400
- t=345600 amount=220 to=addr:alice-savings balance_before=999220
- t=432000 amount=210 to=addr:landlord balance_before=999000
Past payments to this receiver (addr:alice-savings):
- t=0 amount=200 balance_before=1000000
- t=172800 amount=250 balance_before=999650
- t=345600 amount=220 balance_before=999220
Destination seen before: yes
Known destinations on file: none
RUN RULES
The rule review is a heuristic outside the weights. You may only make the decision stricter, never looser than the rule verdict, and you never override a code-level rejection.
1. Program error or nonzero exit code: block.
2. More than 3 withdrawals in 15 minutes moving at least 50% of the balance: block.
3. At least 50x the median past amount to a never-seen destination: block.
4. At least 10x the median past amount: hold.
5. New destination and at least 3x the median: hold.
6. Otherwise: allow.
Rule verdict: allow
Rule reason: Valid by code and consistent with history.
median past amount: 205
ratio to median: 1.02
destination seen before: yes
transactions in 15 minutes: 1
window outflow: 210
window outflow fraction: 0
REQUIRED REPLY
Reply with one JSON object and no other text. The keys are decision and reason.
{"decision": "allow" or "hold" or "block", "reason": "short reason"}
SIMULATED model output, no real LLM was called:
{"decision":"allow","reason":"The program exited 0 in 12 compute units. The amount is in line with what this wallet usually sends, and the receiver is an address it already pays. Allow it."}Wallet X asks to withdraw 20500 from the vault. The program exits 0, so the code allows it. The amount is 100x the usual size, and the destination has never been seen. Addresses below are placeholders.
SYSTEM
You are the transaction reviewer for a wallet. The sBPF virtual machine has already run the program exactly. Decide allow, hold or block based on everything you know about the sender and the receiver and the run rules. You may only make the decision stricter, never looser than the rule verdict, and you never override a code-level rejection.
TRANSACTION REQUEST
Wallet X has requested calling this Solana program "Vault with withdrawal limit" (program id: vault).
Wallet X would like to withdraw 20500. Based on everything you know about the context and the run rules, should that be allowed?
Sender (wallet): Wallet X
Receiver (destination): addr:9qZ...never-seen
Amount: 20500
Time (seconds): 864000
Balance before this request: 1000000
VM RESULT
The sBPF virtual machine has already run the program exactly. These numbers are that result.
status: ok
exit code: 0
compute units: 13
return data: 0000000000000000
SENDER AND RECEIVER HISTORY
Past requests from this sender, oldest first:
- t=0 amount=200 to=addr:alice-savings balance_before=1000000
- t=86400 amount=150 to=addr:landlord balance_before=999800
- t=172800 amount=250 to=addr:alice-savings balance_before=999650
- t=259200 amount=180 to=addr:grocer balance_before=999400
- t=345600 amount=220 to=addr:alice-savings balance_before=999220
- t=432000 amount=210 to=addr:landlord balance_before=999000
Past payments to this receiver (addr:9qZ...never-seen):
- none
Destination seen before: no
Known destinations on file: none
RUN RULES
The rule review is a heuristic outside the weights. You may only make the decision stricter, never looser than the rule verdict, and you never override a code-level rejection.
1. Program error or nonzero exit code: block.
2. More than 3 withdrawals in 15 minutes moving at least 50% of the balance: block.
3. At least 50x the median past amount to a never-seen destination: block.
4. At least 10x the median past amount: hold.
5. New destination and at least 3x the median: hold.
6. Otherwise: allow.
Rule verdict: block
Rule reason: 100x the usual amount to a destination never seen before. Valid by code, but a sensible operator would stop it.
median past amount: 205
ratio to median: 100
destination seen before: no
transactions in 15 minutes: 1
window outflow: 20500
window outflow fraction: 0.021
REQUIRED REPLY
Reply with one JSON object and no other text. The keys are decision and reason.
{"decision": "allow" or "hold" or "block", "reason": "short reason"}
SIMULATED model output, no real LLM was called:
{"decision":"block","reason":"The program allowed this withdrawal (exit code 0), but 20500 is 100x the usual amount and the destination has never been seen before. Block it."}The other three scenarios use the same prompt builder and the same simulated backend (SIMULATED model output, no real LLM was called):
- Rapid drain:
{"decision":"block","reason":"The program allowed this withdrawal, but the history shows 4 withdrawals in 15 minutes moving 96% of the balance. Block it."} - Spend cap, exit code 1:
{"decision":"block","reason":"The program itself exited with code 1. A code-level rejection is never overridden. Block it."} - Moderate amount, new address:
{"decision":"hold","reason":"The program allowed the call, but the amount is about 4.9x the usual size and the destination has never been seen before. Hold it for a person to confirm."}
git clone https://github.com/Verafyai/svmLLM && cd svmLLM
python3 -m venv .venv && . .venv/bin/activate
python3 -m pip install -r requirements.txt
python -m svllm.demo fibonacci 10 # weights machine vs reference, with the trace
python -m svllm.judge # judgement-filter examples
(cd xcheck && cargo build --release) # optional: solana-sbpf 0.12 cross-check binary
python run_tests.py # all suites; writes results/
python export_site.py # regenerate site/js/data.js from results/
node tests/js_parity.mjs # browser interpreter, judge, and prompt vs PythonN_RANDOM and N_FULL environment variables size the random and full-mode suites.
| Suite | Cases | Reference vs solana-sbpf | Weights machine vs reference | Steps executed on weights |
|---|---|---|---|---|
| Conformance (named, per opcode and error) | 135 | 134/134 | 135/135 | 1,508 |
| Demo presets (10 programs x 3) | 30 | 30/30 | 30/30 | 2,764 |
| Random programs (seed 1337) | 1000 | 1000/1000 | 1000/1000 | 44,756 |
- Full mode (no lazy branch skipping) equals lazy mode: 32/32 (first 16 conformance + first 16 random).
- Random-program outcomes: ok 551, DivideByZero 201, ExceededMaxInstructions 159, AccessViolation 89.
- Browser JS interpreter vs Python reference: 565/565 cases, assembler 10/10, judge 5/5, prompt text 5/5, simulated replies 5/5 (
tests/js_parity.mjs). - Weights: 1,130,672 constant parameters, none trained.
Cross-check details: xcheck/ drives Anza's solana-sbpf 0.12 interpreter with SBPFVersion::V0, r1 at the input region and only the top 128 bytes of the first stack frame mapped, so its memory model matches the toy. Status, r0, CU and the input window are compared (its registers after a run are not exposed reliably). The weights machine is compared against the reference on status, r0, CU, all 11 registers and both memory windows. The one conformance case not run on solana-sbpf is misc/call-stub, because call is a local stub.
A static demo site (plain HTML, CSS and JS, no build step) lives in site/ (serve it with python -m http.server -d site). It has the ten demos with presets, a Run button, step traces and CU counts, a paste-your-own box (256 instructions, 8 KB), the judgement-filter examples, a prompt viewer for the simulated judge, and an animated walkthrough that steps through real recorded traces of the weights machine. Results are labeled honestly: live runs use a JavaScript port of the reference interpreter in your browser; weights results are precomputed traces from the PyTorch runs (results/weights_traces.json). The judge reply on the site is simulated.
This is a demo that exact computation can live in a model's weights. It is not a node replacement.
Realistic today
- Sidecar next to the chain software. A node operator runs the model beside their own client and uses it to review, flag or hold transactions in their own RPC endpoint, mempool view or wallet guard. It affects only that operator's infrastructure, not consensus. This is the realistic path today.
- Off-chain checker and audit. Re-run a transaction on the weights machine, compare it to the reference result, and attach the review to an audit trail.
- Judgement layer in front of a wallet or multisig. The exact VM result plus the review step decides allow, hold or block before a signer approves.
Research direction: part of consensus
Making the weights machine part of consensus would need:
- Determinism. GPU floating point can differ across hardware and software versions. The weights here use exact lookup-table style math with values that stay exactly 0 or 1, but a full Qwen-sized model is a much bigger problem.
- Cost and speed. Each sBPF instruction is one full pass through the circuits, and every validator would have to repeat every pass. On one CPU thread that is about 5 ms for a simple instruction and roughly 65 ms for div or mod.
- Compute limits. The model run would need to be metered and bounded by compute units like any other execution.
The judgement filter is illustrative heuristics on synthetic histories, not a security product.
- Reduced subset. No ALU32 (SBPF v0 ALU32 sign extension quirks are not modeled), no syscalls (
callzeroes r0), no accounts, no CPI, no internal function calls, no SBPF v1+ instructions. - Memory is two 128-byte windows and programs are at most 256 slots.
- CU model is 1 per instruction. Real Solana pricing for syscalls and CPI is not modeled.
- It is slow: CPU PyTorch on one thread, about 5 ms per simple instruction and roughly 65 ms for div or mod (lazy mode; full mode is slower). A research demo, not a validator.
- The driver loop and the circuit sequencing are Python. The judgement filter is heuristics, not learned and not in the weights.
- Not yet inside a pretrained LLM.
- The judge examples and the demo site use a simulated reply (
SIMULATED model output, no real LLM was called). No language model is called unless you passcompletetoLLMJudge. A single model that both runs the program and writes the review has not been built or tested.
- Accounts: serialize Solana account data into the input window and enforce owner, signer and writable rules in the circuits.
- Syscalls:
sol_log,sol_memcpy, hashing, each as additional FFN circuits. - CPI and internal calls: a call stack of frames in DeltaNet memory.
- A real CU model: per instruction class and per syscall, matching the runtime's cost tables.
Untested, not a claim. Merge the hand-weighted sBPF machine into a Qwen-class model, so one model does both the language judgement and the exact execution. This has not been built or tested. Its size and speed would be dominated by the base model: Qwenputer's checkpoint is 18.8 GB and needs a 24 GB GPU, while the SVM machine adds about 1.1M parameters.
- Get a single 24 GB GPU (the Qwenputer post uses an L4). The checkpoint is 18.8 GB, and the post reports about 3,200 CPU tokens per second for its batched executor on an L4.
- Reproduce Qwenputer first, using the commands from the post:
git clone https://github.com/divagr18/qwenputer && cd qwenputer/experimental/riscv && uv sync --extra hf hf download divagr1925/Qwen3.5-4B-Qwenputer --local-dir qwenputer-rv uv run python -m chipformer.rv.chat qwenputer-rv --firmware tests/data/rv/calc-bridge.elf \ --ask "Use calc: what is 123456789 * 987654321?"
- Two routes for sBPF. (a) Firmware route, no new weights: write an sBPF interpreter in C, build it for RV32IM, and run it as firmware on the CPU that already sits in Qwenputer's weights. Validate with this repo's conformance and random suites. (b) Native route: express SVMLLM's circuits in Qwenputer's circuit compiler (attention heads for fetch, FFN units for the ALU tables, DeltaNet heads for program, registers and memory), emitting ADDR, VAL and STEP tokens per instruction, as extra layers alongside the RISC-V ones.
- Run
run_tests.pyagainst the embedded version and require the same pass rates. Then wire the judgement filter to Qwen itself throughLLMJudge(complete=...), so one model runs the program exactly and reviews the result.
Drop me some Solana if you want to support me to keep working on this.
86pEFG22pV4hd9wH48TSnATLo8U9bAraM8C58NAmPtuy
- Qwenputer by divagr1925: a RISC-V CPU compiled into Qwen3.5-4B with Gated DeltaNet layers as exact RAM. Announcement, weights on Hugging Face. SVMLLM borrows the central idea (gate 1, beta 1, one-hot keys) and applies it to sBPF.
- Anza's solana-sbpf for the reference VM used in cross-checking.
- Sibling project: EVMLLM, an Ethereum VM executed by transformer weights (evmllm.rexstjohn.com).
MIT. See LICENSE.