OKFgen generates portable Open Knowledge Format (OKF) v0.2 bundles from your documentation, schemas, source code, and URLs. It uses LangChain so you can choose a hosted open-source model, a local Ollama model, or a proprietary provider without changing the generation pipeline.
The npm package and installed command are both okfgen.
- Collects source material from files, directories, or HTTP(S) URLs.
- Detects existing OKF v0.1, v0.2, and conformant unversioned bundles at the output path and includes them as update context.
- Sends a structured generation or improvement prompt to the selected LangChain chat model.
- Parses the model response into a validated bundle plan.
- Renders Markdown concepts with YAML frontmatter deterministically.
- Creates progressive-disclosure
index.mdfiles and maintains an optionallog.mdhistory. - Validates the finished bundle against the OKF v0.2 conformance rules and optional metadata guidance.
- Optionally opens a searchable document explorer with an interactive relationship graph.
The model never writes files directly. This keeps frontmatter, paths, reserved filenames, and output boundaries under CLI control.
npm install -g okfgenNode.js 20 or newer is required. For local development:
git clone https://github.com/Arindam200/OKFgen.git
cd OKFgen
npm install
npm run build
npm linkRun okfgen with a terminal attached to open the interactive command center:
okfgenOn its first run, the persistent shell shows the OKFgen wordmark and guides you through provider, API-key, and model setup before opening the startup panel. Later launches reuse that saved setup. Direct interactive commands show the same branding and context before their prompts. Type / to see live command suggestions with short descriptions; the list filters as you continue typing. Every action in the persistent shell uses a slash command:
/generate [request]starts the guided generation flow/update [request]refreshes the last generated bundle with its remembered sources/view [directory]opens a bundle explorer/validate [directory]validates a bundle/lint [directory]checks editorial and graph quality/migrate [directory]migrates a recognized bundle to OKF v0.2/providerslists model providers and credential variables/provider [name]changes the provider for the current session/model [id]changes the model for the current session/api-keysecurely enters or replaces the current provider credential/statusshows effective configuration and where each value came from/config savepersists provider/model defaults;/config resetclears saved provider settings and API keys, then starts guided provider setup again/commands(or/help) shows syntax, examples, and hints/exitcloses the shell
Quoted arguments work as expected, for example /generate "Document our payments API" --source ./docs. You can still run okfgen generate directly for the original one-shot guided flow. It lets you select a provider, choose a model, paste a missing API key into a masked prompt, add sources, choose an output directory, and review a final configuration panel before generation. If the output directory already contains a recognized OKF bundle, the panel switches to update mode and reports how many existing concepts will be improved.
API keys entered during a run stay in memory unless you explicitly choose to save them.
OKFgen resolves settings in this order: command flags, exported terminal environment, ~/.okfgen/.env, project configuration, then interactive prompts and provider defaults. If no provider is configured, Nebius is used. If exactly one supported provider credential is exported, the interactive CLI selects that provider automatically. /status reports the effective provider, model, credential status, and source without displaying secrets.
To remove a credential saved by OKFgen and redo the initial provider setup, run /config reset in the interactive shell. This clears all OKFgen-managed values in ~/.okfgen/.env (including saved API keys) and immediately opens the guided provider setup. Credentials exported by your terminal are intentionally left untouched; remove one with unset OPENAI_API_KEY (or the corresponding provider variable) before resetting if you also want it unavailable to the new setup.
export OPENROUTER_API_KEY="..."
export OKFGEN_PROVIDER="openrouter"
export OKFGEN_MODEL="openai/gpt-oss-120b"
okfgenWhen a key is entered through a masked prompt or /api-key, OKFgen asks whether to save it. Saving is opt-in. The ~/.okfgen directory is protected with mode 0700 and its .env file with 0600 on supported platforms. Exported terminal variables always override saved values.
Managed settings are OKFGEN_PROVIDER, OKFGEN_MODEL, OKFGEN_BASE_URL, and OKFGEN_RETRY_ATTEMPTS. Provider requests default to three retries; the retry setting accepts values from 0 through 10. To prevent a checked-out repository from redirecting credentials or source material, project-level baseUrl is accepted only for loopback Ollama endpoints. Trust any remote endpoint explicitly with --base-url or OKFGEN_BASE_URL.
Configure and save a provider/model pair without starting generation:
okfgen providerIn an interactive terminal, this command selects a provider and model, prompts securely for a missing credential, and optionally saves that credential. Provider and model defaults are written to ~/.okfgen/.env. For scripted setup, pass the provider and model explicitly; credentials supplied with --api-key are used for setup but are not persisted non-interactively:
okfgen provider openrouter --model openai/gpt-oss-120b --api-key "$OPENROUTER_API_KEY"| Provider | Default model | Environment variable | Notes |
|---|---|---|---|
| Nebius Token Factory | openai/gpt-oss-120b |
NEBIUS_API_KEY |
Hosted default; interactive selection uses the live model catalog |
| OpenRouter | openai/gpt-oss-120b |
OPENROUTER_API_KEY |
Open and proprietary models through one router |
| Ollama | qwen3:8b |
None | Local models; Ollama must be running |
| OpenAI | gpt-5.4-mini |
OPENAI_API_KEY |
OpenAI API models |
| Anthropic | claude-sonnet-4-6 |
ANTHROPIC_API_KEY |
Claude API models |
Set a provider key before an automated run:
export NEBIUS_API_KEY="..."--api-key is available for one-off runs, but environment variables are safer because shell history and CI logs can expose command arguments.
For Nebius, an interactive model-selection flow validates the API key by loading the current Token Factory model catalog. The searchable selector shows readable names such as GPT OSS 120B (OpenAI) while retaining the exact model ID as a hint. New project configurations pin the live-tested openai/gpt-oss-120b, so they work consistently in automation. Nebius generation requests time out after five minutes instead of hanging indefinitely. A non-interactive Nebius run without a configured model still requires --model.
Create a reusable project configuration:
okfgen initThis writes okfgen.config.yml. Generation automatically discovers that file in the current directory or a parent directory:
# Nebius is the hosted default. Set NEBIUS_API_KEY in your environment.
provider: nebius
model: openai/gpt-oss-120b
# To use a local model instead, replace the two lines above with:
# provider: ollama
# model: qwen3:8b
# Other supported providers: openrouter, openai, anthropic
# Add files, directories, or URLs that OKFgen should use as source material.
sources:
- ./docs
output: ./okfgen-bundle
log: truePaths are resolved relative to the configuration file. Command flags override environment or saved settings, which override project configuration. Use --config <file> to select another file and okfgen init --force to replace an existing one.
okfgen generate "Document our payments API" \
--provider nebius \
--model openai/gpt-oss-120b \
--source ./docs ./openapi.yaml \
--output ./payments-okfgenOptions:
-p, --provider:nebius,openrouter,ollama,openai, oranthropic-m, --model: provider model ID--api-key: one-off provider key-s, --source: one or more files, directories, or HTTP(S) URLs--config: explicitokfgen.config.ymlpath-o, --output: output directory, default./okfgen-bundle--base-url: override an OpenAI-compatible or Ollama endpoint--force: allow writing into a non-empty directory that is not an existing OKF bundle--prune: explicitly delete existing concepts omitted from the generated plan--no-log: skiplog.md
Without a TTY, generate prints a compact JSON summary, making it suitable for CI.
Use --print to force this one-shot behavior even when a terminal is attached.
Run the same generation command with an existing OKF bundle as the output directory:
okfgen generate "Refresh this knowledge from the latest source material" \
--source ./docs \
--output ./payments-okfgenOKFgen automatically supplies the current bundle to the model as context and asks for a complete improved plan. It updates changed concepts, adds new concepts, preserves omitted concepts, rebuilds indexes, and appends a timestamped, per-document change summary to the existing log.md. Unknown frontmatter and unrelated files are preserved. Use --prune only when omitted concepts should be deleted. --force is not required for recognized OKF bundles.
Preview or apply a safe root-version migration:
okfgen migrate ./legacy-bundle --dry-run
okfgen migrate ./legacy-bundleMigration preserves legacy timestamp and # Citations content rather than inventing generation or provenance metadata. Subsequent meaningful edits use v0.2 generated and sources fields.
okfgen validate ./payments-okfgen
okfgen validate ./payments-okfgen --jsonValidation checks frontmatter, required type fields, reserved files, index/log structure, and reports broken internal links as warnings.
Run editorial and graph-quality checks in addition to OKF conformance validation:
okfgen lint ./payments-okfgen
okfgen lint ./payments-okfgen --strict
okfgen lint ./payments-okfgen --jsonLinting detects duplicate concept titles, orphan concepts, thin content, skipped heading levels, broken Markdown links, and broken heading anchors. Warnings are informational by default; --strict treats them as failures for CI.
okfgen view ./payments-okfgenThe explorer runs locally and provides two connected views:
- Document renders sanitized Markdown with searchable concept navigation.
- Graph visualizes every concept as a node and every internal Markdown link as a directed edge. Selecting a node opens its document.
- Theme defaults to a clean light interface and includes a persistent dark mode toggle.
Use --port and --host to control the local server, or --no-open to start it without launching a browser:
okfgen view ./payments-okfgen --port 4400 --no-openTo open the explorer immediately after generation:
okfgen generate "Document this repository" --source . --viewokfgen providersConfigure the default provider and model without starting generation. Omit the provider in a terminal for guided selection, or provide it with --model for scripted setup:
okfgen provider
okfgen provid
82B0
er ollama --model qwen3:8bAn output directory looks like this:
payments-okfgen/
├── index.md
├── log.md
├── api/
│ ├── index.md
│ └── authentication.md
└── schemas/
├── index.md
└── payments.md
Every concept document contains OKF frontmatter with a non-empty type, followed by ordinary Markdown. New concepts record generated, start as status: draft, and use sources for provenance when source evidence is available. The root index declares okf_version: "0.2"; nested indexes contain linked directory listings.
import { generateBundle } from "okfgen";
const result = await generateBundle({
provider: "nebius",
model: "openai/gpt-oss-120b",
apiKey: process.env.NEBIUS_API_KEY,
request: "Document the catalog represented by these files",
sources: ["./catalog"],
outputDirectory: "./catalog-okfgen",
});
console.log(result.mode, result.validation.valid, result.files);The exported API also includes createChatModel, renderBundle, validateBundle, migrateBundle, viewer helpers, provider metadata, and the Zod schemas.
The published package includes SKILL.md, which gives Codex, Claude Code, Cursor, and other coding agents a safe workflow for generating, validating, and visualizing OKF bundles. Agents can read that file directly from the repository or installed package.
- Source input is capped at 1 MB per run.
- Hidden directories,
.git, dependencies, and build output are skipped during directory ingestion. - Remote sources use a 15-second timeout and are size-checked after download.
- Concept paths are constrained to remain inside the output directory, and writes refuse bundle-internal symbolic-link traversal.
- Automatic updates recognize v0.1, v0.2, and structurally conformant unversioned bundles; other non-empty directories remain protected unless
--forceis explicit. - Updates preserve omitted concepts by default.
--pruneremoves only omitted OKF Markdown concepts and generated indexes; other files are preserved. - API keys are persisted only after explicit confirmation, in the private
~/.okfgen/.envfile; exported terminal values take precedence. - Model output is parsed and validated before any files are written.
npm run check
npm pack --dry-runnpm run check runs the unit tests, typecheck, build, and the stress harness. The stress harness drives the real CLI against temporary local fake-model and viewer servers. It covers provider configuration, initialization, model-response repair, generation, safe updates, explicit pruning, validation, strict linting, migration, credential redaction, and the explorer.
Before publishing, confirm the package name is available and run:
npm login
npm publish --access publicMIT. See LICENSE.
Built with love by Arindam.