8000 GitHub - NB-Corp/nb-extract: Keep the content. Lose the clutter. An Agent Skill for turning web pages and text into Markdown, powered by nb-search. · GitHub
[go: up one dir, main page]

Skip to content

About

Keep the content. Lose the clutter. An Agent Skill for turning web pages and text into Markdown, powered by nb-search.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

nb-extract — Keep the content. Lose the clutter.

Web pages. Documents. Markdown your AI can use.

TypeScript API Agent Skill Node.js

Get started · Choose an engine · Agent Skill

English · 简体中文

Give nb-extract a URL, a file or some text. Get Markdown back—or save a document with the resources its extraction engine provides. No search credentials needed for local conversion.

Use the TypeScript API inside your app, run the CLI, or let an assistant use the Skill. Defuddle handles web content, optional MarkItDown handles local documents, and MinerU connects to cloud document parsing.

Web page or file → nb-extract and a selected engine → document.md

Get started

Node.js ≥ 24.15. Install the CLI from the official npm registry:

npm install -g @nb-corp/nb-extract@0.1.1 --registry=https://registry.npmjs.org
nb-extract "https://nodejs.org/en/about/releases/" --out ./article

Open article/document.md. --out creates a new directory and does not overwrite or merge an existing one. An interrupted save may leave files behind; inspect them before retrying into a fresh directory. Without --out, stdout is Markdown. Add --json for { markdown, source, engine, warnings, assets }; asset bytes are base64 in CLI JSON.

For a source checkout, pnpm is needed only to install dependencies and build:

git clone https://github.com/NB-Corp/nb-extract.git
cd nb-extract
pnpm install --frozen-lockfile
pnpm build
node scripts/nb-extract.mjs "https://nodejs.org/en/about/releases/" --out ./article

Choose an engine

Input / purpose Engine Setup
Web page or local HTML Defuddle, selected automatically Included; does not execute page scripts
Markdown or plain text Direct, selected automatically Included; no network for local input
Office document or PDF MarkItDown, selected automatically Optional Python ≥ 3.10 with the matching format dependencies
Scanned PDF, OCR, complex layout Explicit --engine mineru MinerU account and MINERU_TOKEN; cloud processing
Existing reader or rendered page Explicit --engine nb-search Your existing nb-search CLI, pipeline and profile

Auto never uploads local documents or falls back to a cloud converter. Defuddle and MarkItDown return Markdown without exporting image files; their warnings make this explicit. MinerU preserves useful resources from its result archive. Format and layout fidelity depend on the selected engine.

For local Office/PDF conversion, install only the formats you need in a Python environment:

python -m venv .venv
.venv/bin/python -m pip install "markitdown[docx,xlsx,xls,pptx,pdf]"
node scripts/nb-extract.mjs ./report.docx --python .venv/bin/python --out ./report

On Windows, use .venv/Scripts/python.exe in place of .venv/bin/python. DOCX and text-based PDF have been exercised locally; scanned content needs an OCR-capable engine.

For MinerU, set MINERU_TOKEN in your environment, then choose it explicitly:

node scripts/nb-extract.mjs ./scan.pdf --engine mineru --out ./scan

This uploads the file to MinerU. URL inputs are sent to MinerU for retrieval. Its service limits, charges and data handling terms apply. MINERU_BASE_URL selects a compatible hosted API, not the different self-hosted MinerU protocol. The default overall timeout is 600 seconds; --timeout is in seconds. Stopping local waiting does not cancel a remote task.

Use it in your app

npm install @nb-corp/nb-extract@0.1.1 --registry=https://registry.npmjs.org
import { extract, saveResult } from '@nb-corp/nb-extract';

const result = await extract({
  text: '<article><h1>Example</h1><p>Keep the source.</p></article>',
  format: 'html'
});
console.log(result.markdown);
// Optional: await saveResult(result, './new-output-directory');

The SDK accepts a URL/path string or { url }, { path }, { text, format }. It returns Markdown, source metadata, warnings and actual assets as Uint8Array; it does not automatically save the final result. MarkItDown conversion uses temporary files and attempts to clean them up afterward. Call saveResult to save the final document and resources. Options include engine, timeoutMs, signal, python and provider settings. See the typed interface.

Give your assistant the Skill

Point the assistant's Skill loader at the installed package root: after a global install, run npm root -g and use <global-root>/@nb-corp/nb-extract; for a project-local install, use node_modules/@nb-corp/nb-extract. A source checkout root also works. Each contains SKILL.md and the launcher; then reload skills. Hosts with explicit invocation can use $nb-extract:

Convert this document to Markdown. Save the result and any available images in a new folder.

The Skill includes commands for local documents, cloud OCR and existing nb-search profiles. For a JavaScript-rendered page, select the bridge with --engine nb-search --pipeline browser.render; it handles the existing async job without another job system. Run node "<skill_dir>/scripts/nb-extract.mjs" --help for all options.

Already have the source? Use nb-extract. Need to discover sources or run research? Use nb-search.

MIT License · Report an issue · Built by NB-Corp

About

Keep the content. Lose the clutter. An Agent Skill for turning web pages and text into Markdown, powered by nb-search.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

0