8000 GitHub - gilf/prompt-chain: Browser-native AI agent runner. Leverages Prompt API for local, private inference and modular skills but also pluggable to other on-device AI options. · GitHub
[go: up one dir, main page]
Skip to content

About

Browser-native AI agent runner. Leverages Prompt API for local, private inference and modular skills but also pluggable to other on-device AI options.

Resources

Readme
MIT license
Activity

Stars

12 stars

Watchers

0 watching

Forks

0 forks
Report repository
BranchesTags

Repository files navigation

Prompt Chain Logo

On-Device Prompt Chain Agent

An interactive, on-device AI agent platform that runs inside a Web Worker. It leverages Prompt API for private, local, and cost-free inference, combining custom tools, modular skills, persistent long-term memory, and a declarative LangChain Expression Language (LCEL) pipeline architecture.

Live Demo

Key Features

  • On-Device LLM Inference: Runs entirely in the browser using Chrome's built-in LanguageModel API (window.LanguageModel), eliminating the need for external API keys or network latency.
  • Pluggable Model Providers (Zero Third-Party Dependencies):
    • Supports Chrome native Built-in AI (ChromeBuiltInAIPlugin), local Ollama REST servers (OllamaPlugin), in-browser/Web Worker Transformers.js ONNX models (TransformersJSPlugin), and custom functional backends (CustomModelPlugin).
    • Modular plugin system (BaseModelPlugin) allows switching between on-device Gemini Nano, local open-weight models (Llama 3, Mistral, Qwen 2.5), and custom LLMs without external npm runtime dependencies.
  • Composable Chains & Universal Agent Runtime (LCEL):
    • Features declarative primitive composition via src/runnables/ (RunnableSequence, RunnableParallel, .pipe(), .bind()).
    • createAgentWorker() acts as a Universal Agent Runtime Host. It accepts either legacy tool arrays (to spin up default ReAct loops via ReActAgentExecutor) or any custom Runnable chain topology.
  • Asynchronous Web Worker Architecture:
    • prompt-chain-host.js runs on the main browser thread to manage the LLM session.
    • prompt-chain-worker.js runs in a background thread to orchestrate the agent loop, execute tools, and handle errors, keeping the user interface completely responsive.
  • Dynamic Skill & Tool Retrieval (Lightweight RAG): Matches the user prompt against loaded skills and tools using a token-overlap scorer, feeding only relevant context to the prompt and preserving token limits.
  • Typed Message History & Roles (LangChain Standard):
    • Structures memory using standardized message objects (HumanMessage, AIMessage, SystemMessage, ToolMessage) defined in messages.js.
    • Uses agent-memory.js to persist object-oriented message schemas directly in IndexedDB.
    • Fully Configurable: Conversation history can be toggled on/off at agent creation (createAgentWorker(tools, skills, callbacks, { enableHistory: false })) or on a per-invocation basis (host.runAgent(prompt, sessionId, { enableHistory: false })) for pure stateless/single-turn execution.
    • Implements automatic conversation summarization (defined in utils.js) once the chat history exceeds 5 turns, ensuring the context window remains optimized.
  • Complex Structured Tool Schemas (Multi-Parameter Tools):
    • Tools extend Runnable and accept structured JSON Schema parameter definitions. Supports both legacy string inputs and complex multi-parameter objects (e.g., bookFlight({ origin: "NYC", dest: "LAX", passengers: 2 })).
    • Automatically validates required arguments prior to execution and generates precise self-correction feedback observations when parameters are missing.
  • Event Callbacks & Token-by-Token Streaming:
    • Implements a global CallbackManager emitting structured lifecycle hooks (on_chain_start, on_llm_start, on_llm_new_token, on_tool_start, etc.).
    • Uses Chrome's session.promptStreaming() API across the Web Worker boundary to render live, token-by-token reasoning updates in the UI.
  • Dynamic Context Management & Rolling Summarization (RunnableTokenBuffer):
    • Bridges Chrome Prompt API token monitoring (session.measureContextUsage(), session.contextUsage, session.contextWindow) across the Web Worker boundary.
    • Features RunnableTokenBuffer LCEL primitive with a configurable watermark threshold (default 85% of context window capacity).
    • Automatically truncates verbose single-turn tool observations and triggers rolling summarization of past conversation turns to ensure continuous ReAct loops without quota errors.
  • Human-in-the-Loop (HITL) Interruption & Checkpointing (RunnableInterrupt):
    • Implements an asynchronous safety rail for sensitive operations (e.g., modifying databases, booking flights, or external network requests).
    • When the agent attempts to invoke a tool flagged with { requiresApproval: true }, the execution graph suspends and serializes its exact loop state (including sanitized context and turn history) into IndexedDB (checkpoints store).
    • Emits a real-time userApprovalRequired event to the host UI, displaying an interactive approval card where developers can review or edit JSON tool parameters.
  • Hybrid Local/Cloud Fallback & Readiness Routing (RunnableFallback):
    • Download State Monitoring: Hooks into Chrome's LanguageModel.create({ monitor(m) }) to track on-device model downloading, emitting real-time modelDownloadProgress events (loaded, total) to display UI progress bars.
    • Composable Readiness & Retry Routing: Wraps the inference runtime in a RunnableFallback pipeline ([localLLM, cloudFallbackLLM]). If the local Prompt API is unavailable, rate-limited, or exceeds configurable self-correction retry limits (maxSelfCorrectionAttempts, default 2 attempts), execution transparently switches to CloudFallbackLLMRunnable (or remote fetch endpoints) without altering agent business logic or swallowing HITL interruptions.
  • Native Structured Output Enforcement & Schema Grammar (StructuredOutputRunnable):
    • Integrates JSON Schema parameter and output constraints directly into Chrome Prompt API options (responseSchema and responseConstraint).
    • Features a zero-dependency client-side recursive JSON Schema validator (validateJSONSchema) producing exact JSON pointer error paths (e.g., Error at /toolInput/passengers: expected number, got string).
    • Wraps agent inference steps inside StructuredOutputRunnable. If schema violations occur, exact pointer paths are intercepted and automatically injected into the ReAct self-correction loop for pinpoint model self-repair.
  • On-Device Vector RAG & Semantic Tool/Skill Retrieval (IndexedDBVectorStore & RunnableRetriever):
    • Features zero-dependency local vector search backed by IndexedDB (IndexedDBVectorStore) with exact mathematical cosine similarity ranking.
    • Exposes an extensible EmbeddingsPlugin interface allowing developers to plug in any custom client-side embedding generator (such as Chrome's built-in embedding API or Transformers.js WebGPU models).
    • Includes RunnableRetriever for declarative LCEL RAG pipelines.
    • Upgraded ToolRetriever and SkillRetriever to perform dynamic semantic pruning, filtering manifests of 50+ tools/skills down to the Top-K most relevant items before LLM prompt construction.
  • StateGraph & Multi-Agent Supervisor Swarms (StateGraph & AgentSupervisor):
    • Brings LangGraph-style cyclical state graphs, conditional routing, and custom chan BD0 nel state reducers directly to on-device Web Workers.
    • Features AgentSupervisor and createAgentSupervisor to dynamically evaluate team state and route tasks across specialized AI worker runnables using strict JSON schema output enforcement.
  • Cross-Session Episodic Memory & Semantic Fact Indexing (RunnableEpisodicMemory):
    • Features zero-dependency persistent long-term semantic memory (RunnableEpisodicMemory) that indexes declarative user preferences, profiles, and historical facts across multiple agent turns and sessions using IndexedDB (episodes store) and cosine vector similarity.
    • Can be piped directly into any LCEL sequence (memory.pipe(agentExecutor)) to automatically enrich incoming user prompts with relevant historical facts ([SEMANTIC MEMORY (Top 5 Facts across sessions)]), or invoked as ReAct structured tools (memory.getTools()) allowing the LLM to autonomously execute remember, recall, and forget.
  • Enterprise Observability & Tracing (LangSmith / OpenTelemetry Equivalent):
    • Upgrades flat callback logs into standardized hierarchical OpenTelemetry trace trees (Trace, Span). Automatically tracks parent-child span links (parentSpanId), timestamps, execution durations (durationMs), status codes (SpanStatus.OK / ERROR), and custom attributes.
    • Features built-in multi-destination exporters (ConsoleTraceExporter, IndexedDBTraceExporter, and OTLPTraceExporter for HTTP OTLP telemetry dashboards).
    • Includes a live Interactive Trace Explorer Dashboard in the host UI (index.html) with expandable parent-child tree views and one-click OTLP JSON downloads.
  • Download Readiness & User Activation Gatekeeper:
    • ChromeBuiltInAIPlugin.checkAvailability() / PromptChainHost.checkAvailability() safely returns 'readily' | 'after-download' | 'unavailable' without throwing errors when the Prompt API is missing or disabled.
    • Automatically satisfies Chrome's mandatory user activation gesture requirement for model downloads via ensureUserActivation (activationButton and activationHint options), displaying the UI button and awaiting a click gesture before triggering download.
    • Normalizes download progress events ({ resource, loaded, total, percent }) and automatically drives <progress> elements (downloadProgress option), switching to an indeterminate state while unpacking the model.
  • AbortSignal Propagation & Execution Cancellation:
    • Full cancellation support across all execution layers via native AbortSignal and untilAborted().
    • Tools receive { signal } in Tool.execute(params, { signal }) and cancel promptly on abort without dangling promises.
    • PromptChainHost.runAgent(..., { signal }) and host.abortAgent(sessionId) immediately stop ongoing background Web Worker agent reasoning loops.
  • Zero-Dependency Streaming HTML Sanitizer (StreamingHtmlSanitizer):
    • Prevents split-tag XSS injection across streaming chunk boundaries via pendingTagStart, buffering incomplete tag boundaries until the closing > arrives.
    • Preserves markdown code fences (```) intact (splitByCodeFences) so code examples are not corrupted by HTML sanitization.
    • Automatically neutralizes dangerous protocol injections (javascript:, vbscript:, data:).
  • Chrome Summarizer API Compaction & Permanent Anchoring (BuiltInAISummarizer):
    • Integrates Chrome's native Summarizer API (window.ai.summarizer).
    • Implements session.compact(), summarizing historical turns and re-anchoring them into initialPrompts. Unlike rolling history turns, Chrome Prompt API guarantees that initialPrompts are permanently pinned and immune to runtime overflow evictions.
  • Native Prompt API Structured Tool Calling:
    • Supports declaring tools directly to LanguageModel.create({ tools }), enabling Chrome Built-in AI's native tool-call and tool-response content types.
    • Bridges native structured tool calls directly into ReActAgentExecutor and JSONOutputParserRunnable, bypassing text regex parsing and eliminating hallucinated action strings.
  • Interactive UI Stream: A sleek interface built with HTML/CSS that displays real-time agent reasoning steps (Thoughts, Actions, and Observations), live token generation, download progress bars, interactive HITL approval cards, and trace hierarchies.

File Directory & Architecture

  • index.html & styles.css: The frontend user interface containing input fields, suggestion chips, reasoning stream log viewports, interactive HITL approval cards, download progress bars, and loaded skills indicators.
  • src/index.js: Main package entry point exporting all library modules.
  • src/runnables/: Modular LangChain Expression Language (LCEL) primitives:
    • runnable.js: Base abstract class with .pipe() and .bind().
    • runnable-sequence.js: Sequential chaining (RunnableSequence.from).
    • runnable-parallel.js: Parallel graph branching (RunnableParallel).
    • runnable-lambda.js: Function wrapping (RunnableLambda).
    • runnable-passthrough.js: Identity mapping & assignment (RunnablePassthrough.assign).
    • runnable-token-buffer.js: Context window watermark truncation & summarization.
    • runnable-interrupt.js & interrupt-exception.js: Human-in-the-Loop suspension rails.
    • runnable-fallback.js: Hybrid local/cloud model fallback routing.
    • structured-output-runnable.js & validate-json-schema.js: JSON Schema validation and pinpoint self-repair.
    • runnable-retriever.js: Declarative LCEL vector and semantic retriever primitive.
    • runnable-episodic-memory.js: Persistent cross-session semantic facts and profile memory (RunnableEpisodicMemory).
    • llm-runnable.js: Adapter runnable wrapping prompt inference functions (LLMRunnable).
    • cloud-fallback-llm-runnable.js: Remote cloud API fallback runnable (CloudFallbackLLMRunnable).
    • json-output-parser-runnable.js: Output parsing runnable extracting JSON from raw LLM responses (JSONOutputParserRunnable).
    • state-graph.js: LangGraph-style cyclical state graphs (StateGraph, CompiledStateGraph) with conditional routing and reducers.
    • agent-supervisor.js: LLM-powered multi-agent supervisor router (AgentSupervisor, createAgentSupervisor).
  • src/retrievers/:
    • indexeddb-vector-store.js: Zero-dependency local vector store backed by IndexedDB (IndexedDBVectorStore), pluggable embedding interface (EmbeddingsPlugin), and exact cosineSimilarity calculation.
    • semantic-retriever.js: Unified base retriever class (SemanticRetriever) providing vector cosine pruning and token overlap fallback.
  • src/observability/:
    • trace.js: Core hierarchical primitives (SpanStatus, Span, Trace, and hex ID generation).
    • tracer.js: Active span stack manager and trace execution singleton (Tracer).
    • exporters.js: Standardized trace exporters (SpanExporter, ConsoleTraceExporter, IndexedDBTraceExporter, OTLPTraceExporter).
    • index.js: Observability package module entry point.
  • src/skills/:
    • skill.js: Dynamic skill loader and markdown frontmatter parser.
    • skill-retriever.js: Semantic skill retriever extending SemanticRetriever.
  • src/tools/:
    • tool.js: Enterprise tool runnable primitive (Tool) encapsulating executable logic, parameter schemas, and HITL approval flags.
    • tool-retriever.js: Semantic tool retriever extending SemanticRetriever.
  • src/core/:
    • prompt-chain-host.js: Main thread session manager, event dispatcher, and Prompt API host bridge.
    • prompt-chain-worker.js: Universal Web Worker agent runtime host & worker RPC communication controller.
    • react-agent-executor.js: Core ReAct reasoning agent executor (ReActAgentExecutor) managing multi-turn reasoning loops, tool retries, schema self-correction, and HITL interruptions.
    • callbacks.js: Global CallbackManager for structured event emitting and cross-thread token streaming.
    • messages.js: Standard LangChain typed message classes (HumanMessage, AIMessage, SystemMessage, ToolMessage).
    • prompt-template.js: LCEL-pipeable prompt formatting component.
    • agent-memory.js: IndexedDB persistent conversation storage manager.
  • src/models/:
    • base-model-plugin.js: Abstract base class defining standard contract (init, generate, measureContextUsage, getContextStats, destroy).
    • chrome-built-in-ai-plugin.js: Native Chrome window.LanguageModel (Gemini Nano) inference plugin with activation gatekeeper, download tracking, compaction, and native tools.
    • built-in-ai-summarizer.js: Chrome Built-in Summarizer API integration (BuiltInAISummarizer) for on-device context compaction.
    • ollama-plugin.js: Native fetch() REST API integration for local Ollama servers (supporting /api/chat, /api/generate, NDJSON streaming, and JSON schema formatting).
    • transformers-js-plugin.js: In-browser / Web Worker ONNX inference using Hugging Face Transformers.js supporting custom pipelines, lazy loaders, and TextStreamer token callbacks.
    • custom-model-plugin.js: Flexible functional wrapper allowing custom generateFn, initFn, and measureContextFn callbacks.
  • src/examples/:
    • my-agent.js: Default Web Worker entry point running global tools and dynamic skills.
    • custom-runner-demo.js: Demonstration of custom linear QA topologies.
    • supervisor-demo.js: Demonstration of an LLM-supervised multi-agent swarm (Researcher + MathExpert).
  • tests/: Complete automated test suites covering LCEL runnables, HITL interrupts, callbacks, token buffers, structured tools, fallback routing, structured output, on-device vector RAG, pluggable model plugins (test-model-plugins.js), cyclical state graph / multi-agent supervisor swarms (test-state-graph.js), user activation, HTML sanitization, abort signals, and Summarizer compaction (test-new-features.js), and configurable conversation history (test-configurable-history.js).

LCEL Chaining & Runnable Usage Guide

1. Sequential Chaining (RunnableSequence & .pipe())

Compose multiple runnables or functions step-by-step:

import { RunnableSequence, RunnableLambda } from './src/index.js';

// Using .pipe()
const addOne = new RunnableLambda(async (x) => x + 1);
const multiplyTwo = new RunnableLambda(async (x) => x * 2);
const chain = addOne.pipe(multiplyTwo);

console.log(await chain.invoke(3)); // Output: 8

// Or declarative array syntax:
const arrayChain = RunnableSequence.from([
    async (text) => text.trim(),
    async (text) => text.toUpperCase()
]);
console.log(await arrayChain.invoke("  hello world  ")); // Output: "HELLO WORLD"

2. Parallel Branching (RunnableParallel)

Execute multiple runnables concurrently on the same input object:

import { RunnableParallel } from './src/index.js';

const parallel = new RunnableParallel({
    charCount: async (text) => text.length,
    wordCount: async (text) => text.split(/\s+/).filter(Boolean).length
});

const stats = await parallel.invoke("Prompt Chain runs on device!");
// Output: { charCount: 28, wordCount: 5 }

3. State Enrichment (RunnablePassthrough.assign)

Attach computed properties to an incoming input dictionary without mutating or dropping existing fields:

import { RunnablePassthrough } from './src/index.js';

const enrichChain = RunnablePassthrough.assign({
    timestamp: async () => new Date().toISOString(),
    normalizedTopic: async (input) => input.topic.toLowerCase()
});

const enriched = await enrichChain.invoke({ topic: "AI Agents", user: "Alice" });
// Output: { topic: "AI Agents", user: "Alice", timestamp: "...", normalizedTopic: "ai agents" }

4. Structured Output Enforcement (StructuredOutputRunnable)

Force an LLM or pipeline step to produce strictly validated JSON matching a JSON Schema:

import { StructuredOutputRunnable } from './src/index.js';

const schema = {
    type: "object",
    properties: {
        city: { type: "string" },
        temp: { type: "number" }
    },
    required: ["city", "temp"]
};

// Wraps any runnable; intercepts markdown fences and validates properties
const structuredChain = new StructuredOutputRunnable(mockLLMRunnable, schema);
const res = await structuredChain.invoke("The weather in Tokyo is 22C");
// Output: { success: true, parsed: { city: "Tokyo", temp: 22 } }

5. Readiness & Self-Repair Fallbacks (RunnableFallback)

Automatically route to a backup model or cloud endpoint if local on-device inference fails or exceeds self-correction limits:

import { RunnableFallback } from './src/index.js';

const robustLLM = new RunnableFallback([
    localOnDeviceLLM, // Tries Chrome window.LanguageModel first
    cloudFallbackLLM  // Switches to cloud API if local model fails or is rate-limited
], {
    onFallback: async (err, failedRunnable, nextRunnable) => {
        console.warn(`Local model failed (${err.message}), falling back to remote endpoint...`);
    }
});

6. Default ReAct Agent Mode

Passing an array of tools to createAgentWorker automatically instantiates the built-in ReActAgentExecutor:

import { Tool, createAgentWorker } from './src/index.js';

const calcTool = new Tool("Calculator", "Evaluates math", expr => eval(expr));
createAgentWorker([calcTool]); // Runs standard 7-turn ReAct reasoning loop

7. Custom Agent Topologies (Bypassing ReAct)

You can pass any custom Runnable chain directly into createAgentWorker():

import { RunnableSequence, RunnableLambda, createAgentWorker } from './src/index.js';

const directAnswerChain = RunnableSequence.from([
    new RunnableLambda(async ({ userPrompt, logToMain }) => {
        logToMain("Thought: Bypassing ReAct loop for linear execution...");
        return `Answer directly: ${userPrompt}`;
    }),
    myLLMRunnable // Any custom model wrapper or pipeline step
]);

8. Human-in-the-Loop Interruption (requiresApproval)

Flag sensitive tools with { requiresApproval: true } to suspend execution before high-impact operations occur:

// Worker side (my-agent.js)
const bookFlightTool = new Tool(
    "bookFlight",
    "Books a flight ticket.",
    async ({ origin, dest, passengers }) => `Booked flight to ${dest}!`,
    flightSchema,
    { requiresApproval: true } // Execution pauses right before running this tool
);

// Host UI side (index.html)
window.addEventListener(CallbackEvents.eventDispatch, (e) => {
    if (e.detail.event === CallbackEvents.userApprovalRequired) {
        const { checkpointId, toolName, toolInput } = e.detail;
        // Prompt human user for review or modifications...
        host.resume(checkpointId, approvedToolInput); // Rehydrate state and complete execution
    }
});

9. StateGraph & Multi-Agent Supervisor Swarms

Orchestrate complex cyclical workflows and multi-agent swarms using LangGraph-style state graphs and LLM supervisors:

import { Tool, StateGraph, START, END, createAgentSupervisor, createAgentWorker, RunnableLambda } from './src/index.js';

// 1. Define specialized worker runnables
const researcherAgent = new RunnableLambda(async (state) => {
    const res = await searchTool.invoke(state.userPrompt);
    return { messages: [`Researcher: ${res}`] };
});

const mathAgent = new RunnableLambda(async (state) => {
    const res = await calcTool.invoke("542 * 13");
    return { messages: [`MathExpert: ${res}`], finalAnswer: res };
});

// 2. Create an LLM Supervisor Router
const supervisor = createAgentSupervisor({
    agents: [
        { name: "Researcher", description: "Searches documentation and specs" },
        { name: "MathExpert", description: "Performs mathematical calculations" }
    ]
});

// 3. Build the cyclical StateGraph with state reducers
const graph = new StateGraph({
    reducers: { messages: (old, add) => (old || []).concat(add) }
});

graph.addNode("supervisor", supervisor);
graph.addNode("Researcher", researcherAgent);
graph.addNode("MathExpert", mathAgent);

// 4. Connect cyclical routing: Supervisor -> Worker -> Supervisor -> END
graph.setEntryPoint("supervisor");
graph.addConditionalEdges("supervisor", (state) => state.next, {
    "Researcher": "Researcher",
    "MathExpert": "MathExpert",
    "FINISH": END
});
graph.addEdge("Researcher", "supervisor");
graph.addEdge("MathExpert", "supervisor");

// 5. Host compiled swarm inside Web Worker runtime!
createAgentWorker(graph.compile());

10. Enterprise Observability & OpenTelemetry Tracing (Tracer & Exporters)

Bridge flat lifecycle events into hierarchical parent-child span trees with multi-destination export (Console DevTools, local IndexedDB, or remote OTLP collectors like Jaeger / OpenTelemetry Collector):

import { Tracer, ConsoleTraceExporter, IndexedDBTraceExporter, OTLPTraceExporter, CallbackManager, createAgentWorker } from './src/index.js';

// 1. Initialize hierarchical Tracer with desired exporters
const tracer = new Tracer("ProductionAgentTracer")
    .addExporter(new ConsoleTraceExporter()) // Styled parent-child tree logs in DevTools
    .addExporter(new IndexedDBTraceExporter("AgentMemoryDB", "traces")) // Persistent local storage
    .addExporter(new OTLPTraceExporter({ endpointUrl: "http://localhost:4318/v1/traces" })); // OTLP HTTP JSON

// 2. Attach Tracer to CallbackManager (or pass directly to worker runtime)
const callbacks = new CallbackManager().attachTracer(tracer);

// 3. Spans (`AgentExecution` -> `LLMInference` / `Tool.Calculator`) are generated automatically!
createAgentWorker(myAgentRunnable, [], callbacks);

11. Cross-Session Episodic Memory & Semantic Profile Indexing (RunnableEpisodicMemory)

Enable long-term fact retention across sessions. RunnableEpisodicMemory stores facts in IndexedDB (episodes store) with cosine semantic similarity indices. It can be used in two complementary ways:

A. ReAct Structured Tools (memory.getTools())

Equip ReAct agents with structured tools (remember, recall, forget) so the LLM can autonomously manage and recall user profiles and facts:

import { Tool, createAgentWorker, RunnableEpisodicMemory } from './src/index.js';

const episodicMemory = new RunnableEpisodicMemory({ dbName: "AgentMemoryDB", storeName: "episodes" });

// Equip worker with standard tools AND episodic memory tools
createAgentWorker([
    fetchTool,
    mathTool,
    ...episodicMemory.getTools()
]);

B. LCEL Pipeline Prompt Enrichment (memory.pipe(agentExecutor))

Automatically enrich incoming user queries with relevant historical facts before they reach the LLM:

import { RunnableEpisodicMemory, ReActAgentExecutor } from './src/index.js';

const episodicMemory = new RunnableEpisodicMemory({ dbName: "AgentMemoryDB", storeName: "episodes" });
const agentExecutor = new ReActAgentExecutor(toolsArray);

// Pipe semantic memory directly into agent loop:
// Enriches state.userPrompt with "[SEMANTIC MEMORY (Top 5 Facts across sessions): ...]"
const memoryAwareAgent = episodicMemory.pipe(agentExecutor);

12. Pluggable Model Plugins (Ollama, Transformers.js, Chrome Built-in AI, Custom)

Easily swap model inference providers when instantiating PromptChainHost:

A. Ollama Local REST Server (OllamaPlugin)

Connect to a local Ollama instance (http://localhost:11434) using native fetch() and NDJSON streaming:

import { PromptChainHost, OllamaPlugin } from '@gilfink/prompt-chain';

const host = new PromptChainHost('./worker.js', {
    model: new OllamaPlugin({
        model: 'llama3',
        baseUrl: 'http://localhost:11434',
        apiEndpoint: '/api/chat'
    })
});

await host.init("You are a helpful assistant.");
const response = await host.runAgent("Analyze user sentiment.");

B. Transformers.js In-Browser ONNX Models (TransformersJSPlugin)

Run Hugging Face models directly inside Web Workers using ONNX runtime and Transformers.js:

import { PromptChainHost, TransformersJSPlugin } from '@gilfink/prompt-chain';

const host = new PromptChainHost('./worker.js', {
    model: new TransformersJSPlugin({
        modelId: 'Xenova/Qwen1.5-0.5B-Chat',
        pipelineLoader: async (task, modelId) => {
            const { pipeline } = await import('@xenova/transformers');
            return await pipeline(task, modelId);
        }
    })
});

C. Custom Model Callbacks (CustomModelPlugin)

Supply your own custom generate and token streaming functions:

import { PromptChainHost, CustomModelPlugin } from '@gilfink/prompt-chain';

const host = new PromptChainHost('./worker.js', {
    model: new CustomModelPlugin({
        generateFn: async (payload, onToken) => {
            if (onToken) onToken("Streaming token...");
            return JSON.stringify({ thought: "Custom LLM reasoning", finalAnswer: "Result" });
        }
    })
});

13. Download Readiness, User Activation & Progress UX

Chrome Prompt API downloads require an active user activation gesture if the model needs to be fetched from the network. ChromeBuiltInAIPlugin and PromptChainHost manage this automatically:

import { PromptChainHost, ChromeBuiltInAIPlugin } from '@gilfink/prompt-chain';

// 1. Check availability gracefully without throwing
const availability = await PromptChainHost.checkAvailability();
console.log(`Prompt API state: ${availability}`); // 'readily' | 'after-download' | 'unavailable'

// 2. Initialize host with activation button & download progress element
const host = new PromptChainHost('./worker.js', {
    model: new ChromeBuiltInAIPlugin({
        activationButton: document.getElementById('download-btn'),
        activationHint: document.getElementById('download-hint'),
        downloadProgress: document.getElementById('progress-bar'),
        onDownloadProgress: ({ loaded, total, percent }) => {
            console.log(`Downloading Gemini Nano: ${percent}% (${loaded}/${total})`);
        }
    })
});

await host.init("You are an on-device assistant.");

14. Execution Cancellation & AbortSignal Propagation

Halt long-running agent executions or in-flight tool operations instantly without hanging background Web Workers:

// A. Pass standard AbortSignal to host.runAgent()
const controller = new AbortController();
document.getElementById('stop-btn').addEventListener('click', () => controller.abort());

try {
    await host.runAgent("Process massive dataset", "session_1", { signal: controller.signal });
} catch (err) {
    console.log("Agent run halted cleanly:", err.message);
}

// B. Or call host.abortAgent() directly:
host.abortAgent("session_1");

// C. Inside tools, executeFn receives context: { signal } automatically:
const slowFetchTool = new Tool("slowFetch", "Fetches remote data", async (input, { signal }) => {
    const response = await fetch(input.url, { signal }); // Cancelled immediately if agent is stopped
    return await response.text();
});

15. Zero-Dependency Streaming HTML Sanitization

Safely stream HTML chunks directly into DOM elements without risk of split-tag XSS injection or destroying code blocks:

import { StreamingHtmlSanitizer } from '@gilfink/prompt-chain';

const sanitizer = new StreamingHtmlSanitizer({ ignoreFencedCode: true });
const container = document.getElementById('output');

// In token stream callback:
function onToken(delta) {
    const safeHtml = sanitizer.push(delta);
    if (safeHtml) {
        container.insertAdjacentHTML('beforeend', safeHtml);
    }
}

// At end of stream:
const remaining = sanitizer.flush();
if (remaining) {
    container.insertAdjacentHTML('beforeend', remaining);
}

16. Chrome Summarizer API & Permanent Context Compaction

Reclaim context capacity by summarizing long turns using Chrome's native Summarizer API and anchoring the summary into initialPrompts:

import { ChromeBuiltInAIPlugin, BuiltInAISummarizer } from '@gilfink/prompt-chain';

// Verify Summarizer API readiness
const summarizerReady = await BuiltInAISummarizer.checkAvailability();

const plugin = new ChromeBuiltInAIPlugin();
await plugin.init("You are a helpful assistant");

// Compact conversation history turns:
// Summarizes past dialogue and re-anchors the summary into initialPrompts,
// which Chrome guarantees are permanently protected against runtime context eviction.
const { summary, session } = await plugin.compact(conversationTurns);
console.log("Compacted Summary:", summary);

17. Configurable Conversation History & Stateless Execution

By default, agents persist conversation turns in IndexedDB (AgentMemoryDB). For single-turn tools, stateless calculators, or privacy-sensitive agents, history can be disabled either at worker instantiation or per invocation:

A. Disabling History at Agent Creation (Stateless Worker)

// In worker (e.g. my-agent.js)
createAgentWorker([calcTool, fetchTool], [], null, {
    enableHistory: false // Agent will run purely stateless without querying or saving history
});

B. Per-Invocation History Override

// In main thread (host UI)
// Run a one-off private query without loading or saving history
const answer = await host.runAgent("What is 542 * 13?", "session_123", {
    enableHistory: false // Overrides the agent's default for this run only
});

How to Setup Chrome Built-in AI for old Chrome versions

Prompt API is enabled since Chrome 148. If you are using previous versions of Chrome, please do the following to enable it:

  1. Open Google Chrome.
  2. Navigate to chrome://flags/#optimization-guide-on-device-model and set it to Enabled BypassPrefRequirement (or Enabled).
  3. Navigate to chrome://flags/#prompt-api-for-gemini-nano and set it to Enabled.
  4. Relaunch Chrome.
  5. Wait for the on-device model to download in the background (you can verify it by opening DevTools console and checking if window.LanguageModel is defined).

How to Run the Project

Because the project loads ES6 modules (import/export) and spins up Web Workers dynamically, opening index.html directly from your file system (file:// protocol) will fail due to CORS security policies. You must serve it using a local web server.

Option 1: Using Node.js (npx)

npx serve

Option 2: Using Python

python -m http.server 8000

License

This project is licensed under the MIT License.

About

Browser-native AI agent runner. Leverages Prompt API for local, private inference and modular skills but also pluggable to other on-device AI options.

Resources

Readme
MIT license
Activity

Stars

12 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages
0