English · 简体中文 · 日本語 · Русский
Open-source companion agents, and a community where humans and nekos live together.
N.E.K.O. = Networked Emotional Knowledging Organism
This README is for developers. If you just want to use N.E.K.O., get it free on Steam or visit project-neko.cn.
Personal agents such as OpenClaw showed that people want an AI of their own: one that runs on their machine, remembers them, acts for them, and grows closer the longer they spend together. N.E.K.O. brings that idea into companionship in the broadest sense. Your companion talks with you in realtime voice, sees your screen, remembers the days you have shared, plays games with you and takes part in your social life. Under the hood is a local runtime, open-sourced under Apache 2.0, that integrates realtime voice and vision, a biomimetic long-term memory, embodied avatars, proactive behavior, agent tools and a plugin SDK. Around it, a warm community is taking shape: users write plugins, forge and share story cards, and send their companions to visit one another.
- Built-in two-player games: air basketball, badminton, drawing guess, soccer, and watch-together. A mini-game SDK and a game-vision service let developers add more.
- Plays alongside you: in Overcooked! 2, Slay the Spire 2 and Minecraft she is your player 2 and plays the game with you. In War Thunder, Strinova, World of Warships and KARDS she is your copilot: community plugins read the game's telemetry or screen so she can give live callouts and commentary, and she never cheats. Any auto-play is opt-in. The bundled Minecraft plugin connects to a separately started local Minecraft agent server (setup).
- Takes the controls: when you ask, plugins let her play by herself in Plants vs. Zombies, Titanfall 2, Half-Life & Portal and Subnautica; you can take over at any time.
- Knows your game: companions for Genshin Impact, Honkai: Star Rail, Civilization VI and RimWorld read the game state so she can chat about your run and give advice.
- Screen awareness: she can watch whatever you are playing and chime in.
- Visits: two users' companions meet in a shared room, talk, and each brings the memory home.
- Nekoverse, the companion community:
- story cards forged in the app from your companion's memories of your days together, then collected, nurtured and shared;
- a feed curated by YUI, weekly picks of hidden gems, creator rankings, and a points store.
- Group-aware memory: the companion can keep separate memories per person and per group. Visits use this today, and group-chat plugins can build on it.
market.project-neko.cn had 100+ plugins, 10K+ downloads and 50+ active developers as of October 2026. A few community favorites:
| Plugin | What it adds |
|---|---|
| War Thunder Catgirl Copilot | Reads the game's local telemetry and turns it into battle callouts and commentary |
| Slay the Spire 2 Companion | Overlays her live commentary on the game as scrolling comments, rates each turn's risk and suggests plays; auto-play is optional |
| PVZ Agent | She plays Plants vs. Zombies herself: watches the board, says her plan out loud and makes the moves, and you can steer her |
| Strinova Companion | Plays Strinova alongside you with voice callouts, starting and stopping with the game |
| Live Streaming Assistant | Handles live comments, viewer profiles and commentary on streaming platforms |
| Catgirl Diary | She keeps a daily diary of small moments and moods |
Any developer can publish with the plugin SDK and the neko-plugin CLI.
- Today:
- chat and long-term memory sync between devices (opt-in);
- Steam Cloud saves;
- the mobile app is in closed beta.
- Planned: companion smart hardware, including but not limited to mirrors, plush dolls, drones and robots. The goal is for the same companion and memory to carry over to each device.
| Area | Support |
|---|---|
| Models | Mainstream Realtime APIs and ChatCompletion-compatible APIs, a free hosted route that needs no key, and local or self-hosted deployment |
| Voice | Mainstream cloud TTS / ASR plus local engines (CosyVoice, GPT-SoVITS, faster-whisper), with voice cloning |
| Avatars | Live2D, VRM, MMD, PNGTuber, plus a desktop widget mode |
| Platforms | Windows / macOS / Linux desktop (also on Steam), Docker, source; mobile in closed beta |
| UI languages | English, Simplified and Traditional Chinese, Japanese, Korean, Russian, Spanish, Portuguese |
Realtime conversation
- Realtime voice over Realtime APIs (OpenAI-protocol and Gemini profiles). Text chat is a separate mode built on standard ChatCompletion; a failed voice session reports an error rather than switching modes on its own.
- Screen and window understanding through a dedicated vision model.
- Voice stack:
- mainstream cloud TTS / ASR plus local engines (CosyVoice, GPT-SoVITS, faster-whisper);
- voice cloning (MiniMax, Qwen / CosyVoice);
- CAM++ speaker verification;
- an optional offline wake word (sherpa-onnx).
- Proactive behavior: the companion starts conversations from what is on screen, from trending feeds (Bilibili, Tieba, Twitch, YouTube), from personal updates and from music. It also sends break reminders and mini-game invitations. A BM25 anti-repetition corpus keeps it from repeating itself.
Biomimetic memory: three domains, five layers
The design follows how people learn from experience: different kinds of experience end up in different cognitive structures. Experience flows through five layers:
Recent → Summary → Fact → Reflection → Model
It is organized into three domains:
| Domain | About | Ends in | Status |
|---|---|---|---|
| Formative | who I am | Self-model: a stable, consistent self-image | ✅ Shipped |
| Social | others and relationships | Person models: traits, preferences and relationships of others, scoped per person and per group | ✅ Shipped |
| Experiential | how to act | Skill / Policy: better ways of doing things | 🚧 Early: agent tool-routing lessons learned from user corrections |
Memories enter the prompt in three ways:
- Always in context: the self-model, summary and recent history.
- On-demand retrieval: the model calls a
recall_memorytool. Lookup runs keyword search (BM25) and vector search in parallel and merges the two result lists (reciprocal-rank fusion). It also supports time-window recall. - Condition-triggered: reflections are raised in proactive chat, and a high bar decides when a memory is added to a turn automatically.
Supporting machinery:
- Evidence scores. Every fact and reflection has a score that rises when the user confirms it and falls when they dispute it, and both counts decay over time. The score moves a reflection from pending t 8000 o confirmed and then into the persona.
- Contradiction correction. Conflicting persona entries are queued for an LLM to correct.
- Crash safety. An append-only event log, a task outbox that re-runs unfinished work after a restart, and idempotent writes.
- Embedding fallback. A local ONNX embedding model, with automatic keyword-only fallback on low-memory machines.
- Deterministic import from OpenClaw / Hermes Markdown and Molili exports.
Avatars, agent, plugins and cloud
- Avatars:
- Live2D, VRM, MMD and PNGTuber, with emotion mapping;
- a desktop widget mode;
- VMC-protocol pose output to external apps;
- Steam Workshop sharing for models, voices and character cards.
- Agent:
- channels: computer use, browser use, MCP, an OpenClaw-compatible external agent adapter, and user plugins;
- the companion keeps talking to you while a task runs.
- Plugins:
- a plugin SDK with a
neko-pluginCLI and a Vue plugin manager; - a plugin market and a knowledge-pack market;
- built-in plugins for web search, MCP, music, reminders, WeChat, Minecraft and more.
- a plugin SDK with a
- Cloud: a free hosted route covers realtime voice, text, vision, agent models and TTS, with no key required.
From the start, the goal of Project N.E.K.O. has been a living AI-companion metaverse built by its community. In that metaverse the same companion follows you across desktop, phone, games and devices; companions meet one another; and the whole network stays connected to the real world.
Personal agents have made "an AI of your own" mainstream. Our job is to bring that stack into everyday companionship, and to provide what a general-purpose agent does not:
| Pillar | Today | Planned |
|---|---|---|
| Seamless multi-device sync: one companion, one memory, every device | Desktop chat and long-term memory sync, Steam Cloud saves; mobile app in closed beta | Full mobile release |
| Plugin market and cloud services: external compute including, but not limited to, LLM, voice, decision-making and visual recognition | Plugin market, knowledge-pack market, Steam Workshop, free hosted voice / text / vision / TTS route | More hosted capabilities |
| Companion social platform | Visits, community hub | Daily analysis, chat rooms |
| Companion smart hardware: including, but not limited to, mirrors, plush dolls, drones and robots | — | In planning |
How N.E.K.O. differs from neighboring tools:
| Compared with | They are | N.E.K.O. |
|---|---|---|
| General agents (OpenClaw, Hermes, …) | Task engines: instruction → work → done | Relationship-first. It can drive such agents as its hands, while staying with you during the task |
| Roleplay front ends (SillyTavern, …) | Text RP clients that you wire to models and keep consistent by hand with lorebooks | Integrated end to end: native voice and vision, an embodied avatar, persistent memory with a sense of time, and the ability to act on your computer |
- ✅ v0.7: agent capabilities
- ✅ v0.8: memory system, desktop pet mode, built-in two-player mini-games
- ✅ v0.9: Linux support, mobile closed beta, companion network infrastructure (community)
- ✅ v0.9.1: current release
- 🎯 v1.0: targeted before the end of 2026. Completes the community and plugin-market features and the remaining optimization work.
- 🔭 After 1.0 (planned):
- the generalized skill / policy layer of experiential memory;
- companion social-platform features (daily analysis, chat rooms);
- in-house models;
- companion hardware.
Related project: K.U.R.O. is the first AI-native indie game built on the N.E.K.O. stack. It is in development, and a demo is available on its releases page.
Requirements:
- Python 3.11 exactly;
- uv;
- Node.js
^22.12.0 || >=24.0.0(needed to build the frontends).
# Partial clone: the history holds large retired binaries (~390 MB .git), so fetch blobs on demand
git clone --filter=blob:none https://github.com/Project-N-E-K-O/N.E.K.O.git
cd N.E.K.O
uv sync
# Unpack built-in avatars and build the React chat window and the Vue plugin manager
./build_frontend.sh # Windows: .\build_frontend.bat
# Start memory, main and agent services together
uv run python launcher.py
Open the URL the launcher prints (normally http://127.0.0.1:48911) and configure providers at /api_key. Splitting services by hand is only for diagnostics. See Development Setup and Manual Source Setup.
Docker (Linux): requires Docker Compose v2.24 or newer. In docker/, copy env.template to .env, set your provider and API key there (NEKO_CORE_API / NEKO_CORE_API_KEY), then run docker compose up -d. By default it listens on HTTP 48911 and HTTPS 48912; override them with NEKO_HTTP_PORT / NEKO_HTTPS_PORT. For volumes, TLS, image tags, upgrading from the old mount layout and troubleshooting, see the Docker guide and docker/README_Docker.md.
The developer docs at project-neko.online are the reference for everything beyond this page:
- Getting started
- Project structure
- Architecture
- API reference
- Configuration
- Plugin development
- Deployment
- Contributing
Read CONTRIBUTING.md first. Repository rules for AI-assisted development live under .agent/; see the setup guide.
- Bugs and scoped proposals go to GitHub Issues.
- Discussion happens on Discord and in the developer QQ group 1092413003.
N.E.K.O. enables token-usage telemetry by default. It sends no conversation content and identifies a device only by a pseudonymous ID (see below). It is used to track version compatibility, model-usage distribution and error rates.
Opt out: set DO_NOT_TRACK=1 (or NEKO_DO_NOT_TRACK=1). Remote reporting is then fully disabled; token usage is still recorded locally in token_usage.json, which is not uploaded. The variable is read when a process starts, so restart N.E.K.O. after setting it; no rebuild is needed. With Docker, add it to docker/.env and recreate the container with docker compose up -d.
| ✅ Collected | ❌ Not collected |
|---|---|
| LLM token usage (prompt / cached / completion) | Conversation content, text, voice, images |
| Model name and call type (conversation / memory, etc.) | User name, API keys, GitHub ID |
| Call and error counts | IP address, location, MAC, hardware serials |
| App version, A/B branch, locale, time zone, distribution channel (source / release / steam) | File paths, cookies, browser fingerprints |
| Pseudonymous device ID (see note below) | Any other PII that can identify a person (apart from the Steam64 ID below) |
| Device hardware profile: OS, CPU architecture, RAM tier, CPU tier | |
| Usage metrics: feature-use counts, latency and session-length histograms, error classes, coarse settings state (on / off, bucketed intervals) and avatar type | |
| Only in release builds, when the Steamworks SDK initializes and you are logged in to the Steam client: your Steam64 user ID | Any other account ID (GitHub / Google / OpenAI, etc.) |
Discrete events such as crash fingerprints (a hash of the traceback) and onboarding steps are written only to local events-YYYYMMDD.jsonl files. They are not uploaded, and local cleanup deletes files older than 7 days.
Pseudonymous device ID: a one-way SHA-256 hash. The raw inputs (machine ID, MAC, install path) are never sent.
- Primary path:
SHA256(OS_machine_id ‖ namespace).- Fallback path:
SHA256(uuid.getnode() ‖ install path ‖ namespace).- Every report carries both IDs (the current one and the one from the older algorithm), so the server can link them to the same device.
The primary ID stays the same as long as the OS machine ID does; the fallback ID can change if the network adapter (
uuid.getnode()) or the install path changes. Because it is stable, under GDPR / PIPL it is pseudonymous data, not fully anonymous data. The server uses it to group reports from the same device: active-device counts (DAU / WAU / MAU), retention, version-compatibility attribution, and per-device attributes such as the hardware profile and, when reported, the Steam64 ID.Steam64: the public numeric ID that the Steam client exposes to every third-party SDK after you log in. It is the number at the end of your Steam profile URL. It contains no email, phone number or real name, but it stays the same across sessions.
The code decides when it is sent:
app/main_server/__init__.pycallsinitialize_steamworks()at startup._get_telemetry_metadata()inutils/token_tracker/telemetry.pyreads the ID only in a release build, when the SDK is available andUsers.GetSteamID()returns a non-zero value.- Source runs never send it: for any non-release build the function returns
("source", ""), even with the Steam client open.To prevent it, set
DO_NOT_TRACK=1orNEKO_DO_NOT_TRACK=1; these variables are the only way to turn telemetry off. Quitting the Steam client only leaves the Steam64 field empty for that run: the rest of the payload is still sent, and a Steam64 ID reported earlier stays on the server.
Implementation:
- Client metadata and reporting:
utils/token_tracker/telemetry.pyandutils/token_tracker/reporting.py. - Server transport and storage:
local_server/telemetry_server/README.md. Requests are HMAC-SHA256 signed, accepted only within ±5 minutes to block replays, and rate-limited to 120 per hour per device. Raw events are append-only; an admin cleanup endpoint deletes old ones (default cutoff 180 days), and how often it runs is up to the server operator; the same cleanup also removes per-day usage-metric rows; device and daily token-usage aggregate tables are updated over time, and the daily aggregates are kept indefinitely. Deleting an account puts its Steam64 ID on a denylist and removes the stored Steam64 links; raw events already received are not rewritten and keep the ID until the cleanup removes them. - Reporting runs from the same 60 s background loop that saves token data locally (each process also saves once at shutdown), and a separate interval check keeps each server process to at most about one report every 60 s.