Explore India state by state with a chat-guided language atlas built on Next.js and the Agora Conversational AI Engine. The experience combines a clickable India map, state-specific phrase cards, a spin-the-map view, live transcript, and real-time pipeline latency via AGENT_METRICS (Agent Toolkit).
Getting started is quick and easy: install the CLI (skip if you already have it), bind an Agora project, install dependencies, and run the app.
-
Install the Agora CLI and sign in (skip if
agorais already on your PATH):curl -fsSL https://raw.githubusercontent.com/AgoraIO/cli/main/install.sh | sh -s -- --add-to-path agora login -
Bind and run If you cloned this repository directly, bind your Agora project and write
.env.local:agora project use <your-project> agora project env write .env.local pnpm install pnpm dev
-
Open http://localhost:3000, choose a state on the map, and click Start voice guide.
If the agent does not join or transcripts do not appear, run agora project doctor --deep to check credentials, feature enablement, network reachability, and local env binding.
Use this path if you already cloned this repo (for example to contribute or fork):
agora login
agora project use <your-project>
pnpm install
agora project env write .env.local
agora project doctor --deep
pnpm devDeploy this as a standard Next.js app on Vercel and set the environment variables below:
agora project use <your-project>
agora project env write .env.local
rg "^(NEXT_PUBLIC_AGORA_APP_ID|NEXT_AGORA_APP_CERTIFICATE)=" .env.localCopy those two values into Vercel Project Settings -> Environment Variables.
Defined in env.local.example.
| Variable | Required | Default | Notes |
|---|---|---|---|
NEXT_PUBLIC_AGORA_APP_ID |
✅ | — | Agora Console → Project → App ID. |
NEXT_AGORA_APP_CERTIFICATE |
✅ | — | Agora Console → Project → App Certificate. Server-side only. |
NEXT_PUBLIC_AGENT_UID |
123456 |
Must match the agentUid in app/api/invite-agent/route.ts. |
|
NEXT_AGENT_GREETING |
— | Override the agent's opening line. | |
NEXT_SARVAM_API_KEY |
— | Sarvam ASR key. Required together with NEXT_SARVAM_SUBSCRIPTION_KEY to enable Sarvam. |
|
NEXT_SARVAM_SUBSCRIPTION_KEY |
— | Sarvam TTS subscription key. Required together with NEXT_SARVAM_API_KEY. |
The default agent configuration in app/api/invite-agent/route.ts uses Agora-managed STT, LLM, and TTS, with the selected state's language and context passed into the conversation. No extra vendor API keys are required for the base experience. If you set both Sarvam env vars, the agent switches STT/TTS to Sarvam for the supported Indic languages.
Default vs BYOK — the quickstart ships with Agora-managed inference for a zero-key install. Switch to BYOK below when you need to bring your own provider quotas, models, or vendors.
# Dev
pnpm dev # start the Next.js dev server
# Quality
pnpm run lint # eslint
pnpm run typecheck # tsc --noEmit
pnpm run doctor # local prereqs + env binding
# CI / pre-ship
pnpm run verify:api # API contract checks
pnpm run build # production build
pnpm run verify # doctor + lint + typecheck + verify:api + buildRun pnpm run verify before shipping changes — it covers local prerequisites, lint, type safety, the core API route contracts, and the production build.
The browser fetches a combined RTC + RTM token (buildTokenWithRtm) from this app, joins the channel using a single RTC client, and uses RTM as the data channel for transcript, agent state, metrics, and error events. The Conversational AI Engine joins the same channel as the configured NEXT_PUBLIC_AGENT_UID and runs the STT → LLM → TTS pipeline in Agora Cloud.
- a Bhaasha-style atlas shell with a clickable India map and spin-wheel discovery mode
- browser RTC audio plus RTM transcript and state events
- server routes for token generation, invite, and stop
- state-specific prompt and STT language selection when a user starts a voice guide
AgentVisualizerfor agent state and a built-in transcript panel for live turns- per-stage latency header driven by
AGENT_METRICS - Agora-managed default STT, LLM, and TTS configuration
- Sarvam STT/TTS automatically enabled when Sarvam env vars are configured
These captures live in art/ and are ready for GitHub preview:
Watch the short demo video: art/demo.mov
- The browser requests an RTC + RTM token from
/api/generate-agora-token. - The backend invites an Agora cloud agent with
/api/invite-agent, including the selected state's language context. - The browser joins the channel and publishes mic audio.
- The client receives transcript, agent state, and
AGENT_METRICS(per-stage latency) events over RTM. - On end, the client calls
/api/stop-conversation, logs out RTM, and unmounts the call view so Agora React hooks clean up RTC publish/join and the local microphone track.
The quickstart defaults to Agora-managed inference. To bring your own keys, uncomment the matching blocks in app/api/invite-agent/route.ts and add the corresponding env vars.
# Deepgram STT
NEXT_DEEPGRAM_API_KEY=...
# OpenAI-compatible LLM
NEXT_LLM_URL=https://api.openai.com/v1/chat/completions
NEXT_LLM_API_KEY=...
# ElevenLabs TTS
NEXT_ELEVENLABS_API_KEY=...
NEXT_ELEVENLABS_VOICE_ID=...app/api/generate-agora-token/route.ts— issues RTC + RTM tokensapp/api/invite-agent/route.ts— starts the agent session and configures the pipelineapp/api/stop-conversation/route.ts— stops the agent sessioncomponents/LandingPage.tsx— entry point: atlas shell, map selection, token fetch, RTM login, conversation lifecyclecomponents/BhaashaIndiaMap.tsx— clickable India map loaded frompublic/bhaasha-map.svgcomponents/BhaashaSpinWheel.tsx— spin-to-discover state wheelcomponents/BhaashaStatePanel.tsx— selected-state language card and voice CTAcomponents/ConversationComponent.tsx— RTC client, transcript state,AGENT_METRICS, mic releasecomponents/QuickstartConversationLayout.tsx— in-call header, transcript rail, controls dockcomponents/QuickstartPipelineMetrics.tsx— per-stage latency chips in the headercomponents/QuickstartTranscriptPanel.tsx— live transcript railcomponents/QuickstartPreCallCard.tsx— pre-call hero cardlib/conversation.ts— transcript normalization and visualizer state mappingAGENTS.md— primary agent-facing guide
- Agent does not join or transcripts are missing: run
agora project doctor --deep. pnpm run doctorfails: runagora project env write .env.local, then retry.- Manual clone / env values:
agora project use <your-project>thenagora project env write .env.local. - RTM login fails: keep
app/api/generate-agora-token/route.tsonRtcTokenBuilder.buildTokenWithRtm— RTC-only tokens will not satisfyrtm.login. - Transcript speakers inverted: check the
uid === "0"remap incomponents/ConversationComponent.tsx. - Agent never appears in channel: ensure
NEXT_PUBLIC_AGENT_UIDmatches the value used inapp/api/invite-agent/route.ts.
Pull requests welcome — see CONTRIBUTING.md for development setup and conventions.
Please do not open public issues for security reports. Email security@agora.io with details and reproduction steps.
Released under the MIT License.