8000 GitHub - petrhora/alveusgg: alveussanctuary.org / alveus.gg - community-built, open-source website for Alveus Sanctuary, built with Next.js and Tailwind. · GitHub
[go: up one dir, main page]
Skip to content
petrhora
alveusgg
Public
forked from alveusgg/alveusgg

About

alveussanctuary.org / alveus.gg - community-built, open-source website for Alveus Sanctuary, built with Next.js and Tailwind.

www.alveussanctuary.org

Resources

Readme
License

Contributing

Contributing
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository
BranchesTags
 
 

Repository files navigation

Alveus Sanctuary website

This is the community-built, open-source website for Alveus Sanctuary. You can access the site at alveussanctuary.org (or alveus.gg).

See also

Repository overview

This repository holds more than the public website. It also contains the admin dashboard, the stream overlays, a Twitch chat bot, and the donations service behind features like the pixel mural. It is a pnpm workspace monorepo, and most work happens in apps/website.

Path What it is Runs on
apps/website Next.js app: public pages, admin dashboard, stream overlays, and the API (tRPC and REST) that the other apps talk to Vercel
apps/database Prisma schema and client (@alveusgg/database), shared by the website and the chat bot library
apps/chatbot Twitch chat bot that lets mods send "live now" notifications from chat Docker
apps/donations/worker Donations manager: takes in donations from Twitch Charity, PayPal, and Neon CRM, saves them via the website, and runs the pixel mural Cloudflare Workers
apps/donations/core Shared donation types library
apps/donations/neon-crm-api Neon CRM API client and webhook schemas library
docs Deep dives into specific features, e.g. file uploads and attachments

Systems overview

flowchart TB
  chat(["Twitch chat"])
  people(["Visitors, staff & mods<br/>browser, PWA, stream overlays"])
  donations(["Donation sources<br/>Twitch Charity, PayPal, Neon CRM"])

  subgraph repo["This repository"]
    chatbot["<b>Chat bot</b><br/>apps/chatbot<br/><i>Docker</i>"]
    worker["<b>Donations worker</b><br/>apps/donations/worker<br/><i>Cloudflare Workers</i>"]
    website["<b>Website & API</b><br/>apps/website<br/><i>Vercel</i>"]
  end

  data["@alveusgg/data<br/>ambassadors & enclosures"]

  subgraph services["Infrastructure & external services"]
    db[("MySQL<br/>schema in apps/database")]
    storage[("S3 storage<br/>file uploads")]
    upstash["Upstash<br/>QStash, Redis"]
    push["Web push services"]
    apis["External APIs<br/>Twitch, YouTube, Discord,<br/>Weather, Cloudflare Stream"]
  end

  chat <-- "commands" --> chatbot
  people -- "pages, tRPC" --> website
  people -- "live pixels" --> worker
  donations -- "webhooks, sync" --> worker
  chatbot -- "send notification" --> website
  worker -- "new donations" --> website
  data -. "npm package" .-> website
  chatbot --> db
  website --> db & storage & upstash & push & apis
Loading
  • Website (apps/website) is the hub. It serves the pages and the tRPC API, signs people in with Twitch, keeps data in MySQL and uploads in S3-compatible storage, and sends notifications through web push and Discord. A Vercel cron job calls /api/cron every 10 minutes to run the scheduled tasks, such as syncing calendar events and retrying notification pushes.
  • Chat bot (apps/chatbot) listens to Twitch chat. When a mod sends a notification command, the bot passes it to the website API. It reads its own Twitch token and user roles from the shared database.
  • Donations worker (apps/donations/worker) receives donation webhooks (Twitch Charity via EventSub, PayPal IPN, Neon CRM) and syncs with Neon CRM every hour. It queues the donations, saves them via the website's tRPC API, and keeps the pixel mural state that browsers follow live over a WebSocket.
  • Ambassador and enclosure data comes from @alveusgg/data, which is installed from the GitHub Package Registry. This is why the setup below needs a GitHub token.

Finding your way around apps/website/src

  • pages/: most site pages (Next.js Pages Router), plus API routes in pages/api/
  • app/: App Router route handlers for newer API endpoints, RSS feeds, and the OAuth endpoints
  • components/: React components, grouped by feature (admin/, overlay/, content/, …)
  • server/: server-only code, including tRPC routers (server/trpc/router/), database access (server/db/), and clients for external APIs (server/apis/)
  • data/: static content and configuration, such as navigation, staff, events, and scheduled tasks
  • sw/: the service worker that shows push notifications
  • env/: the environment variable schema (keep it in sync with .env.example)

Tech stack

For development:

  • Node.js (see engines in package.json)
  • pnpm with workspaces
  • Prettier (code formatting)
  • ESLint (code linting)
  • Vitest (unit tests)
  • Docker Compose (local MySQL, Redis, and S3)

Website stack (based on the T3 Stack):

  • TypeScript
  • Next.js (framework)
  • tRPC (typesafe API)
  • Prisma (database ORM)
  • Auth.js aka NextAuth.js (auth via OAuth)
  • Tailwind CSS (styling)

Other apps:

  • Chat bot: Node.js with Twurple
  • Donations worker: Cloudflare Workers with Hono, Durable Objects, and Queues

Hosting (production):

  • Vercel (website)
  • PlanetScale (MySQL database)
  • DigitalOcean Spaces (S3-compatible storage)
  • Upstash QStash (queue for push notifications) and Upstash Redis (rate limiting, OAuth codes)
  • Cloudflare Workers (donations worker) and Cloudflare Stream (low-latency cams)
  • A Docker host (chat bot)

External APIs

  • Twitch: OAuth (sign-in), Helix API, chat, and EventSub (charity donations)
  • YouTube Data API
  • Discord (calendar events and webhooks)
  • Weather.com / Wunderground (on-site weather station)
  • Neon CRM and PayPal (donations)

How to contribute

Hey there! Welcome to Alveus.gg! There's a few ways that you can help contribute.

  1. If you find a bug - you can fill out a bug report
  2. If you have an idea that would make Alveus better - please fill out an idea issue
  3. If you have development experience, take a look at our issues labeled good first issue, read our contributing guide and agree to our code of conduct before you get started.

Development setup

Note

If you only want to work on the front end, you may skip the prerequisite and skip setting up a database or file storage (step 5). But you may encounter some errors when running the website without a database or file storage.

Prerequisite

If you aren't working on features related to Twitch authentication, you can set the DISABLE_ADMIN_AUTH to true in your apps/website/.env file (see step 6 and 6.i) in order to use the admin dashboard immediately. Otherwise, create a Twitch application, setting the OAuth callback to be http://localhost:3000/api/auth/callback/twitch. Note down your client ID and client secret.

Local development

  1. Install Node.js using a version manager like nvm or fnm. Run nvm install/fnm install in the project directory to install and nvm use/fnm use to use the correct Node.js version. Alternatively see nodejs.org for other options to download and install Node.js. You can see what version we need in package.json under engines.
  2. Use corepack enable to use pnpm as the package manager. Corepack is a tool provided with Node.js that makes it easy for you to use the specific package manager we expect.
  3. Authenticate with the GitHub Package Registry: npm login --auth-type=legacy --registry=https://npm.pkg.github.com
    1. Use your GitHub username (lowercase) as the username when prompted
    2. Create a GitHub personal access token (classic) with the read:packages scope and use it as the password when prompted
  4. Install dependencies: pnpm install --frozen-lockfile
  5. Run docker compose up -d from within apps/website to start a local MySQL database, Redis, and an S3 bucket.
  6. Copy apps/website/.env.example to apps/website/.env and open your copy in a text editor and fill it:
    1. If your feature is not related to Twitch authentication, you can set DISABLE_ADMIN_AUTH to true in order to use the admin dashboard without authentication. Otherwise, fill in TWITCH_CLIENT_ID and TWITCH_CLIENT_SECRET as mentioned above.
    2. The vapid keys for web notifications have to be generated using pnpx web-push generate-vapid-keys
    3. The Next Auth secret (NEXTAUTH_SECRET), Action API secret (ACTION_API_SECRET), TRPC API secret (TRPC_API_SHARED_KEY) and Vercel Cron Secret (CRON_SECRET) have to be filled with 32-byte Base64-encoded secrets. See Generate secrets below.
    4. The data encryption passphrase (DATA_ENCRYPTION_PASSPHRASE) has to be filled with a 24-byte Base64-encoded secret. See Generate secrets below.
    5. You may define a Weather API key (WEATHER_API_KEY) to use the weather API and stream overlay. See Weather API below.
    6. If you are using Twitch authentication, you may define privileged users once they have signed in in the SUPER_USER_IDS variable with their CUID (using comma separated values)
  7. Copy apps/database/.env.example to apps/database/.env
  8. Copy apps/donations/worker/.env.example to apps/donations/worker/.env
    1. Copy your TRPC_API_SHARED_KEY from apps/website/.env as the value for SHARED_KEY
    2. Set a secret for Twitch EventSub as the value for TWITCH_SUBSCRIPTION_SECRET. See Twitch EventSub secrets for more info. This is used to verify that incoming EventSub webhooks are actually from Twitch.
  9. Push the database schema to the new database using pnpm prisma db push from within apps/database.
  10. Start the dev server using pnpm dev from within apps/website
  11. The website should be running at http://localhost:3000/ (open in browser)
  • Learn more about the stack at Create T3 App - Introduction
  • You can use the Prisma Studio to view your database. Launch it with pnpm prisma studio from within apps/database
  • To change the database schema, see apps/database/README.md
  • You can access a direct MySQL CLI to the database with docker compose exec db sh -c 'MYSQL_PWD=$MYSQL_ROOT_PASSWORD mysql alveusgg'
  • If you're using VSCode, add "typescript.tsdk": "node_modules/typescript/lib" + "eslint.workingDirectories": [{ "pattern": "apps/*" }] to .vscode/s 8000 ettings.json to ensure you're using the correct TypeScript version + ESLint working directories

Generate secrets

We use Base64-encoded random strings for various secrets. To generate these secrets you can use OpenSSL or Python. OpenSSL should be preinstalled on most Unix-like systems (Linux, macOS, WSL). If neither is installed on your system, you may need to install it yourself.

  • Using OpenSSL:
    • Generate a 32-byte secret: openssl rand -base64 32
    • Generate a 24-byte secret: openssl rand -base64 24
  • Using Python: You may need to call python3 instead depending on your installation.
    • Generate a 32-byte secret: python -c "import os, base64; print(base64.b64encode(os.urandom(32)).decode('utf-8'))"
    • Generate a 24-byte secret: python -c "import os, base64; print(base64.b64encode(os.urandom(24)).decode('utf-8'))"

Weather API

Alveus operates a weather station on-site that uploads data to wunderground.com. Wunderground provide an API via api.weather.com that we use to display the weather for the stream overlay (and in the API for the Twitch chat bot). Accessing the API required an API key, which is only granted to users that have a weather station on wunderground.com.

It seems that anyone can sign up for a Wunderground account and register a weather station on the site, at which point you'll be granted access to the API, without ever needing to actually connect a weather station and upload data. It seems you can use this to get an API key to develop with the weather API locally.

In production, we use the actual API key for the account that is associated with the Alveus weather station, which does upload data to the site.

Cloudflare Stream

We use Cloudflare Stream to host a low-latency variation of the live cams specifically for camera operators. To do that, we need to do some setup with Cloudflare.

  1. Login to the Cloudflare dashboard and navigate to Stream -> Live Inputs and click "Create Live Input"

  2. Tick the "Require Signed URLs" box and leave the rest of the defaults

  3. Click "Create Live Input" button

  4. Make a note of the "Live Input ID", this will be used to configure the CF_STREAM_LOLA_VIDEO_ID environment variable.

  5. Make a note of the host, this will be used to configure the CF_STREAM_HOST environment variable. It will look like customer-<unique-customer-id>.cloudflarestream.com and can be found under Connection Info -> WebRTC (WHIP) -> WebRTC (WHIP) URL.

  6. Create an API token with the Stream:Edit permission. This is sometimes referred to as "Stream" under the "Account" dropdown.

  7. Use that token to make a request to create a set of signing keys for Cloudflare Stream. This is global to the account, not just this specific stream.

  8. Make a note of the id and jwk values, these will be used to configure the CF_STREAM_KEY_ID and CF_STREAM_KEY_JWK environment variables.

  9. Ensure that the following environment variables are set in apps/website/.env:

    • CF_STREAM_KEY_ID
    • CF_STREAM_KEY_JWK
    • CF_STREAM_LOLA_VIDEO_ID
    • CF_STREAM_HOST

Neon Donation Embed

For the Neon CRM donation embed to load successfully, the site needs to be running with SSL on a domain that is trusted by Neon CRM. To aid local development with this, we've configured local.alveussanctuary.org to be trusted by Neon CRM and set it to resolve to 127.0.0.1 (localhost). Next.js can be started with HTTPS support bound to that hostname via pnpm dev:next --experimental-https --hostname local.alveussanctuary.org --port 443 in apps/website, allowing https://local.alveussanctuary.org to be accessed to test the Neon CRM donation embed locally.

Chat bot and donations worker (optional)

You only need these when working on the chat bot or on donations and the pixel mural.

  • Chat bot: Copy apps/chatbot/.env.example to apps/chatbot/.env. Set API_SECRET to the website's ACTION_API_SECRET, and fill in the same Twitch client ID and secret as the website. The bot reads its Twitch token from the database, but a normal sign-in only grants basic scopes. So first sign in to your local website as the bot account (BOT_USER_ID) using the Log in button under Provide auth on the admin Twitch API page (/admin/twitch), which also grants the chat scopes (chat:read and chat:edit). That page is admin-only, so set DISABLE_ADMIN_AUTH to true (see step 6.i) or add your user to SUPER_USER_IDS before opening it. Then start it using pnpm start from within apps/chatbot.
  • Donations worker: After step 8 above, start it using pnpm dev from within apps/donations/worker. It runs at http://localhost:8787. To have the worker save donations to your local website, set SITE_URL to http://localhost:3000 in apps/donations/worker/.env. To connect the website to the worker for the pixel mural, set NEXT_PUBLIC_DONATIONS_MANAGER_URL to http://localhost:8787 in apps/website/.env.

Production deployment

Website

The stack should work on any Node.js server or Next.js capable hosting provider and any MySQL server, but has only been tested on Vercel (and PlanetScale) for now.

  1. Create a twitch application (see Development setup above)
  2. Create a PlanetScale account or provide your own MySQL server, that should give you two DSN for the main and shadow database (something like mysql://user:pass@us-east.connect.psdb.cloud/alveusgg?sslaccept=strict and mysql://user:pass@us-east.connect.psdb.cloud/alveusgg/shadow?sslaccept=strict)
  3. Set up some S3-compatible storage for file uploads (e.g. DigitalOcean Spaces, Backblaze R2 or AWS S3)
  4. Go through the apps/website/.env.example and create your own apps/website/.env.production (see Development setup above) and also:
    1. Fill the Prisma section with the database info (DSN)
    2. Fill in the S3 section with your S3-compatible storage info
  5. Push the database schema to the new database using pnpm prisma db push from within apps/database.
  6. Get your own domain (optional)
  7. Create a Vercel account
  8. Create a new Vercel project with these settings:
    • General:
      • Framework Preset: Next.js, leave the other build/dev settings on the default option
      • Root directory: apps/website
      • Node.js Version: See engines in package.json for the required version
    • Domains: add your domains
    • Git: connect your Git repo
    • Environment Variables: Copy your apps/website/.env.production here, and add GH_PKG_REGISTRY_TOKEN: a GitHub personal access token with the read:packages scope. The install command in apps/website/vercel.json uses it to install @alveusgg/data.

Chat bot and donations worker

Both deploy automatically through GitHub Actions when a push to main (production) or preview changes their files (each workflow has a paths filter). They can also be started manually from the Actions tab.

About

alveussanctuary.org / alveus.gg - community-built, open-source website for Alveus Sanctuary, built with Next.js and Tailwind.

www.alveussanctuary.org

Resources

Readme
License

Contributing

Contributing
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages
0