8000 Add integration tests for HTTP routes and API endpoints · Issue #13 · jra3/mulm · GitHub
[go: up one dir, main page]
Skip to content

Add integration tests for HTTP routes and API endpoints #13

New issue
New issue
Open
Open
Add integration tests for HTTP routes and API endpoints#13
Labels
enhancementNew feature or requestpriority:mediumMedium priority - important but not urgenttestingTesting, test coverage, and quality assurance

Description

@jra3
jra3
opened on Jun 12, 2025

Problem Description

The application currently lacks comprehensive integration tests for HTTP routes and API endpoints. This makes it difficult to catch regressions and ensure that the entire request/response cycle works correctly.

Proposed Solution

Implement integration tests that cover all major HTTP routes and API endpoints, testing the full stack from HTTP request to database queries to response rendering.

Implementation Tasks

  • Set up integration test framework and utilities
  • Test Species Explorer search endpoint with various parameters
  • Test sorting functionality with different sort fields
  • Test error handling for invalid inputs
  • Test pagination when implemented
  • Test response formats (HTML vs JSON if applicable)
  • Add tests for edge cases (empty results, special characters, etc.)
  • Set up CI to run integration tests

Reference

This issue was identified in PR #9 review comments as part of the Species Explorer feature development.

Impact

Integration tests will provide confidence in deployments and help catch issues that unit tests might miss.

Activity

  1. jra3
    added
    enhancementNew feature or request
    on Jun 12, 2025
  2. jra3
    added
    testingTesting, test coverage, and quality assurance
    priority:mediumMedium priority - important but not urgent
    on Jun 25, 2025

  3. jra3 commented on Jun 26, 2025

    @jra3
    jra3
    on Jun 26, 2025
    OwnerAuthor

    The foundation for comprehensive integration testing has been significantly strengthened by recent database testing infrastructure improvements (commit 0931a4a).

    ✅ Current Integration Test Infrastructure

    Database Integration Tests:

    • src/__tests__/members.ts - Comprehensive member and roster testing with real database operations
    • src/__tests__/species.ts - Species explorer and data management testing
    • Robust testDbHelper.helper.ts with utilities for creating test data and managing database lifecycle

    Test Quality Improvements:

    • In-memory SQLite databases for fast, isolated testing
    • Proper foreign key constraint enforcement
    • Utilities for creating realistic test data (createTestMembers, createTestSpeciesName)
    • Suite-level database fixtures with automatic setup/teardown
    • All 92 tests passing consistently

    🎯 HTTP Route Testing Assessment

    Current State:
    The existing database integration tests provide excellent coverage of the data layer that underlies all HTTP routes. The infrastructure is now solid enough that adding HTTP-level integration tests would be straightforward.

    Options for HTTP Route Testing:

    1. Extend existing test infrastructure - The testDbHelper could easily be used with supertest or similar HTTP testing tools
    2. Focus on critical user journeys - Member registration, submission creation, admin approval workflows
    3. API endpoint validation - Ensure all routes return expected status codes and data formats

    🤔 Recommendation

    Option A: Close this issue since we now have robust integration testing of the core business logic and data layer.

    Option B: Update this issue to be more specific about HTTP-level testing requirements and user journey validation.

    The database integration tests we now have provide significantly more value than typical HTTP route tests, as they validate the actual business logic and data integrity. HTTP route tests would primarily validate request/response handling rather than core functionality.

    What do you think? Should we focus on other priorities, or are there specific HTTP routes/user journeys that would benefit from dedicated integration tests?

  4. jra3 commented on Oct 8, 2025

    @jra3
    jra3
    on Oct 8, 2025
    OwnerAuthor

    Comprehensive E2E & Integration Testing Plan

    🎯 Overview

    Build a robust E2E and integration testing suite that runs in GitHub Actions, testing actual web flows with HTTP requests, database interactions, and full request/response cycles.

    📋 Implementation Plan

    Phase 1: E2E Test Infrastructure (Foundation)

    1. Create E2E test utilities (src/__tests__/e2eHelper.ts)

      • Test app factory that creates Express app instance with all routes
      • Supertest integration for HTTP testing
      • Test database setup using existing setupTestDatabase()< 8000 /li>
      • Session/auth helpers for authenticated requests
      • Cookie jar management for multi-request flows

    2. Create test config (src/config.test.json)

      • Test-specific configuration (no real OAuth, SMTP, R2 credentials needed)
      • Mock/stub external services (email, R2 storage)

    Phase 2: Critical User Journey Tests (High-Value E2E)

    1. Authentication flows (src/__tests__/e2e/auth.test.ts)

      • Sign up → email verification → login flow
      • Password reset flow
      • Google OAuth flow (mocked)
      • Session persistence across requests
      • Rate limiting enforcement

    2. Submission creation flow (src/__tests__/e2e/submission-flow.test.ts)

      • Anonymous user: create draft → sign up → complete submission
      • Authenticated user: full submission creation
      • Tank preset creation and reuse
      • Image upload flow (with R2 mocking)
      • Form validation and error handling

    3. Admin approval workflow (src/__tests__/e2e/admin-workflow.test.ts)

      • Member submits → appears in queue
      • Admin approves → updates member points
      • Witness confirmation flow
      • Request changes → member resubmits
      • Denial workflow

    4. Member progression (src/__tests__/e2e/member-progression.test.ts)

      • Multiple submissions → level advancement
      • Specialty award triggers
      • Points accumulation
      • Standings page updates

    Phase 3: HTTP Route Integration Tests (API Coverage)

    1. Public routes (src/__tests__/integration/public-routes.test.ts)

      • GET / (homepage with activity feed)
      • GET /standings (all programs)
      • GET /species (species explorer)
      • GET /member/:id (member profiles)
      • Verify correct status codes, response types, data structure

    2. Auth routes (src/__tests__/integration/auth-routes.test.ts)

      • POST /auth/signup (validation, duplicate handling)
      • POST /auth/login (success, failure, rate limiting)
      • POST /auth/forgot-password
      • GET /auth/logout
      • Dialog endpoints (HTMX partials)

    3. Submission routes (src/__tests__/integration/submission-routes.test.ts)

      • GET /submissions/new (form rendering)
      • POST /submissions (create with validation)
      • GET /submissions/:id (view permissions)
      • PATCH /submissions/:id (update draft)
      • DELETE /submissions/:id (delete own draft)

    4. Admin routes (src/__tests__/integration/admin-routes.test.ts)

      • GET /admin/queue (access control)
      • POST /admin/submissions/:id/approve
      • POST /admin/submissions/:id/confirm-witness
      • GET /admin/members (member management)
      • PATCH /admin/members/:id

    5. API endpoints (src/__tests__/integration/api-routes.test.ts)

      • GET /api/members/search (typeahead)
      • GET /api/species/search (typeahead)
      • Response format validation (JSON structure)
      • Edge cases (special characters, empty results)

    Phase 4: GitHub Actions Workflow (CI/CD)

    1. Enhanced CI workflow (.github/workflows/ci.yml)

      • Add E2E test job (separate from unit tests for clarity)
      • Run integration tests with full app stack
      • Test in Node 20.x (current production version)
      • Generate test coverage report
      • Store test artifacts (screenshots, logs on failure)

    2. Docker test environment (optional, docker-compose.test.yml)

      • If E2E tests need nginx/SSL simulation
      • Run tests against containerized app
      • Verify production-like environment

    Phase 5: Test Quality & Maintenance

    1. Test documentation (update CLAUDE.md)

      • How to run E2E tests locally
      • How to debug failing tests
      • Test patterns and conventions
      • CI workflow explanation

    2. Coverage targets

      • Critical user journeys: 100% coverage
      • All routes: HTTP 200/400/401/403/404 cases
      • Edge cases: validation, auth, rate limiting
      • HTMX partial rendering

    🔧 Technical Details

    Existing Foundation:

    • ✅ supertest already installed
    • ✅ testDbHelper provides database fixtures
    • ✅ In-memory SQLite for fast, isolated tests
    • ✅ Current tests (rateLimiter) show pattern

    New Dependencies:

    • None! Everything needed is already installed

    Key Testing Patterns:

    // E2E test example
const { app, agent } = await createTestApp();
const response = await agent
  .post('/auth/signup')
  .send({ email: 'test@example.com', password: 'Secure123!' })
  .expect(200);

// Follow redirect and verify session
const profileResponse = await agent
  .get('/me')
  .expect(200);

    Mocking Strategy:

    • Database: Real SQLite (in-memory)
    • Email: Mock nodemailer transport
    • R2 Storage: Mock S3 client
    • OAuth: Mock token exchange
    • External APIs: Mock responses

    📊 Success Metrics

    • All critical user journeys covered by E2E tests
    • 95%+ route coverage with integration tests
    • Tests run in <2 minutes in CI
    • Zero flaky tests (deterministic, isolated)
    • Clear failure messages for debugging

    🚀 Rollout Strategy

    1. Phase 1-2 first → Get E2E infrastructure + auth flows working
    2. Phase 3 → Add route integration tests incrementally
    3. Phase 4 → CI integration (run in GitHub Actions)
    4. Phase 5 → Documentation and maintenance

    Estimated effort: 2-3 days of focused work

    This plan leverages the existing robust database testing infrastructure and extends it to cover full HTTP request/response cycles, user journeys, and real-world workflows.

  5. jra3 commented on Oct 23, 2025

    @jra3
    jra3
    on Oct 23, 2025
    OwnerAuthor

    Significant Progress on Integration Testing

    While full HTTP route testing isn't yet complete, we've made major progress on integration testing for the admin approval workflow:

    New Integration Tests Added (66 tests)

    1. Witness Operations (witness-operations.test.ts) - 20 tests

      • Database-level integration tests for witness confirmation/decline
      • State machine validation
      • Transaction integrity

    2. Request Changes Workflow (request-changes.test.ts) - 22 tests

      • Full workflow integration testing
      • State validation across submission lifecycle
      • Multi-admin coordination

    3. Edit Approved Submissions (edit-approved-submission.test.ts) - 24 tests

      • Complex field update integration
      • Approval state preservation
      • Multi-step edit workflows

    Test Coverage Summary

    • 587 total tests (up from 521)
    • All passing with zero failures
    • ~31% → ~35% estimated code coverage

    Remaining Work

    Still need HTTP route-level integration tests for:

    • Admin routes (approval handlers, queue displays)
    • Submission routes (CRUD operations)
    • Authentication routes (login, OAuth, WebAuthn)
    • Account routes (password changes, settings)

    Keeping this issue open to track the remaining HTTP route testing work.

  6. jra3 commented on Oct 13, 2026

    @jra3
    jra3
    on Oct 13, 2026
    OwnerAuthor

    The October maintainability review produced bounded implementation proposals for the remaining testability foundations:

    These complement the existing handler/render and database tests; they do not claim that the full route coverage described in this issue is complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpriority:mediumMedium priority - important but not urgenttestingTesting, test coverage, and quality assurance

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions
      0