An automation framework that takes an OpenAPI/Swagger spec, uses a local LLM (via Ollama) to generate positive, negative, boundary, and required-field test cases, orchestrates CRUD-lifecycle execution against the target API, and reports the results through a Flask + React web app for managing projects, generations, executions, and environments.
- Backend: Python 3, Flask, Flask-CORS, PyJWT, PyYAML, Requests
- Frontend: React 18, Vite, MUI, Axios, React Router, Recharts
- LLM: Ollama (local), default model
qwen3.5:9b - Storage: flat JSON files under
framework_data/(no external database)
flowchart TD
A[Swagger / OpenAPI spec<br/>file or URL] --> B[Parser<br/>change detection via hash diff]
B --> C[Testcase Generator]
C -->|builds prompt| D[Ollama LLM]
D -->|parses response| C
C --> E[Generated test cases<br/>positive / negative / boundary / required-field]
E --> F[Execution Runner]
F --> G[Lifecycle Executor<br/>CREATE → READ → UPDATE → READ → DELETE → READ]
G --> H[Validators<br/>per HTTP method + data consistency]
H --> I[Execution Report<br/>JSON + HTML]
I --> L[(Stored on disk:<br/>latest report + history)]
L --> M[Execution history view<br/>per project, latest + past runs]
subgraph WEB[" Web app: Flask API + React SPA "]
J[Project / Generation / Execution / Environment management] --> A
K[Scenario Job Queue<br/>async human-written scenarios] --> C
M
end
Through the web app, users manage projects, upload Swagger specs, trigger generations and executions, define environments, submit human-written scenarios as async background jobs (via the scenario job queue), review/approve generated test cases, and view execution reports — both the latest run and historical runs per project.
Identity & change detection: the parser computes a hash per API endpoint and compares it against the previously parsed spec, so only new or changed endpoints are regenerated. Lifecycle execution stores created resources in a resource context so dependent steps (READ/UPDATE/DELETE) can reuse real resource IDs from prior responses.
app/ Flask application (routes, controllers, middleware, storage, scenario job queue)
Parser/ Swagger/OpenAPI parsing and change detection
testcase_generator/ LLM-driven test case generation (prompts, parsing, retries, scenario-based generation)
execution/ Test execution engine (HTTP execution, lifecycle orchestration, validators)
storage/ JSON persistence for test cases and execution reports
prompts/ Markdown prompt templates used for LLM calls
llm/ LLM client abstraction (Ollama connector)
frontend/ React SPA (Vite, MUI)
configs/ settings.py — LLM, paths, users, JWT, logging config
Utils/ Shared utilities (logging, resource grouping)
server.py Flask web server entry point
- Python 3.x and pip
- Node.js and npm (for the frontend)
- Ollama installed and running locally, with the configured model pulled:
Ollama must be reachable at
ollama pull qwen3.5:9bhttp://localhost:11434(configurable inconfigs/settings.py).
-
Create and activate a virtual environment
python -m venv venv- Windows:
venv\Scripts\activate - macOS/Linux:
source venv/bin/activate
- Windows:
-
Install backend dependencies
pip install -r requirements.txt -
Start Ollama and make sure the model from
configs/settings.py(qwen3.5:9bby default) is pulled. -
Run the web app (with the virtual environment active)
python server.pyThe Flask API starts on
http://localhost:8080. -
Run the frontend (dev mode)
cd frontend npm install npm run dev -
Log in with one of the default accounts from
configs/settings.py(change these, and the JWT secret, before any production use):admin/admin123(admin role)viewer/viewer123(viewer role)
With the backend running, the full REST API documentation (parameters, request/response schemas, and status codes for every endpoint) is available at:
http://localhost:8080/swagger/
The raw OpenAPI spec (used by the UI above) is served as JSON at:
http://localhost:8080/swagger.json
Most endpoints require a Bearer JWT — use the "Authorize" button in the UI and paste the token returned from POST /api/auth/login as Bearer <token>. Doc content lives separately from the route code, under docs/swagger/.
- Features — detailed description of every feature in the app
- Technical Reference — class diagrams, sequence diagrams, data models
A standalone Flask test app — a "School API" demoing students, teachers, and classes — lives at tests/school-api. It runs independently of the main app, with its own virtual environment, port, and in-memory data.
cd tests/school-api
python -m venv venv
venv\Scripts\activate # Windows
# source venv/bin/activate # macOS/Linux
pip install -r requirements.txt
python server.py
The API starts on http://localhost:8070. Swagger UI is at http://localhost:8070/swagger/ (spec at /swagger.json). See tests/school-api/README.md for the hardcoded user accounts and full endpoint details.