Synthetic OMOP generator. Produces test datasets for the Daphne federated cohort discovery platform.
pip install -e .# 1. Generate synthetic data files
somop generate --config configs/conditions.yaml
# 2. Spin up the full stack (Postgres + omop-lite loader + Bunny)
somop run \
--config configs/conditions.yaml \
--collection-id <bunny-collection-uuid> \
--api-url http://host.docker.internal:8100/api/v1 \
--api-username admin@example.com \
--api-password secretsomop run is self-contained: it spins up a local Postgres container, loads the OMOP data using omop-lite, then starts two Bunny instances — all in a single docker-compose.run.yaml. Ctrl+C stops and removes all containers.
somop generate --config configs/conditions.yaml [OPTIONS]| Option | Default | Description |
|---|---|---|
--config |
(required) | Path to YAML config |
--out-dir |
from config | Override output directory |
--n-people |
from config | Override person.n_people |
--seed |
from config | Override random seed |
--chunk-size |
from config | Override chunk size |
--concepts |
— | Path to CONCEPT.csv to copy into the data directory |
Output files (tab-separated .csv) are written to the out_dir from the config (default: .):
PERSON.csv · CONDITION_OCCURRENCE.csv · DRUG_EXPOSURE.csv · MEASUREMENT.csv · OBSERVATION.csv · PROCEDURE_OCCURRENCE.csv · SPECIMEN.csv · DEATH.csv · LOCATION.csv
Generates a docker-compose.run.yaml containing all services and starts them. No prior somop load needed.
somop run --config configs/conditions.yaml \
--collection-id <uuid> \
--api-url http://host.docker.internal:8100/api/v1 \
--api-username admin@example.com \
--api-password secret| Option | Default | Description |
|---|---|---|
--config |
(required) | Path to YAML config |
--collection-id |
(required) | Bunny collection UUID |
--api-url |
(required) | Daphne API base URL |
--api-username / TASK_API_USERNAME |
(required) | Daphne API username |
--api-password / TASK_API_PASSWORD |
(required) | Daphne API password |
--db-name |
config file stem | Postgres database name |
--db-password / DB_PASSWORD |
postgres |
Postgres password |
--db-url / DATABASE_URL |
— | Use an external Postgres instead of a container (postgresql://user:pass@host:port/db) |
--db-port |
— | Expose the containerised Postgres on this host port |
--drop-db / --no-drop-db |
drop | When using --db-url: drop and recreate the DB before loading |
--concepts |
— | Path to CONCEPT.csv to copy into the data directory before loading |
--bunny-build |
— | Path to a local Bunny build context (Dockerfile directory) instead of pulling the remote image |
--bunny-a-interval |
5 |
Polling interval (seconds) for the bunny-a task resolver |
--bunny-b-interval |
20 |
Polling interval (seconds) for the bunny-b task resolver |
--compose-out |
<out_dir>/docker-compose.run.yaml |
Where to write the compose file |
With an external Postgres (e.g. a persistent dev DB):
somop generate --config configs/mortality_conditions.yaml --concepts ./data/CONCEPT.csv
somop run \
--config configs/mortality_conditions.yaml \
--collection-id <uuid> \
--api-url http://host.docker.internal:8100/api/v1 \
--api-username <username> \
--api-password <password> \
--db-url postgresql://postgres:postgres@host.docker.internal:5435/mortality_conditions \
--bunny-build ../bunny/hutch-bunnyUse this when you want to load data into a long-lived database independently of running Bunny — for example to inspect the schema, reload with a different dataset, or share a DB across multiple runs.
somop load --config configs/conditions.yaml [--db-url ...] [--concepts ...]| Option | Default | Description |
|---|---|---|
--config |
(required) | Path to YAML config |
--db-name |
config file stem | Postgres database name |
--db-password / DB_PASSWORD |
postgres |
Postgres password |
--db-url / DATABASE_URL |
— | Use an existing Postgres instead of a container |
--db-port |
— | Expose the containerised Postgres on this host port |
--drop-db / --no-drop-db |
drop | Drop and recreate the DB before loading |
--concepts |
— | Path to CONCEPT.csv to copy into the data directory |
--compose-out |
<out_dir>/docker-compose.load.yaml |
Where to write the compose file |
Use a multi-config YAML to define N datasets, each with its own data config and Bunny collection ID. somop multi run produces a single docker-compose.multi.yaml with all services namespaced per dataset (db-<name>, loader-<name>, bunny-a-<name>, bunny-b-<name>).
somop multi generate --config configs/multi_example.yaml
somop multi run --config configs/multi_example.yamlMulti-config format (configs/multi_example.yaml):
api_url: http://host.docker.internal:8100/api/v1
api_username: admin@example.com
api_password: secret
db_password: postgres
# bunny_build: ../bunny/hutch-bunny
# concepts: ./data/CONCEPT.csv
bunny_a_interval: 5 # optional: default polling interval (seconds) for all bunny-a instances
bunny_b_interval: 20 # optional: default polling interval (seconds) for all bunny-b instances
datasets:
- name: conditions
config: configs/conditions.yaml
collection_id: <uuid-1>
- name: mortality
config: configs/mortality_conditions.yaml
collection_id: <uuid-2>
- name: diabetes
config: configs/uk_t2d_primary_care.yaml
collection_id: <uuid-3>
db_port: 5435 # optional: expose this DB on a host port
db_password: secret # optional: override global db_password
bunny_a_interval: 2 # optional: override global bunny_a_interval for this dataset
bunny_b_interval: 10 # optional: override global bunny_b_interval for this datasetapi_url, api_username, api_password, db_password, bunny_build, and concepts are global defaults. Per-dataset overrides: db_name, db_password, db_url, db_port, drop_db.
somop multi generate accepts --concepts FILE to override the path set in the config.
somop multi run accepts --compose-out FILE to write the compose file to a custom path.
seed: 42
out_dir: ./data/my_dataset
chunk_size: 100_000
person:
n_people: 5000
genders:
- concept_id: 8507 # male
p: 0.5
- concept_id: 8532 # female
p: 0.5
age_dist: normal # normal | lognormal | uniform
age_param1: 40.0 # mean
age_param2: 18.0 # sd
min_age: 0.0
max_age: 110.0
condition:
items:
- concept_id: 198185
p: 0.60
drug_exposure:
items:
- concept_id: 1503297
p: 0.30
measurement:
items:
- concept_id: 3955314
unit_concept_id: 12310002
p: 0.70
dist: lognormal # adds value_as_number; normal | lognormal | uniform
param1: 7.0 # mu
param2: 0.7 # sigma
observation:
items:
- concept_id: 4275495
p: 0.40
procedure:
items:
- concept_id: 4047494
p: 0.25
specimen:
items:
- concept_id: 4001225
p: 0.20
death:
p: 0.05
causes:
- concept_id: 4306655
p: 1.0
death_type_concept_id: 32519 # EHR
location:
# Either list locations inline…
items:
- location_id: 1
city: London
country_source_value: GBR
latitude: 51.5074
longitude: -0.1278
# …or point at a pre-built LOCATION table (used only when `items` is empty):
# prebuilt_file: ./LOCATION.csv # e.g. a UK table from the cohort-discovery-service-docs repo
interactions:
after_drug_exposure:
condition: 1.2 # multiply condition p by 1.2 for persons with a drug exposure
measurement: 1.5
after_condition:
observation: 1.3The OMOP LOCATION table gives each person a geographic point (latitude/longitude).
BUNNY uses those coordinates to answer distance queries — "people within N km of
here" — so a dataset needs a populated LOCATION table for location-based cohort
discovery to work.
There are two ways to supply one, via the location block in a config:
1. Inline — list the locations you want directly:
location:
enabled: true
items:
- location_id: 1
city: London
latitude: 51.5074
longitude: -0.12782. Pre-built table — point at a ready-made TSV LOCATION file:
location:
enabled: true
prebuilt_file: ./LOCATION.csv # a LOCATION.csv you downloaded (see below)items takes precedence: prebuilt_file is only read when items is empty. In both
cases somop generate assigns each synthetic person a random location_id from the
table.
You don't have to build a LOCATION table. Pre-built UK tables (whole-UK and per nation)
using real population-weighted centroids for each small-area geography are published
in the Cohort Discovery Service docs repo.
Download one and point prebuilt_file at it.
Because each row's location_source_value matches the code that
uk-postcode-mapper emits, the same tables
also let custodians attach location to real OMOP data by blurring a postcode to its
LSOA/Data Zone and joining on the code — no postcode is ever stored. See the
Location & Geo-radius Filtering
page for the privacy rationale.