ORCA recipes for Biomedical Data Fabric (BDF) data processing on Nextflow Tower.
| Modality | Recipe | Pipeline | Status |
|---|---|---|---|
| WES (whole exome sequencing) | sarek_wes_somatic_workflow.py | nf-core/sarek v3.1.2 | ✓ |
| WGS (whole genome sequencing) | sarek_wgs_workflow.py | nf-core/sarek v3.1.2 | ✓ |
| Bulk RNA-seq | rnaseq_workflow.py | nf-core/rnaseq v3.11.2 | ✓ |
| Single-cell RNA-seq | scrnaseq_workflow.py | nf-core/scrnaseq v4.1.0 | ✓ |
| Spatial transcriptomics | External | nf-core/spatialvi dev | ✓ |
Each recipe follows a standardized four-step workflow orchestrated by ORCA (Nextflow Tower):
- fetch_samplesheet: Download samplesheet from Synapse to S3
- synstage: Download input files (FASTQ, BAM, etc.) from Synapse to S3
- pipeline: Run nf-core pipeline on staged data
- synindex: Upload results back to Synapse
Install dependencies first (in your environment):
pip install py-orca synapseclientSet environment variables:
export TOWER_ACCESS_TOKEN="<your-tower-token>" # Get from https://tower.sagebionetworks.org/user/api-tokens
export TOWER_WORKSPACE="sage-bionetworks/ntap-add5-project"
export TOWER_API_ENDPOINT="https://tower.sagebionetworks.org/api"TOWER_ACCESS_TOKEN: Personal API token from Seqera Tower. Generate at https://tower.sagebionetworks.org/user/api-tokens (requires login)
Run a recipe (all steps by default):
python recipes/sarek_wes_somatic_workflow.py
python recipes/sarek_wgs_workflow.py
python recipes/rnaseq_workflow.py
python recipes/scrnaseq_workflow.pyRun specific steps:
python recipes/rnaseq_workflow.py fetch_samplesheet synstage
python recipes/rnaseq_workflow.py rnaseq synindex
python recipes/scrnaseq_workflow.py synstage scrnaseqIncrement run number to preserve previous outputs:
python recipes/rnaseq_workflow.py --run-number 2-
sarek_wes_somatic_workflow.py: WES (whole exome) variant calling (tumor vs. matched normal)
- Pipeline: nf-core/sarek v3.1.2
- Config: GATK.GRCh38, WES + Agilent V6 intervals, callers: strelka, mutect2, vep
- Based on JHU NF1 Biobank release-2 (JH_batch1)
- Input samplesheet: syn52236715
- Output folder: syn76340288
- Note: Standalone (does not use base_rna.py)
- Usage: See docstring in
sarek_wes_somatic_workflow.py
-
sarek_wgs_workflow.py: WGS (whole genome) variant calling (tumor vs. matched normal)
- Pipeline: nf-core/sarek v3.1.2
- Config: GATK.GRCh38, whole genome calling, callers: strelka, mutect2, vep
- Derived from sarek_wes_somatic_workflow.py
- Input samplesheet: syn77362192
- Output folder: syn77361958
- Note: Standalone (does not use base_rna.py)
- Usage: See docstring in
sarek_wgs_workflow.py
-
rnaseq_workflow.py: Bulk RNA-seq alignment and quantification
- Pipeline: nf-core/rnaseq v3.11.2
- Config: GRCh38, STAR + Salmon quantification
- Based on NF-OSI Nextflow Data Processing standard
- Input samplesheet: syn76923670
- Output folder: syn76921355
- Uses: base_rna.py (shared module)
- Usage: See docstring in
rnaseq_workflow.py
-
scrnaseq_workflow.py: Single-cell RNA-seq processing
- Pipeline: nf-core/scrnaseq v4.1.0
- Config: GRCh38 reference (10x CellRanger pre-built), CellRanger (alignment) + CellBender (ambient RNA correction)
- Input samplesheet: syn76926335
- Output folder: syn76921961
- Uses: base_rna.py (shared module)
- Usage: See docstring in
scrnaseq_workflow.py
- WES Somatic (sarek_wes_somatic_workflow.py): Self-contained with direct Tower orchestration
- WGS Somatic (sarek_wgs_workflow.py): Self-contained with direct Tower orchestration
- RNA Modalities (rnaseq, scrnaseq): Use shared base_rna.py for standardized workflow
Shared infrastructure for RNA modalities (rnaseq, scrnaseq). Manages Tower authentication, dataset configuration, samplesheet staging, workflow orchestration, and parameter management.
Science parameters are defined in config/:
rnaseq.params.json: nf-core/rnaseq settings (aligner, genome, reference)scrnaseq.params.json: nf-core/scrnaseq settings (protocol, genome, reference)
To add new datasets or modify pipeline parameters:
- Update
generate_datasets()in recipe file to specify Synapse IDs and run configurations - Modify
config/[modality].params.jsonfor science parameters - Adjust
prepare_[pipeline]_launch_info()for pipeline-specific settings