A Reference pipeline to annotate euKaryotes at high resolution
The Hiller Lab at the Senckenberg Research Institute
usage .
pipeline .
us
Note
Requirements: Nextflow ≥ 25.04.6, Docker or Apptainer, Java.
git clone https://github.com/hillerlab/ark.git
cd arkEdit params.json (set relevant options/filters), then:
# Docker
nextflow run main.nf -params-file params.json -profile docker
# Apptainer / Singularity
nextflow run main.nf -params-file params.json -profile apptainerSmoke test:
nextflow run main.nf -profile test,apptainerNote
You can also specify these options directly in params.json.
To restart from LIMA outputs, point the input at 02_LIMA and provide the same
primer FASTA used for demultiplexing:
nextflow run src/main.nf -params-file src/params.json -profile docker \
--entrypoint refine --global_input_dir /path/to/02_LIMA \
--global_primers /path/to/primers.fastaThe refine checkpoint reads every *.bam and its matching *.bam.pbi, creating
missing indexes, and skips CCS, Skera, and LIMA. Each BAM is refined independently,
preserving its primer-pair group. BAM names must end in
.<5p_primer>--<3p_primer>.bam.
To restart from refined reads, point the input at 03_ISOSEQ_REFINE; no primer
FASTA is needed:
nextflow run src/main.nf -params-file src/params.json -profile docker \
--entrypoint cluster --global_input_dir /path/to/03_ISOSEQ_REFINEThe cluster checkpoint reads every *.bam, creating missing indexes, and skips
CCS, Skera, LIMA, and refine. BAM names must end in _flnc.bam; the rest of the
name is kept as the sample ID, so a full run's
movie.IsoSeqX_bc01_5p--IsoSeqX_3p_flnc.bam stays
movie.IsoSeqX_bc01_5p--IsoSeqX_3p.
cluster_mode controls clustering of those refined reads: per_sample
keeps each input primer-pair group separate; multi_sample clusters all groups
together under global_prefix; both produces both sets using the same refined
reads. Sample IDs retain the full primer pair, for example
movie.IsoSeqX_bc01_5p--IsoSeqX_3p or movie.NEB_5p--NEB_Clontech_3p.
Their tissue names depend on your experimental barcode assignments; the pipeline
does not infer them. Each input BAM/primer pair is treated as a separate sample,
including when files come from different sequencing runs.
cluster_engine decides whether refined reads are clustered before alignment: none
(default) aligns them directly, as PacBio's current Kinnex workflow does, and
transcript models are built after alignment; isoseq (legacy) runs
isoseq cluster2 first and needs tagged PacBio BAMs. The cluster entrypoint
reads *_flnc.bam only.
For flnc, put FASTA/FASTQ files (.fasta/.fastq, optionally .gz) in
global_input_dir. Public SRA/ENA runs come in every processing state, so
iso-fastx inspect classifies each file from its first reads and routes it:
| State | Looks like | What happens |
|---|---|---|
ccs |
primers at the read ends, both orientations | lima --isoseq orients and clips (kit detected, or global_primers) |
mixed |
primer-free, some reads start with polyT | iso-fastx orient flips the reversed reads |
fl / flnc |
primer-free, oriented, with or without polyA | used as is |
clustered |
transcript/N, PB.x.y names |
used as is; every distinct chain is kept |
subreads, ambiguous |
raw subreads, partial primers, long arrays | the run stops and says why |
--flnc_input_state forces one state for all files (ccs then needs
--global_primers). cluster_mode applies as before: per_sample aligns each
file separately; multi_sample concatenates all files per hq/singleton class
into one pooled sample named global_prefix; both produces both sets. Do not
name an input sample global_prefix: its reads would merge with the pool.
After polyA segmentation and fusion detection, reconstruct_engine chain
(default) collapses the reads of each sample and chromosome into transcript
models by intron chain (collapse chain). Only models reach ORF prediction;
each keeps a real read as its representative and carries its support as
#CN<n>. reconstruct_preset sets how much evidence a novel model needs:
sensitive only removes redundancy (every distinct chain is kept), balanced
(default) needs 2 molecules per novel chain, strict 3. Every read ends up in
a model or in 07B_RECONSTRUCT/chain/<sample>/*.excluded.bed with its reason.
reconstruct_engine none restores per-read ORF calling.
A helper sh script is provided to run the pipeline on a SLURM cluster. See details below.
Click to expand
Edit the path variables at the top of assets/hpc/ark.sh (cache dir, container image, manifest path), then submit:
sbatch --array=1-<N> ark.shEach array task spawns one Nextflow head job that submits all compute as child SLURM jobs.
PREDICT_ORFS run as SLURM job arrays. Partition routing, array sizes, and resource tiers are documented inline in nextflow.config — edit there to match your cluster.
results/
├── 00_?/ *bed
└── pipeline_info/ timeline, trace, DAG
| File | What |
|---|---|
params.json |
Genome paths, alignment settings, checkpoints — per run |
nextflow.config |
Compute resources, profiles, container, SLURM — rarely |