Repository navigation
Multi-sample runs: UPDATE_FROM_OME duplicates markers when samples differ in exposure time #165
Description
Description of the bug
Multi-sample runs fail because the shared markers.csv is inflated with duplicate marker rows. The root cause is that UPDATE_FROM_OME dedupes markers on a map that still includes per-sample exposure metadata, so any channel whose exposure time differs across samples produces one row per distinct exposure. Single-sample runs are unaffected, which is why existing test data never caught it.
MCQUANT aborts with:
Exception: The number of channels in markers.csv doesn't match the image
The image is fine — instrumenting n_channels() in-container returns the correct count, and that count matches the rows of the input markers.csv. But the UPDATE_FROM_OME generated markers.csv has 64 rows instead of 44: the 44 correct unique marker_names, ragged-duplicated (DNA8 ×4, DNA3 ×3, GFAP ×3, several ×2).
The samples were imaged with different per-channel exposure times.
modules/local/omevalidationreads<Plane ExposureTime>into each marker's metadata (exposure_time,exposure_time_unit).subworkflows/local/update_from_ome/main.nfbuilds one shared markersheet across all samples and dedupes it with.unique()on maps that still carryexposure_time.
So any (cycle, channel) whose exposure differs across samples is never collapsed, yielding one row per distinct exposure.
In the triggering run, of 44 (cycle, channel) pairs, 16 have more than one distinct exposure time across the 4 samples. Those extras sum to 20, giving 44 + 20 = 64. Per-marker multiplicity matches exactly (e.g. cycle 8 / ch 29 has 4 distinct exposures → DNA8 ×4).
The guard meant to catch this ("Inconsistent marker exposure data across samples") never fires: it .maps over the single toSortedList emission as if it were one marker (entry.cycle_number on a List), so the condition is never evaluated.
Scope / other triggers
The deeper flaw is that the shared markersheet merges per-sample OME metadata and then dedupes on the whole map, so any per-sample-varying field inflates it. Today the only such fields are exposure_time / exposure_time_unit (everything else comes from the single shared markers.csv), so exposure is the practical trigger — including the sub-case where some samples have a null exposure and others don't.
A related path produces the same symptom: if samples disagree on a cycle's channel_count, CHANNEL_DELTA_INDEX/agg gets duplicate cycle keys and combine(agg, by:0) fans the markers out. The per-sample count check in fix 3 below covers all of these at once.
Suggested fixes:
Generate the markersheet per sample. Run UPDATE_FROM_OME (and the markers.csv construction) per meta.id, so each MCQUANT receives a sheet built from and validated against its own image. This treats differing per-sample acquisition metadata (e.g. exposure times) as legitimate and removes the cross-sample .unique() that causes the inflation, covering the exposure, channel_count, and any-future-field triggers in one change.
- Key the markers.csv collectFile/combine in workflows/mcmicro.nf:178 by meta.id (today it broadcasts one shared sheet to every sample).
- The cross-sample consistency guard (update_from_ome L90-99) is then unnecessary; if enforcing identical panels across a cohort is still wanted, make it an explicit opt-in check (and fix it — it currently no-ops over the toSortedList emission).
- Defense-in-depth: a PRELUDE assertion that each sample's marker count equals its image SizeC, failing with a clear message.
- Independently, make MCQUANT's error report both counts + the marker file path.
Command used and terminal output
Relevant files
No response
System information
- nf-core/mcmicro
dev@54a9c89 - Nextflow 25.10.6
Activity
AlexBarbera commentedon Jul 24, 2026
Hi @adamjtaylor, thanks for the insight.
Could you share the markersheet/samplesheet you used for this bug so we could run it on our end? Thanks
AlexBarbera commentedon Jul 24, 2026
Upon further review it does look like your fork is very behind the
devbranch and many changes has been made since.The underlying issue may well be valid, but it's difficult to evaluate the proposed changes in their current state.
Could you please also update your branch/fork to the current HEAD and rebase the changes on top of the latest code.