Repository navigation
Hi-C: skip mate rescue when the insert-size distribution is uninformative #11
Description
For Hi-C libraries the mate is effectively anywhere in the genome, so the insert-size distribution carries no information about where the mate should be. Mate rescue still runs, and it spends a large amount of time on rescues that were never going to succeed.
The size of the effect turned up while ablating the window-narrowing gate at pe.c:352:
| gate level | real WGS | real Hi-C |
|---|---|---|
| stock | — | — |
drop n_good == 1 |
-1.47% | -6.80% |
relax len>>1 -> len>>2 |
-0.42% | -1.94% |
| both | -3.99% | -16.24% |
keep only max_ug >= 10 |
-5.54% | -44.92% |
Hi-C mapping phase went 164.9 s -> 90.8 s at the most aggressive level.
That ablation is not the change being proposed here. Every level of it moves confident placements on real data (at the most aggressive level, 1 in 5,390 reads at MAPQ >= 10 on Hi-C), which is not worth paying for the speed, and loosening a window gate globally penalises every library type to fix one.
The interesting part is what the -44.92% implies: nearly all of that work was futile. If the pair orientation and insert-size distribution observed for a library are uninformative -- as they are for Hi-C, where the two ends are ligation partners rather than fragment ends -- then attempting rescue at all is close to 7391 pure cost. Detecting that regime and skipping rescue in it costs WGS nothing, since a normal library's insert-size distribution is tight and informative, and the check would be a property of the inferred distribution rather than a user-supplied flag.
Sketch, not a design:
- minibwa already infers per-orientation insert-size statistics for pairing.
- Add a test for "uninformative": e.g. the inferred distribution is so broad relative to the reference span, or so evenly split across orientations, that a rescue window would cover an unusable fraction of the genome.
- When it holds, skip
mb_mateswfor that library rather than narrowing its window.
Open questions before this is worth implementing:
- What is the right statistic, and does it separate Hi-C from a genuinely wide but still informative WGS library (long-insert mate-pair, for instance)?
- Should it be automatic, or opt-in via a flag, given that a wrong automatic decision silently changes output for a whole run?
- What does skipping rescue cost Hi-C in recall? The speed number says the work is mostly futile, but "mostly" needs to be quantified against a truth set before this ships.
Full ablation data and the reasoning for rejecting the window-gate change is in reports/2026-08-01-mate-rescue-window-gate.md; the apparatus is on branch perf/rescue-window-gate.
Activity
nh13 commentedon Aug 20, 2026
Closing: minibwa already ships this, and the number that motivated it was a benchmark misconfiguration.
--hicalready skips all mate rescue.map-main.c:506-507setsMB_F_PRIMARY5 | MB_F_NO_PAIRING;map-algo.c:755gates the entire paired-end block on!(opt->flag & (MB_F_NO_PAIRING|MB_F_NO_ALN)); that block holds the only call tomb_pair, which is the only caller ofmb_matesw. So-Palone suffices -- rescue lives inside the pairing block, which is why there is no separate skip-rescue flag.Measured on 100k HG002 Hi-C pairs against GRCh38:
--hicProper-paired going to exactly zero confirms the pairing block never runs.
The -44.92% came from the benchmark, not the gate.
minibwa-bench'shic-1Mdeclaredpreset = "sr"and no extra flags, so the Hi-C arm ran without--hic-- a configuration nobody should use for the assay. A second defect in the same path made it worse: therunverb had no--dataset/--presetargument at all, so every cell used the defaultsrpreset regardless of what the config declared. Both fixed (minibwa-bench f8a7abd + a2d4758);--datasetis now required for mapping runs.The per-read futility gate is also already there.
pe.c:358runs the rescue SW only whenmax_ug >= 10 || max_ug >= n_kmer * 0.33-- the same predicate, by De Morgan and with identical constants (10, 0.33), thatbwa-mem3later shipped as--rescue-skip. The ablation ladder only ever varied the narrowing gate atpe.c:352, one line above it.Nothing to implement and nothing to file upstream. Full write-up in
reports/2026-08-08-hic-mate-rescue-futility.md.