Near-optimal sampling schemes
Ragnar Groot Koerkamp
2025-02-27 Thu 00:00
Created: 2025-03-05 Wed 02:08
1. Warming up: A cute prolblem
- Given a string, choose one character.
CABAACBD
- Given a rotation, choose one character.
ACBDCABA
- Can we always choose the same character?
- Yes: e.g. the smallest rotation (bd-anchor):
CABAACBDACBDCABA
1.1. This talk: what if one character is hidden?
- Given a string (length w), choose one character.
CABAACBDX
- Given a rotation (of the hidden w+1 string), choose one character.
ACBDXCABA
- Can we always choose the same character?
- Maybe?
CABAACBDACBDXCAB🤔
1.2. The answer is no!
CABAACBDX rotations:
CABAACBD.........ABAACBDX.........BAACBDXC.........AACBDXCA.........ACBDXCAB....<— theAis not here.....CBDXCABA.........BDXCABAA.........DXCABAAC.........XCABAACB
In the w+1 rotations, we need at least 2 samples.
2. Minimizer schemes
- Minimizer scheme: Given a window of w k-mers, pick the (leftmost) smallest one
- according to some order Ok
- k=1, w=5:
CABCA
- k=2, w=5:
CABCAC......ABCACC......BCACCX......CACCXY......ACCXYZ......CCXYZX
- Density: 3/10
CABCACCXYZX
2.1. The situation, 1 year ago
2.2. The mod-minimizer
2.3. A near-tight lower bound
2.4. The current picture
2.5. Greedymini
2.6. Minimizer density lower bound
Density of minimizer scheme is ≥1/σk:
sample exactly every
AAAk-mer, and nothing else.- k=1: density at least 1/σ=1/4.
3. Sampling schemes: more general
- Any function f:Σw+k−1→{0,…,w−1}
- We fix k=1 from now: f:Σw→{0,…,w−1}
3.1. Bidirectional anchors
- Pick the start of the smallest rotation
EADCAE.......ADCAEB.......DCAEBE.......CAEBEC.......AEBECD.......EBECDC.......BECDCD
3.2. Limitations of bd-anchors
- Lexicographic is bad:
AAAABCD....AAABCDE....AABCDEF....ABCDEFG
- Comparing rotations is unstable:
-
AABACD.. .ABACDA...BACDAE
-
- Avoid last r positions.
3.3. Bd-anchor density for k=1
4. Smallest-unique-substring anchors
- Idea: instead of smallest rotation: smallest suffix.
- What about
CABA: isABAorAsmaller?- We choose
ABAsmaller for stability.
- We choose
ABis the smallest unique substring.- Stable:
-
AABACD.. .ABACDA...BACDAE
-
4.1. Sus-anchor density
4.2. ABB order
AAAAis BAD:- small strings overlap
- small strings cluster
- We want the opposite!
ABB order:
Afollowed by many non-Ais smallest:ABBBBBBBBB- no overlap
- no clustering
4.3. Sus-anchor, ABB order
4.4. Anti-lex
Anti-lexicographic order:
Asmall, followed by largest possible suffix:AZZZZZis minimal- no overlap
- no clustering
4.5. Sus-anchor, anti-lex order
5. Understanding the lower bound
- To reach lower bound: exactly 2 samples in every w+1 cycle.
5.1. Failure mode
0010101 cycle:
-
001010...... .010101.......101010.......010100.......101001.......010010.......100101- The
01010sus is not overlap free- Just like how
AAAis not overlap free
- Just like how
Goal: find two non-overlapping substrings.
5.2. Can we design a perfectly optimal scheme?
Goal:
For every w+1 window, find two non-overlapping small strings.
- Instead of
011...11, search00...0011...11- Also non-overlapping, and more signal.
- Still not optimal.
- Tried many things. No general solution found yet.
6. Thanks to my co-authors!
- Giulio Ermanno Pibiri
- Bryce Kille
- Daniel Liu
- Igor Martayan
Near-optimal sampling schemes Ragnar Groot Koerkamp 2025-02-27 Thu 00:00 Created: 2025-03-05 Wed 02:08