Highlight on CuriousCoding

CI for Rust testing and releasing

Sun, 25 Jan 2026 00:00:00 +0100

Table of Contents

1 Testing
2 Releasing libraries
- 2.1 Changelog
- 2.2 cargo release
- 2.3 crates.io
3 Releasing binaries
- 3.1 Crates.io
- 3.2 The pain of AVX2
  - 3.2.1 ensure_simd
- 3.3 Profile selection
- 3.4 GitHub
- 3.5 cargo binstall
- 3.6 PyPI
- 3.7 Bioconda
4 Conclusion

This post will collect template files for setting up GitHub CI for testing and releasing Rust libraries and binaries to:

crates.io (libraries, binaries)
GitHub releases
cargo binstall
PyPI
Bioconda

We also have some workarounds for dealing with AVX2 SIMD instructions.

Trying to understand DDR memory

Tue, 20 Jan 2026 00:00:00 +0100

Table of Contents

1 Questions
2 A load of articles/blogs/pages to read
- 2.1 Wikipedia articles
- 2.2 More posts
- 2.3 Notes
- 2.4 My own RAM
- 2.5 Continued notes
- 2.6 Address mapping notation
- 2.7 Intel spec
- 2.8 Rank interleaving
- 2.9 Nontemporal reads/writes
3 reMap: using Performance counters
4 Sudoku
- 4.1 Step 1: DRAM addressing functions
- 4.2 Step 2: row/column bits
- 4.3 Step 3: validation
- 4.4 Step 4: which function is what?
- 4.5 Refreshes
- 4.6 Consecutive Accesses
5 Sudoku, now with only 1 DIMM
- 5.1 setup
- 5.2 1. reverse functions
- 5.3 2. identify bits
- 5.4 3. validate mapping
- 5.5 4. decompose functions
6 Final results
7 decode-dimms
- 7.1 Bank groups
- 7.2 Refresh
- 7.3 Random access throughput
8 CPU benchmarks
- 8.1 cpu-benchmarks
  - 8.1.1 random access throughput 1 DIMM
  - 8.1.2 random access throughput 2 DIMM
- 8.2 memory-read-experiment
  - 8.2.1 strided reading 1 DIMM
  - 8.2.2 strided reading 2 DIMM
9 tinymembench
10 Remaining questions

These are chronological (and thus, only lightly organized) notes on my attempt to understand how DDR4 and DDR5 RAM memory work.

Asymptotic elevators

Mon, 22 Dec 2025 00:00:00 +0100

I was listening to an episode of the well there’s your problem podcast about pencil towers (youtube), and it had a section on how elevators are a problem because they require a lot of space. So here’s a mathematical version of that.

Problem statement Link to heading

Given are $n$ people that need to go to floors $1, 2, \dots, n$.
Elevators have constant acceleration, and must be standing still to enter/exit.¹
Possible elevator configurations:
1. single elevator over entire height
2. partition the height in disjoint intervals, and then one elevator per interval
3. double-deck: a single elevator that is $h$ floors high
Not allowed: two free-moving elevators above each other that make sure to never bump into each other.
Elevators have infinite capacity.
There are $k$ elevator shafts.
Question: How much total travel time do you need to get everyone home, if everybody arrives at the same time.
Harder(?): What if the people arrive in a random permutation (1 per time step), and their clock starts ticking as soon as they arrive?

Observations Link to heading

Going $n$ floors up takes at least $O(\sqrt n)$ time.
Total travel time is at least $\sum_{i=0}^n O(\sqrt i) = O(n \sqrt n)$
$1$ elevator carrying everyone going 1 step at a time: $O(n^2)$ total time
$2$-elevator sqrt-decomposition: one elevator stops every $\sqrt n$ floors, and then a (set of) second elevators for the final up to $\sqrt n$ floors.
- first elevator: $n$ people times $n/(\sqrt n)/2$ ‘big steps’ on average times $\sqrt{\sqrt n}$ time per big step is $O(n^{7/4})$
- second elevator: $n$ people times $(\sqrt n)$ ‘small steps’ on average times $1$ per small step is $O(n^{3.2})$
- so overall the big steps dominate
$2$-elevator, one that stops every $B$ floors and then $B$ that stop every one floor:
- the first one: $n \cdot (n/B) \cdot \sqrt B = n^2/\sqrt B$
- the second one: $n \cdot B = nB$.
- solve for equality: $B = n/\sqrt B$ => $B = n^{2/3}$
- so $n^{1+2/3}$ solution
$\lg n$ elevator binary tree:
- $2^k \leq n \leq 2^{k+1}$
- Take first elevator to $2^k$ if needed: time $\sqrt{2^k} = O(\sqrt n)$
- There take second elevator that goes up $2^{k-1}$ floors if needed.
- Then $2^{k-2}$ levels up
- and so on until the last floor.
- Total time per person averages $\sqrt{2^k}/2 + \sqrt{2^{k-1}}/2 + \dots + \sqrt{2}/2 + \sqrt{1}/2 = O(\sqrt{2^k}) = O(\sqrt n)$, so up-to-a-constant optimal total travel time $O(n \sqrt n)$.

Open questions:

Overview of static data structures

Wed, 17 Dec 2025 00:00:00 +0100

Table of Contents

1 Classification of static data structures
2 Space lower bounds and practical approaches
- 2.1 Rank
- 2.2 Rank + Select
- 2.3 Minimal perfect hash function (MPHF)
- 2.4 TODO Monotone MPHF
- 2.5 TODO Order-preserving MPHF
- 2.6 Static retrieval: Static function with static values
- 2.7 Updatable retrieval: Static function with mutable values
- 2.8 Static set (membership)
- 2.9 Static ordered set
- 2.10 Static dictionary: static keys and values
- 2.11 Updatable dictionary with mutable values
- 2.12 TODO Dynamic dictionary with mutable keys and values
- 2.13 TODO Static filter
- 2.14 TODO Ordered static/updatable/dynamic dictionary?
3 Summary table

\[ \newcommand{\K}{\mathbb K} \newcommand{\V}{\mathbb V} \newcommand{\c}[1]{\mathbf{\mathsf{#1}}} \]

Three log scientist

Tue, 12 Aug 2025 00:00:00 +0200

A rating system for theoretical computer scientists. The more logarithms there are (i.e. the more “$\log$” before your variables), the higher your reputation will be. No-log theoretical computer scientists are virtually non-existent, as virtually all non-trivial algorithms require use of logarithms. Most are one-log scientists. In the old times (well, I’m young, so these look like old times to me at least), one would occasionally find a piece of code done by a three-log scientist and shiver with awe.

Beyond Global Alignment

Mon, 24 Mar 2025 00:00:00 +0100

Table of Contents

1 Variants of semi-global alignment
2 Fast text searching
- 2.1 Skip-cost for overlap alignments
- 2.2 Results
3 Mapping using A*Map
- 3.1 Seeding
- 3.2 Chaining
- 3.3 Aligning
- 3.4 A*Map
- 3.5 Results

This is Chapter 5 of my thesis (Groot Koerkamp 2025).

Summary

So far, we have considered only algorithms for global alignment. In this chapter, we consider semi-global alignment and its variants instead, where a pattern (query) is searched in a longer string (reference). There are many flavours of semi-global alignment, depending on the (relative) sizes of the inputs. We list these variants, and introduce some common approaches to solve this problem.

SimdSketch: a fast bucket sketch

Sun, 09 Mar 2025 00:00:00 +0100

Table of Contents

1 Jaccard similarity
2 Hash schemes
- 2.1 MinHash
- 2.2 $s$-mins sketch
- 2.3 Bottom-$s$ sketch
- 2.4 FracMinHash
- 2.5 Bucket sketch
- 2.6 Mod-bucket hash (new?)
- 2.7 Variants
3 Compressing sketches
- 3.1 $b$-bit hashing
  - 3.1.1 Accounting for collisions
- 3.2 HyperMinHash
4 Densification strategies
5 SimdSketch
6 Evaluation
- 6.1 Setup
  - 6.1.1 Tools
  - 6.1.2 Inputs
  - 6.1.3 Parameters
  - 6.1.4 Metrics
- 6.2 Raw results
- 6.3 Correlation
- 6.4 Comparison speed
- 6.5 Low-similarity data
7 Discussion
8 TODO / Future work

\[ \newcommand{\sketch}{\mathsf{sketch}} \]

Thesis: Optimal Throughput Bioinformatics

Sun, 23 Feb 2025 00:00:00 +0100

Table of Contents

Abstract
1 Introduction
2 Discussion
- 2.1 Pairwise Alignment
- 2.2 Low Density Minimizers
- 2.3 High Throughput Bioinformatics
- 2.4 Propositions

This post contains the abstract, introduction, and conclusion of my thesis (Groot Koerkamp 2025a). Individual chapters are based either on blog posts or papers and linked from the introduction.

A History of Pairwise Alignment

Sat, 22 Feb 2025 00:00:00 +0100

Table of Contents

1 A Brief History
- 1.1 A*PA
- 1.2 A*PA2
- 1.3 Overview
2 Problem Statement
3 Alignment types
4 Cost Models
- 4.1 Minimizing Cost versus Maximizing Score
5 The Classic DP Algorithms
6 Linear Memory using Divide and Conquer
7 Dijkstra’s Algorithm and A*
8 Computational Volumes and Band Doubling
9 Diagonal Transition
10 Parallelism
11 LCS and Contours
12 Some Tools
13 Subquadratic Methods and Lower Bounds
14 Summary

This is Chapter 2 of my thesis (Groot Koerkamp 2025), to introduce the first part on Pairwise Alignment.

Low Density Minimizers

Fri, 21 Feb 2025 00:00:00 +0100

Table of Contents

1 Theory of Sampling Schemes
- 1.1 Introduction
- 1.2 Overview
- 1.3 Theory of sampling schemes
- 1.4 Notation
- 1.5 Types of sampling schemes
- 1.6 Computing the density
- 1.7 The density of random minimizers
- 1.8 Universal hitting sets
- 1.9 Asymptotic results
- 1.10 Variants
2 Lower Bounds on Sampling Scheme Density
3 Practical Sampling Schemes
- 3.1 Variants of lexicographic minimizers
  - Evaluation
- 3.2 UHS-inspired schemes
- 3.3 Syncmer-based schemes
- 3.4 Open-closed minimizer
  - Evaluation
- 3.5 Mod-minimizer
  - Theoretical density
  - Evaluation
- 3.6 Discussion
4 Towards Optimal Selection Schemes
- 4.1 Bidirectional anchors
- 4.2 Sus-anchors
  - Evaluation
- 4.3 Discussion

This is Part 2 of my thesis (Groot Koerkamp 2025), containing chapters 6 to 9 on Low Density Minimizers.

High Throughput Bioinformatics

Thu, 20 Feb 2025 00:00:00 +0100

Table of Contents

1 Introduction
- 1.1 Overview
2 Optimizing Compute Bound Code: Random Minimizers
- 2.1 Avoiding Branch Misses
- 2.2 SIMD: Processing In Parallel
- 2.3 Instruction Level Parallelism
- 2.4 Input Format
3 Optimizing Memory Bound Code: Minimal Perfect Hashing
- 3.1 Using Less Memory
- 3.2 Reducing Memory Accesses
- 3.3 Interleaving Memory Accesses
- 3.4 Batching, Streaming, and Prefetching

This is Chapter 10 of my thesis (Groot Koerkamp 2025a), to introduce the last part on High Throughput Bioinformatics.

PtrHash: Minimal Perfect Hashing at RAM Throughput

Mon, 03 Feb 2025 00:00:00 +0100

Table of Contents

Abstract
1 Introduction
2 Related work
3 PtrHash
- 3.1 Overview
- 3.2 Construction
- 3.3 Bucket Assignment Functions
- 3.4 Remapping using CacheLineEF
4 Results
- 4.1 Construction
  - 4.1.1 Bucket Functions
  - 4.1.2 Tuning Parameters for Construction
  - 4.1.3 Remap
- 4.2 Comparison to Other Methods
5 Conclusions and Future Work
Appendix A: Query Throughput
Appendix B: Sharding
- Evaluation

This is the HTML version of my SEA 2025 paper on PtrHash (DOI, PDF). The original development-log can be found here.

SimdMinimizers: Computing random minimizers, fast

Fri, 12 Jul 2024 00:00:00 +0200

Table of Contents

1 Introduction
- 1.1 Results
2 Random minimizers
3 Algorithms
4 Analysing what we have so far
5 Rolling our own hash
6 SIMD sliding window
- 6.1 Results
  - Human genome results
7 Extending into something useful
- 7.1 Collecting minimizer positions
- 7.2 Deduplicating the minimizer positions
- 7.3 Super-k-mers
- 7.4 Canonical k-mers
- 7.5 AntiLex hash
8 Conclusion
- 8.1 Future work

A 90 min recording of a talk I gave on this post can be found here.

A*PA2: Up to 19x faster exact global alignment

Sat, 23 Mar 2024 00:00:00 +0100

Table of Contents

Abstract
1 Introduction
- 1.1 Contributions
- 1.2 Previous work
  - 1.2.1 Needleman-Wunsch
  - 1.2.2 Graph algorithms
  - 1.2.3 Computational volumes
  - 1.2.4 Parallelism
  - 1.2.5 Tools
2 Preliminaries
3 Methods
- 3.1 Band-doubling
- 3.2 Blocks
- 3.3 Memory
- 3.4 SIMD
- 3.5 SIMD-friendly sequence profile
- 3.6 Traceback
- 3.7 A*
  - 3.7.1 Bulk-contours update
  - 3.7.2 Pre-pruning
- 3.8 Determining the rows to compute
  - 3.8.1 Sparse heuristic invocation
- 3.9 Incremental doubling
4 Results
- 4.1 Setup
- 4.2 Comparison with other aligners
- 4.3 Effects of methods
5 Discussion
Acknowledgements
Conflict of interest
6 Appendix
- 6.1 Bitpacking
- 6.2 Comparison with other aligners
- 6.3 Effects of methods

\begin{equation*} \newcommand{\g}{g^*} \newcommand{\h}{h^*} \newcommand{\f}{f^*} \newcommand{\cgap}{c_{\textrm{gap}}} \newcommand{\xor}{\ \mathrm{xor}\ } \newcommand{\and}{\ \mathrm{and}\ } \newcommand{\st}[2]{\langle #1, #2\rangle} \newcommand{\matches}{\mathcal M} \end{equation*}

String algorithm visualizations

Tue, 08 Nov 2022 00:00:00 +0100

Select the algorithm to visualize
Click the buttons, or click the canvas and use the indicated keys

Suffix-array construction is explained here and BWT is explained here.

Source code is on GitHub.

Algorithm
String
Query

Delay (s)

28000x speedup with Numba.CUDA

Mon, 24 May 2021 00:00:00 +0200

Xrefs: r/CUDA, Numba discourse