Results on CuriousCoding

SimdSketch: a fast bucket sketch

Sun, 09 Mar 2025 00:00:00 +0100

Table of Contents

1 Jaccard similarity
2 Hash schemes
- 2.1 MinHash
- 2.2 $s$-mins sketch
- 2.3 Bottom-$s$ sketch
- 2.4 FracMinHash
- 2.5 Bucket sketch
- 2.6 Mod-bucket hash (new?)
- 2.7 Variants
3 Compressing sketches
- 3.1 $b$-bit hashing
  - 3.1.1 Accounting for collisions
- 3.2 HyperMinHash
4 Densification strategies
5 SimdSketch
6 Evaluation
- 6.1 Setup
  - 6.1.1 Tools
  - 6.1.2 Inputs
  - 6.1.3 Parameters
  - 6.1.4 Metrics
- 6.2 Raw results
- 6.3 Correlation
- 6.4 Comparison speed
- 6.5 Low-similarity data
7 Discussion
8 TODO / Future work

\[ \newcommand{\sketch}{\mathsf{sketch}} \]

Static search trees: 40x faster than binary search

Wed, 18 Dec 2024 00:00:00 +0100

Table of Contents

1 Introduction
- 1.1 Problem statement
- 1.2 Motivation
- 1.3 Recommended reading
- 1.4 Binary search and Eytzinger layout
- 1.5 Hugepages
- 1.6 A note on benchmarking
- 1.7 Cache lines
- 1.8 S-trees and B-trees
2 Optimizing find
- 2.1 Linear
- 2.2 Auto-vectorization
- 2.3 Trailing zeros
- 2.4 Popcount
- 2.5 Manual SIMD
3 Optimizing the search
- 3.1 Batching
- 3.2 Prefetching
- 3.3 Pointer arithmetic
  - 3.3.1 Up-front splat
  - 3.3.2 Byte-based pointers
  - 3.3.3 The final version
- 3.4 Skip prefetch
- 3.5 Interleave
4 Optimizing the tree layout
- 4.1 Left-tree
- 4.2 Memory layouts
- 4.3 Node size $B=15$
  - 4.3.1 Data structure size
- 4.4 Summary
5 Prefix partitioning
- 5.1 Full layout
- 5.2 Compact subtrees
- 5.3 The best of both: compact first level
- 5.4 Overlapping trees
- 5.5 Human data
- 5.6 Prefix map
- 5.7 Summary
6 Multi-threaded comparison
7 Conclusion
- 7.1 Future work
  - 7.1.1 Branchy search
  - 7.1.2 Interpolation search
  - 7.1.3 Packing data smaller
  - 7.1.4 Returning indices in original data
  - 7.1.5 Range queries
  - 7.1.6 Sorting queries
  - 7.1.7 Suffix array searching

In this post, we will implement a static search tree (S+ tree) for high-throughput searching of sorted data, as introduced on Algorithmica. We’ll mostly take the code presented there as a starting point, and optimize it to its limits. For a large part, I’m simply taking the ‘future work’ ideas of that post and implementing them. And then there will be a bunch of looking at assembly code to shave off all the instructions we can. Lastly, there will be one big addition to optimize throughput: batching.

A lemma on suffix array searching

Sat, 05 Oct 2024 00:00:00 +0200

Table of Contents

1 Suffix arrays
2 Searching methods
3 Analysing the faster search

We’ll prove that using the “faster” binary search algorithm (see 2.2) that tracks the LCP with the left and right boundary of the remaining search interval has amortized runtime

\[ O\Big(\lg_2(n) + |P| + |P| \cdot \lg_2(Occ(P))\Big), \] when $P$ is a randomly sampled fixed-length pattern from the text and $Occ(P)$ counts the number of occurrences of $P$ in the text.

Perfect NtHash for Robust Minimizers

Sun, 31 Dec 2023 00:00:00 +0100

Table of Contents

NtHash
Minimizers
- Robust minimizers
Is NtHash injective on kmers?
- Searching for a collision
- Proving perfection
Alternatives
SmHasher results

NtHash Link to heading

NtHash (Mohamadi et al. 2016) is a rolling hash suitable for hashing any kind of text, but made for DNA originally. For a string of length $k$ it is a $64$ bit value computed as:

Tensor embedding preserves Hamming distance

Fri, 14 Oct 2022 00:00:00 +0200

Table of Contents

Definitions
Proof of Lemma 1
TODO Proof of Lemma 2

This is a proof that Tensor Embedding (Joudaki, Rätsch, and Kahles 2020) with $ℓ^2$-norm preserves the Hamming distance.

This is in collaboration with Amir Joudaki.

\begin{equation*} \newcommand{\I}{\mathcal I} \newcommand{\EE}{\mathbb E} \newcommand{\var}{\operatorname{Var}} \end{equation*}

Definitions Link to heading

Notation

The alphabet is $\Sigma$, of size $|\Sigma| = \sigma$.
The set of indices is $\I := \{(i_1, \dots, i_t) \in [n]^t: i_1 < \dots < i_t\}$.
Given a string $a_1\dots a_n = a\in \Sigma^n$, we define the $I$-index as $a_I = (a_{i_1}, \dots, a_{i_t})$.
We write $[ X ]$ for the indicator variable of event $X$, which is $1$ when $X$ holds and $0$ otherwise.

Definition 1: Tensor embedding

Given $a\in \Sigma^n$, the tensor embedding $T_a$ is the $\sigma^t$ tensor given by $T_a[s] = \sum_{I\in \I} [A_I = s]$ for each $s\in \Sigma^t$.

The normalized tensor embedding distance $d_{te}$ between two sequences $a$ and $b$ is defined as

28000x speedup with Numba.CUDA

Mon, 24 May 2021 00:00:00 +0200

Backlinks: r/CUDA, Numba discourse