Skip to content

Optimize uint64_reverse_bits, rand_int, and count_lt/gt#5311

Open
mulugetam wants to merge 1 commit into
facebookresearch:mainfrom
mulugetam:utils_optim
Open

Optimize uint64_reverse_bits, rand_int, and count_lt/gt#5311
mulugetam wants to merge 1 commit into
facebookresearch:mainfrom
mulugetam:utils_optim

Conversation

@mulugetam

Copy link
Copy Markdown
Contributor

Replace three utility functions with faster algorithms. All changes are semantically equivalent (same outputs for all inputs).

Changes:

faiss/utils/byteswap.h (new)
Portable byteswap32/byteswap64 wrappers (__builtin_bswap* on GCC/Clang, _byteswap_* on MSVC), following the pattern of popcount.h.

faiss/utils/hamming.cpp
uint64_reverse_bits: replace a 64-iteration serial bit-shift loop with byteswap64 + 3 parallel nibble/crumb/bit mask-shift pairs. The loop has 64 data-dependent operations; the new version has 7 independent operations and runs in ~2 cycles.

faiss/utils/random.cpp
rand_int(int max) in RandomGenerator and SplitMix64RandomGenerator: replace rng() % max with Lemire's fast-range (uint64_t(rng()) * uint64_t(max)) >> 32. Eliminates integer division and modulo bias.

faiss/utils/utils.cpp
count_lt / count_gt: replace O(p) linear scan with O(log n) std::lower_bound on the already-sorted input arrays. lower_bound compiles to branchless cmov instructions, avoiding the branch mispredictions and O(p) scan latency that hurt the linear scan at high hit rates. In practice n equals k, which defaults to 1024 in range_search_gpu, putting every call in a regime where binary search wins at all hit rates.

benchmark: https://gist.github.com/mulugetam/524854f16b20f236e645f2cb6da24fc0

Results (measured on Sapphire Rapids):

reverse_bits:

| Function                        | Before    | After    | Speedup |
|---------------------------------|-----------|----------|---------|
| uint64_reverse_bits             | 34.2 ns   | 2.99 ns  |  11.4x  |

Lemire's fast range:

| Function                        | Before    | After    | Speedup |
|---------------------------------|-----------|----------|---------|
| rand_int(max)                   |  3.96 ns  | 2.35 ns  |   1.7x  |

count_lt:

| n       | hit=5%  | hit=50% | hit=95% |
|---------|---------|---------|---------|
| 10      |  0.19x  |  1.24x  |  1.64x  |
| 100     |  0.53x  |  4.30x  |  6.14x  |
| 1000    |  2.62x  | 28.4x   | 49.9x   |
| 10000   | 19.7x   | 188x    | 356x    |
| 100000  | 157x    | 1512x   | 3101x   |

count_gt:

| n       | hit=5%  | hit=50% | hit=95% |
|---------|---------|---------|---------|
| 10      |  0.22x  |  1.19x  |  1.50x  |
| 100     |  0.62x  |  4.28x  |  6.22x  |
| 1000    |  2.70x  | 28.2x   | 48.9x   |
| 10000   | 20.4x   | 184x    | 353x    |
| 100000  | 157x    | 1488x   | 3077x   |

faiss/utils/byteswap.h (new)
  Portable byteswap32/byteswap64 wrappers (GCC __builtin_bswap* /
  MSVC _byteswap_*), following the pattern of popcount.h.

faiss/utils/hamming.cpp
  uint64_reverse_bits: replace 64-iteration serial bit-shift loop with
  byteswap64 + 3 parallel nibble/crumb/bit mask-shift pairs. The loop
  has 64 data-dependent operations; the new version has 7 independent
  operations that run in ~2 cycles.

faiss/utils/random.cpp
  rand_int(int max) in RandomGenerator and SplitMix64RandomGenerator:
  replace mt() % max with fast-range
  (uint64_t(rng()) * uint64_t(max)) >> 32.
  Eliminates integer division and modulo bias.

faiss/utils/utils.cpp
  count_lt / count_gt: replace O(p) linear scan with O(log n)
  std::lower_bound on the already-sorted input arrays.
  lower_bound compiles to branchless cmov instructions; the linear
  scan suffers branch misprediction and full O(p) latency at high
  hit rates.

Benchmark results (measured on Sapphire Rapids):

reverse_bits:

| Function                        | Before    | After    | Speedup |
|---------------------------------|-----------|----------|---------|
| uint64_reverse_bits             | 34.2 ns   | 2.99 ns  |  11.4x  |

Lemire's fast range:

| Function                        | Before    | After    | Speedup |
|---------------------------------|-----------|----------|---------|
| rand_int(max)                   |  3.96 ns  | 2.35 ns  |   1.7x  |

count_lt:

| n       | hit=5%  | hit=50% | hit=95% |
|---------|---------|---------|---------|
| 10      |  0.19x  |  1.24x  |  1.64x  |
| 100     |  0.53x  |  4.30x  |  6.14x  |
| 1000    |  2.62x  | 28.4x   | 49.9x   |
| 10000   | 19.7x   | 188x    | 356x    |
| 100000  | 157x    | 1512x   | 3101x   |

count_gt:

| n       | hit=5%  | hit=50% | hit=95% |
|---------|---------|---------|---------|
| 10      |  0.22x  |  1.19x  |  1.50x  |
| 100     |  0.62x  |  4.28x  |  6.22x  |
| 1000    |  2.70x  | 28.2x   | 48.9x   |
| 10000   | 20.4x   | 184x    | 353x    |
| 100000  | 157x    | 1488x   | 3077x   |

Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com>
@meta-codesync

meta-codesync Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

@alibeklfc has imported this pull request. If you are a Meta employee, you can view this in D108627773.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants