Restructure benchmarks: add C++/Python/GPU suites, relocate legacy

[mulugetam]

This PR replaces the standalone benchmark scripts in benchs/ and perf_tests/ with three structured suites (C++, Python, and GPU) organized under a consistent kern_ / codec_ / index_ naming taxonomy. Nearly all existing benchmarks are covered by the new suites, and the default parameter values from the legacy benchmarks are preserved (with a few exceptions). Every executable and pytest suite supports --help for discovering available parameters and usage (bench_fw drivers are the exception; they use positional arguments documented in their README).

All changes are contained within the benchs/ directory, with the exception of CMakeLists.txt, INSTALL.md, and README.md.

I wasn't sure whether to open this as an issue, a discussion, or a PR. I went with a PR so the code can be reviewed.

Motivation

The existing benchmarks were a flat directory of ~50 scripts with inconsistent naming, no shared parameter infrastructure, and no machine-readable output (some with hardcoded parameters). This made regression tracking difficult (e.g. when benchmarking across different platforms/microarchitectures). The new suites provide:

• Uniform naming (bench_{kern,codec,index}_<feature>) across C++ and Python
• CLI-overridable parameter sweeps (comma-separated lists) for both suites
• JSON output for regression tracking (Google Benchmark JSON / pytest-benchmark JSON)
• Shared dataset loading with automatic skip when data is absent
• One CMake target (benchmarks) to build the entire C++ + GPU suite

Benchmark suites

• benchs/cpp/ — 35 Google Benchmark executables registering 5,146 benchmark cases, covering kernel-level operations (distances, hamming, heap, partitioning, sorting, FWHT, fp16, visited-table, result-handlers), codec-level operations (PQ, SQ, RQ, k-means, SA decode, PQ ADC/fastscan), and index-level operations (flat, IVF, fastscan, graph, binary, composite, RaBitQ, Panorama, polysemous, big-batch, IO, RCQ, IVF factory with autotune). CLI-overridable parameter sweeps via bench_cli_utils.h. Index benchmarks optionally load SIFT1M (or any .fvecs/.bvecs dataset) via bench_dataset_utils.h and report recall. See benchs/README.md for more details.

• benchs/python/ — 18 pytest-benchmark files registering 4,209 benchmark cases, mirroring the C++ coverage through the public Python API with the same overridable parameter model (conftest.py / bench_utils.py). Includes a bench_fw/ subdirectory housing the end-to-end evaluation framework (moved from the repo root) with relative imports and four driver scripts (codecs, IVF, optimize, range), excluded from pytest collection. See benchs/python/README.md for more details.

• benchs/gpu/ — 5 Google Benchmark executables for GPU indexes (flat+IVFPQ, IVF 1-billion, hybrid CPU/GPU, cuVS IVFFlat/IVFPQ), built only when FAISS_ENABLE_GPU=ON.

• benchs/legacy/ — all original benchs/ scripts and perf_tests/ moved here, preserving their CMake gating and CI paths.

All index-level benchmarks (C++ and Python) run on synthetic data by default and additionally register real-data cases (SIFT1M or any .fvecs/.bvecs dataset pointed to by --data_dir) that report recall and skip automatically when the dataset is absent. Kernel and codec benchmarks (e.g. bench_kern_distances, bench_kern_heap, bench_codec_pq) use only synthetic data, since their throughput characteristics are data-distribution-independent.

Tested the following

• cmake -B build -DFAISS_BUILD_BENCHMARKS=ON && cmake --build build --target benchmarks compiles all 35 C++ executables (GPU skipped with FAISS_ENABLE_GPU=OFF)
• Each build/benchs/cpp/bench_* runs with --benchmark_list_tests=true
• cd benchs/python && pytest --collect-only discovers all 18 Python benchmark modules and excludes bench_fw/
• cd benchs/python && pytest bench_index_flat.py passes
• Legacy benchmarks at benchs/legacy/ build via FAISS_ENABLE_EXTRAS=ON (individual targets, EXCLUDE_FROM_ALL)
• CI perf_tests path (benchs/legacy/perf_tests/) resolves correctly in .github/actions/build_cmake/action.yml