Build pytantan from nextgenusfs/pytantan fork with SIMD disabled - #69
Merged
Conversation
Replaces the previous Dockerfile-level source-patch dance with a clean, upstream-able fix: * pixi.toml now resolves pytantan from nextgenusfs/pytantan@90818ff (feat/disable-simd branch). That fork adds a PYTANTAN_DISABLE_SIMD CMake option and a PYTANTAN_SIMD runtime env-var override. * Dockerfile sets CMAKE_ARGS=-DPYTANTAN_DISABLE_SIMD=ON before 'pixi install --locked', so uv compiles a generic-only pytantan wheel during the normal pixi install step. Build tools (git, build-essential, cmake, zlib1g-dev, binutils) are installed before pixi install so the source build can run. * Removes the ~100-line post-install rebuild block: the AVX2-bearing wheel is never installed in the first place, so there is nothing to uninstall and replace. This also eliminates the pixi-self-healing risk that was causing the previous fix to silently revert. * Verifier RUN is kept as a safety net: asserts no avx*/sse4*/neon* .so files in pytantan/platform/ and zero AVX/AVX2/ AVX512 instruction hits in any pytantan .so. Verified locally on linux/amd64 under Rosetta 2: - platform/ contains only generic.abi3.so - 'from pytantan.lib import RepeatFinder' succeeds - mask_repeats() runs end-to-end without SIGILL
nextgenusfs
marked this pull request as ready for review
May 26, 2026 03:58
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
PR #68 attempted to scrub AVX2 from
pytantanby source-patching itsCMakeLists.txtduring the Docker build and rebuilding the wheel. The build-time verifier passed, but the published:latestimage still containedavx2.so/sse4.sofiles and SIGILL'd onimport pytantanunder Rosetta 2. The most likely cause was pixi/uv self-healing the environment back to the PyPI wheel between the in-build verifier and theCOPY --from=buildstep — the source patch was applied to a copy of pytantan that was then overwritten.Rather than continue fighting the patch-after-install pattern, this PR moves the fix upstream into a fork of pytantan and changes the funannotate2 build to consume that fork as its only source for pytantan.
What
pytantan side (separate repo, nextgenusfs/pytantan@90818ff)
Two independent escape hatches were added to the fork on branch
feat/disable-simd:PYTANTAN_DISABLE_SIMDCMake option (defaultOFF). WhenON, skips theFindSSE4/FindAVX2/FindNEONincludes and forcesHAVE_SSE2=HAVE_SSE4=HAVE_AVX2=HAVE_NEON=OFF, so only the generic Cython extension is compiled.PYTANTAN_SIMDenv-var override read atlib.pyximport time. Accepted values:generic(ornone/off/0),sse2,sse4,avx2,neon. Overrides the archspec-detected runtime support flags. Lets users force the generic backend on a SIMD-enabled wheel without rebuilding.These are additive and intended to be upstreamed to
althonos/pytantanonce we've validated the funannotate2 image.funannotate2 side (this PR)
pixi.toml:pytantan = "==0.1.3"is replaced withpixi.lock: regenerated. Bothlinux-64andosx-arm64now reference the git URL.Dockerfile: net-93lines.git build-essential cmake zlib1g-dev binutils ca-certificates) are installed beforepixi install --lockedso uv has what it needs to build pytantan from source.ENV CMAKE_ARGS="-DPYTANTAN_DISABLE_SIMD=ON"is set beforepixi install, which scikit-build-core honors during the source build.RUNis kept as a safety net: it asserts noavx*/sse4*/neon*.sofiles inpytantan/platform/and zero AVX/AVX2/AVX512 instruction hits in any pytantan.so.Local verification (linux/amd64 under Rosetta 2)
The
buildstage was built withdocker buildx build --platform linux/amd64 --target build. The in-build verifier output:Runtime smoke test against the built image:
Only
generic.abi3.sois present inplatform/— noavx2.so, nosse4.so, noneon.so.RepeatFinderinstantiates andmask_repeats()runs end-to-end without SIGILL under Rosetta 2.Follow-up
main(push=true) so:latestis rebuilt from the new Dockerfile.althonos/pytantanwith the two escape hatches.Pull Request opened by Augment Code with guidance from the PR author