Conversation
The dev requirements now install current flake8, mypy and bandit directly. The dead plugins flake8-mypy, pytest-pep8 and bandit-high-entropy-string are removed, and mypy runs standalone instead of through flake8. Tool options move from the Makefile into setup.cfg, and scripts/check.sh becomes the shared entry point for host, VM and CI runs. The mypy stubs live in stubs/ and gain hand-written type stubs for the jail and freebsd_sysctl packages, which ship no py.typed marker. setup.py reads requirements.txt with a plain file parser because modern pip no longer exposes parse_requirements, and the supported Python version is 3.11 or newer. The Travis CI configuration is deleted since the service no longer exists for open source.
Modules whose libzfs use is limited to type annotations import it under typing.TYPE_CHECKING, combined with deferred annotation evaluation via the annotations future import. Quoted forward references now have matching imports in the TYPE_CHECKING block, which modern mypy requires for resolution. libioc.helpers imports the jail package inside mount and umount because current glibc lacks the sysctl symbol that the jail package touches at import time. libioc.ResourceSelector imports its runtime dependencies libioc.errors and libioc.helpers explicitly instead of relying on transitive imports, and defers libioc.Datasets to the one method using it. Fstab and ResolverProp inherit from collections.abc.MutableSequence, since the collections aliases were removed in Python 3.10. Several annotations referenced classes under outdated names and now point to libioc.events.IocEvent, libioc.events.Scope, libioc.ResourceUpdater.Updater, libioc.Provisioning.Provisioner and JailConfigProperties. get_os_version no longer passes re.MULTILINE as the buffering argument of open. The result imports the pure-logic modules on Linux without any FreeBSD dependency.
tests/linux_shims provides import-time replacements for py-libzfs and py-jail on hosts where the FreeBSD packages are unavailable. conftest.py detects the platform, skips every ZFS or jail dependent fixture visibly and keeps the release mirror cache server FreeBSD-only. The mocked distribution downloads releases from the FreeBSD archive mirror because the tested releases are end of life, overridable through LIBIOC_TEST_MIRROR. Fetching release updates is disabled unless LIBIOC_TEST_FETCH_UPDATES=1, since the freebsd-update servers no longer serve end-of-life releases. The logger fixture writes into the pytest temporary directory instead of /var/log/iocage, and the pool fixture skips instead of exiting the interpreter. The Fstab parser tests run against a mocked host, so that they no longer construct a ZFS handle. test_VNET survives collection on hosts without /sbin/kldstat, and the Storage test class is collected again after gaining the missing Test prefix and imports. New unit tests cover the helper parsers, the nested config data dict, the filter terms and the resource limit value parsing. On Linux the suite currently reports 63 passed and 53 skipped.
The scripts in tools/vm download the archived 13.5-RELEASE VM image, verify it against a pinned checksum, provision it over the serial console and prepare the guest with the packages, the fdescfs mount and the file-backed ZFS pool that the test suite expects. FreeBSD 13.5 is the newest release whose binary packages are still available while remaining close to the FreeBSD 12.1 environment of the original CI, and the guest package cache is mirrored to the host because the repository serves a branch past its end of life. The test runner synchronizes the repository into the guest and offers tiered targets from an import sweep up to the full suite, plus an end-to-end jail lifecycle smoke test. The host has no KVM device, so QEMU runs in TCG software emulation and the provisioning script switches the boot console to the serial port at the loader prompt before persisting it in loader.conf. A GitHub Actions workflow replaces the retired Travis CI setup and runs the static checks and the platform-independent tests on Linux.
The strict mypy configuration now checks all 73 source files without errors on current mypy, after roughly 550 findings. Most fixes are annotation corrections: generator methods that yield arbitrary IocEvent subclasses are annotated as such, signatures widened to the types their call sites actually pass, class-object attributes declared as typing.Type, implicit-Optional defaults made explicit and outdated typeshed signatures updated. Deliberate deviations from dict or list supertypes and dynamic metaprogramming carry narrowly scoped type-ignore comments with justifications. IocEvent.fail and its error attribute accept exceptions and strings in addition to booleans, matching all callers. The sha1 call generating jail identifiers passes usedforsecurity=False, and SecureTarfile requests the fully_trusted extraction filter explicitly because its own member validation guards the archive content. The static check toolchain is now pinned to the exact versions the fixes were verified with.
When a .venv directory exists in the repository root, the check script prepends it to the PATH, so that the static checks run without activating the environment first.
The official VM images route the first boot to the emulated VGA console and the serial port stays silent, which the earlier expect-based provisioning could not handle. The provisioning script now boots the machine daemonized, types the setup commands blindly into the VGA console through the QEMU monitor with monitor_type.py, persists the serial console in loader.conf for all future boots and verifies the result over SSH. The guest setup pipes a POSIX script into sh on the guest, because root's login shell on FreeBSD is csh and complex quoted ssh command strings break there. The VM control script uses the pc machine type and exposes the monitor socket, so that screenshots of the VGA console are available for debugging.
Current mypy depends on Rust-based build backends that have no FreeBSD wheels, so the FreeBSD guest cannot install the full development toolchain. requirements-test.txt carries the pytest packages alone, requirements-dev.txt includes them for development hosts, and the VM test runner and the CI unit job install only the test set. The VM test runner pipes every guest command through a POSIX shell, since root's login shell on FreeBSD is csh, and the import sweep lives in its own file instead of an inline python -c string.
The Storage test class had never been collected since it lacked the Test prefix, and its assertions encoded a basejail-by-default world that does not match the code: new jails use standalone storage, basejail_type is None until a jail is a basejail, and devfs is mounted by default per mount_devfs=1. Enabling basejail on an existing jail does not regenerate the fstab basejail lines, so the corresponding start test is marked xfail and the gap is documented in the mirror issue tracker. The smoke test asserts a shell echo instead of ps, whose exit code depends on visible processes inside the fresh jail. The VM test runner keeps the guest release cache across syncs.
Modern pkg refuses an ABI override without OSVERSION and warns loudly, which the strict stderr filter treated as a fatal error, so every pkg environment now sets OSVERSION derived from the release major version. pkg fetches .pkg archives instead of .txz nowadays, and the latest-archive lookup accepts both extensions. The pkg bootstrap wrapper inside a pristine jail fetches Latest/pkg.pkg from the repository instead of accepting a direct archive path, so building the mirror index publishes the newest pkg archive under that name. The puppet provisioner installs puppet8, since puppet6 left the package repositories.
The rlimits property defaults to None, and the previous strict is-True check meant that configured pcpu, memoryuse or similar limits were silently never applied unless rlimits was also set explicitly. An unset rlimits now means automatic behavior: rules are applied whenever limit properties are present, while an explicit False still disables them. The guest setup persists kern.racct.enable and if_epair in loader.conf so that the resource limit and VNET tests run after every boot, and rc.local reimports the file-backed pool, which is not always reimported automatically. The VM readme states the measured wall-clock times instead of estimates.
Unmounting a jail dataset can briefly race with the stopping jail and fail with a busy error, so the fixture retries the destruction a few times before giving up.
The jail sdist declares a setuptools build backend but is installed without build isolation because its setup.py imports pip, and Python 3.13 environments no longer ship setuptools by default.
actions/checkout moves to v7 and actions/setup-python to v6, which both run on the current Node runtime and silence the deprecation notice in the workflow logs.
isinstance comparisons against True and False become bare isinstance calls, which is semantically identical and lets mypy narrow types naturally instead of requiring casts behind every guard. ListableResource is generic over its resource type, so jail and release collections yield precisely typed elements without downcasts at the call sites. JailGenerator accepts any HostGenerator, matching what every caller passes, and Datasets parameterizes its dict base with the value type it holds. The freebsd_sysctl stubs declare that next and children instantiate the receiver class, matching the implementation, which types subclass iteration without casts. parse_user_input gained overloads describing its passthrough behavior, to_string accepts any sequence, and the config Prototype declares the data types its subclasses actually exchange. The remaining casts each carry a comment naming the dynamic construct or cross-function invariant that the type system cannot express. In total the cast count drops from 102 to 62, verified by the strict mypy run, the Linux test subset and the full suite on FreeBSD 13.5.
The BASIC-CI image ships with a serial console, DHCP, growfs and an sshd that accepts root with an empty password on the first boot, so the whole provisioning runs over SSH and the QEMU monitor typing helper is gone. Provisioning installs the SSH key, switches PermitRootLogin to prohibit-password and closes the empty-password access, while the forwarded port only ever binds to 127.0.0.1. Three behaviors cost debugging time and are documented in the scripts: sshd accepts the initial none authentication while the root password is empty, so provisioning state is probed through the guest configuration; a remote pipeline must not close its reader early, because the resulting SIGPIPE in sshd -T is reported as a failure by root's csh; and the first boot reboots once after growing the root filesystem, which both the firstboot-marker gate and the verification window account for. QEMU uses KVM whenever /dev/kvm is usable and falls back to TCG emulation, the serial console mirrors into a logfile that provisioning prints when SSH never comes up, and the shared QEMU invocation lives in config.sh. The guest setup reboots once when the resource accounting tunable is not active yet, and rc.local reimports the file-backed pool. Resource limits in auto mode now require an rctl-enabled kernel, since kern.features.rctl only reports compiled-in support and clearing limits on a racct-disabled kernel would fail every jail stop. The full suite passes on the BASIC-CI guest with 115 passed and one expected failure.
The freebsd job consumes the shared setup-vm and run actions from gronke/freebsd-ci to boot a FreeBSD 13.5 guest on the Linux runner, prepares it with the packages, the fdescfs mount and the file-backed pool the suite expects, and runs pytest against the pool. The actions follow main until their interface settles behind a version tag. Resource limit tests skip on this guest because enabling resource accounting requires a boot-time tunable, which the graceful rctl detection reports as unsupported.
The differential backup implementation shells out to rsync, exactly as the local guest setup already installs it.
The generic plumbing for fetching, provisioning and controlling the FreeBSD VM now comes from the freebsd-ci repository, which 00-bootstrap.sh pins by commit into the ignored tools/vm/freebsd-ci directory. tools/vm/config.sh exports the libioc defaults, the 13.5 release and the cache location, before delegating to the shared configuration, so the existing cache and snapshots keep working unchanged. Only the libioc-specific pieces remain here: the guest setup with the package set, pool and tunables the suite expects, the tiered test runner, the smoke test and the import sweep. CI already consumes the same tooling as composite actions.
The bootstrap pin moves to the upstream commit that provides setup.sh, run.sh and the guest verbs. The guest setup now composes those helpers for packages, kernel modules, the boot tunable, fdescfs, the pool and the package cache mirror, keeping only the libioc package set, the pool name and the venv here. The test runner executes guest commands through the shared run.sh, while the repository still syncs incrementally with rsync, because the suite's release download cache inside the guest must survive between runs.
Updates to the shared tooling now arrive without changes to this repository, matching the workflow that already consumes the composite actions on main. FREEBSD_CI_REF still accepts a commit to reproduce a historic state, and both references move to the v1 tag once upstream cuts it. Vendoring was considered and rejected, since it would restore the duplication the extraction removed; the mirrored images and packages in the cache remain the end-of-life insurance.
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This branch revives the dormant code base with a modernized static-check toolchain, a platform-independent test subset that runs on Linux, and a QEMU harness that runs the full suite on FreeBSD 13.5.
Toolchain
The dead packages flake8-mypy, pytest-pep8 and bandit-high-entropy-string are gone, mypy runs standalone against hand-written stubs for jail and freebsd_sysctl, and the toolchain versions are pinned.
setup.py reads requirements with a plain file parser instead of private pip internals.
A GitHub Actions workflow replaces the retired Travis CI configuration.
Portability
Type-only libzfs imports moved under typing.TYPE_CHECKING, and modules broken since Python 3.10 (collections.MutableSequence base classes) are fixed.
tests/linux_shims provides import-time stand-ins for py-libzfs and py-jail, so the logic-tier tests collect and run on any host.
Functional repairs
Package mirroring and installation into jails works with current pkg again: OSVERSION accompanies every ABI override, .pkg archives are recognized alongside .txz, and the mirror index publishes Latest/pkg.pkg for the bootstrap wrapper.
The puppet provisioner installs puppet8, since puppet6 left the package repositories.
Resource limits apply automatically when limit properties are configured, matching the behavior the tests always expected.