The problem
A @kills registration names the file to mutate and the test that must fail against the mutation. Where the two are the same file, the check cannot run: it reports the mutation as broken and the decorated test fails, whatever the test asserts.
AssertionError: <Outcome.BROKEN: 'broken'> != <Outcome.KILLED: 'killed'> : Mutation(...) was broken:
ModuleNotFoundError: No module named 'tests.update_time.test_visibility.VisibilityTest';
'tests.update_time.test_visibility' is not a package
This is not a rare shape. A check written as a test — the architecture rules, the visibility rules — lives in the module that tests it, so its guard is exactly the case the harness cannot express. It came up twice while refactoring the marker language: for the settings rule in tests/update_time/test_architecture.py and for the private-import rule in tests/update_time/test_visibility.py. Both mutations were run by hand with just mutate instead, which proves the guard once and keeps nothing.
The cause
_run_test installs the mutated source under the module's name and then asks the loader to resolve the test by its dotted name:
sys.modules[_module_name(mutation.path)] = _executed_module(mutated, mutation.path)
unittest.TestLoader().loadTestsFromName(mutation.test)
loadTestsFromName resolves a dotted name by importing the longest prefix it can. When the mutated module is the test's own, the prefix it tries to import runs through that module, which is installed as a plain module object with no __path__, so the import raises rather than backing off to attribute lookup.
The work
Resolve the test against the module object the harness already built, rather than through the loader's import, when the mutated module is the one the test lives in. TestLoader.loadTestsFromName takes a module argument for exactly this: the remainder of the dotted name — the class and the method — is then looked up as attributes of the module handed to it.
What this must not change
- What each outcome means.
STALE and BROKEN keep judging the mutation, KILLED and SURVIVED the test.
- That the mutation is applied in memory, so an interrupted run leaves no file broken.
- The modules a run leaves behind:
_run_test still restores sys.modules to what it found.
- Mutating a source module from a test elsewhere, which is how every registration in the suite works today.
Documentation
None. @kills gains no argument and no rule, so nothing a reader of the README or the changelog can observe changes.
Open questions
- Should a registration whose file and test disagree about the module keep going through the loader, or should both paths resolve against the executed module?
- Is a test that mutates its own module worth a case in
tests/test_mutation.py, which today covers a subject module mutated from a test beside it?
The problem
A
@killsregistration names the file to mutate and the test that must fail against the mutation. Where the two are the same file, the check cannot run: it reports the mutation asbrokenand the decorated test fails, whatever the test asserts.This is not a rare shape. A check written as a test — the architecture rules, the visibility rules — lives in the module that tests it, so its guard is exactly the case the harness cannot express. It came up twice while refactoring the marker language: for the settings rule in
tests/update_time/test_architecture.pyand for the private-import rule intests/update_time/test_visibility.py. Both mutations were run by hand withjust mutateinstead, which proves the guard once and keeps nothing.The cause
_run_testinstalls the mutated source under the module's name and then asks the loader to resolve the test by its dotted name:loadTestsFromNameresolves a dotted name by importing the longest prefix it can. When the mutated module is the test's own, the prefix it tries to import runs through that module, which is installed as a plain module object with no__path__, so the import raises rather than backing off to attribute lookup.The work
Resolve the test against the module object the harness already built, rather than through the loader's import, when the mutated module is the one the test lives in.
TestLoader.loadTestsFromNametakes amoduleargument for exactly this: the remainder of the dotted name — the class and the method — is then looked up as attributes of the module handed to it.What this must not change
STALEandBROKENkeep judging the mutation,KILLEDandSURVIVEDthe test._run_teststill restoressys.modulesto what it found.Documentation
None.
@killsgains no argument and no rule, so nothing a reader of the README or the changelog can observe changes.Open questions
tests/test_mutation.py, which today covers a subject module mutated from a test beside it?