Avoid needless completeness queries and cap CI database concurrency - #7185
Conversation
Add an optional database-independent precheck hook to CompletenessChecker, running after $or/$and decomposition and before any table lookup or null-count query. Register an NFBound precheck that recognizes a root discriminant range lying above the Galois root discriminant range, so the intrinsically impossible query no longer waits on the expensive null-data probe that CI's statement timeout cancels. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The devmirror shards share one database; cap concurrent matrix jobs at six so functional CI does not double as an uncontrolled load test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
psycodict predicates involving None carry SQL-null semantics ({"$ne": None}
means IS NOT NULL), which to_rset misreads as empty numeric sets: None maps to
the whole real line, so its complement is empty. The precheck could therefore
declare a nonempty search (e.g. grd IS NOT NULL, or "non-null or at most 30")
intrinsically impossible without touching the database.
Conservatively bypass the numeric precheck whenever None appears anywhere in
the rd/grd expression tree, using a sentinel so an omitted constraint stays
distinguishable from an explicit null predicate. The numeric contradiction
fast path is unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Pushed 2e6a262 fixing a correctness issue in the precheck: psycodict predicates carrying SQL-null semantics ( The precheck now conservatively declines any |
Summary
CompletenessChecker.root-discriminant constraints before querying for null stored values.
the database and that compatible intervals are not short-circuited.
devmirror database.
Motivation
With the statement timeout introduced in #7181,
UtilsTest.test_completerepeatedly failed while executing this preliminary null-data query
(e.g. https://github.com/LMFDB/lmfdb/actions/runs/31268853079/job/93204410081):
The original query is mathematically impossible: its root-discriminant interval
lies strictly above its Galois-root-discriminant interval. The existing
NFBoundlogic detects that contradiction, but only after the generic null-dataprobe.
This PR introduces a narrow precheck contract for conclusions that are valid
independently of database contents. It does not move ordinary completeness
guarantees ahead of null checking: those can depend on all relevant columns
having been computed, so they stay behind the null-data check.
Safety
Noneanywhere in therd/grdvalue, e.g.
{"$ne": None}forIS NOT NULL) bypass the numeric precheckentirely: the real-number model cannot represent them (
to_rsetreadsNoneas the whole real line, so{"$ne": None}would collapse to theempty set and misreport a nonempty search as impossible). A sentinel keeps
an omitted constraint distinguishable from an explicit null predicate, and
a regression test covers four null-predicate shapes, including the
{"$or": [{"$ne": None}, {"$lte": 30}]}collapse.Verification
tests pass (22 tests total in
test_utils.py).results_completeon the contradictory query against the realdatabase with psycodict statement logging enabled executes zero SQL
statements and returns in under a millisecond, with the exact existing
result triple.
tox -e tests -- lmfdb/tests/test_utils.pypasses (21 tests, 19.7s), withUtilsTest.test_completeat 14.9s.PGOPTIONS='-c statement_timeout=120s'(the pre-Raise the CI database statement timeout to 10 minutes #7184 threshold) and with
PGOPTIONS='-c statement_timeout=10min'.jobs.test.strategyretainsfail-fast: falseand theunchanged matrix expression, with
max-parallel: 6added andtimeout-minutes: 60untouched.git diff --check,pyflakes,pylint(repo flags) andruff check --preview --select=E722pass on the edited files.(run 31300780554,
16 minutes total) and the null-predicate fix
(run 31332813750).
Details below are from the first run:
devmirror belyi higher_genus_w_automorphisms hypergm lattice maass_forms tests users, the one runninglmfdb/tests/test_utils.py) passed.UtilsTest.test_completetook 9.22s in CI (it previously stalled on the canceled null probe until the 120s statement timeout); the two new regression tests took 0.00s.QueryCanceledappears anywhere in that job's log.max-parallellimited execution without suppressing jobs (observed 6 in progress + 6 queued, with the second wave starting as first-wave jobs finished).🤖 Generated with Claude Code