Skip to content

perf(filters): skip empty name keys in values() -- 9% faster completion - #2126

Closed
dexhunter wants to merge 1 commit into
davidhalter:masterfrom
dexhunter:perf/filters-skip-empty-name-keys
Closed

dexhunter wants to merge 1 commit into
davidhalter:masterfrom
dexhunter:perf/filters-skip-empty-name-keys

Conversation

@dexhunter

Copy link
Copy Markdown

TL;DR

  • This PR makes ParserTreeFilter.values() and ParserTreeFilter._filter faster, so name completion gets faster.
  • values() now skips name keys that have no definitions and looks up the module cache once.
  • Measured on one CPU core with a completion benchmark (9 stdlib modules plus Interpreter completions): 462.7 ms -> 419.4 ms, 9.4% faster.
  • Tests: pytest test/ shows the same 44 failures as master (environment-dependent), 3883 passed.

What & Why

Most keys in used_names are plain usages, not definitions. Before, values() still called _get_definition_names() and _filter() for each of these keys. Each call did a weak dict lookup, a super() call, two list builds and a generator, only to return an empty list.

Now values() gets the per-module definition cache once. Then it skips keys whose definition tuple is empty. _filter() returns early for empty input and checks until_position and reachability in one list comprehension. _check_flows() sorts only when there are two or more names, and it returns a list.

The output stays identical. Every _filter() override (ClassFilter, SelfAttributeFilter, FixtureFilter, StubFilter) returns nothing for empty input. The cache keeps the same keys and values. Names keep the same order. The flow checks run in the same order and stop at the same name.

Changes

  • _AbstractUsedNamesFilter.values(): get the definition cache once, fill it in the same way, and skip empty keys.
  • ParserTreeFilter._filter(): return early for no names; use one list comprehension; no super() call.
  • ParserTreeFilter._check_flows(): sort only for two or more names; build and return a list.

Results

Median times on one dedicated CPU core, Python 3.12, InterpreterEnvironment.

Situation Before After Change
Completion workload (9 stdlib modules + Interpreter completions) 462.7 ms 419.4 ms -9.4%
Script.complete() at end of argparse.py 19.79 ms 18.25 ms -7.8%
Script.complete() at end of bisect.py 13.27 ms 12.80 ms -3.5%
Interpreter completions (C, C.foo, C., os.path.) 34.96 ms 33.07 ms -5.4%
Interpreter("str.").complete() 28.91 ms 24.98 ms -13.6%
Script.infer() on one local name (does not use values()) 2.39 ms 2.37 ms no change

Why this is safe and easy to maintain

The change is small and stays in jedi/inference/filters.py. It keeps the same structure and only removes extra work. The completion results, their order and the cache contents do not change. test/test_api/test_interpreter.py, test_classes.py, test_signature.py, test_imports.py and the completion integration tests cover this code.

Validation

  • pytest test/test_api/test_interpreter.py test/test_inference/test_signature.py test/test_api/test_classes.py test/test_inference/test_imports.py: 485 passed, 5 skipped, 4 xfailed (same as master).
  • pytest test/: 44 failed, 3883 passed, 111 skipped, 5 xfailed. The 44 failures are the same test IDs on master (environment-dependent test_integration pytest/conftest cases and test_namespace_package.py::test_relative_import).
  • The completion results of the workload above are identical before and after (names, complete, name_with_symbols, order).
  • Short reproduction:
python -m timeit -s "import jedi, argparse; s = open(argparse.__file__).read(); n = s.count(chr(10)) + 1; jedi.Script(s, path=argparse.__file__).complete(n, 0)" "jedi.Script(s, path=argparse.__file__).complete(n, 0)"

AI assistance

Codex (gpt-6.1-sol) generated the candidate changes, and Weco Observe tracked and benchmarked them. Claude Code selected this candidate and ran the tests and benchmarks above. It also rejected a faster candidate because it crashed on empty modules.

Weco share link: https://dashboard.weco.ai/share/FvOQKGvG0Xt8f_28B1bRfiP_tq4kRO_z

Most used-name keys in a module are plain usages with no definitions.
_AbstractUsedNamesFilter.values() still ran _get_definition_names() and
ParserTreeFilter._filter() for each of them, only to get an empty list.

Look up the per-module definition cache once in values() and skip keys
without definitions. Return early from _filter() for empty input, check
until_position and reachability in one pass, and sort in _check_flows()
only when there are two or more names. Results and their order are
unchanged.

Completion benchmark (9 stdlib modules plus Interpreter completions):
462.7 ms -> 419.4 ms.

Co-Authored-By: Aiden <aiden@weco.ai>
@davidhalter davidhalter closed this Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants