Bench large_array_vs_numpy.py is raising a TypeError

[FrancescAlted]

With latest numexpr:

> python bench/large_array_vs_numpy.py
<snip>
Exception in thread Thread-16 (benchmark_numexpr_re_evaluate):
Traceback (most recent call last):
  File "/Users/faltet/miniforge3/envs/blosc2/lib/python3.13/threading.py", line 1043, in _bootstrap_inner
    self.run()
    ~~~~~~~~^^
  File "/Users/faltet/miniforge3/envs/blosc2/lib/python3.13/threading.py", line 994, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/faltet/blosc/numexpr/bench/large_array_vs_numpy.py", line 96, in benchmark_numexpr_re_evaluate
    time_taken = timeit.timeit(
        lambda: ne.re_evaluate(
    ...<2 lines>...
        number=num_runs,
    )
  File "/Users/faltet/miniforge3/envs/blosc2/lib/python3.13/timeit.py", line 237, in timeit
    return Timer(stmt, setup, timer, globals).timeit(number)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/Users/faltet/miniforge3/envs/blosc2/lib/python3.13/timeit.py", line 180, in timeit
    timing = self.inner(it, self.timer)
  File "<timeit-src>", line 6, in inner
  File "/Users/faltet/blosc/numexpr/bench/large_array_vs_numpy.py", line 97, in <lambda>
    lambda: ne.re_evaluate(
            ~~~~~~~~~~~~~~^
        local_dict={"a": a[start:end], "b": b[start:end], "c": c[start:end]}
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ),
    ^
  File "/Users/faltet/blosc/numexpr/numexpr/necompiler.py", line 1051, in re_evaluate
    args = getArguments(argnames, local_dict, global_dict, _frame_depth=_frame_depth)
  File "/Users/faltet/blosc/numexpr/numexpr/necompiler.py", line 774, in getArguments
    for name in names:
                ^^^^^
TypeError: 'NoneType' object is not iterable
numexpr time (threaded with re_evaluate over 32 chunks with 2 threads): 2.748033 seconds
numexpr speedup: 5.48x

This used to work before. This benchmark was introduced in PR #496, so there should be something that we broke recently. @emmaai if you have some clue on what's going on, that would be great.

[emmaai]

I'll have a look in the next few days. I haven't got on 3.13 yet.

[27rabbitlt]

Just more information: tested with python 3.10, checkedout to the commit @a99412e9 that PR #496 was merged:

~/Work/Workplace/Python/numexpr @a99412e9 *1 ?6                                                                      ne 13:04:56
❯ python bench/large_array_vs_numpy.py
Benchmarking Expression 1:
NumPy time (threaded over 32 chunks with 2 threads): 4.709605 seconds
Exception in thread Thread-4 (benchmark_numexpr_re_evaluate):
Traceback (most recent call last):
  File "/Users/liuteng/opt/anaconda3/envs/ne/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/Users/liuteng/opt/anaconda3/envs/ne/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/liuteng/Work/Workplace/Python/numexpr/bench/large_array_vs_numpy.py", line 94, in benchmark_numexpr_re_evaluate
    time_taken = timeit.timeit(
  File "/Users/liuteng/opt/anaconda3/envs/ne/lib/python3.10/timeit.py", line 234, in timeit
    return Timer(stmt, setup, timer, globals).timeit(number)
  File "/Users/liuteng/opt/anaconda3/envs/ne/lib/python3.10/timeit.py", line 178, in timeit
    timing = self.inner(it, self.timer)
  File "<timeit-src>", line 6, in inner
  File "/Users/liuteng/Work/Workplace/Python/numexpr/bench/large_array_vs_numpy.py", line 95, in <lambda>
    lambda: ne.re_evaluate(
  File "/Users/liuteng/Work/Workplace/Python/numexpr/numexpr/necompiler.py", line 1002, in re_evaluate
    args = getArguments(argnames, local_dict, global_dict, _frame_depth=_frame_depth)
  File "/Users/liuteng/Work/Workplace/Python/numexpr/numexpr/necompiler.py", line 760, in getArguments
    for name in names:
TypeError: 'NoneType' object is not iterable
numexpr time (threaded with re_evaluate over 32 chunks with 2 threads): 2.472811 seconds
numexpr speedup: 1.90x

[27rabbitlt]

If we change this line in benchmark https://github.com/pydata/numexpr/blob/master/bench/large_array_vs_numpy.py#L81 from

if index == 0:

to

if index == 0 or index == 1:

problem seems to be solved.

Here index refers to the chunk index (we divide the large array into 32 chunks, and assign even index chunks to thread 0, odd index chunks to thread 1), and supposedly we want to evaluate the first chunk, then we can re_evaluate from then on. However since the cahce is thread local, the argument name cache is not shared between two threads (thread 0 and thread 1). So it seems that we need to evaluate chunk 0 for thread 0 and chunk 1 for thread 1.