Update Python Dependencies by github-actions[bot] · Pull Request #40960 · apache/beam

github-actions · 2026-11-03T02:49:08Z

This PR was created by automation - by default this does not execute tests due to a limitation in GitHub Actions. To review this PR, please (1) review the contents to make sure all dependency upgrades are formatted correctly, (2) close and reopen the PR (3) Approve/merge if tests pass successfully.

github-actions · 2026-11-03T03:11:26Z

Assigning reviewers:

R: @claudevdm for label python.
Backup reviewer: shunping (if the main reviewer is unresponsive, you can escalate to them by tagging their username)

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

tvalentyn · 2026-11-04T02:17:13Z

27.04 RuntimeError: Could not retrieve licences for packages ['arro3-core'] in Python3.14 environment.

tvalentyn · 2026-11-04T02:27:27Z

          git add -A
-          git diff-index --quiet HEAD || gitdiff=$? || echo $?
-          if [[ $gitDiff != 0 ]]; then
+          if ! git diff --cached --quiet; then


note: changes to this file have been merged already: #40959

tvalentyn · 2026-11-04T15:48:36Z

Attempting a rerun of Py314 precommit suite that failed with:

2026-11-04T02:45:41.1805010Z =================================== FAILURES ===================================
2026-11-04T02:45:41.1806134Z _ PipelineBasedStreamingInsertTest.test_failure_in_some_rows_does_not_duplicate_1 _
2026-11-04T02:45:41.1807842Z [gw5] linux -- Python 3.14.10 /runner/_work/beam/beam/sdks/python/test-suites/tox/py314/build/srcs/sdks/python/target/.tox-py314-cloud/py314-cloud/bin/python
2026-11-04T02:45:41.1809373Z 
2026-11-04T02:45:41.1809692Z >   return self.do_fn_invoker.invoke_process(windowed_value)
2026-11-04T02:45:41.1810322Z 
2026-11-04T02:45:41.1810665Z apache_beam/runners/common.py:1516: 
2026-11-04T02:45:41.1811512Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2026-11-04T02:45:41.1812689Z apache_beam/runners/common.py:918: in apache_beam.runners.common.PerWindowInvoker.invoke_process
2026-11-04T02:45:41.1813648Z     self._invoke_process_per_window(
2026-11-04T02:45:41.1814783Z apache_beam/runners/common.py:1063: in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
2026-11-04T02:45:41.1815953Z     self.process_method(*args_for_process, **kwargs_for_process),
2026-11-04T02:45:41.1816824Z target/.tox-py314-cloud/py314-cloud/lib/python3.14/site-packages/apache_beam/transforms/core.py:2128: in <lambda>
2026-11-04T02:45:41.1817895Z     wrapper = lambda x, *args, **kwargs: [fn(x, *args, **kwargs)]
2026-11-04T02:45:41.1818946Z                                           ^^^^^^^^^^^^^^^^^^^^^^
2026-11-04T02:45:41.1819930Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2026-11-04T02:45:41.1820458Z 
2026-11-04T02:45:41.1820684Z actual = ['value1', 'value3']
2026-11-04T02:45:41.1821392Z equals_fn = <function row_namedtuple_equals_fn at 0x7cbbf3dc6a30>
2026-11-04T02:45:41.1821929Z 
2026-11-04T02:45:41.1822262Z     def _equal(actual, equals_fn=equals_fn):
2026-11-04T02:45:41.1822847Z       expected_list = list(expected)
2026-11-04T02:45:41.1823327Z     
2026-11-04T02:45:41.1823869Z       # Try to compare actual and expected by sorting. This fails with a
2026-11-04T02:45:41.1824732Z       # TypeError in Python 3 if different types are present in the same
2026-11-04T02:45:41.1825612Z       # collection. It can also raise false negatives for types that don't have
2026-11-04T02:45:41.1826498Z       # a deterministic sort order, like pyarrow Tables as of 0.14.1
2026-11-04T02:45:41.1827220Z       if not equals_fn:
2026-11-04T02:45:41.1827554Z         equals_fn = row_namedtuple_equals_fn
2026-11-04T02:45:41.1828091Z         try:
2026-11-04T02:45:41.1828429Z           sorted_expected = sorted(expected)
2026-11-04T02:45:41.1829345Z           sorted_actual = sorted(actual)
2026-11-04T02:45:41.1829754Z           if sorted_expected == sorted_actual:
2026-11-04T02:45:41.1830101Z             return
2026-11-04T02:45:41.1830388Z         except TypeError:
2026-11-04T02:45:41.1830679Z           pass
2026-11-04T02:45:41.1830976Z       # Slower method, used in two cases:
2026-11-04T02:45:41.1831475Z       # 1) If sorted expected != actual, use this method to verify the inequality.
2026-11-04T02:45:41.1832149Z       #    This ensures we don't raise any false negatives for types that don't
2026-11-04T02:45:41.1832642Z       #    have a deterministic sort order.
2026-11-04T02:45:41.1833109Z       # 2) As a fallback if we encounter a TypeError in python 3. this method
2026-11-04T02:45:41.1833624Z       #    works on collections that have different types.
2026-11-04T02:45:41.1834026Z       unexpected = []
2026-11-04T02:45:41.1834474Z       for element in actual:
2026-11-04T02:45:41.1836081Z         found = False
2026-11-04T02:45:41.1836641Z         for i, v in enumerate(expected_list):
2026-11-04T02:45:41.1837283Z           if equals_fn(v, element):
2026-11-04T02:45:41.1837856Z             found = True
2026-11-04T02:45:41.1838362Z             expected_list.pop(i)
2026-11-04T02:45:41.1839088Z             break
2026-11-04T02:45:41.1839545Z         if not found:
2026-11-04T02:45:41.1840054Z           unexpected.append(element)
2026-11-04T02:45:41.1840652Z       if unexpected or expected_list:
2026-11-04T02:45:41.1841346Z         msg = 'Failed assert: %r == %r' % (expected, actual)
2026-11-04T02:45:41.1842016Z         if unexpected:
2026-11-04T02:45:41.1842691Z           msg = msg + ', unexpected elements %r' % unexpected
2026-11-04T02:45:41.1843352Z         if expected_list:
2026-11-04T02:45:41.1843936Z           msg = msg + ', missing elements %r' % expected_list
2026-11-04T02:45:41.1844654Z >       raise BeamAssertException(msg)
2026-11-04T02:45:41.1867721Z E       apache_beam.testing.util.BeamAssertException: Failed assert: ['value5', 'value1'] == ['value1', 'value3'], unexpected elements ['value3'], missing elements ['value5']
2026-11-04T02:45:41.1869197Z 
2026-11-04T02:45:41.1869957Z target/.tox-py314-cloud/py314-cloud/lib/python3.14/site-packages/apache_beam/testing/util.py:205: BeamAssertException
2026-11-04T02:45:41.1870722Z 
2026-11-04T02:45:41.1871076Z During handling of the above exception, another exception occurred:
2026-11-04T02:45:41.1871653Z 
2026-11-04T02:45:41.1872592Z a = (<apache_beam.io.gcp.bigquery_test.PipelineBasedStreamingInsertTest testMethod=test_failure_in_some_rows_does_not_duplicate_1>,)
2026-11-04T02:45:41.1873451Z kw = {}
2026-11-04T02:45:41.1873609Z 
2026-11-04T02:45:41.1873754Z     @wraps(func)
2026-11-04T02:45:41.1874073Z     def standalone_func(*a, **kw):
2026-11-04T02:45:41.1874504Z >       return func(*(a + p.args), **p.kwargs, **kw)
2026-11-04T02:45:41.1874911Z                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-11-04T02:45:41.1875179Z 
2026-11-04T02:45:41.1875565Z target/.tox-py314-cloud/py314-cloud/lib/python3.14/site-packages/parameterized/parameterized.py:620: 
2026-11-04T02:45:41.1876257Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2026-11-04T02:45:41.1877128Z target/.tox-py314-cloud/py314-cloud/lib/python3.14/site-packages/apache_beam/io/gcp/bigquery_test.py:3062: in test_failure_in_some_rows_does_not_duplicate
2026-11-04T02:45:41.1878006Z     with beam.Pipeline(runner='BundleBasedDirectRunner') as p:
2026-11-04T02:45:41.1878712Z          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-11-04T02:45:41.1879575Z target/.tox-py314-cloud/py314-cloud/lib/python3.14/site-packages/apache_beam/pipeline.py:656: in __exit__
2026-11-04T02:45:41.1880534Z     self.result.wait_until_finish()
2026-11-04T02:45:41.1881570Z target/.tox-py314-cloud/py314-cloud/lib/python3.14/site-packages/apache_beam/runners/direct/direct_runner.py:652: in wait_until_finish
2026-11-04T02:45:41.1882814Z     self._executor.await_completion()
2026-11-04T02:45:41.1883900Z target/.tox-py314-cloud/py314-cloud/lib/python3.14/site-packages/apache_beam/runners/direct/executor.py:415: in await_completion
2026-11-04T02:45:41.1884994Z     self._executor.await_completion()
2026-11-04T02:45:41.1886075Z target/.tox-py314-cloud/py314-cloud/lib/python3.14/site-packages/apache_beam/runners/direct/executor.py:462: in await_completion
2026-11-04T02:45:41.1887168Z     raise update.exception
2026-11-04T02:45:41.1888158Z target/.tox-py314-cloud/py314-cloud/lib/python3.14/site-packages/apache_beam/runners/direct/executor.py:353: in call
2026-11-04T02:45:41.1889529Z     self.attempt_call(
2026-11-04T02:45:41.1890780Z target/.tox-py314-cloud/py314-cloud/lib/python3.14/site-packages/apache_beam/runners/direct/executor.py:396: in attempt_call
2026-11-04T02:45:41.1892006Z     evaluator.process_element(value)
2026-11-04T02:45:41.1893911Z target/.tox-py314-cloud/py314-cloud/lib/python3.14/site-packages/apache_beam/runners/direct/transform_evaluator.py:933: in process_element
2026-11-04T02:45:41.1895522Z     self.runner.process(element)
2026-11-04T02:45:41.1896437Z apache_beam/runners/common.py:1513: in apache_beam.runners.common.DoFnRunner.process
2026-11-04T02:45:41.1897410Z     def process(self, windowed_value):
2026-11-04T02:45:41.1898345Z apache_beam/runners/common.py:1518: in apache_beam.runners.common.DoFnRunner.process
2026-11-04T02:45:41.1899576Z     self._reraise_augmented(exn, windowed_value)
2026-11-04T02:45:41.1900588Z apache_beam/runners/common.py:1629: in apache_beam.runners.common.DoFnRunner._reraise_augmented
2026-11-04T02:45:41.1901316Z     raise new_exn
2026-11-04T02:45:41.1901771Z apache_beam/runners/common.py:1516: in apache_beam.runners.common.DoFnRunner.process
2026-11-04T02:45:41.1902494Z     return self.do_fn_invoker.invoke_process(windowed_value)
2026-11-04T02:45:41.1903121Z apache_beam/runners/common.py:918: in apache_beam.runners.common.PerWindowInvoker.invoke_process
2026-11-04T02:45:41.1903703Z     self._invoke_process_per_window(
2026-11-04T02:45:41.1904498Z apache_beam/runners/common.py:1063: in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
2026-11-04T02:45:41.1905186Z     self.process_method(*args_for_process, **kwargs_for_process),
2026-11-04T02:45:41.1905895Z target/.tox-py314-cloud/py314-cloud/lib/python3.14/site-packages/apache_beam/transforms/core.py:2128: in <lambda>
2026-11-04T02:45:41.1906609Z     wrapper = lambda x, *args, **kwargs: [fn(x, *args, **kwargs)]
2026-11-04T02:45:41.1907052Z                                           ^^^^^^^^^^^^^^^^^^^^^^
2026-11-04T02:45:41.1907524Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2026-11-04T02:45:41.1907817Z 
2026-11-04T02:45:41.1907948Z actual = ['value1', 'value3']
2026-11-04T02:45:41.1908363Z equals_fn = <function row_namedtuple_equals_fn at 0x7cbbf3dc6a30>
2026-11-04T02:45:41.1909015Z 
2026-11-04T02:45:41.1909184Z     def _equal(actual, equals_fn=equals_fn):
2026-11-04T02:45:41.1909546Z       expected_list = list(expected)
2026-11-04T02:45:41.1909879Z     
2026-11-04T02:45:41.1910231Z       # Try to compare actual and expected by sorting. This fails with a
2026-11-04T02:45:41.1910754Z       # TypeError in Python 3 if different types are present in the same
2026-11-04T02:45:41.1911304Z       # collection. It can also raise false negatives for types that don't have
2026-11-04T02:45:41.1911838Z       # a deterministic sort order, like pyarrow Tables as of 0.14.1
2026-11-04T02:45:41.1912329Z       if not equals_fn:
2026-11-04T02:45:41.1912640Z         equals_fn = row_namedtuple_equals_fn
2026-11-04T02:45:41.1912972Z         try:
2026-11-04T02:45:41.1913245Z           sorted_expected = sorted(expected)
2026-11-04T02:45:41.1913593Z           sorted_actual = sorted(actual)
2026-11-04T02:45:41.1914190Z           if sorted_expected == sorted_actual:
2026-11-04T02:45:41.1914717Z             return
2026-11-04T02:45:41.1915050Z         except TypeError:
2026-11-04T02:45:41.1915350Z           pass
2026-11-04T02:45:41.1915646Z       # Slower method, used in two cases:
2026-11-04T02:45:41.1916262Z       # 1) If sorted expected != actual, use this method to verify the inequality.
2026-11-04T02:45:41.1916995Z       #    This ensures we don't raise any false negatives for types that don't
2026-11-04T02:45:41.1917488Z       #    have a deterministic sort order.
2026-11-04T02:45:41.1917946Z       # 2) As a fallback if we encounter a TypeError in python 3. this method
2026-11-04T02:45:41.1918654Z       #    works on collections that have different types.
2026-11-04T02:45:41.1919103Z       unexpected = []
2026-11-04T02:45:41.1919407Z       for element in actual:
2026-11-04T02:45:41.1919719Z         found = False
2026-11-04T02:45:41.1920036Z         for i, v in enumerate(expected_list):
2026-11-04T02:45:41.1920397Z           if equals_fn(v, element):
2026-11-04T02:45:41.1920729Z             found = True
2026-11-04T02:45:41.1921036Z             expected_list.pop(i)
2026-11-04T02:45:41.1921338Z             break
2026-11-04T02:45:41.1921965Z         if not found:
2026-11-04T02:45:41.1922583Z           unexpected.append(element)
2026-11-04T02:45:41.1922960Z       if unexpected or expected_list:
2026-11-04T02:45:41.1923356Z         msg = 'Failed assert: %r == %r' % (expected, actual)
2026-11-04T02:45:41.1923742Z         if unexpected:
2026-11-04T02:45:41.1924084Z           msg = msg + ', unexpected elements %r' % unexpected
2026-11-04T02:45:41.1924465Z         if expected_list:
2026-11-04T02:45:41.1924807Z           msg = msg + ', missing elements %r' % expected_list
2026-11-04T02:45:41.1925196Z >       raise BeamAssertException(msg)
2026-11-04T02:45:41.1926119Z E       apache_beam.testing.util.BeamAssertException: Failed assert: ['value5', 'value1'] == ['value1', 'value3'], unexpected elements ['value3'], missing elements ['value5'] [while running 'assert_that/Match']
2026-11-04T02:45:41.1926909Z 
2026-11-04T02:45:41.1927363Z target/.tox-py314-cloud/py314-cloud/lib/python3.14/site-packages/apache_beam/testing/util.py:205: BeamAssertException
2026-11-04T02:45:41.1928126Z ------------------------------ Captured log call -------------------------------

tvalentyn · 2026-11-04T17:08:21Z

R: @Abacn

github-actions · 2026-11-04T17:09:06Z

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

tvalentyn and others added 2 commits November 2, 2026 17:38

Authenticate before pushing.

14bb487

Update Python Dependencies

ac42282

github-actions Bot added the python label Nov 3, 2026

github-actions Bot requested a review from claudevdm November 3, 2026 03:11

github-actions Bot added the Next Action: Reviewers label Nov 3, 2026

derrickaw mentioned this pull request Nov 3, 2026

Authenticate before pushing. #40959

Merged

Add license for arro3-core

64db60f

github-actions Bot added build docker size/xl XL: >=1000 lines changed labels Nov 4, 2026

lints

a413b75

tvalentyn reviewed Nov 4, 2026

View reviewed changes

tvalentyn mentioned this pull request Nov 4, 2026

[release-2.78.0] Update pillow version to remediate some known vulnerabitlites and reduce the vuln checker alerts. #40996

Merged

tvalentyn approved these changes Nov 4, 2026

View reviewed changes

tvalentyn merged commit 579b6c1 into master Nov 4, 2026
90 of 91 checks passed

tvalentyn deleted the weekly_update_python_dependencies_1793674144 branch November 4, 2026 17:33

tvalentyn mentioned this pull request Nov 5, 2026

Fix flaky PipelineBasedStreamingInsertTest.test_failure_in_some_rows_does_not_duplicate #41004

Open

Navigation Menu

Repository navigation

Update Python Dependencies - #40960

Update Python Dependencies#40960
tvalentyn merged 4 commits into
masterfrom
weekly_update_python_dependencies_1793674144

github-actions Bot commented Nov 3, 2026

Uh oh!

github-actions Bot commented Nov 3, 2026

Uh oh!

tvalentyn commented Nov 4, 2026

Uh oh!

tvalentyn Nov 4, 2026 •

edited

Loading

Uh oh!

tvalentyn commented Nov 4, 2026

Uh oh!

tvalentyn commented Nov 4, 2026

Uh oh!

github-actions Bot commented Nov 4, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

github-actions Bot commented Nov 3, 2026

Uh oh!

github-actions Bot commented Nov 3, 2026

Uh oh!

tvalentyn commented Nov 4, 2026

Uh oh!

tvalentyn Nov 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tvalentyn commented Nov 4, 2026

Uh oh!

tvalentyn commented Nov 4, 2026

Uh oh!

github-actions Bot commented Nov 4, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

tvalentyn Nov 4, 2026 •

edited

Loading