Skip to content

Update python#1465

Open
renovate-bot wants to merge 1 commit into
GoogleCloudPlatform:mainfrom
renovate-bot:renovate/python
Open

Update python#1465
renovate-bot wants to merge 1 commit into
GoogleCloudPlatform:mainfrom
renovate-bot:renovate/python

Conversation

@renovate-bot

@renovate-bot renovate-bot commented Sep 23, 2024

Copy link
Copy Markdown
Contributor

ℹ️ Note

This PR body was truncated due to platform limits.

This PR contains the following updates:

Package Change Age Confidence
Werkzeug (changelog) ==3.1.6==3.1.8 age confidence
accelerate ==v1.0.1==1.14.0 age confidence
aiohappyeyeballs (changelog) ==2.6.1==2.7.1 age confidence
arxiv ==2.1.3==2.4.1 age confidence
bitsandbytes (changelog) ==0.44.1==0.49.2 age confidence
cachetools (changelog) ==5.5.0==5.5.2 age confidence
certifi ==2024.8.30==2024.12.14 age confidence
charset-normalizer (changelog) ==3.4.0==3.4.7 age confidence
click (changelog) ==8.1.7==8.4.2 age confidence
datasets ==3.1.0==3.6.0 age confidence
einops ==0.8.0==0.8.2 age confidence
elasticsearch ==8.15.0==8.19.3 age confidence
emoji ==2.14.1==2.15.0 age confidence
fast-dash ==0.2.13==0.5.5 age confidence
fastapi (changelog) ==0.114.0==0.139.0 age confidence
fastapi (changelog) ==0.115.0==0.139.0 age confidence
fastapi (changelog) ==0.115.13==0.139.0 age confidence
fastparquet ==2024.5.0==2024.11.0 age confidence
fsspec (changelog) ==2026.3.0==2026.6.0 age confidence
gcsfs ==2024.9.0==2024.12.0 age confidence
google-adk (changelog) ==1.16.0==1.36.1 age confidence
google-api-core ==2.22.0==2.31.0 age confidence
google-api-core ~=2.28.1~=2.31.0 age confidence
google-auth (source) ~=2.34.0~=2.55.1 age confidence
google-auth (source) ==2.27.0==2.55.1 age confidence
google-cloud-access-context-manager (source) ==0.2.1==0.6.0 age confidence
google-cloud-asset (source) ==3.27.0==3.30.1 age confidence
google-cloud-bigquery ==3.29.0==3.42.1 age confidence
google-cloud-core (source) ==2.5.1==2.6.0 age confidence
google-cloud-monitoring (source) ==2.23.0==2.31.0 age confidence
google-cloud-org-policy (source) ==1.11.1==1.18.0 age confidence
google-cloud-os-config (source) ==1.18.0==1.25.0 age confidence
google-cloud-pubsub (source) ~=2.23.0~=2.39.0 age confidence
google-cloud-storage (source) ==2.18.2==2.19.0 age confidence
google-resumable-media (source) ==2.8.2==2.10.0 age confidence
google_adk (changelog) ==1.2.1==1.36.1 age confidence
google_cloud_bigquery ==3.29.0==3.42.1 age confidence
googleapis-common-protos (source) ==1.65.0==1.75.0 age confidence
gradio ==6.10.0==6.19.0 age confidence
grpc-google-iam-v1 ==0.13.1==0.14.4 age confidence
grpcio ==1.62.3==1.81.1 age confidence
grpcio-status ==1.62.3==1.81.1 age confidence
huggingface_hub ==0.24.7==0.36.2 age confidence
k8s-agent-sandbox ==0.4.6==0.5.0 age confidence
langchain-weaviate ==0.0.2==0.0.7 age confidence
llmcompressor ==0.8.1==0.12.0 age confidence
mlflow ==3.11.0rc1==3.14.0 age confidence
numpy (changelog) ==2.4.4==2.5.1 age confidence
pandas ==2.2.2==2.3.3 age confidence
peft ==v0.13.2==0.19.1 age confidence
pgvector ==0.3.2==0.5.0 age confidence
propcache ==0.4.1==0.5.2 age confidence
proto-plus ==1.25.0==1.28.0 age confidence
psutil ==6.1.0==6.1.1 age confidence
psycopg2-binary (source, changelog) ==2.9.9==2.9.12 age confidence
pyasn1-modules (changelog) ==0.4.1==0.4.2 age confidence
pyasn1_modules (changelog) ==0.4.1==0.4.2 age confidence
pycparser ==2.22==2.23 age confidence
pydantic (changelog) ==2.11.7==2.13.4 age confidence
pymupdf (changelog) ==1.24.10==1.28.0 age confidence
pytz ==2026.1.post1==2026.2 age confidence
qdrant-client ==1.11.1==1.18.0 age confidence
redis (changelog) ==5.0.8==5.3.1 age confidence
requests (changelog) ==2.33.0==2.34.2 age confidence
requests (changelog) ==2.33.1==2.34.2 age confidence
rsa ==4.9==4.9.1 age confidence
ruff (source, changelog) >=0.6,<=0.6.4>=0.15,<=0.15.20 age confidence
scikit-learn (changelog) ==1.5.1==1.9.0 age confidence
scipy ==1.17.1==1.18.0 age confidence
six ==1.16.0==1.17.0 age confidence
streamlit (changelog) ==1.54.0==1.59.0 age confidence
tensorflow ==2.17.0==2.21.0 age confidence
thejsonlogger ==0.0.3==0.0.5 age confidence
transformers ==5.0.0==5.13.0 age confidence
transformers ==v4.46.1==4.57.6 age confidence
trl ==v0.11.4==0.29.1 age confidence
typing-extensions (changelog) ==4.15.0==4.16.0 age confidence
uvicorn (changelog) ==0.30.6==0.50.2 age confidence
uvicorn (changelog) ==0.34.0==0.50.2 age confidence
weaviate-client ==4.7.1==4.22.0 age confidence
yarl ==1.23.0==1.24.2 age confidence

Warning

Some dependencies could not be looked up. Check the Dependency Dashboard for more information.


Release Notes

pallets/werkzeug (Werkzeug)

v3.1.8

Compare Source

Released 2026-04-02

  • Request.host and get_host return the empty string if the header is
    missing or has invalid characters. :issue:3142

v3.1.7

Compare Source

Released 2026-03-23

  • parse_list_header preserves partially quoted items, discards empty
    items, and returns empty for unclosed quoted values. :pr:3128
  • WWWAuthenticate.to_header does not produce a trailing space when there
    are no parameters. :issue:3127
  • Transfer-Encoding is parsed as a set. :pr:3134
  • Request.host, get_host, and host_is_trusted validate the
    characters of the value. An empty value is no longer allowed. A Unix socket
    server address is ignored. The trusted_list argument to
    host_is_trusted is optional. :pr:3113
  • Fix multipart form parser handling of newline at boundary. :issue:3088
  • Response.make_conditional sets the Accept-Ranges header even if it
    is not a satisfiable range request. :issue:3108
  • merge_slashes merges any number of consecutive slashes. :issue:3121
huggingface/accelerate (accelerate)

v1.14.0: : AMD ROCm support, FSDP2 hardening

Compare Source

FSDP2 Improvements

This release brings a large batch of FSDP2 fixes and quality-of-life improvements: correct dtype handling on load, sharding of embeddings/norms, QLoRA crash prevention, and a more robust auto-wrap policy.

AMD ROCm support

Accelerate now works end-to-end on AMD ROCm devices. Thanks @​Abdennacer-Badaoui!

Neuron

Further Neuron improvements to reduce recompilation and cover missing device cases.

Quantization & Offloading

We improved offloading support for quantized models, including Torchao, int8, and tied-weight handling.

Data Loading

  • Feat: Support dynamic batch size in BatchSamplerShard with even_batches by @​yuxinyuan in #​3969
  • Fix iterable dataset sharding condition when n_shards == num_processes by @​SunMarc in #​3958
  • Fix implicit padding in split_between_processes when apply_padding=False and num_samples < num_processes by @​3manifold in #​4052

Minor fixes

Full Changelog: huggingface/accelerate@v1.13.0...v1.14.0

v1.13.0: : Neuron support, IPEX removal, and distributed training fixes

Compare Source

AWS Neuron support

We now have support for AWS Neuron (Trainium/Inferentia) devices. Thanks @​michaelbenayoun for adding this.

XPU Improvements

We've removed IPEX dependency and improved device-agnostic code for XPU.

FSDP2 Improvements

We've added a bunch of important fixes for FSDP2 users: upcasting only grad-requiring params, better tied embedding errors, DCP optimizer loading, bf16 optimizer step crash fix, and torch < 2.7.0 compatibility.

DeepSpeed Sequence Parallelism

We've added several fixes to the DeepSpeed + Sequence Parallelism integration introduced in v1.12.0, including evaluation support during SP training and proper process group handling.

FP8

We've enhanced FP8 training. Thanks @​shimizust for fixing torchao support.

Performance

Accelerate now imports faster by deferring heavy dependencies, and torch.compile hooks are disabled lazily.

Minor fixes

v1.12.0: : Deepspeed Ulysses/ALST

Compare Source

Deepspeed Ulysses/ALST integration

Deepspeed Ulysses/ALST is an efficient way of training on long sequences by employing sequence parallelism and attention head parallelism. You can learn more about this technology in this paper https://arxiv.org/abs/2506.13996 or this deepspeed tutorial https://www.deepspeed.ai/tutorials/ulysses-alst-sequence-parallelism/.

0d8bd9e0

To enable Deepspeed Ulysses, you first need to create ParallelismConfig and setting sp related args:

parallelism_config = ParallelismConfig(
    sp_backend="deepspeed",
    sp_size=2,
    sp_handler=DeepSpeedSequenceParallelConfig(...),
)

Then, you need to make sure to compute the correct loss as described on our docs

        ...
        losses_per_rank = torch.distributed.nn.functional.all_gather(loss, group=sp_group)
        good_tokens = (shift_labels != -100).view(-1).sum()
        good_tokens_per_rank = torch.distributed.nn.functional.all_gather(good_tokens, group=sp_group)
        total_loss = sum(
            losses_per_rank[rank] * good_tokens_per_rank[rank]
            for rank in range(sp_world_size)
            if good_tokens_per_rank[rank] > 0
        )
        total_good_tokens = sum(good_tokens_per_rank)
        loss = total_loss / max(total_good_tokens, 1)

Thanks @​S1ro1 for starting this work and for @​stas00 for finishing this work. Also thanks @​kashif for adding docs and reviewing/testing this PR !

This feature will also be available in HF Trainer thanks for this PR from @​stas00: huggingface/transformers#41832

Minor changes

New Contributors

Full Changelog: huggingface/accelerate@v1.11.0...v1.12.0

v1.11.0: : TE MXFP8, FP16/BF16 with MPS, Python 3.10

Compare Source

TE MXFP8 support

We've added support for MXFP8 in our TransformerEngine integration. To use that, you need to set use_mxfp8_block_scaling in fp8_config. See nvidia docs [here]. (https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/examples/fp8_primer.html#MXFP8-and-block-scaling)

FP16/BF16 Training for MPS devices

BF16 and FP16 support for MPS devices is finally here. You can now pass mixed_precision = "fp16" or "bf16" when training on a mac (fp16 requires torch 2.8 and bf16 requires torch 2.6)

FSDP updates

The following PRs add respectively support to ignored_params and no_sync() for FSDPv2:

Mixed precision can now be passed as a dtype string from accelerate cli flag or fsdp_config in accelerate config file:

Nd-parallel updates

Some minor updates concerning nd-parallelism.

Bump to Python 3.10

We've dropped support for python 3.9 as it reached EOL in October.

Lots of minor fixes:

Note

PR body was truncated to here.

@renovate-bot renovate-bot requested a review from a team as a code owner September 23, 2024 00:47
@forking-renovate forking-renovate Bot added the lang: python Issues specific to Python. label Sep 23, 2024
@olivi-eh

Copy link
Copy Markdown
Collaborator

Looks like most of the issues look like this:

6.059 The conflict is caused by:
6.059     langchain 0.3.0 depends on langchain-core<0.4.0 and >=0.3.0
6.059     langchain-community 0.3.0 depends on langchain-core<0.4.0 and >=0.3.0
6.059     langchain-google-vertexai 1.0.10 depends on langchain-core<0.3 and >=0.2.33

Might need to manually bump langchain-google-vertexai, latest is 2.0.1

@renovate-bot renovate-bot force-pushed the renovate/python branch 7 times, most recently from 2e4015a to 9815aef Compare September 26, 2024 18:40
@renovate-bot renovate-bot force-pushed the renovate/python branch 15 times, most recently from 89c302d to b8a5279 Compare October 4, 2024 03:20
@renovate-bot renovate-bot force-pushed the renovate/python branch 12 times, most recently from 096a5d6 to 9beb11e Compare October 17, 2024 00:55
@renovate-bot renovate-bot force-pushed the renovate/python branch 10 times, most recently from c7294de to 089f9c7 Compare October 24, 2024 17:06
@renovate-bot renovate-bot force-pushed the renovate/python branch 6 times, most recently from 16286e7 to 5492174 Compare October 28, 2024 16:18
@forking-renovate

forking-renovate Bot commented Jul 4, 2025

Copy link
Copy Markdown

⚠️ Artifact update problem

Renovate failed to update an artifact related to this branch. You probably do not want to merge this PR as-is.

♻ Renovate will retry this branch, including artifacts, only when one of the following happens:

  • any of the package files in this branch needs updating, or
  • the branch becomes conflicted, or
  • you click the rebase/retry checkbox if found above, or
  • you rename this PR's title to start with "rebase!" to trigger it manually

The artifact failure details are included below:

File name: cost-optimization/hpa-config-recommender/requirements.txt
Command failed: pip-compile --generate-hashes requirements.in
  ERROR: Cannot install -r requirements.in (line 10), -r requirements.in (line 11), -r requirements.in (line 12), -r requirements.in (line 13), -r requirements.in (line 15), -r requirements.in (line 19), -r requirements.in (line 5), -r requirements.in (line 7), -r requirements.in (line 8), -r requirements.in (line 9) and protobuf==5.29.6 because these package versions have conflicting dependencies.
Discarding protobuf==5.29.6 (from -r requirements.txt (line 1181)) to proceed the resolution
  ERROR: Cannot install -r requirements.in (line 10), -r requirements.in (line 11), -r requirements.in (line 12), -r requirements.in (line 13), -r requirements.in (line 15), -r requirements.in (line 19), -r requirements.in (line 5), -r requirements.in (line 7), -r requirements.in (line 8), -r requirements.in (line 9) and protobuf==5.29.6 because these package versions have conflicting dependencies.
Traceback (most recent call last):
  File "/opt/containerbase/tools/pip-tools/7.5.3/3.11.15/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 99, in resolve
    result = self._result = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
  File "/opt/containerbase/tools/pip-tools/7.5.3/3.11.15/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers/resolution.py", line 601, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/containerbase/tools/pip-tools/7.5.3/3.11.15/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers/resolution.py", line 542, in resolve
    raise ResolutionImpossible(self.state.backtrack_causes)
pip._vendor.resolvelib.resolvers.exceptions.ResolutionImpossible: [RequirementInformation(requirement=SpecifierRequirement('protobuf==5.29.6'), parent=None), RequirementInformation(requirement=SpecifierRequirement('protobuf<8.0.0,>=5.29.6'), parent=LinkCandidate('https://files.pythonhosted.org/packages/86/40/9bdbb60b03a332bd45acb8703da08bbc27d991d35286b62e42acc86d243a/google_api_core-2.31.0-py3-none-any.whl (from https://pypi.org/simple/google-api-core/) (requires-python:>=3.10)')), RequirementInformation(requirement=SpecifierRequirement('protobuf<8.0.0,>=4.25.8'), parent=LinkCandidate('https://files.pythonhosted.org/packages/e7/c8/e2645aa8ed02fd4c7a2f59d68783b65b1f3cbdfe39a6308e156509d1fee8/googleapis_common_protos-1.75.0-py3-none-any.whl (from https://pypi.org/simple/googleapis-common-protos/) (requires-python:>=3.9)')), RequirementInformation(requirement=SpecifierRequirement('protobuf<8.0.0,>=4.25.8'), parent=LinkCandidate('https://files.pythonhosted.org/packages/7c/20/b122d4626976acb81132036d2ad1bb35a1a8775fceb837ec30964622516a/proto_plus-1.28.0-py3-none-any.whl (from https://pypi.org/simple/proto-plus/) (requires-python:>=3.10)')), RequirementInformation(requirement=SpecifierRequirement('protobuf<8.0.0,>=4.25.8'), parent=LinkCandidate('https://files.pythonhosted.org/packages/c6/13/47719751cc2864939baea3a0c8e23351706392f1efd05c55dfc4cd6e4a34/google_cloud_access_context_manager-0.6.0-py3-none-any.whl (from https://pypi.org/simple/google-cloud-access-context-manager/) (requires-python:>=3.9)')), RequirementInformation(requirement=SpecifierRequirement('protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<7.0.0,>=3.20.2'), parent=LinkCandidate('https://files.pythonhosted.org/packages/78/69/93134bd9f355abbb8f7341b9781d4d74d956e35b0c0f1702b838d2b0c498/google_cloud_asset-3.30.1-py3-none-any.whl (from https://pypi.org/simple/google-cloud-asset/) (requires-python:>=3.7)')), RequirementInformation(requirement=SpecifierRequirement('protobuf<8.0.0,>=4.25.8'), parent=LinkCandidate('https://files.pythonhosted.org/packages/7f/00/3a585409ea6be01ccd0b56adc1798e74c25da7e2a4dbb7c2a2f62856a4c0/google_cloud_org_policy-1.18.0-py3-none-any.whl (from https://pypi.org/simple/google-cloud-org-policy/) (requires-python:>=3.10)')), RequirementInformation(requirement=SpecifierRequirement('protobuf<8.0.0,>=4.25.8'), parent=LinkCandidate('https://files.pythonhosted.org/packages/63/ba/b017ffb5aa22e8f796790f27dd7e0bf19d037b037b9f489b3bbe31e718ad/google_cloud_os_config-1.25.0-py3-none-any.whl (from https://pypi.org/simple/google-cloud-os-config/) (requires-python:>=3.10)')), RequirementInformation(requirement=SpecifierRequirement('protobuf<8.0.0,>=4.25.8'), parent=LinkCandidate('https://files.pythonhosted.org/packages/89/22/c2dd50c09bf679bd38173656cd4402d2511e563b33bc88f90009cf50613c/grpc_google_iam_v1-0.14.4-py3-none-any.whl (from https://pypi.org/simple/grpc-google-iam-v1/) (requires-python:>=3.9)')), RequirementInformation(requirement=SpecifierRequirement('protobuf<8.0.0,>=4.25.8'), parent=LinkCandidate('https://files.pythonhosted.org/packages/55/30/aa6635296da9c1c14d2e64f64e1cacd4f4debf8ab7e646c0559545f0f70d/google_cloud_monitoring-2.31.0-py3-none-any.whl (from https://pypi.org/simple/google-cloud-monitoring/) (requires-python:>=3.10)')), RequirementInformation(requirement=SpecifierRequirement('protobuf<8.0.0,>=6.33.5'), parent=LinkCandidate('https://files.pythonhosted.org/packages/e5/5e/5abfec5f7e89d3b7993d57cfb025ca5f968a2c18656d7fcda2b6919440b9/grpcio_status-1.81.1-py3-none-any.whl (from https://pypi.org/simple/grpcio-status/) (requires-python:>=3.10)'))]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/containerbase/tools/pip-tools/7.5.3/3.11.15/bin/pip-compile", line 6, in <module>
    sys.exit(cli())
             ^^^^^
  File "/opt/containerbase/tools/pip-tools/7.5.3/3.11.15/lib/python3.11/site-packages/click/core.py", line 1569, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/containerbase/tools/pip-tools/7.5.3/3.11.15/lib/python3.11/site-packages/click/core.py", line 1490, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/containerbase/tools/pip-tools/7.5.3/3.11.15/lib/python3.11/site-packages/click/core.py", line 1353, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/containerbase/tools/pip-tools/7.5.3/3.11.15/lib/python3.11/site-packages/click/core.py", line 907, in invoke
    return callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/containerbase/tools/pip-tools/7.5.3/3.11.15/lib/python3.11/site-packages/click/decorators.py", line 34, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/containerbase/tools/pip-tools/7.5.3/3.11.15/lib/python3.11/site-packages/piptools/scripts/compile.py", line 481, in cli
    results = resolver.resolve(max_rounds=max_rounds)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/containerbase/tools/pip-tools/7.5.3/3.11.15/lib/python3.11/site-packages/piptools/resolver.py", line 642, in resolve
    is_resolved = self._do_resolve(
                  ^^^^^^^^^^^^^^^^^
  File "/opt/containerbase/tools/pip-tools/7.5.3/3.11.15/lib/python3.11/site-packages/piptools/resolver.py", line 677, in _do_resolve
    resolver.resolve(
  File "/opt/containerbase/tools/pip-tools/7.5.3/3.11.15/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 108, in resolve
    raise error from e
pip._internal.exceptions.DistributionNotFound: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies lang: python Issues specific to Python.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants