[Feature] Build for pcc64el

[Drzhivago264]

Motivation

Hello,
I just got my hand on some v100 on the old IBM power 9 system.
I would appreciate if lmdeploy can be built on the machine.
Can anyone point me to the correct direction to pull this off.
I appreciate your help.

Related resources

No response

Additional context

No response

[windreamer]

For curiosity， how did you drive V100 on PPC? Does any nvidia driver ready in PPC?

And if you just need to run PyTorch Engine, I think you just need to make triton work on your platform.

[Drzhivago264]

I seem to me that IBM came up with a server call AC922 with 6 V100 SXM2 sockets a decade ago. At the moment, Power 11 is still a big thing in corporate environment, I know some dudes threw the whole trading core into an Oracle DB on pcc64el machine and it beats AMD and Intel chip by a wide margin. If you take the bet and go the extra mile, you can have a Power 9 with 6 x V100 for the price of a scooter now.

There are pytorch, cuda and ubuntu builds for it. Vllm recently came up with a pcc64el release, yet getting vllm to work with v100 is quite tricky.

The Triton part seems to be a problem, I will try it later today.

*Edit: just check the tritton part, there seems to be a working solution, I believe following lmdeploy-jeson step may get the thing going.

[windreamer]

OK. Looking forward for your results. I believe you can follow the standard build process for ppc. And if you can not make TurboMind working, you can try to use DISABLE_TURBOMIND env variable to get a workaround.

[windreamer]

You may try out this patch to enable TurboMind compilation in ppc64le:

--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -155,9 +155,9 @@ endif()
 set(CXX_STD "17" CACHE STRING "C++ standard")
 # enable gold linker for binary and .so
 if(NOT MSVC)
-  find_program(GOLD_PATH ld.gold REQUIRED)
+  find_program(GOLD_PATH ld.gold)
   if(NOT GOLD_PATH)
-    message(FATAL_ERROR "GNU gold linker is required but not found. "
+    message(WARNING "GNU gold linker not found. "
                          "Please install binutils-gold package.")
   endif()
   set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fuse-ld=gold")
@@ -232,6 +232,24 @@ elseif(ARCH STREQUAL "aarch64")
   if (NOT CMAKE_CUDA_ARCHITECTURES)
     set(CMAKE_CUDA_ARCHITECTURES 72-real 87-real)  # Jetson
   endif()
+elseif(ARCH STREQUAL "ppc64le")
+  if (NOT CMAKE_CUDA_ARCHITECTURES)
+    set(CMAKE_CUDA_ARCHITECTURES "")
+    # IBM PowerPC GPUs: V100, A100, A30, H100
+    if (${CMAKE_CUDA_COMPILER_VERSION} VERSION_GREATER_EQUAL "10")
+      list(APPEND CMAKE_CUDA_ARCHITECTURES 70-real)  # V100
+    endif()
+    if (${CMAKE_CUDA_COMPILER_VERSION} VERSION_GREATER_EQUAL "11")
+      list(APPEND CMAKE_CUDA_ARCHITECTURES 80-real)  # A100
+    endif()
+    if (${CMAKE_CUDA_COMPILER_VERSION} VERSION_GREATER_EQUAL "11.1")
+      list(APPEND CMAKE_CUDA_ARCHITECTURES 86-real)  # A30
+    endif()
+    if (${CMAKE_CUDA_COMPILER_VERSION} VERSION_GREATER_EQUAL "11.8")
+      list(APPEND CMAKE_CUDA_ARCHITECTURES 90-real)  # H100
+    endif()
+  endif()
 else()
   message(FATAL_ERROR "Unsupported Architecture:" ${ARCH})
 endif()

[Drzhivago264]

Update, I managed to install all the dependencies of internLM except for triton. Now I will try to compile triton from source.
In order to get here I have done:

1. Install cuda 12.2 + driver 535 from official Nvidia library
2. Install missing dependencies from https://github.com/open-ce:
   2.1 conda python=3.10 => python=3.11
   2.2 ray -> conda install -c https://ftp.osuosl.org/pub/open-ce/current/ ray-all
   2.3 torch -> conda install -c https://ftp.osuosl.org/pub/open-ce/current/ pytorch
   2.4 datasets -> conda install -c https://ftp.osuosl.org/pub/open-ce/current/ datasets
   2.5 pyarrow -> conda install -c https://ftp.osuosl.org/pub/open-ce/current/ pyarrow
   2.6 tiktoken -> conda install -c https://ftp.osuosl.org/pub/open-ce/current/ titoken
   2.7 pydantic -> conda install -c https://ftp.osuosl.org/pub/open-ce/current/ pydantic
   2.8 xgrammar -> pip install xgrammar==0.1.14 (newer version fail to build)

pip install git+https://github.com/InternLM/lmdeploy.git
Collecting git+https://github.com/InternLM/lmdeploy.git
Cloning https://github.com/InternLM/lmdeploy.git to /tmp/pip-req-build-xdd2in1t
Running command git clone --filter=blob:none --quiet https://github.com/InternLM/lmdeploy.git /tmp/pip-req-build-xdd2in1t
Resolved https://github.com/InternLM/lmdeploy.git to commit 958acbd
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Collecting accelerate>=0.29.3 (from lmdeploy==0.12.0)
Using cached accelerate-1.12.0-py3-none-any.whl.metadata (19 kB)
Requirement already satisfied: aiohttp in /home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages (from lmdeploy==0.12.0) (3.10.2)
Requirement already satisfied: cloudpickle in /home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages (from lmdeploy==0.12.0) (2.2.1)
Collecting einops (from lmdeploy==0.12.0)
Using cached einops-0.8.2-py3-none-any.whl.metadata (13 kB)
Requirement already satisfied: fastapi in /home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages (from lmdeploy==0.12.0) (0.109.1)
Collecting fire (from lmdeploy==0.12.0)
Using cached fire-0.7.1-py3-none-any.whl.metadata (5.8 kB)
Collecting mmengine-lite (from lmdeploy==0.12.0)
Using cached mmengine_lite-0.10.7-py3-none-any.whl.metadata (20 kB)
Requirement already satisfied: numpy in /home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages (from lmdeploy==0.12.0) (1.26.0)
Collecting openai (from lmdeploy==0.12.0)
Using cached openai-2.17.0-py3-none-any.whl.metadata (29 kB)
Collecting openai_harmony (from lmdeploy==0.12.0)
Using cached openai_harmony-0.0.8-cp38-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.metadata (8.0 kB)
Collecting partial_json_parser (from lmdeploy==0.12.0)
Using cached partial_json_parser-0.2.1.1.post7-py3-none-any.whl.metadata (6.1 kB)
Collecting peft<=0.14.0 (from lmdeploy==0.12.0)
Using cached peft-0.14.0-py3-none-any.whl.metadata (13 kB)
Requirement already satisfied: pillow in /home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages (from lmdeploy==0.12.0) (10.0.1)
Requirement already satisfied: prometheus_client in /home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages (from lmdeploy==0.12.0) (0.14.1)
Requirement already satisfied: protobuf in /home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages (from lmdeploy==0.12.0) (4.21.12)
Collecting pybase64 (from lmdeploy==0.12.0)
Using cached pybase64-1.4.3-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl.metadata (8.7 kB)
Requirement already satisfied: pydantic>2.0.0 in /home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages (from lmdeploy==0.12.0) (2.5.3)
Collecting pyzmq (from lmdeploy==0.12.0)
Using cached pyzmq-27.1.0.tar.gz (281 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: ray in /home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages (from lmdeploy==0.12.0) (2.35.0)
Collecting safetensors (from lmdeploy==0.12.0)
Using cached safetensors-0.7.0-cp38-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.metadata (4.1 kB)
Requirement already satisfied: sentencepiece in /home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages (from lmdeploy==0.12.0) (0.1.99)
Collecting shortuuid (from lmdeploy==0.12.0)
Using cached shortuuid-1.0.13-py3-none-any.whl.metadata (5.8 kB)
Collecting tiktoken (from lmdeploy==0.12.0)
Using cached tiktoken-0.12.0.tar.gz (37 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: torch<=2.8.0,>=2.0.0 in /home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages (from lmdeploy==0.12.0) (2.1.2)
Requirement already satisfied: torchvision<=0.23.0,>=0.15.0 in /home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages (from lmdeploy==0.12.0) (0.16.2)
Collecting transformers<5.0.0 (from lmdeploy==0.12.0)
Downloading transformers-4.57.6-py3-none-any.whl.metadata (43 kB)
INFO: pip is looking at multiple versions of lmdeploy to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement triton<=3.4.0,>=3.0.0; sys_platform == "linux" and "aarch64" not in platform_machine and "arm" not in platform_machine (from lmdeploy) (from versions: none)
ERROR: No matching distribution found for triton<=3.4.0,>=3.0.0; sys_platform == "linux" and "aarch64" not in platform_machine and "arm" not in platform_machine

[windreamer]

You can try this to remove triton from lmdeploy dependencies:

diff --git a/requirements/runtime_cuda.txt b/requirements/runtime_cuda.txt
index 03376695..0ced5821 100644
--- a/requirements/runtime_cuda.txt
+++ b/requirements/runtime_cuda.txt
@@ -24,6 +24,6 @@ tiktoken
 torch<=2.8.0,>=2.0.0
 torchvision<=0.23.0,>=0.15.0
 transformers
-triton<=3.4.0,>=3.0.0; sys_platform == "linux" and "aarch64" not in platform_machine and "arm" not in platform_machine
+triton<=3.4.0,>=3.0.0; sys_platform == "linux" and "aarch64" not in platform_machine and "arm" not in platform_machine and platform_machine != "ppc64le"
 uvicorn
 xgrammar

[windreamer]

I just launched a docker with qemu to emulate a ppc64le environment, and I am using this patch:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index f29eae90..9e061957 100755
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -161,9 +161,9 @@ endif()
 set(CXX_STD "17" CACHE STRING "C++ standard")
 # enable gold linker for binary and .so
 if(NOT MSVC)
-  find_program(GOLD_PATH ld.gold REQUIRED)
+  find_program(GOLD_PATH ld.gold)
   if(NOT GOLD_PATH)
-    message(FATAL_ERROR "GNU gold linker is required but not found. "
+         message(WARNING "GNU gold linker is required but not found. "
                          "Please install binutils-gold package.")
   endif()
   set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fuse-ld=gold")
@@ -252,6 +252,23 @@ elseif(ARCH STREQUAL "aarch64")
   if (NOT CMAKE_CUDA_ARCHITECTURES)
     set(CMAKE_CUDA_ARCHITECTURES 72-real 87-real)  # Jetson
   endif()
+elseif(ARCH STREQUAL "ppc64le")
+  if (NOT CMAKE_CUDA_ARCHITECTURES)
+    set(CMAKE_CUDA_ARCHITECTURES "")
+    # IBM PowerPC GPUs: V100, A100, A30, H100
+    if (${CMAKE_CUDA_COMPILER_VERSION} VERSION_GREATER_EQUAL "10")
+      list(APPEND CMAKE_CUDA_ARCHITECTURES 70-real)  # V100
+    endif()
+    if (${CMAKE_CUDA_COMPILER_VERSION} VERSION_GREATER_EQUAL "11")
+      list(APPEND CMAKE_CUDA_ARCHITECTURES 80-real)  # A100
+    endif()
+    if (${CMAKE_CUDA_COMPILER_VERSION} VERSION_GREATER_EQUAL "11.1")
+      list(APPEND CMAKE_CUDA_ARCHITECTURES 86-real)  # A30
+    endif()
+    if (${CMAKE_CUDA_COMPILER_VERSION} VERSION_GREATER_EQUAL "11.8")
+      list(APPEND CMAKE_CUDA_ARCHITECTURES 90-real)  # H100
+    endif()
+  endif()
 else()
   message(FATAL_ERROR "Unsupported Architecture:" ${ARCH})
 endif()
diff --git a/requirements/runtime_cuda.txt b/requirements/runtime_cuda.txt
index c55b6646..3695e91d 100644
--- a/requirements/runtime_cuda.txt
+++ b/requirements/runtime_cuda.txt
@@ -16,7 +16,7 @@ protobuf
 pybase64
 pydantic>2.0.0
 pyzmq
-ray
+ray>=2.0; platform_machine != "ppc64le"
 safetensors
 sentencepiece
 shortuuid
@@ -24,6 +24,6 @@ tiktoken
 torch<=2.8.0,>=2.0.0
 torchvision<=0.23.0,>=0.15.0
 transformers<5.0.0
-triton<=3.4.0,>=3.0.0; sys_platform == "linux" and "aarch64" not in platform_machine and "arm" not in platform_machine
+triton<=3.4.0,>=3.0.0; sys_platform == "linux" and "aarch64" not in platform_machine and "arm" not in platform_machine and platform_machine != "ppc64le"
 uvicorn
 xgrammar

I think I can build a wheel without turbomind, but haven't tested if the wheel is good. And for turbomind, I got strange 'signal 11 (Invalid memory reference)'. I think this is due to qemu is not a real machine.

[Drzhivago264]

Thank you, it seems to work. I also provided some more tricks to get around dependency problems as per my previous comment. I will test it further tonight. Just a side question, how different does triton and turbomind make to inference speeds?

Do you get this kind of output?

``
DISABLE_TURBOMIND=1 python setup.py build
/home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages/setuptools/init.py:85: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!

    ********************************************************************************
    Requirements should be satisfied by a PEP 517 installer.
    If you are using pip, you can try `pip install --use-pep517`.
    ********************************************************************************

!!
dist.fetch_build_eggs(dist.setup_requires)
/home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages/setuptools/_distutils/dist.py:268: UserWarning: Unknown distribution option: 'tests_require'
warnings.warn(msg)
[02/06/26 10:40:26] WARNING toml section missing PosixPath('pyproject.toml') does not contain a tool.setuptools_scm section pyproject_reading.py:215
running build
running build_py
copying lmdeploy/api.py -> build/lib/lmdeploy
copying lmdeploy/archs.py -> build/lib/lmdeploy
copying lmdeploy/utils.py -> build/lib/lmdeploy
copying lmdeploy/pipeline.py -> build/lib/lmdeploy
copying lmdeploy/tokenizer.py -> build/lib/lmdeploy
copying lmdeploy/version.py -> build/lib/lmdeploy
copying lmdeploy/main.py -> build/lib/lmdeploy
copying lmdeploy/profiler.py -> build/lib/lmdeploy
copying lmdeploy/messages.py -> build/lib/lmdeploy
copying lmdeploy/init.py -> build/lib/lmdeploy
copying lmdeploy/logger.py -> build/lib/lmdeploy
copying lmdeploy/model.py -> build/lib/lmdeploy
copying lmdeploy/turbomind/turbomind.py -> build/lib/lmdeploy/turbomind
copying lmdeploy/turbomind/supported_models.py -> build/lib/lmdeploy/turbomind
copying lmdeploy/turbomind/tokenizer_info.py -> build/lib/lmdeploy/turbomind
copying lmdeploy/turbomind/init.py -> build/lib/lmdeploy/turbomind
copying lmdeploy/metrics/loggers.py -> build/lib/lmdeploy/metrics
copying lmdeploy/metrics/stats.py -> build/lib/lmdeploy/metrics
copying lmdeploy/metrics/init.py -> build/lib/lmdeploy/metrics
copying lmdeploy/metrics/metrics_processor.py -> build/lib/lmdeploy/metrics
copying lmdeploy/serve/init.py -> build/lib/lmdeploy/serve
copying lmdeploy/cli/chat.py -> build/lib/lmdeploy/cli
copying lmdeploy/cli/serve.py -> build/lib/lmdeploy/cli
copying lmdeploy/cli/utils.py -> build/lib/lmdeploy/cli
copying lmdeploy/cli/lite.py -> build/lib/lmdeploy/cli
copying lmdeploy/cli/entrypoint.py -> build/lib/lmdeploy/cli
copying lmdeploy/cli/cli.py -> build/lib/lmdeploy/cli
copying lmdeploy/cli/init.py -> build/lib/lmdeploy/cli
copying lmdeploy/vl/engine.py -> build/lib/lmdeploy/vl
copying lmdeploy/vl/utils.py -> build/lib/lmdeploy/vl
copying lmdeploy/vl/init.py -> build/lib/lmdeploy/vl
copying lmdeploy/vl/constants.py -> build/lib/lmdeploy/vl
copying lmdeploy/pytorch/block.py -> build/lib/lmdeploy/pytorch
copying lmdeploy/pytorch/ray.py -> build/lib/lmdeploy/pytorch
copying lmdeploy/pytorch/model_inputs.py -> build/lib/lmdeploy/pytorch
copying lmdeploy/pytorch/utils.py -> build/lib/lmdeploy/pytorch
copying lmdeploy/pytorch/envs.py -> build/lib/lmdeploy/pytorch
copying lmdeploy/pytorch/distributed.py -> build/lib/lmdeploy/pytorch
copying lmdeploy/pytorch/consts.py -> build/lib/lmdeploy/pytorch
copying lmdeploy/pytorch/config.py -> build/lib/lmdeploy/pytorch
copying lmdeploy/pytorch/messages.py -> build/lib/lmdeploy/pytorch
copying lmdeploy/pytorch/init.py -> build/lib/lmdeploy/pytorch
copying lmdeploy/lite/defaults.py -> build/lib/lmdeploy/lite
copying lmdeploy/lite/init.py -> build/lib/lmdeploy/lite
copying lmdeploy/turbomind/deploy/loader.py -> build/lib/lmdeploy/turbomind/deploy
copying lmdeploy/turbomind/deploy/module.py -> build/lib/lmdeploy/turbomind/deploy
copying lmdeploy/turbomind/deploy/policy.py -> build/lib/lmdeploy/turbomind/deploy
copying lmdeploy/turbomind/deploy/converter.py -> build/lib/lmdeploy/turbomind/deploy
copying lmdeploy/turbomind/deploy/config.py -> build/lib/lmdeploy/turbomind/deploy
copying lmdeploy/turbomind/deploy/parameter.py -> build/lib/lmdeploy/turbomind/deploy
copying lmdeploy/turbomind/deploy/init.py -> build/lib/lmdeploy/turbomind/deploy
copying lmdeploy/turbomind/deploy/target_model/fp.py -> build/lib/lmdeploy/turbomind/deploy/target_model
copying lmdeploy/turbomind/deploy/target_model/base.py -> build/lib/lmdeploy/turbomind/deploy/target_model
copying lmdeploy/turbomind/deploy/target_model/init.py -> build/lib/lmdeploy/turbomind/deploy/target_model
copying lmdeploy/turbomind/deploy/source_model/internlm2.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/turbomind/deploy/source_model/llama.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/turbomind/deploy/source_model/xcomposer2.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/turbomind/deploy/source_model/llava.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/turbomind/deploy/source_model/gpt_oss.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/turbomind/deploy/source_model/deepseek2.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/turbomind/deploy/source_model/mixtral.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/turbomind/deploy/source_model/qwen.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/turbomind/deploy/source_model/minicpmv.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/turbomind/deploy/source_model/glm4.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/turbomind/deploy/source_model/base.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/turbomind/deploy/source_model/molmo.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/turbomind/deploy/source_model/internvl.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/turbomind/deploy/source_model/init.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/turbomind/deploy/source_model/deepseek_vl.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/turbomind/deploy/source_model/baichuan.py -> build/lib/lmdeploy/turbomind/deploy/source_model
copying lmdeploy/serve/managers/session_manager.py -> build/lib/lmdeploy/serve/managers
copying lmdeploy/serve/managers/init.py -> build/lib/lmdeploy/serve/managers
copying lmdeploy/serve/proxy/proxy.py -> build/lib/lmdeploy/serve/proxy
copying lmdeploy/serve/proxy/init.py -> build/lib/lmdeploy/serve/proxy
copying lmdeploy/serve/proxy/constants.py -> build/lib/lmdeploy/serve/proxy
copying lmdeploy/serve/openai/api_client.py -> build/lib/lmdeploy/serve/openai
copying lmdeploy/serve/openai/protocol.py -> build/lib/lmdeploy/serve/openai
copying lmdeploy/serve/openai/launch_server.py -> build/lib/lmdeploy/serve/openai
copying lmdeploy/serve/openai/serving_completion.py -> build/lib/lmdeploy/serve/openai
copying lmdeploy/serve/openai/api_server.py -> build/lib/lmdeploy/serve/openai
copying lmdeploy/serve/openai/serving_generate.py -> build/lib/lmdeploy/serve/openai
copying lmdeploy/serve/openai/init.py -> build/lib/lmdeploy/serve/openai
copying lmdeploy/serve/openai/harmony_utils.py -> build/lib/lmdeploy/serve/openai
copying lmdeploy/serve/openai/serving_chat_completion.py -> build/lib/lmdeploy/serve/openai
copying lmdeploy/serve/processors/multimodal.py -> build/lib/lmdeploy/serve/processors
copying lmdeploy/serve/processors/init.py -> build/lib/lmdeploy/serve/processors
copying lmdeploy/serve/core/exceptions.py -> build/lib/lmdeploy/serve/core
copying lmdeploy/serve/core/async_engine.py -> build/lib/lmdeploy/serve/core
copying lmdeploy/serve/core/vl_async_engine.py -> build/lib/lmdeploy/serve/core
copying lmdeploy/serve/core/init.py -> build/lib/lmdeploy/serve/core
copying lmdeploy/serve/openai/reasoning_parser/qwen_qwq_reasoning_parser.py -> build/lib/lmdeploy/serve/openai/reasoning_parser
copying lmdeploy/serve/openai/reasoning_parser/reasoning_parser.py -> build/lib/lmdeploy/serve/openai/reasoning_parser
copying lmdeploy/serve/openai/reasoning_parser/init.py -> build/lib/lmdeploy/serve/openai/reasoning_parser
copying lmdeploy/serve/openai/reasoning_parser/deepseek_r1_reasoning_parser.py -> build/lib/lmdeploy/serve/openai/reasoning_parser
copying lmdeploy/serve/openai/tool_parser/qwen3_parser.py -> build/lib/lmdeploy/serve/openai/tool_parser
copying lmdeploy/serve/openai/tool_parser/qwen2d5_parser.py -> build/lib/lmdeploy/serve/openai/tool_parser
copying lmdeploy/serve/openai/tool_parser/utils.py -> build/lib/lmdeploy/serve/openai/tool_parser
copying lmdeploy/serve/openai/tool_parser/tool_parser.py -> build/lib/lmdeploy/serve/openai/tool_parser
copying lmdeploy/serve/openai/tool_parser/init.py -> build/lib/lmdeploy/serve/openai/tool_parser
copying lmdeploy/serve/openai/tool_parser/internlm2_parser.py -> build/lib/lmdeploy/serve/openai/tool_parser
copying lmdeploy/serve/openai/tool_parser/llama3_parser.py -> build/lib/lmdeploy/serve/openai/tool_parser
copying lmdeploy/vl/tools/merge_xcomposer2d5_task.py -> build/lib/lmdeploy/vl/tools
copying lmdeploy/vl/tools/init.py -> build/lib/lmdeploy/vl/tools
copying lmdeploy/vl/model/cogvlm.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/qwen2.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/llava_hf.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/phi3_vision.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/internvl_llava.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/xcomposer2.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/llava.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/interns1_pro.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/internvl3_hf.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/qwen.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/qwen3.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/llava_next.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/minicpmv.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/builder.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/utils.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/base.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/gemma3_vl.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/molmo.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/internvl.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/mllama.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/yi.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/init.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/glm4_1v.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/deepseek_vl2.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/deepseek.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/llama4.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/vl/model/glm4_v.py -> build/lib/lmdeploy/vl/model
copying lmdeploy/pytorch/transformers/init.py -> build/lib/lmdeploy/pytorch/transformers
copying lmdeploy/pytorch/transformers/configuration_deepseek_v32.py -> build/lib/lmdeploy/pytorch/transformers
copying lmdeploy/pytorch/nn/attention.py -> build/lib/lmdeploy/pytorch/nn
copying lmdeploy/pytorch/nn/norm.py -> build/lib/lmdeploy/pytorch/nn
copying lmdeploy/pytorch/nn/nsa.py -> build/lib/lmdeploy/pytorch/nn
copying lmdeploy/pytorch/nn/eplb.py -> build/lib/lmdeploy/pytorch/nn
copying lmdeploy/pytorch/nn/embedding.py -> build/lib/lmdeploy/pytorch/nn
copying lmdeploy/pytorch/nn/activation.py -> build/lib/lmdeploy/pytorch/nn
copying lmdeploy/pytorch/nn/multinomial_sampling.py -> build/lib/lmdeploy/pytorch/nn
copying lmdeploy/pytorch/nn/quant_utils.py -> build/lib/lmdeploy/pytorch/nn
copying lmdeploy/pytorch/nn/utils.py -> build/lib/lmdeploy/pytorch/nn
copying lmdeploy/pytorch/nn/rotary_embedding.py -> build/lib/lmdeploy/pytorch/nn
copying lmdeploy/pytorch/nn/init.py -> build/lib/lmdeploy/pytorch/nn
copying lmdeploy/pytorch/tools/utils.py -> build/lib/lmdeploy/pytorch/tools
copying lmdeploy/pytorch/tools/init.py -> build/lib/lmdeploy/pytorch/tools
copying lmdeploy/pytorch/paging/scheduler.py -> build/lib/lmdeploy/pytorch/paging
copying lmdeploy/pytorch/paging/state_manager.py -> build/lib/lmdeploy/pytorch/paging
copying lmdeploy/pytorch/paging/block_trie.py -> build/lib/lmdeploy/pytorch/paging
copying lmdeploy/pytorch/paging/init.py -> build/lib/lmdeploy/pytorch/paging
copying lmdeploy/pytorch/kernels/dispatcher.py -> build/lib/lmdeploy/pytorch/kernels
copying lmdeploy/pytorch/kernels/w8a8_triton_kernels.py -> build/lib/lmdeploy/pytorch/kernels
copying lmdeploy/pytorch/kernels/init.py -> build/lib/lmdeploy/pytorch/kernels
copying lmdeploy/pytorch/adapter/init.py -> build/lib/lmdeploy/pytorch/adapter
copying lmdeploy/pytorch/adapter/adapter.py -> build/lib/lmdeploy/pytorch/adapter
copying lmdeploy/pytorch/configurations/cogvlm.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/minicpm3.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/llava_hf.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/chatglm.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/llama.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/gpt_oss.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/interns1_pro.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/internvl3_hf.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/gemma.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/qwen.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/qwen3_next.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/builder.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/default.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/glm4.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/sdar.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/utils.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/internvl.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/qwen3_vl.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/init.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/deepseek_vl2.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/deepseek_v2.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/configurations/llama4.py -> build/lib/lmdeploy/pytorch/configurations
copying lmdeploy/pytorch/third_party/flash_attn_interface.py -> build/lib/lmdeploy/pytorch/third_party
copying lmdeploy/pytorch/third_party/init.py -> build/lib/lmdeploy/pytorch/third_party
copying lmdeploy/pytorch/models/internlm2.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/interns1_pro_ts.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/cogvlm.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/q_modules.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/qwen2_reward.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/mistral.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/internlm2_ve.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/internvl_patch.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/qwen2.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/minicpm3.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/whisper.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/minicpmv26.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/llama.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/phi3_moe.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/deepseek_v32.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/internlm.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/llava.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/phi3_v.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/gpt_oss.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/interns1_pro.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/qwen2_5_vl.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/internvl3_hf.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/mixtral.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/internlm2_reward.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/gemma.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/qwen.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/qwen3_next.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/qwen3.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/siglip.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/qwen2_moe.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/llama_eagle3.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/glm4.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/sdar.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/patch.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/sdar_moe.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/starcoder2.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/deepseek_mtp.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/gemma3_vl.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/qwen3_vl_moe.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/llama_eagle.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/qwen3_moe.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/internvl.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/qwen2_vl.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/qwen3_vl.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/phi3.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/init.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/glm4_1v.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/internlm3.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/glm4_moe.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/deepseek_vl2.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/module_map.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/baichuan.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/deepseek_v2.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/chatglm2.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/deepseek.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/models/llama4.py -> build/lib/lmdeploy/pytorch/models
copying lmdeploy/pytorch/strategies/init.py -> build/lib/lmdeploy/pytorch/strategies
copying lmdeploy/pytorch/multimodal/data_type.py -> build/lib/lmdeploy/pytorch/multimodal
copying lmdeploy/pytorch/multimodal/image_type.py -> build/lib/lmdeploy/pytorch/multimodal
copying lmdeploy/pytorch/multimodal/init.py -> build/lib/lmdeploy/pytorch/multimodal
copying lmdeploy/pytorch/weight_loader/model_weight_loader.py -> build/lib/lmdeploy/pytorch/weight_loader
copying lmdeploy/pytorch/weight_loader/init.py -> build/lib/lmdeploy/pytorch/weight_loader
copying lmdeploy/pytorch/spec_decode/spec_agent.py -> build/lib/lmdeploy/pytorch/spec_decode
copying lmdeploy/pytorch/spec_decode/base.py -> build/lib/lmdeploy/pytorch/spec_decode
copying lmdeploy/pytorch/spec_decode/reject_sampler.py -> build/lib/lmdeploy/pytorch/spec_decode
copying lmdeploy/pytorch/spec_decode/init.py -> build/lib/lmdeploy/pytorch/spec_decode
copying lmdeploy/pytorch/disagg/config.py -> build/lib/lmdeploy/pytorch/disagg
copying lmdeploy/pytorch/disagg/messages.py -> build/lib/lmdeploy/pytorch/disagg
copying lmdeploy/pytorch/disagg/init.py -> build/lib/lmdeploy/pytorch/disagg
copying lmdeploy/pytorch/devices/init.py -> build/lib/lmdeploy/pytorch/devices
copying lmdeploy/pytorch/devices/device_manager.py -> build/lib/lmdeploy/pytorch/devices
copying lmdeploy/pytorch/engine/guided_process.py -> build/lib/lmdeploy/pytorch/engine
copying lmdeploy/pytorch/engine/engine.py -> build/lib/lmdeploy/pytorch/engine
copying lmdeploy/pytorch/engine/cache_engine.py -> build/lib/lmdeploy/pytorch/engine
copying lmdeploy/pytorch/engine/request.py -> build/lib/lmdeploy/pytorch/engine
copying lmdeploy/pytorch/engine/config_builder.py -> build/lib/lmdeploy/pytorch/engine
copying lmdeploy/pytorch/engine/input_process.py -> build/lib/lmdeploy/pytorch/engine
copying lmdeploy/pytorch/engine/logits_process.py -> build/lib/lmdeploy/pytorch/engine
copying lmdeploy/pytorch/engine/engine_checker.py -> build/lib/lmdeploy/pytorch/engine
copying lmdeploy/pytorch/engine/engine_loop.py -> build/lib/lmdeploy/pytorch/engine
copying lmdeploy/pytorch/engine/base.py -> build/lib/lmdeploy/pytorch/engine
copying lmdeploy/pytorch/engine/engine_instance.py -> build/lib/lmdeploy/pytorch/engine
copying lmdeploy/pytorch/engine/inputs_maker.py -> build/lib/lmdeploy/pytorch/engine
copying lmdeploy/pytorch/engine/init.py -> build/lib/lmdeploy/pytorch/engine
copying lmdeploy/pytorch/backends/flash_attention.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/attention.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/norm.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/moe.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/nsa.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/embedding.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/selector.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/activation.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/qmodules.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/multinomial_sampling.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/blockedf8_modules.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/awq_modules.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/base.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/deepep_moe_checker.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/token_dispatcher.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/rotary_embedding.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/init.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/graph_runner.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/linear.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/apply_rotary_emb.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/backends/lora.py -> build/lib/lmdeploy/pytorch/backends
copying lmdeploy/pytorch/check_env/cuda.py -> build/lib/lmdeploy/pytorch/check_env
copying lmdeploy/pytorch/check_env/deeplink.py -> build/lib/lmdeploy/pytorch/check_env
copying lmdeploy/pytorch/check_env/base.py -> build/lib/lmdeploy/pytorch/check_env
copying lmdeploy/pytorch/check_env/dist.py -> build/lib/lmdeploy/pytorch/check_env
copying lmdeploy/pytorch/check_env/triton.py -> build/lib/lmdeploy/pytorch/check_env
copying lmdeploy/pytorch/check_env/init.py -> build/lib/lmdeploy/pytorch/check_env
copying lmdeploy/pytorch/check_env/model.py -> build/lib/lmdeploy/pytorch/check_env
copying lmdeploy/pytorch/check_env/triton_custom_add.py -> build/lib/lmdeploy/pytorch/check_env
copying lmdeploy/pytorch/check_env/transformers.py -> build/lib/lmdeploy/pytorch/check_env
copying lmdeploy/pytorch/check_env/torch.py -> build/lib/lmdeploy/pytorch/check_env
copying lmdeploy/pytorch/check_env/adapter.py -> build/lib/lmdeploy/pytorch/check_env
copying lmdeploy/pytorch/nn/linear/default.py -> build/lib/lmdeploy/pytorch/nn/linear
copying lmdeploy/pytorch/nn/linear/utils.py -> build/lib/lmdeploy/pytorch/nn/linear
copying lmdeploy/pytorch/nn/linear/awq.py -> build/lib/lmdeploy/pytorch/nn/linear
copying lmdeploy/pytorch/nn/linear/base.py -> build/lib/lmdeploy/pytorch/nn/linear
copying lmdeploy/pytorch/nn/linear/blocked_fp8.py -> build/lib/lmdeploy/pytorch/nn/linear
copying lmdeploy/pytorch/nn/linear/w8a8.py -> build/lib/lmdeploy/pytorch/nn/linear
copying lmdeploy/pytorch/nn/linear/init.py -> build/lib/lmdeploy/pytorch/nn/linear
copying lmdeploy/pytorch/nn/linear/lora.py -> build/lib/lmdeploy/pytorch/nn/linear
copying lmdeploy/pytorch/nn/moe/default.py -> build/lib/lmdeploy/pytorch/nn/moe
copying lmdeploy/pytorch/nn/moe/base.py -> build/lib/lmdeploy/pytorch/nn/moe
copying lmdeploy/pytorch/nn/moe/blocked_fp8.py -> build/lib/lmdeploy/pytorch/nn/moe
copying lmdeploy/pytorch/nn/moe/w8a8.py -> build/lib/lmdeploy/pytorch/nn/moe
copying lmdeploy/pytorch/nn/moe/init.py -> build/lib/lmdeploy/pytorch/nn/moe
copying lmdeploy/pytorch/paging/block_manager/base_block_manager.py -> build/lib/lmdeploy/pytorch/paging/block_manager
copying lmdeploy/pytorch/paging/block_manager/window_block_manager.py -> build/lib/lmdeploy/pytorch/paging/block_manager
copying lmdeploy/pytorch/paging/block_manager/init.py -> build/lib/lmdeploy/pytorch/paging/block_manager
copying lmdeploy/pytorch/paging/block_manager/default_block_manager.py -> build/lib/lmdeploy/pytorch/paging/block_manager
copying lmdeploy/pytorch/paging/seq_states/states.py -> build/lib/lmdeploy/pytorch/paging/seq_states
copying lmdeploy/pytorch/paging/seq_states/init.py -> build/lib/lmdeploy/pytorch/paging/seq_states
copying lmdeploy/pytorch/paging/eviction_helper/base_eviction_helper.py -> build/lib/lmdeploy/pytorch/paging/eviction_helper
copying lmdeploy/pytorch/paging/eviction_helper/recompute_eviction_helper.py -> build/lib/lmdeploy/pytorch/paging/eviction_helper
copying lmdeploy/pytorch/paging/eviction_helper/init.py -> build/lib/lmdeploy/pytorch/paging/eviction_helper
copying lmdeploy/pytorch/kernels/dlinfer/flash_attention.py -> build/lib/lmdeploy/pytorch/kernels/dlinfer
copying lmdeploy/pytorch/kernels/dlinfer/pagedattention.py -> build/lib/lmdeploy/pytorch/kernels/dlinfer
copying lmdeploy/pytorch/kernels/dlinfer/rms_norm.py -> build/lib/lmdeploy/pytorch/kernels/dlinfer
copying lmdeploy/pytorch/kernels/dlinfer/fused_rotary_emb.py -> build/lib/lmdeploy/pytorch/kernels/dlinfer
copying lmdeploy/pytorch/kernels/dlinfer/activation.py -> build/lib/lmdeploy/pytorch/kernels/dlinfer
copying lmdeploy/pytorch/kernels/dlinfer/apply_rotary_pos_emb.py -> build/lib/lmdeploy/pytorch/kernels/dlinfer
copying lmdeploy/pytorch/kernels/dlinfer/awq_kernels.py -> build/lib/lmdeploy/pytorch/kernels/dlinfer
copying lmdeploy/pytorch/kernels/dlinfer/fill_kv_cache.py -> build/lib/lmdeploy/pytorch/kernels/dlinfer
copying lmdeploy/pytorch/kernels/dlinfer/fused_moe.py -> build/lib/lmdeploy/pytorch/kernels/dlinfer
copying lmdeploy/pytorch/kernels/dlinfer/init.py -> build/lib/lmdeploy/pytorch/kernels/dlinfer
copying lmdeploy/pytorch/kernels/dlinfer/w8a8_kernels.py -> build/lib/lmdeploy/pytorch/kernels/dlinfer
copying lmdeploy/pytorch/kernels/dlinfer/linear.py -> build/lib/lmdeploy/pytorch/kernels/dlinfer
copying lmdeploy/pytorch/kernels/dlinfer/moe_gating_topk_softmax.py -> build/lib/lmdeploy/pytorch/kernels/dlinfer
copying lmdeploy/pytorch/kernels/cuda/flatten_kv_cache.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/fused_moe_ep.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/blocked_fp8_fused_moe.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/pagedattention.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/rms_norm.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/blocked_gemm_fp8.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/activation.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/multinomial_sampling.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/fused_lora.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/apply_rotary_pos_emb.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/utils.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/bitonic_topk.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/w8a8_triton_kernels.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/awq_kernels.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/fill_kv_cache.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/fused_moe.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/w8a8_fused_moe.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/init.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/flashattention.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/cuda/ds_index.py -> build/lib/lmdeploy/pytorch/kernels/cuda
copying lmdeploy/pytorch/kernels/default/multinomial_sampling.py -> build/lib/lmdeploy/pytorch/kernels/default
copying lmdeploy/pytorch/kernels/default/init.py -> build/lib/lmdeploy/pytorch/kernels/default
copying lmdeploy/pytorch/kernels/default/w8a8_kernels.py -> build/lib/lmdeploy/pytorch/kernels/default
copying lmdeploy/pytorch/third_party/deep_gemm/init.py -> build/lib/lmdeploy/pytorch/third_party/deep_gemm
copying lmdeploy/pytorch/models/utils/micro_batch.py -> build/lib/lmdeploy/pytorch/models/utils
copying lmdeploy/pytorch/models/utils/cudagraph.py -> build/lib/lmdeploy/pytorch/models/utils
copying lmdeploy/pytorch/models/utils/multimodal.py -> build/lib/lmdeploy/pytorch/models/utils
copying lmdeploy/pytorch/models/utils/init.py -> build/lib/lmdeploy/pytorch/models/utils
copying lmdeploy/pytorch/models/utils/model.py -> build/lib/lmdeploy/pytorch/models/utils
copying lmdeploy/pytorch/strategies/ar_spec/model_agent.py -> build/lib/lmdeploy/pytorch/strategies/ar_spec
copying lmdeploy/pytorch/strategies/ar_spec/cudagraph.py -> build/lib/lmdeploy/pytorch/strategies/ar_spec
copying lmdeploy/pytorch/strategies/ar_spec/engine.py -> build/lib/lmdeploy/pytorch/strategies/ar_spec
copying lmdeploy/pytorch/strategies/ar_spec/sampling.py -> build/lib/lmdeploy/pytorch/strategies/ar_spec
copying lmdeploy/pytorch/strategies/ar_spec/model_inputs.py -> build/lib/lmdeploy/pytorch/strategies/ar_spec
copying lmdeploy/pytorch/strategies/ar_spec/init.py -> build/lib/lmdeploy/pytorch/strategies/ar_spec
copying lmdeploy/pytorch/strategies/ar_spec/sequence.py -> build/lib/lmdeploy/pytorch/strategies/ar_spec
copying lmdeploy/pytorch/strategies/base/model_agent.py -> build/lib/lmdeploy/pytorch/strategies/base
copying lmdeploy/pytorch/strategies/base/cudagraph.py -> build/lib/lmdeploy/pytorch/strategies/base
copying lmdeploy/pytorch/strategies/base/engine.py -> build/lib/lmdeploy/pytorch/strategies/base
copying lmdeploy/pytorch/strategies/base/sampling.py -> build/lib/lmdeploy/pytorch/strategies/base
copying lmdeploy/pytorch/strategies/base/model_inputs.py -> build/lib/lmdeploy/pytorch/strategies/base
copying lmdeploy/pytorch/strategies/base/init.py -> build/lib/lmdeploy/pytorch/strategies/base
copying lmdeploy/pytorch/strategies/base/sequence.py -> build/lib/lmdeploy/pytorch/strategies/base
copying lmdeploy/pytorch/strategies/ar/model_agent.py -> build/lib/lmdeploy/pytorch/strategies/ar
copying lmdeploy/pytorch/strategies/ar/cudagraph.py -> build/lib/lmdeploy/pytorch/strategies/ar
copying lmdeploy/pytorch/strategies/ar/engine.py -> build/lib/lmdeploy/pytorch/strategies/ar
copying lmdeploy/pytorch/strategies/ar/sampling.py -> build/lib/lmdeploy/pytorch/strategies/ar
copying lmdeploy/pytorch/strategies/ar/model_inputs.py -> build/lib/lmdeploy/pytorch/strategies/ar
copying lmdeploy/pytorch/strategies/ar/init.py -> build/lib/lmdeploy/pytorch/strategies/ar
copying lmdeploy/pytorch/strategies/ar/sequence.py -> build/lib/lmdeploy/pytorch/strategies/ar
copying lmdeploy/pytorch/strategies/dllm/model_agent.py -> build/lib/lmdeploy/pytorch/strategies/dllm
copying lmdeploy/pytorch/strategies/dllm/cudagraph.py -> build/lib/lmdeploy/pytorch/strategies/dllm
copying lmdeploy/pytorch/strategies/dllm/engine.py -> build/lib/lmdeploy/pytorch/strategies/dllm
copying lmdeploy/pytorch/strategies/dllm/sampling.py -> build/lib/lmdeploy/pytorch/strategies/dllm
copying lmdeploy/pytorch/strategies/dllm/model_inputs.py -> build/lib/lmdeploy/pytorch/strategies/dllm
copying lmdeploy/pytorch/strategies/dllm/unmasking.py -> build/lib/lmdeploy/pytorch/strategies/dllm
copying lmdeploy/pytorch/strategies/dllm/init.py -> build/lib/lmdeploy/pytorch/strategies/dllm
copying lmdeploy/pytorch/strategies/dllm/sequence.py -> build/lib/lmdeploy/pytorch/strategies/dllm
copying lmdeploy/pytorch/spec_decode/proposers/eagle.py -> build/lib/lmdeploy/pytorch/spec_decode/proposers
copying lmdeploy/pytorch/spec_decode/proposers/base.py -> build/lib/lmdeploy/pytorch/spec_decode/proposers
copying lmdeploy/pytorch/spec_decode/proposers/deepseek_mtp.py -> build/lib/lmdeploy/pytorch/spec_decode/proposers
copying lmdeploy/pytorch/spec_decode/proposers/eagle3.py -> build/lib/lmdeploy/pytorch/spec_decode/proposers
copying lmdeploy/pytorch/spec_decode/proposers/init.py -> build/lib/lmdeploy/pytorch/spec_decode/proposers
copying lmdeploy/pytorch/disagg/conn/protocol.py -> build/lib/lmdeploy/pytorch/disagg/conn
copying lmdeploy/pytorch/disagg/conn/engine_conn.py -> build/lib/lmdeploy/pytorch/disagg/conn
copying lmdeploy/pytorch/disagg/conn/proxy_conn.py -> build/lib/lmdeploy/pytorch/disagg/conn
copying lmdeploy/pytorch/disagg/conn/init.py -> build/lib/lmdeploy/pytorch/disagg/conn
copying lmdeploy/pytorch/disagg/backend/backend.py -> build/lib/lmdeploy/pytorch/disagg/backend
copying lmdeploy/pytorch/disagg/backend/mooncake.py -> build/lib/lmdeploy/pytorch/disagg/backend
copying lmdeploy/pytorch/disagg/backend/base.py -> build/lib/lmdeploy/pytorch/disagg/backend
copying lmdeploy/pytorch/disagg/backend/init.py -> build/lib/lmdeploy/pytorch/disagg/backend
copying lmdeploy/pytorch/disagg/backend/dlslime.py -> build/lib/lmdeploy/pytorch/disagg/backend
copying lmdeploy/pytorch/engine/model_agent/agent.py -> build/lib/lmdeploy/pytorch/engine/model_agent
copying lmdeploy/pytorch/engine/model_agent/profiler.py -> build/lib/lmdeploy/pytorch/engine/model_agent
copying lmdeploy/pytorch/engine/model_agent/inputs_maker.py -> build/lib/lmdeploy/pytorch/engine/model_agent
copying lmdeploy/pytorch/engine/model_agent/init.py -> build/lib/lmdeploy/pytorch/engine/model_agent
copying lmdeploy/pytorch/engine/executor/base_worker.py -> build/lib/lmdeploy/pytorch/engine/executor
copying lmdeploy/pytorch/engine/executor/ray_executor.py -> build/lib/lmdeploy/pytorch/engine/executor
copying lmdeploy/pytorch/engine/executor/uni_executor.py -> build/lib/lmdeploy/pytorch/engine/executor
copying lmdeploy/pytorch/engine/executor/mp_executor.py -> build/lib/lmdeploy/pytorch/engine/executor
copying lmdeploy/pytorch/engine/executor/base.py -> build/lib/lmdeploy/pytorch/engine/executor
copying lmdeploy/pytorch/engine/executor/dist_utils.py -> build/lib/lmdeploy/pytorch/engine/executor
copying lmdeploy/pytorch/engine/executor/init.py -> build/lib/lmdeploy/pytorch/engine/executor
copying lmdeploy/pytorch/engine/mp_engine/base_worker.py -> build/lib/lmdeploy/pytorch/engine/mp_engine
copying lmdeploy/pytorch/engine/mp_engine/zmq_rpc.py -> build/lib/lmdeploy/pytorch/engine/mp_engine
copying lmdeploy/pytorch/engine/mp_engine/base.py -> build/lib/lmdeploy/pytorch/engine/mp_engine
copying lmdeploy/pytorch/engine/mp_engine/zmq_engine.py -> build/lib/lmdeploy/pytorch/engine/mp_engine
copying lmdeploy/pytorch/engine/mp_engine/init.py -> build/lib/lmdeploy/pytorch/engine/mp_engine
copying lmdeploy/pytorch/engine/mp_engine/ray_engine.py -> build/lib/lmdeploy/pytorch/engine/mp_engine
copying lmdeploy/pytorch/backends/dlinfer/flash_attention.py -> build/lib/lmdeploy/pytorch/backends/dlinfer
copying lmdeploy/pytorch/backends/dlinfer/attention.py -> build/lib/lmdeploy/pytorch/backends/dlinfer
copying lmdeploy/pytorch/backends/dlinfer/norm.py -> build/lib/lmdeploy/pytorch/backends/dlinfer
copying lmdeploy/pytorch/backends/dlinfer/moe.py -> build/lib/lmdeploy/pytorch/backends/dlinfer
copying lmdeploy/pytorch/backends/dlinfer/op_backend.py -> build/lib/lmdeploy/pytorch/backends/dlinfer
copying lmdeploy/pytorch/backends/dlinfer/activation.py -> build/lib/lmdeploy/pytorch/backends/dlinfer
copying lmdeploy/pytorch/backends/dlinfer/qmodules.py -> build/lib/lmdeploy/pytorch/backends/dlinfer
copying lmdeploy/pytorch/backends/dlinfer/awq_modules.py -> build/lib/lmdeploy/pytorch/backends/dlinfer
copying lmdeploy/pytorch/backends/dlinfer/rotary_embedding.py -> build/lib/lmdeploy/pytorch/backends/dlinfer
copying lmdeploy/pytorch/backends/dlinfer/init.py -> build/lib/lmdeploy/pytorch/backends/dlinfer
copying lmdeploy/pytorch/backends/dlinfer/linear.py -> build/lib/lmdeploy/pytorch/backends/dlinfer
copying lmdeploy/pytorch/backends/dlinfer/apply_rotary_emb.py -> build/lib/lmdeploy/pytorch/backends/dlinfer
copying lmdeploy/pytorch/backends/cuda/flash_attention.py -> build/lib/lmdeploy/pytorch/backends/cuda
copying lmdeploy/pytorch/backends/cuda/warmup_manager.py -> build/lib/lmdeploy/pytorch/backends/cuda
copying lmdeploy/pytorch/backends/cuda/norm.py -> build/lib/lmdeploy/pytorch/backends/cuda
copying lmdeploy/pytorch/backends/cuda/op_backend.py -> build/lib/lmdeploy/pytorch/backends/cuda
copying lmdeploy/pytorch/backends/cuda/nsa.py -> build/lib/lmdeploy/pytorch/backends/cuda
copying lmdeploy/pytorch/backends/cuda/activation.py -> build/lib/lmdeploy/pytorch/backends/cuda
copying lmdeploy/pytorch/backends/cuda/qmodules.py -> build/lib/lmdeploy/pytorch/backends/cuda
copying lmdeploy/pytorch/backends/cuda/multinomial_sampling.py -> build/lib/lmdeploy/pytorch/backends/cuda
copying lmdeploy/pytorch/backends/cuda/blockedf8_modules.py -> build/lib/lmdeploy/pytorch/backends/cuda
copying lmdeploy/pytorch/backends/cuda/awq_modules.py -> build/lib/lmdeploy/pytorch/backends/cuda
copying lmdeploy/pytorch/backends/cuda/token_dispatcher.py -> build/lib/lmdeploy/pytorch/backends/cuda
copying lmdeploy/pytorch/backends/cuda/init.py -> build/lib/lmdeploy/pytorch/backends/cuda
copying lmdeploy/pytorch/backends/cuda/graph_runner.py -> build/lib/lmdeploy/pytorch/backends/cuda
copying lmdeploy/pytorch/backends/cuda/apply_rotary_emb.py -> build/lib/lmdeploy/pytorch/backends/cuda
copying lmdeploy/pytorch/backends/cuda/lora.py -> build/lib/lmdeploy/pytorch/backends/cuda
copying lmdeploy/pytorch/backends/default/norm.py -> build/lib/lmdeploy/pytorch/backends/default
copying lmdeploy/pytorch/backends/default/moe.py -> build/lib/lmdeploy/pytorch/backends/default
copying lmdeploy/pytorch/backends/default/op_backend.py -> build/lib/lmdeploy/pytorch/backends/default
copying lmdeploy/pytorch/backends/default/embedding.py -> build/lib/lmdeploy/pytorch/backends/default
copying lmdeploy/pytorch/backends/default/activation.py -> build/lib/lmdeploy/pytorch/backends/default
copying lmdeploy/pytorch/backends/default/multinomial_sampling.py -> build/lib/lmdeploy/pytorch/backends/default
copying lmdeploy/pytorch/backends/default/awq_modules.py -> build/lib/lmdeploy/pytorch/backends/default
copying lmdeploy/pytorch/backends/default/token_dispatcher.py -> build/lib/lmdeploy/pytorch/backends/default
copying lmdeploy/pytorch/backends/default/rotary_embedding.py -> build/lib/lmdeploy/pytorch/backends/default
copying lmdeploy/pytorch/backends/default/init.py -> build/lib/lmdeploy/pytorch/backends/default
copying lmdeploy/pytorch/backends/default/linear.py -> build/lib/lmdeploy/pytorch/backends/default
copying lmdeploy/pytorch/backends/default/apply_rotary_emb.py -> build/lib/lmdeploy/pytorch/backends/default
copying lmdeploy/pytorch/backends/dlinfer/maca/op_backend.py -> build/lib/lmdeploy/pytorch/backends/dlinfer/maca
copying lmdeploy/pytorch/backends/dlinfer/maca/init.py -> build/lib/lmdeploy/pytorch/backends/dlinfer/maca
copying lmdeploy/pytorch/backends/dlinfer/camb/op_backend.py -> build/lib/lmdeploy/pytorch/backends/dlinfer/camb
copying lmdeploy/pytorch/backends/dlinfer/camb/init.py -> build/lib/lmdeploy/pytorch/backends/dlinfer/camb
copying lmdeploy/pytorch/backends/dlinfer/ascend/op_backend.py -> build/lib/lmdeploy/pytorch/backends/dlinfer/ascend
copying lmdeploy/pytorch/backends/dlinfer/ascend/utils.py -> build/lib/lmdeploy/pytorch/backends/dlinfer/ascend
copying lmdeploy/pytorch/backends/dlinfer/ascend/init.py -> build/lib/lmdeploy/pytorch/backends/dlinfer/ascend
copying lmdeploy/pytorch/backends/cuda/moe/default.py -> build/lib/lmdeploy/pytorch/backends/cuda/moe
copying lmdeploy/pytorch/backends/cuda/moe/blocked_fp8.py -> build/lib/lmdeploy/pytorch/backends/cuda/moe
copying lmdeploy/pytorch/backends/cuda/moe/w8a8.py -> build/lib/lmdeploy/pytorch/backends/cuda/moe
copying lmdeploy/pytorch/backends/cuda/moe/ep_utils.py -> build/lib/lmdeploy/pytorch/backends/cuda/moe
copying lmdeploy/pytorch/backends/cuda/moe/init.py -> build/lib/lmdeploy/pytorch/backends/cuda/moe
copying lmdeploy/pytorch/backends/cuda/attention/fa3.py -> build/lib/lmdeploy/pytorch/backends/cuda/attention
copying lmdeploy/pytorch/backends/cuda/attention/default.py -> build/lib/lmdeploy/pytorch/backends/cuda/attention
copying lmdeploy/pytorch/backends/cuda/attention/init.py -> build/lib/lmdeploy/pytorch/backends/cuda/attention
copying lmdeploy/pytorch/backends/cuda/attention/mla.py -> build/lib/lmdeploy/pytorch/backends/cuda/attention
copying lmdeploy/lite/utils/global_avail.py -> build/lib/lmdeploy/lite/utils
copying lmdeploy/lite/utils/cal_qparams.py -> build/lib/lmdeploy/lite/utils
copying lmdeploy/lite/utils/batch_split.py -> build/lib/lmdeploy/lite/utils
copying lmdeploy/lite/utils/calib_dataloader.py -> build/lib/lmdeploy/lite/utils
copying lmdeploy/lite/utils/collect.py -> build/lib/lmdeploy/lite/utils
copying lmdeploy/lite/utils/init.py -> build/lib/lmdeploy/lite/utils
copying lmdeploy/lite/utils/load.py -> build/lib/lmdeploy/lite/utils
copying lmdeploy/lite/utils/memory_efficient.py -> build/lib/lmdeploy/lite/utils
copying lmdeploy/lite/apis/gptq.py -> build/lib/lmdeploy/lite/apis
copying lmdeploy/lite/apis/calibrate.py -> build/lib/lmdeploy/lite/apis
copying lmdeploy/lite/apis/get_small_sharded_hf.py -> build/lib/lmdeploy/lite/apis
copying lmdeploy/lite/apis/smooth_quant.py -> build/lib/lmdeploy/lite/apis
copying lmdeploy/lite/apis/auto_awq.py -> build/lib/lmdeploy/lite/apis
copying lmdeploy/lite/apis/init.py -> build/lib/lmdeploy/lite/apis
copying lmdeploy/lite/quantization/awq.py -> build/lib/lmdeploy/lite/quantization
copying lmdeploy/lite/quantization/calibration.py -> build/lib/lmdeploy/lite/quantization
copying lmdeploy/lite/quantization/init.py -> build/lib/lmdeploy/lite/quantization
copying lmdeploy/lite/modeling/internlm2_gptq.py -> build/lib/lmdeploy/lite/modeling
copying lmdeploy/lite/modeling/internlm3_gptq.py -> build/lib/lmdeploy/lite/modeling
copying lmdeploy/lite/modeling/init.py -> build/lib/lmdeploy/lite/modeling
copying lmdeploy/lite/quantization/modules/init.py -> build/lib/lmdeploy/lite/quantization/modules
copying lmdeploy/lite/quantization/modules/linear.py -> build/lib/lmdeploy/lite/quantization/modules
copying lmdeploy/lite/quantization/activation/observer.py -> build/lib/lmdeploy/lite/quantization/activation
copying lmdeploy/lite/quantization/activation/init.py -> build/lib/lmdeploy/lite/quantization/activation
copying lmdeploy/lite/quantization/weight/quant_utils.py -> build/lib/lmdeploy/lite/quantization/weight
copying lmdeploy/lite/quantization/weight/init.py -> build/lib/lmdeploy/lite/quantization/weight
copying lmdeploy/lite/quantization/weight/quantizer.py -> build/lib/lmdeploy/lite/quantization/weight
running egg_info
writing lmdeploy.egg-info/PKG-INFO
writing dependency_links to lmdeploy.egg-info/dependency_links.txt
writing entry points to lmdeploy.egg-info/entry_points.txt
writing requirements to lmdeploy.egg-info/requires.txt
writing top-level names to lmdeploy.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
warning: no files found matching 'lmdeploy/lib/.so'
warning: no files found matching 'lmdeploy/lib/.so*'
warning: no files found matching 'lmdeploy/lib/.dll'
warning: no files found matching 'lmdeploy/lib/.pyd'
warning: no files found matching 'lmdeploy/bin/*'
adding license file 'LICENSE'
writing manifest file 'lmdeploy.egg-info/SOURCES.txt'
/home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages/setuptools/command/build_py.py:215: _Warning: Package 'lmdeploy.monitoring' is absent from the packages configuration.
!!

    ********************************************************************************
    ############################
    # Package would be ignored #
    ############################
    Python recognizes 'lmdeploy.monitoring' as an importable package[^1],
    but it is absent from setuptools' `packages` configuration.

    This leads to an ambiguous overall configuration. If you want to distribute this
    package, please make sure that 'lmdeploy.monitoring' is explicitly added
    to the `packages` configuration field.

    Alternatively, you can also rely on setuptools' discovery methods
    (for example by using `find_namespace_packages(...)`/`find_namespace:`
    instead of `find_packages(...)`/`find:`).

    You can read more about "package discovery" on setuptools documentation page:

    - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

    If you don't want 'lmdeploy.monitoring' to be distributed and are
    already explicitly excluding 'lmdeploy.monitoring' via
    `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
    you can try to use `exclude_package_data`, or `include-package-data=False` in
    combination with a more fine grained `package-data` configuration.

    You can read more about "package data files" on setuptools documentation page:

    - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


    [^1]: For Python, any directory (with suitable naming) can be imported,
          even if it does not contain any `.py` files.
          On the other hand, currently there is no concept of package data
          directory, all directories are treated like packages.
    ********************************************************************************

!!
check.warn(importable)
/home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages/setuptools/command/build_py.py:215: _Warning: Package 'lmdeploy.monitoring.grafana.dashboards.config' is absent from the packages configuration.
!!

    ********************************************************************************
    ############################
    # Package would be ignored #
    ############################
    Python recognizes 'lmdeploy.monitoring.grafana.dashboards.config' as an importable package[^1],
    but it is absent from setuptools' `packages` configuration.

    This leads to an ambiguous overall configuration. If you want to distribute this
    package, please make sure that 'lmdeploy.monitoring.grafana.dashboards.config' is explicitly added
    to the `packages` configuration field.

    Alternatively, you can also rely on setuptools' discovery methods
    (for example by using `find_namespace_packages(...)`/`find_namespace:`
    instead of `find_packages(...)`/`find:`).

    You can read more about "package discovery" on setuptools documentation page:

    - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

    If you don't want 'lmdeploy.monitoring.grafana.dashboards.config' to be distributed and are
    already explicitly excluding 'lmdeploy.monitoring.grafana.dashboards.config' via
    `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
    you can try to use `exclude_package_data`, or `include-package-data=False` in
    combination with a more fine grained `package-data` configuration.

    You can read more about "package data files" on setuptools documentation page:

    - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


    [^1]: For Python, any directory (with suitable naming) can be imported,
          even if it does not contain any `.py` files.
          On the other hand, currently there is no concept of package data
          directory, all directories are treated like packages.
    ********************************************************************************

!!
check.warn(importable)
/home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages/setuptools/command/build_py.py:215: _Warning: Package 'lmdeploy.monitoring.grafana.dashboards.json' is absent from the packages configuration.
!!

    ********************************************************************************
    ############################
    # Package would be ignored #
    ############################
    Python recognizes 'lmdeploy.monitoring.grafana.dashboards.json' as an importable package[^1],
    but it is absent from setuptools' `packages` configuration.

    This leads to an ambiguous overall configuration. If you want to distribute this
    package, please make sure that 'lmdeploy.monitoring.grafana.dashboards.json' is explicitly added
    to the `packages` configuration field.

    Alternatively, you can also rely on setuptools' discovery methods
    (for example by using `find_namespace_packages(...)`/`find_namespace:`
    instead of `find_packages(...)`/`find:`).

    You can read more about "package discovery" on setuptools documentation page:

    - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

    If you don't want 'lmdeploy.monitoring.grafana.dashboards.json' to be distributed and are
    already explicitly excluding 'lmdeploy.monitoring.grafana.dashboards.json' via
    `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
    you can try to use `exclude_package_data`, or `include-package-data=False` in
    combination with a more fine grained `package-data` configuration.

    You can read more about "package data files" on setuptools documentation page:

    - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


    [^1]: For Python, any directory (with suitable naming) can be imported,
          even if it does not contain any `.py` files.
          On the other hand, currently there is no concept of package data
          directory, all directories are treated like packages.
    ********************************************************************************

!!
check.warn(importable)
/home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages/setuptools/command/build_py.py:215: _Warning: Package 'lmdeploy.monitoring.grafana.datasources' is absent from the packages configuration.
!!

    ********************************************************************************
    ############################
    # Package would be ignored #
    ############################
    Python recognizes 'lmdeploy.monitoring.grafana.datasources' as an importable package[^1],
    but it is absent from setuptools' `packages` configuration.

    This leads to an ambiguous overall configuration. If you want to distribute this
    package, please make sure that 'lmdeploy.monitoring.grafana.datasources' is explicitly added
    to the `packages` configuration field.

    Alternatively, you can also rely on setuptools' discovery methods
    (for example by using `find_namespace_packages(...)`/`find_namespace:`
    instead of `find_packages(...)`/`find:`).

    You can read more about "package discovery" on setuptools documentation page:

    - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

    If you don't want 'lmdeploy.monitoring.grafana.datasources' to be distributed and are
    already explicitly excluding 'lmdeploy.monitoring.grafana.datasources' via
    `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
    you can try to use `exclude_package_data`, or `include-package-data=False` in
    combination with a more fine grained `package-data` configuration.

    You can read more about "package data files" on setuptools documentation page:

    - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


    [^1]: For Python, any directory (with suitable naming) can be imported,
          even if it does not contain any `.py` files.
          On the other hand, currently there is no concept of package data
          directory, all directories are treated like packages.
    ********************************************************************************

!!
check.warn(importable)
copying lmdeploy/monitoring/docker-compose.yaml -> build/lib/lmdeploy/monitoring
copying lmdeploy/monitoring/prometheus.yaml -> build/lib/lmdeploy/monitoring
copying lmdeploy/monitoring/grafana/dashboards/config/dashboard.yaml -> build/lib/lmdeploy/monitoring/grafana/dashboards/config
copying lmdeploy/monitoring/grafana/dashboards/json/lmdeploy-dashboard.json -> build/lib/lmdeploy/monitoring/grafana/dashboards/json
copying lmdeploy/monitoring/grafana/datasources/datasource.yaml -> build/lib/lmdeploy/monitoring/grafana/datasources
copying lmdeploy/pytorch/disagg/README.md -> build/lib/lmdeploy/pytorch/disagg

``

[Drzhivago264]

@windreamer
Found a small bug in zmq

self._target(self._args, **self._kwargs)
File "/home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages/lmdeploy-0.12.0-py3.11.egg/lmdeploy/pytorch/engine/mp_engine/zmq_engine.py", line 101, in _mp_proc
server = AsyncRPCServer()
^^^^^^^^^^^^^^^^
File "/home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages/lmdeploy-0.12.0-py3.11.egg/lmdeploy/pytorch/engine/mp_engine/zmq_rpc.py", line 41, in init
self.port = self.socket.bind_to_random_port(address)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages/zmq/sugar/socket.py", line 504, in bind_to_random_port
self.bind(f"{addr}:")
File "/home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages/zmq/sugar/socket.py", line 320, in bind
super().bind(addr)
File "zmq/backend/cython/_zmq.py", line 1009, in zmq.backend.cython._zmq.Socket.bind
File "zmq/backend/cython/_zmq.py", line 190, in zmq.backend.cython._zmq._check_rc
zmq.error.ZMQError: No such device (addr='tcp://localhost:*')

this error is cause by 'localhost' in var address => can be fix by changing to 127.0.0.1

ref: https://stackoverflow.com/questions/6024003/why-doesnt-zeromq-work-on-localhost/8958414#8958414

[windreamer]

@windreamer
Found a small bug in zmq

self._target(self._args, **self._kwargs)
File "/home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages/lmdeploy-0.12.0-py3.11.egg/lmdeploy/pytorch/engine/mp_engine/zmq_engine.py", line 101, in _mp_proc
server = AsyncRPCServer()
^^^^^^^^^^^^^^^^
File "/home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages/lmdeploy-0.12.0-py3.11.egg/lmdeploy/pytorch/engine/mp_engine/zmq_rpc.py", line 41, in init
self.port = self.socket.bind_to_random_port(address)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages/zmq/sugar/socket.py", line 504, in bind_to_random_port
self.bind(f"{addr}:")
File "/home/h/anaconda3/envs/lmdeploy/lib/python3.11/site-packages/zmq/sugar/socket.py", line 320, in bind
super().bind(addr)
File "zmq/backend/cython/_zmq.py", line 1009, in zmq.backend.cython._zmq.Socket.bind
File "zmq/backend/cython/_zmq.py", line 190, in zmq.backend.cython._zmq._check_rc
zmq.error.ZMQError: No such device (addr='tcp://localhost:*')

this error is cause by 'localhost' in var address => can be fix by changing to 127.0.0.1

ref: https://stackoverflow.com/questions/6024003/why-doesnt-zeromq-work-on-localhost/8958414#8958414

Thanks for your update. I think there should be some implementation differences across platforms.

ZeroMQ bind() doesn't do DNS resolution — it needs an exact interface or IP. Embedded systems like PPC often can't resolve 'localhost', so use 127.0.0.1 instead.