Skip to content

Commit 31d42bd

Browse files
committed
model-runner: document inference engine support and GitHub issue reporting
Signed-off-by: Dorin Geman <[email protected]>
1 parent a55bf1b commit 31d42bd

File tree

2 files changed

+8
-9
lines changed

2 files changed

+8
-9
lines changed

_vale/config/vocabularies/Docker/accept.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -213,6 +213,7 @@ Visual Studio Code
213213
VMware
214214
vpnkit
215215
vSphere
216+
Vulkan
216217
Vue
217218
Wasm
218219
Wasmtime

content/manuals/ai/model-runner/_index.md

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ params:
66
group: AI
77
weight: 30
88
description: Learn how to use Docker Model Runner to manage and run AI models.
9-
keywords: Docker, ai, model runner, docker desktop, docker engine, llm
9+
keywords: Docker, ai, model runner, docker desktop, docker engine, llm, openai, llama.cpp, vllm, cpu, nvidia, cuda, amd, rocm, vulkan
1010
aliases:
1111
- /desktop/features/model-runner/
1212
- /model-runner/
@@ -34,7 +34,8 @@ with AI models locally.
3434

3535
- [Pull and push models to and from Docker Hub](https://hub.docker.com/u/ai)
3636
- Serve models on OpenAI-compatible APIs for easy integration with existing apps
37-
- Package GGUF files as OCI Artifacts and publish them to any Container Registry
37+
- Support for both llama.cpp and vLLM inference engines (vLLM currently supported on Linux x86_64/amd64 with NVIDIA GPUs only)
38+
- Package GGUF and Safetensors files as OCI Artifacts and publish them to any Container Registry
3839
- Run and interact with AI models directly from the command line or from the Docker Desktop GUI
3940
- Manage local models and display logs
4041
- Display prompt and response details
@@ -68,7 +69,7 @@ Windows(arm64):
6869

6970
Docker Engine only:
7071

71-
- Linux CPU & Linux NVIDIA
72+
- Linux CPU, NVIDIA, AMD and Vulkan
7273
- NVIDIA drivers 575.57.08+
7374

7475
{{< /tab >}}
@@ -83,6 +84,8 @@ initial pull may take some time. After that, they're cached locally for faster
8384
access. You can interact with the model using
8485
[OpenAI-compatible APIs](api-reference.md).
8586

87+
Docker Model Runner supports both [llama.cpp](https://github.com/ggerganov/llama.cpp) and [vLLM](https://github.com/vllm-project/vllm) as inference engines, providing flexibility for different model formats and performance requirements. For more details, see the [Docker Model Runner repository](https://github.com/docker/model-runner).
88+
8689
> [!TIP]
8790
>
8891
> Using Testcontainers or Docker Compose?
@@ -111,14 +114,9 @@ $ ln -s /Applications/Docker.app/Contents/Resources/cli-plugins/docker-model ~/.
111114

112115
Once linked, rerun the command.
113116

114-
### No consistent digest support in Model CLI
115-
116-
The Docker Model CLI currently lacks consistent support for specifying models by image digest. As a temporary workaround, you should refer to models by name instead of digest.
117-
118117
## Share feedback
119118

120-
Thanks for trying out Docker Model Runner. Give feedback or report any bugs
121-
you may find through the **Give feedback** link next to the **Enable Docker Model Runner** setting.
119+
Thanks for trying out Docker Model Runner. To report bugs or request features, [open an issue on GitHub](https://github.com/docker/model-runner/issues). You can also give feedback through the **Give feedback** link next to the **Enable Docker Model Runner** setting.
122120

123121
## Next steps
124122

0 commit comments

Comments
 (0)