WiLoR: End-to-end 3D hand localization and reconstruction in-the-wild

Rolandos Alexandros Potamias¹   Jinglei Zhang²   Jiankang Deng¹   Stefanos Zafeiriou¹

¹Imperial College London, UK
²Shanghai Jiao Tong University, China

CVPR 2025

This is the official implementation of WiLoR, an state-of-the-art hand localization and reconstruction model:

🚀 News: WiLoR can run even faster! 🚀

✅ WiLoR demo now supports half precision inference and depth prunning, resulting in up to 1.6x faster inferece with minimal performance degradation (~Δ-MPJPE 0.05mm). Simply run demo using --fast argument.

python demo.py --img_folder demo_img --out_folder demo_out --save_mesh  --fast

Installation

Original Installation

git clone --recursive https://github.com/rolpotamias/WiLoR.git
cd WiLoR

The code has been tested with PyTorch 2.0.0 and CUDA 11.7. It is suggested to use an anaconda environment to install the the required dependencies:

conda create --name wilor python=3.10
conda activate wilor

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu117
# Install requirements
pip install -r requirements.txt

Download the pretrained models using:

wget https://huggingface.co/spaces/rolpotamias/WiLoR/resolve/main/pretrained_models/detector.pt -P ./pretrained_models/
wget https://huggingface.co/spaces/rolpotamias/WiLoR/resolve/main/pretrained_models/wilor_final.ckpt -P ./pretrained_models/

It is also required to download MANO model from MANO website. Create an account by clicking Sign Up and download the models (mano_v*_*.zip). Unzip and place the right hand model MANO_RIGHT.pkl under the mano_data/ folder. The WiLoR authors also host the same MANO_RIGHT.pkl in their Hugging Face Space:

wget https://huggingface.co/spaces/rolpotamias/WiLoR/resolve/main/mano_data/mano/MANO_RIGHT.pkl -P ./mano_data/

Note that MANO model falls under the MANO license.

Quick Installation

Thanks to @warmshao WiLoR can now be installed using a single pip command:

pip install git+https://github.com/warmshao/WiLoR-mini

Please head to WiLoR-mini for additional details.

Note: the above code is a simplified version of WiLoR and can be used for demo only.

Demo

python demo.py --img_folder demo_img --out_folder demo_out --save_mesh --fast

[🚀 Update March 2026]: Use argument --fast for up to 1.6 faster inference.

Live Intel RealSense

Create the Pixi environment and show the predicted MANO mesh live on the RGB stream of a connected RealSense camera:

pixi install
pixi run realsense

The demo selects the camera automatically when exactly one is connected. If multiple cameras are connected, choose one with pixi run realsense -- --serial SERIAL. Add --fast to use FP16 and WiLoR's depth pruning. The official WiLoR assets (detector.pt, wilor_final.ckpt, and licensed MANO_RIGHT.pkl) must be installed as described above.

For XHand retargeting, the live demo also sends the 21 right-hand MANO joints as JSON UDP packets to 127.0.0.1:5005. Set --udp-port 0 to disable it.

Live Meta Quest RGB camera

With an authorized Quest 3 connected over ADB and the pinned scrcpy server installed, preview the passthrough RGB camera or run WiLoR on its H.264 stream:

pixi run quest-preview
pixi run quest --fast

Run one command at a time. Use --eye right to select the right camera. Joint publishing is disabled by default; add --udp-port 5005 to the WiLoR command to send right-hand joints to the existing retargeter. See the Quest setup and validation guide for tool installation, WiVRn coexistence, timestamp semantics, and test limitations.

Start a local gradio demo

You can start a local demo for inference by running:

python gradio_demo.py

WHIM Dataset

To download WHIM dataset please follow the instructions here

Acknowledgements

Parts of the code are taken or adapted from the following repos:

• HaMeR
• Ultralytics

License

WiLoR models fall under the CC-BY-NC--ND License. This repository depends also on Ultralytics library and MANO Model, which are fall under their own licenses. By using this repository, you must also comply with the terms of these external licenses.

Citing

If you find WiLoR useful for your research, please consider citing our paper:

@misc{potamias2024wilor,
    title={WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild},
    author={Rolandos Alexandros Potamias and Jinglei Zhang and Jiankang Deng and Stefanos Zafeiriou},
    year={2024},
    eprint={2409.12259},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}