A scalable, agentic-first, and HuggingFace-native RL framework for research (9k lines).
Best practices & guides on how to write distributed pytorch training code
From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.
Repo for Qwen Image Finetune
Find why PyTorch training is slow, compare runs, and catch performance regressions in CI.
A native PyTorch backend for AWS Trainium with support for eager execution, torch.compile, distributed training, and custom NKI kernels
Educational distributed training and inference library for heterogeneous local hardware: Mac minis, Raspberry Pis and GPUs over plain Python sockets. FSDP, GRPO, parameter-server, expert and data parallelism.
Write once, run anywhere; ezpz 🍋
A comprehensive hands-on guide to building production-grade distributed applications with Ray - from distributed training and multimodal data processing to inference and reinforcement learning.
Research platform for model training, evaluation, and experimentation across architectures, benchmarks, and recipes.
META LLAMA3 GENAI Real World UseCases End To End Implementation Guide
AWS Neuron's extension of TorchTitan that adds AWS Trainium support with TorchNeuron for training generative AI models
Interactive exercises for understanding compute, communication, and dependencies in distributed LLM training traces.
Fast and easy distributed model training examples.
Forge kernels — fused Triton kernels for faster, leaner LLM fine-tuning. One-call patching into Hugging Face models via forge.patch(model), with FSDP2 multi-GPU support and a growing set of kernels and architectures. Every speedup backed by a committed benchmark. Apache-2.0.
To associate your repository with the fsdp topic, visit your repo's landing page and select "manage topics."