LLM Fine-Tuning 🚀

A hands-on collection of 23 notebooks covering LLM fine-tuning (LoRA, QLoRA, PEFT, RLHF), Retrieval-Augmented Generation (RAG), knowledge graphs, quantization, and LLM evaluation — all runnable on Google Colab.

Fine-tune models like Llama 2, Mistral 7B, Falcon 7B, Gemma 2 9B, Phi-1.5, GPT-NeoX 20B, MPT-30B, Bloom, OPT, Vicuna, and GPT-3.5 Turbo using Hugging Face Transformers, TRL, Unsloth, AutoTrain, bitsandbytes, LangChain, Neo4j, and MLflow.

📂 Repository Structure

LLM-Finetuning/
├── fine-tuning/        # Supervised fine-tuning with LoRA, QLoRA, PEFT & more
├── rlhf/               # Reinforcement Learning from Human Feedback
├── rag/                # Retrieval-Augmented & Cache-Augmented Generation
├── knowledge-graphs/   # Knowledge graph construction & graph-based QA
├── quantization/       # Extreme (1-bit / 2-bit) quantization
└── evaluation/         # LLM & RAG pipeline evaluation with MLflow

🔧 Fine-Tuning

Notebook	Description	Colab
Efficiently Train LLMs with LoRA and Hugging Face	Efficient training of large language models using LoRA and Hugging Face.
Fine-Tune Your Own Llama 2 Model in Colab	Step-by-step guide to fine-tuning your own Llama 2 model in a Colab notebook.
Guanaco Chatbot Demo with LLaMA-7B	Chatbot demo powered by the LLaMA-7B model fine-tuned on the Guanaco dataset.
PEFT Fine-Tune Bloom-560m Tagger	Parameter-efficient fine-tuning of Bloom-560m for tagging tasks.
Fine-Tune Meta OPT-6.1b with bitsandbytes + PEFT	Fine-tuning the Meta OPT-6.1b model with 8-bit quantization and PEFT.
Fine-Tune Falcon-7b with BNB Self-Supervised Training	Fine-tuning Falcon-7b using bitsandbytes self-supervised training.
Fine-Tune Llama 2 with QLoRA	Fine-tuning the Llama 2 7B model using the PEFT library and the QLoRA method.
Stable Vicuna 13B 8-bit in Colab	Fine-tuning and running Stable Vicuna 13B in 8-bit precision.
GPT-NeoX-20B 4-bit Training	Training the GPT-NeoX-20B model using 4-bit quantization with bitsandbytes.
MPT-Instruct-30B Model Training	Training MosaicML's MPT-Instruct-30B for instruction following, QA and text generation.
Fine-Tune Microsoft Phi-1.5b on a Custom Dataset	SFT training of Microsoft Phi-1.5b on the DialogStudio dataset with TRL.
Fine-Tune OpenAI GPT-3.5 Turbo	Fine-tuning GPT-3.5 Turbo on your own data via the OpenAI fine-tuning API.
Fine-Tune Mistral 7B Using AutoTrain	Fine-tuning Mistral 7B with the AutoTrain-Advanced library.
Alpaca + Gemma 2 9B — Unsloth 2x Faster Fine-Tuning	Fine-tuning Gemma 2 9B on the Alpaca dataset with Unsloth for 2x faster training.

🎯 RLHF

Notebook	Description	Colab
RLHF Training for a Custom Dataset with Any Model	Train any LLM with Reinforcement Learning from Human Feedback on a custom dataset.

🔍 RAG (Retrieval-Augmented Generation)

Notebook	Description	Colab
RAG with LangChain	Build a Retrieval-Augmented Generation pipeline using LangChain.
CAG: Optimizing LLMs with Cache-Augmented Generation	Techniques from the Cache-Augmented Generation (CAG) paper to boost LLM efficiency and response speed.

🕸️ Knowledge Graphs

Notebook	Description	Colab
Neo4j and LangChain for Enhanced Question Answering	Build a knowledge graph with Neo4j and LangChain for PDF question answering.
OpenAI Functions: Constructing a Knowledge Graph for QA	Build a knowledge graph from text or PDF documents with OpenAI functions, Neo4j and a LangChain agent.
Convert Any Document to a Knowledge Graph	The easiest way to convert any document into a knowledge graph for your next RAG application, with LangChain and OpenAI.

🗜️ Quantization

Notebook	Description	Colab
HQQ 1-bit / 2-bit Quantization	Train a model with extreme 1-bit and 2-bit quantization using the HQQ framework.

📊 Evaluation

Notebook	Description	Colab
RAG Pipeline Evaluation Using MLflow	Comprehensive guide to evaluating a RAG pipeline with MLflow, following industry best practices.
Evaluate a Hugging Face LLM with mlflow.evaluate()	Evaluate a Hugging Face LLM end-to-end using mlflow.evaluate().

🚀 Getting Started

1. Click the Open in Colab badge next to any notebook — no local setup required (a free Colab T4 GPU is enough for most notebooks).
2. Or clone the repo and run locally:

git clone https://github.com/codeztech-atique/LLM-Finetuning.git
cd LLM-Finetuning

🤝 Contributing

Contributions are welcome! If you'd like to contribute to this project, feel free to open an issue or submit a pull request.

📝 License

This project is licensed under the MIT License.

---

Maintained with ❤️ by codeztech-atique