Guides and Examples

Attention

This site was previously dedicated solely to fine-tuning LLMs. I have since expanded the scope to include other topics. The process of converting the site is still in process, so you might encounter broken links or other issues. Let me know if you do! You can submit an issue with the GitHub button on the top right.

This repository contains a collection of Jupyter notebooks demonstrating various concepts and techniques across different fields. Currently, it includes a series on fine-tuning language models, but it will expand to cover other topics in the future.

AI Training: Fine-Tuning LLMs

The AI Training section currently focuses on fine-tuning language models. It shows how to fine-tune models starting from smaller, single-GPU setups and gradually scaling up to multi-GPU and multi-node configurations.

These examples aim to bridge the gap between single-node single-GPU training and various forms of distributed training, serving as a practical and conceptual guide for scaling up model training.

How to use this repository

The examples in this repository are organized by topic. Within each topic, the notebooks are intended to be read sequentially. Later examples often build on earlier examples and gradually add complexity.

AI Training

• LLM Training/Fine-tuning
  • Fine-Tuning T5-Small on a Single GPU
  • Fine-Tuning GPT-2 on a Single GPU
  • Fine-Tuning TinyLlama on a Single GPU
  • Instruction Tuning Olmo-1b
  • Getting Started with Axolotl: Fine-Tuning Gemma 2B
• Appendix
  • Data Preprocessing

Web Development

• FastHTML
  • Introduction
  • Changing the Appearance of FastHTML Applications and Components
  • HTMX: Server-Driven Interactivity
  • Using a Database