LLM Data, Experimentation, and Inference

This unit guides students through ethical dataset creation, pragmatic engineering for dataset pipelines, and the experimental practices needed to select and optimize large language models. Students will apply experiment management and inference optimization techniques and prepare models for fine-tuning and deployment.

Learning Objectives

• Analyze ethical, legal, and bias-related considerations when building and curating datasets for LLM training and fine-tuning, including privacy, licensing, and mitigation strategies
• Design and implement reproducible, ‘just‑enough’ data engineering pipelines (validation, annotation, storage, and versioning) to prepare datasets for experiments and fine-tuning
• Apply experiment management workflows to track model selection, hyperparameters, datasets, and metrics, and optimize LLM inference using techniques such as batching, quantization, and caching
• Evaluate LLM selection and fine-tuning strategies by running controlled experiments, interpreting results with appropriate metrics, and recommending deployment-ready inference configurations