Efficient Inference and Fine-Tuning

This unit teaches strategies for producing concise outputs, scaling inference efficiently, and adapting large language models with minimal compute and parameter changes. Students will learn model optimization and parameter-efficient fine-tuning techniques, analyze cost/performance tradeoffs, and receive an introduction to model selection to prepare for deployment and evaluation.

Learning Objectives

• Analyze summarization strategies and select appropriate abstractive or extractive methods for targeted use cases
• Apply batch prompting and batching techniques to design inference workflows that reduce latency and per-request cost
• Demonstrate model optimization and parameter-efficient fine-tuning methods (e.g., quantization, pruning, adapter or LoRA-style approaches) on sample models and report measurable effects
• Evaluate cost, latency, and accuracy tradeoffs across optimization and fine-tuning choices and recommend model selection criteria for given application requirements