AI Evaluation and Bake‑off Strategies

This unit synthesizes historical AI breakthroughs and a practical framework for building, testing, and selecting AI solutions for retail analytics. Students learn to design and run model bake‑offs (speech‑to‑text, translation, vision, and generative models), interpret results, and translate outcomes into actionable retail deployments and next steps.

Learning Objectives

• Analyze major AI milestones (speech‑to‑text, translation, game play, transportation, and creative AI) and summarize their implications for retail analytics use cases and timelines
• Design and document a model bake‑off for a specified retail problem, including test datasets, performance metrics, evaluation criteria, and ethical/operational constraints
• Evaluate competing AI service outputs (e.g., speech‑to‑text, translation, or generative models) using quantitative metrics and qualitative criteria, and recommend a winner with justification
• Apply chosen model outputs to create an actionable retail deliverable (such as multilingual customer summaries, automated notes, or prioritized improvement plan) and outline next‑step deployment considerations