Mechanics and Risks of Language Models

This unit examines how rule-based and statistical language systems work, how modern models learn and exhibit emergent behaviors, and why these properties matter for social engineering and deception. Students will study evaluation methods (Turing-style interrogation, CAPTCHA/’bot detection), the role of RLHF, and the ethical and security implications of anthropomorphic and sentience claims.

Learning Objectives

• Analyze the architectures and operational behaviors of rule-based and statistical language models, comparing their strengths, limitations, and vulnerabilities to manipulation or deceptive use.
• Apply evaluation techniques—including prompt analysis, Turing-style interrogation strategies, and CAPTCHA/’bot-detection methods—to distinguish machine-generated from human-generated language and assess evasion risk.
• Demonstrate understanding of emergent properties and reinforcement learning from human feedback (RLHF) by identifying when model outputs produce the illusion of empathy or sentience and explaining the causal factors.
• Evaluate the security, ethical, and social-engineering implications of conversational AI (including democratized access, CAPTCHA evasion, and public claims of sentience) and propose technical controls and policy mitigations.