New (shorter) lecture! Over-optimization, foundations of reward hacking, sycophancy, verbosity, etc. In recording this, I realized that rubrics are going to be prone to overopt in a way like reward models, where RLVR is its own thing. This is mostly fundamentals, history, and reflections!

00:00 Intro & Why We Care About Over-optimization
04:20 Part 1: Over-optimization & Goodhart's Law
09:57 Signatures of Over-Optimization & Misalignment
14:47 Part 2: Beyond "Just Style"
20:13 Llama 4 & Gaming the Leaderboards
23:04 Course Recap & Conclusion

Primarily on Chapter 14.