Hamel Husain@HamelHusainMay 10, 2025
@HamelHusain and @sh_reya are teaching a course about AI evals that starts later this month. (There's also a cohort starting in July.) You should think about taking the course!
If you're building production generative AI applications, evals are critical. Hamel and Shreya are expert practitioners and terrific teachers. I took one of Hamel's courses last year and it was fantastic. Click through Hamel's thread for a link to sign up to this one.
Hamel is joining us in the Voice AI course this Wednesday to give an overview about evals. We'll also have a roundtable Q&A with Hamel and engineers who work on multimodal evals and LLM ops tools at @covaldev, @freeplay_ai, and @wandb.
Hamel Husain@HamelHusainIt is very easy to make mistakes when creating evals for your AI product. @sh_reya and I run through the most common mistakes in this talk (with memes 🌶️!) . Chapter summaries below:
00:51 Foundation model benchmarks are not the same as your application evals
03:00 Generic
