From theory to practice
An idea becomes useful when you can test it. Build simple predictive models and discover where counting reaches its limits.
YOUR STUDY DESK
01
Read the lesson
Follow the ideas, mathematics, and worked examples.
Open lesson 02Open your notebook
Test your understanding with hands-on experiments.
Start experimenting 03Watch the conversation
Learn alongside the recorded session.
View recordingsReadings
Essential
N-gram Language Models
- Andrej Karpathy: The spelled-out intro to language modeling — The inspiration for this chapter. Watch the bigram section (first ~45 minutes) to see the same concepts explained differently.
Statistical NLP History
- Chapter 3: N-gram Language Models from Speech and Language Processing by Jurafsky & Martin — The classic textbook treatment, including smoothing techniques
Optional
Markov Models
- Markov Chains — Interactive visualization of transition matrices and state transitions
Data Sparsity Problem
- The Unreasonable Effectiveness of Data by Halevy, Norvig, Pereira (2009) — Google’s famous paper on why simple models with lots of data often beat sophisticated models with less data