|
Natural Language Processing
Prof. Jason Eisner
Course # 601.465/665 — Fall 2026
|
|
Announcements
- 8/18/26 First class is Mon 8/31, 3pm, Remsen 101. As explained on the syllabus, please keep MWF 3-4:30 pm open to accommodate a variable class schedule as well as office hours after class. Our weekly recitations are Tue 6-7:30 pm in the same room.
- 8/18/26 Please bookmark this
page.
All enrolled students will soon be added to Piazza.
Later, when Homework 1 is due, we will tell you how to join Gradescope.
Key Links
- Syllabus -- reference info about
the course's staff, meetings, office hours, textbooks, goals,
expectations, and policies. May be updated on occasion.
- Piazza
site for discussion and announcements. Sign up, follow, and participate!
- Gradescope
for submitting your homework.
- Office hours for the course staff.
- Video recordings (see policy on syllabus)
Schedule
Warning: The schedule below is adapted from last year's schedule and may still change! Links to future lecture slides, homeworks, and dates currently point to last year's versions. Watch Piazza for important updates, including when assignments are given and when they are due.
What's Important? What's Hard? What's Easy? [1 week]
Mon 8/31:
Wed 9/2:
Fri 9/4:
- Uses of language models
- Language ID
- Text categorization
- Spelling correction
- Segmentation
- Speech recognition
- Machine translation
- Optional reading about n-gram language models: J&M 3 (or M&S 6)
Probabilistic Modeling [1 week]
Mon 9/7 (Labor Day: no class)
Wed 9/9,
Fri 9/11:
- Probability concepts
- Joint & conditional prob
- Chain rule and backoff
- Modeling sequences
- Surprisal, cross-entropy, perplexity
- Optional reading about probability, Bayes' Theorem, information theory: M&S 2; slides by Andrew Moore
- Smoothing n-grams (video lessons, 52 min. total)
- Maximum likelihood estimation
- Bias and variance
- Add-one or add-λ smoothing
- Cross-validation
- Smoothing with backoff
- Good-Turing, Witten-Bell (bonus slides)
- Optional reading about smoothing: M&S 6; J&M 4; Rosenfeld (2000)
- HW2 given: Probabilities
Mon 9/14:
- Bayes' Theorem
- Log-linear models (self-guided interactive visualization with handout)
- Parametric modeling: Features and their weights
- Maximum likelihood and moment-matching
- Non-binary features
- Gradient ascent
- Regularization (L2 or L1) for smoothing and generalization
- Conditional log-linear models
- Application: Language modeling
- Application: Text categorization
- Optional readings about log-linear models: Collins (pp. 1-4), Smith (section 3.5), J&M 5
Grammars and Parsers [3- weeks]
Wed 9/16:
- HW1 due
- In-class discussion of HW1
- Improving CFG with attributes (video lessons, 62 min. total)
- Morphology
- Lexicalization
- Post-processing (CFG-FST composition)
- Tenses
- Gaps (slashes)
- Optional reading about syntactic attributes: J&M 15 (2nd ed.)
Wed 9/16 (continued),
Fri 9/18,
Tue 9/22 (swap Mon 9/21 lecture with recitation):
Wed 9/23,
Fri 9/25:
Mon 9/28:
- HW2 due
- Quick in-class quiz: Log-linear models
- Probabilistic parsing
- PCFG parsing
- Dependency grammar
- Lexicalized PCFGs
- Optional reading on probabilistic parsing: M&S 12, J&M Appendix C
Wed 9/30:
Fri 10/2:
Representing Meaning [1 week]
Mon 10/5,
Wed 10/7,
Mon 10/12:
- HW3 due on Sun 10/11
- Semantics
- What is understanding?
- Lambda terms
- Semantic phenomena and representations
- More semantic phenomena and representations
- Adding semantics to CFG rules
-
Compositional semantics
-
Optional readings on semantics:
- HW5 given: Semantics
Midterm
Fri 10/9:
- Midterm exam (3-4:30, in classroom)
Representing Everything: Deep Learning for NLP [1+ week]
Wed 10/14,
Fri 10/16,
Mon 10/19,
Wed 10/21:
- Back-propagation (video lesson, 33 min.)
- Neural methods
- Vectors, matrices, tensors; PyTorch operations; linear and affine operations
- Log-linear models, temperatures, learned features, nonlinearities
- Vectors as an alternative semantic representation
- Training signals: Categorical labels, similarity, matching
- Encoders and decoders
- End-to-end training, multi-task training, pretraining + fine-tuning
- Self-supervised learning
- word2vec (skip-gram / CBOW)
- Recurrent neural nets (RNNs, BiRNNs, ELMo)
- Optional reading about neural nets and RNNs: J&M 7, 8
Fri 10/23 (fall break: no class)
Unsupervised Learning [1+ week]
Mon 10/26,
Wed 10/28:
Fri 10/30,
Mon 11/2:
Discriminative Modeling [1- week]
Wed 11/4,
Fri 11/6:
Deep Learning for Structured Prediction; Transformers [1- week]
Mon 11/9,
Wed 11/11:
- Neural methods (continued)
- seq2seq: Structure prediction via sequence prediction (or via tagging)
- Decoders: Exact, greedy, beam search, independent, dynamic programming, stochastic, Minimum Bayes Risk (MBR)
- Attention
- Transformers (encoder-decoder, encoder-only (BERT), decoder-only (LM))
- Positional embeddings
- Tokenization
- Parameter-efficient fine tuning, distillation, RLHF (REINFORCE, PPO, DPO)
- Optional Reading on Transformers: The Illustrated Transformer; J&M 9, J&M 11;
GPT-2 spreadsheet
Harnessing Large Language Models [1+ week]
Fri 11/13,
Mon 11/16,
Wed 11/18, Fri 11/20:
NLP Applications [2 weeks]
Mon 11/23,
Wed 11/25,
Fri 11/27:
Mon 11/30,
Wed 12/2,
Fri 12/4,
Mon 12/7,
Wed 12/9,
Fri 12/11:
- HW7 due on Tue 12/1
- Current NLP tasks and competitions
- The NLP research community
- Text annotation tasks
- Other types of tasks
-
Optional reading: Explore links in the "NLP tasks" slides!
- HW8 due on Fri 12/11
Final
Exam period (12/15 - 12/23):
- Final exam review session (date TBA)
- Final exam (Wed 12/16, 6pm-9pm, Remsen 101)
Unofficial Summary of Homework Schedule
These dates were copied from the schedule above, which is subject to change.
Homeworks are due approximately every two weeks, with longer homeworks getting more time. But the
homework periods are generally longer than two weeks -- they overlap. This gives you more flexibility
about when to do each assignment, which is useful if you have other classes and activities.
We assign homework n as soon as you've seen the lectures you need, rather than waiting
until after homework n-1 is due. So you can jump right in while the material is fresh.
- HW1 (grammar): given Wed 9/2, due Wed 9/16
- HW2 (probability): given Fri 9/11, due Mon 9/28
- HW3 (empiricism): given Wed 9/16, due Sun 10/11
- Midterm: Fri 10/9
- HW4 (algorithms): given Wed 9/30, due Mon 10/26
- HW5 (logic): given Fri 10/9, due Mon 11/2
- HW6 (unsupervised learning): given Wed 10/28, due Fri 11/13
- HW7 (discriminative learning): given Fri 11/6, due Tue 12/1
- HW8 (large language models): given Mon 11/16, due Fri 12/11 (last day of class)
Recitation Schedule
Recitations are normally held on Tuesdays (see the syllabus). Enrolled students are expected to attend the recitation and participate in solving practice problems. This will be more helpful than an hour of solo study. The following schedule is subject to change.
Old Materials
Lectures from past years, some still useful:
Old homeworks:
