| Date | Topic | Slides | Slides (Flat) | Readings |
|---|---|---|---|---|
| Tue, 01/07 | Introduction | SLP2: Chapter 1 | ||
| Quiz, Solution | ||||
| Thu, 01/09 | Regular Expressions |
SLP3: Chapter 2.1
Notre Dame CSE 30151 Unit 1 |
||
| Probability, Basic Information Theory | TTIC 31230 Lectures 1 & 8 | |||
| Tue, 01/14 | Statistical Methods | |||
| Thu, 01/16 | Word, Tokenization |
SLP3: Chapters 2.2-2.8
Kudo (2018): will be clearer with knowledge in language modeling Li (2024) |
||
| Tue, 01/21 | Edit Distance, Distributional Lexical Semantics |
SLP3: Chapter 6
Levy and Goldberg (2014) |
||
| Wed, 01/22 | Assignment 1 Released |
PDF
process_childes.py test_process_childes.py |
||
| Thu, 01/23 | Datasets | Gema et al. (2024) | ||
| Tue, 01/28 | Text Classifiers |
PRML: Chapters 4 and 7.1
Peng et al. (2021) : kernel trick in the LLM era |
||
| Thu, 01/30 | Neural Networks: optimization, MLP, CNN, RNN, Transformers |
A summary of
optimizers
Chris Olah's blog on LSTMs Socher et al. (2011) Kim (2014) |
||
| Tue, 02/04 | Neural Networks (cont.) | |||
| Thu, 02/06 | Language Modeling |
SLP 3: Chapter 3
TTIC 31230 Lecture 5 |
||
| Tue, 02/11 | Language Modeling (cont.) | |||
| Wed, 02/12 | Assignment 1 Due | |||
| Thu, 02/13 | Campus Closed: No Class | |||
| Tue, 02/18 | Reading Week, No Class | |||
| Thu, 02/20 | Reading Week, No Class | |||
| Tue, 02/25 | Masked Language Models & Sequence Labeling |
Devlin et al., 2019 SLP 3: Chapter 17.4 |
||
| Thu, 02/27 | Sequence Labeling: Conditional Random Fields | SLP 3: Chapter 23 | ||
| Fri, 02/28 | 1-Page Project Midterm Checkin Due | |||
| Tue, 03/04 | Syntax: Constituency Parsing |
SLP 3: Chapter 18
SLP 3: Appendix D Hale, 2003 |
||
| Wed, 03/05 | Assignment 2 Released | |||
| Thu, 03/06 | Syntax: Dependency Parsing |
SLP 3: Chapter 19
Eisner's Algorithm |
||
| Tue, 03/11 | Semantics: Compositionality, Semantic Role Labeling, Lambda Calculus | SLP 3: Chapter 21 | ||
| Thu, 03/13 | Grounded Semantics | Shi (2024): Chapter 2.2
Mollo and Millière (2023) |
||
| Tue, 03/18 | Pragmatics: Phenomena, Rational Speech Act Model, Pragmatics in Modern LMs |
Frank &
Goodman
(2012)
Wang et al. (2016) |
||
| Thu, 03/20 | Linguistic Typology and Cross-Lingual NLP |
Ponti et al. (2019)
The World Atlas Database |
||
| Tue, 03/25 | Language Models as Human Cognitive Models |
McCoy et al. (2024)
Hu et al. (2024) Lasnik and Lidz (2016) |
||
| Wed, 03/26 | Assignment 2 Due Project Due, Peer Review Starts |
|||
| Thu, 03/27 | How Training Language Models Benefits Linguistic Theories | Misra and Mahowald (2024), Annotated |
Kuribayashi et al. (2024) Misra and Mahowald (2024) |
|
| Tue, 04/01 | Course Review | Final Practice | Final Practice Solution | |
| Thu, 04/03 | Extended Office Hour | |||
| Mon, 04/14 | Project Peer Review Due |