| Date | Topic | Slides | Slides (Flat) | Readings |
|---|---|---|---|---|
| Tue, 01/07 | Introduction | SLP2: Chapter 1 | ||
| Quiz, Solution | ||||
| Thu, 01/09 | Regular Expressions |
SLP3: Chapter 2.1
Notre Dame CSE 30151 Unit 1 |
||
| Probability, Basic Information Theory | TTIC 31230 Lectures 1 & 8 | |||
| Tue, 01/14 | Statistical Methods | |||
| Thu, 01/16 | Word, Tokenization |
SLP3: Chapters 2.2-2.8
Kudo (2018): will be clearer with knowledge in language modeling Li (2024) |
||
| Tue, 01/21 | Edit Distance, Distributional Lexical Semantics |
SLP3: Chapter 6
Levy and Goldberg (2014) |
||
| Wed, 01/22 | Assignment 1 Released |
PDF
process_childes.py test_process_childes.py |
||
| Thu, 01/23 | Datasets | Gema et al. (2024) | ||
| Tue, 01/28 | Text Classifiers |
PRML: Chapters 4 and 7.1
Peng et al. (2021) : kernel trick in the LLM era |
||
| Thu, 01/30 | Neural Networks: optimization, MLP, CNN, RNN, Transformers |
A summary of optimizers
Chris Olah's blog on LSTMs Socher et al. (2011) Kim (2014) |
||
| Tue, 02/04 | Neural Networks (cont.) | |||
| Thu, 02/06 | Language Modeling |
SLP 3: Chapter 3
TTIC 31230 Lecture 5 |
||
| Tue, 02/11 | Sequence Labeling: Hidden Markov Models |
SLP 3: Chapter 17.4
SLP 3: Chapter 23 |
||
| Wed, 02/12 | Assignment 1 Due | |||
| Thu, 02/13 | Neural Sequence Labeling, Conditional Random Fields | SLP 3: Chapter 17.5 | ||
| Tue, 02/18 | Reading Week, No Class | |||
| Thu, 02/20 | Reading Week, No Class | |||
| Tue, 02/25 | Syntax: Weighted Context-Free Grammars, CKY Algorithm |
SLP 3: Chapter 18
SLP 3: Appendix D Hale, 2003 |
||
| Wed, 02/26 | Assignment 2 Released | |||
| Thu, 02/27 | Syntax: Dependency Grammars, Collins' Algorithm |
SLP 3: Chapter 19
Eisner's Algorithm |
||
| Fri, 02/28 | 1-Page Project Midterm Checkin Due | |||
| Tue, 03/04 | Semantics: Compositionality, Semantic Role Labeling, Lambda Calculus | SLP 3: Chapter 21 | ||
| Thu, 03/06 | Grounded Semantics | Shi (2024): Chapter 2.2 | ||
| Tue, 03/11 | Pragmatics: Phenomena, Rational Speech Act Model, Pragmatics in Modern LMs |
Frank &
Goodman
(2012)
Wang et al. (2016) |
||
| Thu, 03/13 | Linguistic Typology and Cross-Lingual NLP |
Ponti et al. (2019)
The World Atlas Database |
||
| Tue, 03/18 | Language Models as Human Cognitive Models |
McCoy et al. (2024)
Hu et al. (2024) |
||
| Wed, 03/19 | Assignment 2 Due | |||
| Thu, 03/20 | How Training Language Models Benefits Linguistic Theories | Kuribayashi et al. (2024) | ||
| Tue, 03/25 | Critical Review of Scientific Papers | |||
| Wed, 03/26 | Project Due, Peer Review Starts | |||
| Thu, 03/27 | Advanced topic/Guest Lecture I | |||
| Tue, 04/01 | Advanced topic/Guest Lecture II | |||
| Thu, 04/03 | Course Review | |||
| Wed, 04/08 | Project Peer Review Due |