PhD Defence • Natural Language Processing • Novel Methods for Natural Language Modeling and Pretraining
Please note: This PhD defence will take place online.
He (Richard) Bai, PhD candidate
David R. Cheriton School of Computer Science
Supervisor: Professor Ming Li
This thesis is about modeling text and speech sequences to achieve lower perplexity, better generation, and benefit downstream language tasks; specifically, we address the problem of modeling natural language sequences (text and speech) with Transformer-based language models. We present three new techniques that improve sequence modeling in different ways.