PhD Defence • Machine Learning | Deep Leaning • Symbolic Regression and Sequence Modelling with Conditional and Dynamic Language Models

Please note: This PhD defence will take place in DC 3317 and online.

Mojtaba Valipour, PhD candidate
David R. Cheriton School of Computer Science

Supervisor: Professor Ali Ghodsi

In an era where the boundaries of machine learning are continuously being pushed, this thesis presents two more advancements in the field of deep learning and artificial intelligence, with a focus on symbolic regression and dynamic training methodologies for neural networks. The first major contribution, SymbolicGPT, introduces a novel approach to symbolic regression using a transformer-based language model. This model significantly outperforms traditional methods by leveraging the strengths of probabilistic language models for improved accuracy and efficiency. The second theme of this thesis revolves around dynamic training methodologies, aimed at enhancing the adaptability and computational efficiency of neural networks under varying constraints. Within this framework, we introduce DyLoRA and SortedNet as key innovations. DyLoRA offers a dynamic, search-free low-rank adaptation technique, enabling models to adjust their complexity on-the-fly without extensive retraining. SortedNet proposes a generalized framework for embedding multiple neural network architectures within a single model, facilitating efficient model selection and adaptation. Extending SortedNet, SortedLLama applies these principles to large language models, demonstrating efficient dynamic inference capabilities.

To attend this PhD defence in person, please go to DC 3317. You can also attend virtually using MS Teams at https://teams.microsoft.com/l/meetup-join/19%3ameeting_MzE4YTdjYmUtNTdiOC00NjZmLTg0ZWEtZmJmMGJkN2JmZDFk%40thread.v2/0?context=%7b%22Tid%22%3a%22723a5a87-f39a-4a22-9247-3fc240c01396%22%2c%22Oid%22%3a%22eb655fbd-d1b9-49bd-9076-57d835f23005%22%7d.