Please note: This PhD defence will take place in DC 2314 and online.
Nolan Shaw, PhD candidate
David R. Cheriton School of Computer Science
Supervisor: Professor Jeff Orchard
This thesis explores two distinct research topics, both applying category theory to machine learning.
The first topic discusses Vector Symbolic Architectures (VSAs). I present the first attempt at formalising VSAs with category theory. VSAs are built to perform symbolic reasoning in high-dimensional vector spaces. I present a brief literature survey demonstrating that the topic is currently completely unexplored. I discuss some desiderata for VSA models, then describe an initial formalisation that covers two of the three desiderata. My formalisation focuses on two of the three primary components of a VSA: binding and bundling, and presents a proof of why element-wise operations constitute the ideal means of performing binding and bundling. The work extends beyond vectors, to any co-presheaves with the desired properties. For example, GHRR representations are captured by this generalisation.
The second line of work discusses, and expands upon, recent work by Milewski in the construction of \enquote{pre-lenses.} This work is motivated by pre-established formalisations of supervised machine learning. From the perspective of category theory, pre-lenses are interesting because they unify the category \Param, or \Learn, with its dual \coPara, or $\Learn^{\operatorname{op}}$. From a computer science perspective, pre-lenses are interesting because they enable programmers to build neural networks with vanilla function composition, and they unify various network features by leveraging the fact that they are profunctors. I replicate Milewski's code, extend it to the non-synthetic data, MNIST, implement re-parameterisations, and describe generative models as dual to discriminative models by way of pre-lenses. This work involved creating a simple dataloader to read in external files, randomising the order that inputs are presented during learning, and fixing some bugs that didn't manifest when training occurred on the very small dataset used by Milewski.
To attend this PhD defence in person, please go to DC 2314. You can also attend virtually on MS Teams.