PhD Seminar • Computational Neuroscience • Toward a Model of Associative Memory via Predictive Coding

Please note: This PhD seminar will take place in DC 3317 and online.

Ehsan Ganjidoost, PhD candidate
David R. Cheriton School of Computer Science

Supervisor: Professor Jeff Orchard

The Predictive Coding Network (PCnet) is equipped with a local learning algorithm that enables it to resemble various applications in the brain, making it a suitable model for associative memory via predictive coding. Realizing a model of associative memory via predictive coding is significant, given the importance of both associative memory and predictive coding in Machine Learning (ML) and computational neuroscience.

The Hopfield Network (HN) uses the energy concept to model associative memory, where each stored pattern corresponds to a local minimum. Although the model benefits from a simple update rule, its capacity to retrieve patterns successfully is limited as the number of patterns increases. The Modern Hopfield Network (MHN) attempted to address this by introducing a new type of energy function, but this resulted in numerical instability. Our experiments showed that the MHN’s capacity is still limited in correlated patterns.

We implemented associative memory via PCnet and compared our model’s capacity with classic Hopfield, pushing storage capacity to the limit. Additionally, we contrasted our model’s retrieval ability with classic HN and MHN in the presence of various noise levels. We observed that PCnet successfully retrieves memories from input corruptions like noise or patching inputs.

Finally, we introduced the adoptive PCnet, which converges to the closest memory with minimal effort. Despite being less complex than the modern Hopfield net, PCnet retains its simplicity and consistency with biological constraints.

To attend this PhD seminar in person, please go to DC 3317. You can also attend virtually using Zoom at https://uwaterloo.zoom.us/j/98054078862.