Revised November 19, 2025
CS 479: Neural Networks
General description
This course surveys computation by models of networks of neurons. It covers a variety of methods for designing and training both feedforward and recurrent neural networks. We will study and implement supervised and unsupervised learning methods, as well as some learning algorithms that conform to the biological constraints of neuroscience. We will examine issues in neural learning, such as generalizability and adversarial inputs, and survey some mitigating measures. Throughout the course, discussions will address the relationship of artificial neural networks and neuroscience.
Logistics
Audience
- Students interested in artificial intelligence, or neuroscience
Normally available
Related courses
- Pre-requisites: (One of STAT 230, STAT 206 with at least 60%, STAT 240)
- Successors: None
- Co-requisites: (One of CS 335, CS 370, CS 371/AMATH 242)
- Anti-requisites: None
- Conflicts: AMATH 449
For official details, see the UW calendar.
Software/hardware used
- Python 3
- Pytorch
- Jupyter notebook
Typical reference(s)
- P Dayan and L.F. Abbott, Theoretical Neuroscience, 2001, Massachusetts Institute of Technology Press
- M Neilsen, Neural Networks and Deep Learning , 2017, Determination press
- I Goodfellow, Y Bengio, and A Courville, Deep Learning , 2016, Massachusetts Institute of Technology Press
- C Eliasmith and C Anderson, Neural Engineering , 2003, Massachusetts Institute of Technology Press
Required preparation
At the start of the course, students should be able to
- Program in a high-procedural programming language and use control instructions, such as loops and conditional statements, basic datatypes, and simple data structures, such as arrays.
- Calculate derivatives, partial derivatives, Taylor series, max/min of multivariate functions, and other basic concepts encountered in calculus.
- Use matrix operations such as inversion and multiplication.
- Compute conditional probabilities using Bayes' theorem.
Learning objectives
At the end of the course, students should be able to
- Write a program to simulate the activity of a network of neurons
- Formulate a neural learning method as gradient-based optimization
- Derive a neural learning method based on energy minimization
- Encode and decode data from the activity of a population of neurons
- Evaluate implementation, designs, and optimization strategies
- Identify some commonalities between artificial neural networks and the brain
Typical syllabus
Networks of neurons (4 hours)
- Neuron models, spiking vs firing-rate
- Activation curves
- Synapses
- Networks of neurons
- Overview of Learning: Supervised/Unsupervised/Reinforcement
Supervised learning (9 hours)
- Training, Validation, Testing
- Universal Approximation Theorem
- Cost functions
- Gradient descent
- Error Backpropagation
- Automatic differentiation
- Overfitting and generalizability (regularization)
- Optimization considerations (vanishing gradients, SGD)
- Convolutional neural networks
- Batch normalization
Unsupervised learning (6 hours)
- Autoencoders
- Variational autoencoders
- Hopfield Networks, Hopfield Energy
Population coding (7 hours)
- Optimal linear Decoding
- Transformations, dyynamics
- Online learning
Recurrent neural networks (3 hours)
- Back prop through time (BPTT)
- Long short-term Memories (LSTM)
- Legender delay networks (time permitting)
Adversarial attacks (2 hours)
- Targeted vs untargeted
- FGSM
- TRADES
Additional topics (5 hours)
- Vector Embeddings (word2vec)
- GANs
- Biological backprop models