Topics, Exams and Homework Assignments

Week Dates Topics Sections Covered Homework Due
1Aug 20—Aug 24 Perceptron, binary classification, Bayesian inference. Prior and posterior likelihood. Constructing maximum likelihood decoders. 39.1, 39.2, 39.3, 39.4
H1:  H1
Sep 4
2Aug 27—Aug 31 Learning by steepest descent. Barzilai-Borwain formula and variable learning rate. Bayes formula with marginalization. Soft-max and decoding of lossy 7-bar display codes. 39.1, 39.2, 39.3, 39.4
H2:  H2
Sep 19
3Sep 3—Sep 7 Multi-layer neural networks. The patternnet architecture. Soft-max layer. Backpropagation as the fundamental training algorithm. 44.1, 44.2, 44.3, 44.4
H3:  H3
Oct 3
4Sep 10—Sep 14 Principal Component Analysis (PCA, KLT). The MNIST data set and its protocol. Binary data formats. Multi-layer neural networks. The patternnet architecture. Soft-max layer. 44.1, 44.2, 44.3, 44.4
5Sep 17—Sep 21 Logistic regression and multi-class logistic regression. Discussion of coordinate-free calculations and Frechet derivative. The derivative, the gradient and the second derivative of the multi-class logistic regression network. 44.1, 44.2, 44.3, 44.4, paper written by instructor
5Oct 15—Oct 19 Using Hopfield networks as associative memory. Convergence of Hopfield network. Improving weights with Hebb rule. 42.4, 42.5, 42.6, 42.7, 42.8, 42.9
H3:  H4
Oct 30
5Oct 22—Oct 26 Message Passing. Parallel programming in MATLAB. Threads, workers, labs, threads. Synchronization of threads with barriers. Collective communications, gop (Global Operation). The spmd (Single Program Multiple Data) block. Implementation of the soldier counting algorithm. 16.1, 16.2, 16.3, 16.4
5Oct 29—Nov 2 More Parallel programming in MATLAB. Modeling with Gaussian Processes. Applications to "Plastic" challenge data. 16.1, 16.2, 16.3, 16.4, 45.1, 45.2, 45.3
5Nov 5—Nov 9 Modeling with Gaussian Processes. The Bayesian point of view on splines. The Gibbs distribution for sampling from a Gaussian process. Filtering Gaussian process, e.g. integration. 45.1, 45.2, 45.3, 45.4, 45.5, 45.6, 45.7
H5:  H5
Nov 28