Essentially, all models are wrong, but some are useful. — George E. P. Box

Week | Dates | Topics | Sections Covered | Homework | Due |
---|---|---|---|---|---|

1 | Aug 20—Aug 24 | Perceptron, binary classification, Bayesian inference. Prior and posterior likelihood. Constructing maximum likelihood decoders. | 39.1, 39.2, 39.3, 39.4 | H1: H1 | Sep 4 |

2 | Aug 27—Aug 31 | Learning by steepest descent. Barzilai-Borwain formula and variable learning rate. Bayes formula with marginalization. Soft-max and decoding of lossy 7-bar display codes. | 39.1, 39.2, 39.3, 39.4 | H2: H2 | Sep 19 |

3 | Sep 3—Sep 7 | Multi-layer neural networks. The patternnet architecture. Soft-max layer. Backpropagation as the fundamental training algorithm. | 44.1, 44.2, 44.3, 44.4 | H3: H3 | Oct 3 |

4 | Sep 10—Sep 14 | Principal Component Analysis (PCA, KLT). The MNIST data set and its protocol. Binary data formats. Multi-layer neural networks. The patternnet architecture. Soft-max layer. | 44.1, 44.2, 44.3, 44.4 | ||

5 | Sep 17—Sep 21 | Logistic regression and multi-class logistic regression. Discussion of coordinate-free calculations and Frechet derivative. The derivative, the gradient and the second derivative of the multi-class logistic regression network. | 44.1, 44.2, 44.3, 44.4, paper written by instructor | ||

5 | Oct 15—Oct 19 | Using Hopfield networks as associative memory. Convergence of Hopfield network. Improving weights with Hebb rule. | 42.4, 42.5, 42.6, 42.7, 42.8, 42.9 | H3: H4 | Oct 30 |

5 | Oct 22—Oct 26 | Message Passing. Parallel programming in MATLAB. Threads, workers, labs, threads. Synchronization of threads with barriers. Collective communications, gop (Global Operation). The spmd (Single Program Multiple Data) block. Implementation of the soldier counting algorithm. | 16.1, 16.2, 16.3, 16.4 | ||

5 | Oct 29—Nov 2 | More Parallel programming in MATLAB. Modeling with Gaussian Processes. Applications to "Plastic" challenge data. | 16.1, 16.2, 16.3, 16.4, 45.1, 45.2, 45.3 | ||

5 | Nov 5—Nov 9 | Modeling with Gaussian Processes. The Bayesian point of view on splines. The Gibbs distribution for sampling from a Gaussian process. Filtering Gaussian process, e.g. integration. | 45.1, 45.2, 45.3, 45.4, 45.5, 45.6, 45.7 | H5: H5 | Nov 28 |