Essentially, all models are wrong, but some are useful. — George E. P. Box

Information Type | Data |
---|---|

Meeting Time | MWF, 11:00—11:50 |

Meeting Room | MATH 124 |

Instructor | Professor Marek Rychlik |

Office | Mathematics 605 |

rychlik@email.arizona.edu | |

Telephone | 1-520-621-6865 |

Homepage | http://alamos.math.arizona.edu/math577 |

Homepage (Mirror) | http://marekrychlik.com/math577 |

Personnel | Day(s) of the Week | Hour | Room | Comment |
---|---|---|---|---|

Marek Rychlik | M | 1:00pm—1:50pm | Mathematics 605 | Regular Office Hours in my office |

Marek Rychlik | W | 12:00pm—12:50pm | Mathematics 605 | Regular Office Hours in my office |

Marek Rychlik | F | 12:00pm—12:50pm | Mathematics 605 | Regular Office Hours in my office |

- Five homework assignments, worth 16% of the course grade each, for a total of 80%
- A take-home Final Project worth 30%.

Homework, and take-home final, shall be submitted as a typed paper, with the exception of these graphs and figures which cannot be easily drawn with software. The work shall be submitted electronically, as a PDF document, through D2L, using the Dropbox feature of D2L.

The student is expected to be comfortable with scientific computing and MATLAB software. Most of the homework will involve writing short (50-200 lines) programs in MATLAB. Some most used MATLAB features will be covered in class.

- General mathematical background representative of a first year graduate student in mathematics, applied mathematics or a related scientific field.
- Familiarity with scientific programming.

Homework is assigned throughout the semester. Two types of homework problems will be assigned:

- Homework which consists of selected exercises in the book.
- Custom homework assignments created by the instructor and involving a programming component in MATLAB.

- Learn connectionist approach to programming, in particular neural computing with various types of neural networks: Hopfield, soft-max, multi-layer, pattern recognition, probabilistic, convolutional, recurrent, long-short memory, self-organizing etc.
- Learn how to scale up computer code to Big Data.
- Learn mathematics related to neural programming and big data, including Bayesian inference, optimization with stochastic gradient type methods, and Monte Carlo.
- Learn image and other signal processing basics.
- Solve representative problems of machine learning with neural networks and MATLAB.

Week | Dates | Topics | Sections Covered |
---|---|---|---|

1 | Aug 20—Aug 24 | Perceptron, binary classification, Bayesian inference. Prior and posterior likelihood. Constructing maximum likelihood decoders. | 39.1, 39.2, 39.3, 39.4 |

2 | Aug 27—Aug 31 | Learning by steepest descent. Barzilai-Borwain formula and variable learning rate. Bayes formula with marginalization. Soft-max and decoding of lossy 7-bar display codes. | 39.1, 39.2, 39.3, 39.4 |

3 | Sep 3—Sep 7 | Multi-layer neural networks. The patternnet architecture. Soft-max layer. Backpropagation as the fundamental training algorithm. | 44.1, 44.2, 44.3, 44.4 |

4 | Sep 10—Sep 14 | Principal Component Analysis (PCA, KLT). The MNIST data set and its protocol. Binary data formats. Multi-layer neural networks. The patternnet architecture. Soft-max layer. | 44.1, 44.2, 44.3, 44.4 |

5 | Sep 17—Sep 21 | Logistic regression and multi-class logistic regression. Discussion of coordinate-free calculations and Frechet derivative. The derivative, the gradient and the second derivative of the multi-class logistic regression network. | 44.1, 44.2, 44.3, 44.4, paper written by instructor |

5 | Oct 15—Oct 19 | Using Hopfield networks as associative memory. Convergence of Hopfield network. Improving weights with Hebb rule. | 42.4, 42.5, 42.6, 42.7, 42.8, 42.9 |

5 | Oct 22—Oct 26 | Message Passing. Parallel programming in MATLAB. Threads, workers, labs, threads. Synchronization of threads with barriers. Collective communications, gop (Global Operation). The spmd (Single Program Multiple Data) block. Implementation of the soldier counting algorithm. | 16.1, 16.2, 16.3, 16.4 |

5 | Oct 29—Nov 2 | More Parallel programming in MATLAB. Modeling with Gaussian Processes. Applications to "Plastic" challenge data. | 16.1, 16.2, 16.3, 16.4, 45.1, 45.2, 45.3 |

5 | Nov 5—Nov 9 | Modeling with Gaussian Processes. The Bayesian point of view on splines. The Gibbs distribution for sampling from a Gaussian process. Filtering Gaussian process, e.g. integration. | 45.1, 45.2, 45.3, 45.4, 45.5, 45.6, 45.7 |

Students are expected to attend every scheduled class and to be familiar with the University Class Attendance policy as it appears in the General Catalog. It is the student's responsibility to keep informed of any announcements, syllabus adjustments or policy changes made during scheduled classes.

Students are expected to behave in accordance with the Student Code of Conduct and the Code of Academic Integrity. The guiding principle of academic integrity is that a student's submitted work must be the student's own. University policies can be found at http://policy.arizona.edu/academic.

See http://policy.web.arizona.edu/threatening-behavior-students. No prohibited behavior will be tolerated.

Students who miss the first two class meetings will be administratively dropped unless they have made other arrangements with the instructor.

Students are expected to be present for all exams. If a verifiable emergency arises which prevents you from taking an in-class exam at the regularly scheduled time, the instructor must be notified as soon as possible, and in any case, prior to the next regularly scheduled class. Make-up exams and quizzes will be administered only at the discretion of the instructor and only under extreme circumstances. If a student is allowed to make up a missed exam, (s)he must take it at a mutually arranged time. No further opportunities will be extended. Failure to contact your instructor as stated above or inability to produce sufficient evidence of a real emergency will result in a grade of zero on the exam. Other remedies, such as adjusting credit for other exams, may be considered.

Disabled students must register with Disability Resources and be identified to the course instructor through the University's online process in order to use reasonable accommodations.

It is the University's goal that learning experiences be as accessible as possible. If you anticipate or experience physical or academic barriers based on disability, please let me know immediately so that we can discuss options. You are also welcome to contact Disability Resources 520-621-3268 to establish reasonable accommodations.

Please be aware that the accessible table and chairs in this room should remain available for students who find that standard classroom seating is not usable.

The grade of "I" will be awarded if all of the following conditions are met:

- The student has completed all but a small portion of the required work.
- The student has scored at least 50% on the work completed.
- The student has a valid reason for not completing the course on time.
- The student agrees to make up the material in a short period of time.
- The student asks for the incomplete before grades are due, 48 hours after the final exam.

The information contained in the course syllabus, other than the grade and absence policies, is subject to change with reasonable advance notice, as deemed appropriate by the instructor.