Data 468, Math 468/568 - Applied Stochastic Processes

Syllabus for Math 468, Section 001, Spring 2022

Location and Times

This course meets TuTh 2-3:15pm. The class will meet as follows:

• Before February 3 February 8, the class will meet on Zoom (due to temporary restrictions on my physical mobility);
• Beginning on February 3 February 8, the class will meet in-person in BIOW 237.

Course Description

Course Prerequisites or Co-requisites

1. DATA 464 or MATH 464 - Theory of Probability.
2. or equivalent coursework with instructor permission.

Instructor and Contact Information

Information	Data
Instructor	Professor Marek Rychlik
Office	Mathematics 605
Telephone	1-520-621-6865
Email	rychlik@arizona.edu
Instructor Homepage/Web Server	http://alamos.math.arizona.edu
Course Homepage	http://alamos.math.arizona.edu/courses/math468
Course Homepage (Mirror)	http://marekrychlik.com/courses/math468

Course Format and Teaching Methods

The course format is that of a conventional lecture, with in-class discussion and additional web-delivered content. All lectures will be recorded and available on Zoom and Panopto.

I will incorporate regular, required programming assignments, as numerical experimentation is essential to understanding and using the course subject matter. Some more advanced programming and simulation exercises will be designed for Math 568 students.

The supported programming languages for homework assignments include:

• R
• Python
• Octave. NOTE: Octave is 90% compatible with MATLAB and programs written in MATLAB will require very few changes to make them work with Octave.

The limited number of programming languages is determined by the use of Gradescope, but I may add additional programming languages, if necessary.

Course Objectives

This course is an introduction to the theory of stochastic processes. Topics to be covered include:

1. Discrete-time finite and countable state space Markov chains
2. Poisson process
3. Continuous-time Markov jump processes
4. Renewal processes

Additional topics may be covered, such as martingales and/or Gaussian processes.

Learning outcomes

For DATA/MATH-468

Upon completion of the course, the student will be able to perform concrete calculations, e.g., transition probabilities, exit times, and stationary distributions involving common stochastic process models like Markov chains, Markov jump processes, the Poisson process, and renewal processes; be able to interpret common stochastic process models and use them to model, e.g., queues.

For MATH-568

In addition to the above, students in MATH-568 are expected to be able to conduct computer simulations of basic stochastic process models; be able to prove some basic results in the theory of stochastic processes.

Absence and Class Participation Policy

Importance of attendance and class participation

Participating in course and attending lectures and other course events are vital to the learning process. As such, attendance is required at all lectures and discussion section meetings. Students who miss class due to illness or emergency are required to bring documentation from their healthcare provider or other relevant, professional third parties. Failure to submit third-party documentation will result in unexcused absences.

Missed Exams

Students are expected to be present for all exams. If a verifiable emergency arises which prevents you from taking an in-class exam at the regularly scheduled time, the instructor must be notified as soon as possible, and in any case, prior to the next regularly scheduled class. Make-up exams and quizzes will be administered only at the discretion of the instructor and only under extreme circumstances. If a student is allowed to make up a missed exam, (s)he must take it at a mutually arranged time. No further opportunities will be extended. Failure to contact your instructor as stated above or inability to produce sufficient evidence of a real emergency will result in a grade of zero on the exam. Other remedies, such as adjusting credit for other exams, may be considered.

COVID-19 related policies

As we enter the Fall semester, the health and wellbeing of everyone in this class is the highest priority. Accordingly, we are all required to follow the university guidelines on COVID-19 mitigation. Please visit http://www.covid19.arizona.edu for the latest guidance.

UA policies

The UA's policy concerning Class Attendance, Participation, and Administrative Drops is available at: http://catalog.arizona.edu/2015-16/policies/classatten.htm The UA policy regarding absences for any sincerely held religious belief, observance or practice will be accommodated where reasonable, http://policy.arizona.edu/human-resources/religious-accommodation-policy . Absences pre-approved by the UA Dean of Students (or Dean Designee) will be honored. See: http://uhap.web.arizona.edu/policy/appointed-personnel/7.04.02

Required Texts or Readings

Required Textbook

Essentials of Stochastic Processes. Richard Durrett, Third Edition

Optional Reference Textbook

Introduction to probability theory. Hoel, Paul G., Stone, Charles J., Port, Sidney C.

Assignments and Examinations

Notes on exam administration

All examinations are planned to be administered during the class time, either in person or on Zoom.

If, due to unforseen circumstances, they cannot be held in person, they are held on Zoom using the "gallery view" mode.The exam papers for not in-person tests will be distributed on-line by D2L and collected electronically using D2L "dropbox" feature.

Exam/assignment listing with date and grade contribution

Exam or Assignment	Date	Grade contribution
Midterm 1	February 15 (Tuesday), 2:00pm - 3:15pm	20%
Midterm 2	April 14 (Thursday), 2:00pm - 3:15pm	20%
Final Examination	May 9 (Monday), 3:30pm - 5:30pm	30%
Homework	See D2L	30%

Homework Assignments

Written homework consists of approximately twelve assignments equally contributing to the grade, each worth 30/12 = 2.5% of the grade. The assignments are posted on line at this link: Homework. The assignment papers are collected via Gradescope, which is cloud-based software for semi-automatic grading. Things to keep in mind:

• The student will submit all graded work through Gradescope.
• In addition, the student is required to submit to D2L the work that produced the answers submitted to Gradescope. The work should be clean and readable, but can be provided in the form of rough solutions ("scratchwork"). I will not grade the scratchwork submitted to D2L, but review it periodically to make sure that the student is following good practices, such as appropriate notation and problem methodology. Gradescope assignments, by their nature, are answer-oriented. To some degree the opportunity for partial credit is limited. Scratchwork shall be submitted to D2L "dropbox" (in 'Assignments' tab) which have the same name as the corresponsing Gradescope assignments.
• Homework must not be submitted through e-mail (no exceptions).
• The D2L "dropbox" will not be used for graded homework. You may submit a record of your work in D2L.
• Due dates will be adjusted in Gradescope as needed and the student is expected to check regularly for those adjustments.

Homework is assigned regularly throughout the semester, for a total of approximately 80 problems. Two types of homework will be assigned:

1. Homework which consists of selected exercises in the required textbook.
2. Custom homework problems composed by the instructor. Some of the custom problems will require programming.

Homework submission requirements

The main requirement is that the solutions must be structured in such a way that Gradescope can read them and that its 'AI' can interpret them. Your homework must be submitted as PDF, even if you use scanner or phone to capture images. Two typical workflows will be as follows:

1. Download the blank assignment (also called a 'template') from Gradescope.
2. Read and understand exactly what answers you need to provide. The space to enter the answer is a blue box, and marked with a label such as 'Q1.1' ("Question 1, part 1").
3. Work out the problem on "paper" (real or virtual), to obtain the answers. They must fit in the designated boxes in the 'template'. The size of the box is a hint from the instructor about the size of the answer (typically a number or a math formula) when entered by hand, using regular character size.
4. The recommended way to fill out the 'template' is paperless, by using suitable software and hardware (digital pen or tablet). I use a free program Xournal for this and it works great. You need to use it in combination with a digital pen or a tablet. It can produce a PDF easily, ready for submission to Gradescope.
5. You can also print the assignment on (real) paper, fill out the answers and scan the marked up document back to PDF format. However, the position of the boxes must be exactly (to a fraction of an inch) as in the original. Also, you may encounter a variety of "quality control" issues, especially if you are using a digital camera to scan the paper solution. All issues can be solved by a mix of the right hardware and software, but may not be the best time investment. The least troublesome way to scan is to use a real, flatbed scanner, e.g. in the library.
6. Upload the resulting document (a PDF of the 'template' marked up with your answers) to Gradescope. Your PDF must contain your name and student id in designated spaces. The Gradescope 'AI' will look for your name and student id, to properly associate it with your account.
7. After grading, the grade will be transmitted to D2L (Brightspace) and will be added to your 'Final Calculated Grade' automatically.
8. Some additional requirements:
   1. Do not reduce handwriting size! Reduce the size of your answer using
      • closed form expressions;
      • appropriate math functions, e.g., absolute value, min and max.
   2. Under no circumstances write outside the provided space (boxes). Gradescope, and the grader only considers the content of the designated boxes.
9. Written scratchwork must be submitted via D2L as instructed above.

Programming and Software

The class will have small programming assignments. It is expected that you will be using software to gain insights into the assigned problems. Most of the course is agnostic with respect to the programming languages or software systems you use to help you with the course. However, the programming assignments must be submitted in formats supported by Gradescope, as discussed earlier. I will be using these programs (easy to download and free to use):

• A computer algebra system (CAS) called Maxima.
• The maxima GUI wxMaxima is a separate program and this is the preferred way to use Maxima if you are a beginner. During the previous semester students successfully applied Maxima to doing their homework, and I will provide examples of Maxima calculations in class.
• The R programming language and environment, with a rich set of tools to do simulations in probability and stochastic processes (and analyze data).
• The GUI (and more for R): RStudio. Makes use of R somewhat more pleasant, especially for a new user.

Final Examination

The final examination is scheduled for: May 9 (Monday), 3:30pm - 5:30pm.

The time, data and general exam rules are set by the University and can be found at these links:

• Final Exam Regulations
• Final Exam Schedule

Grading Scale and Policies

The student in the class normally receives a letter grade A, B, C, D or E.

The cut-offs for the grades are:

Grade	% Range
A	90%+
B	80-90%
C	70-80%
D	60-70%
E	0-60%

Normally, individual tests and assignments will not be "curved". However, grade cut-offs may be lowered at the end of the semester (but not raised!) to reflect the difficulty of the assignments and other factors that may cause abnormal grade distribution.

The grade will be computed by D2L and the partial grade will be updated automatically by the system as soon as the individual grades are recorded.

General UA policy regarding grades and grading systems is available at https://catalog.arizona.edu/policy-type/grade-policies

Classroom Behavior Policy

To foster a positive learning environment, students and instructors have a shared responsibility. We want a safe, welcoming and inclusive environment where all of us feel comfortable with each other and where we can challenge ourselves to succeed. To that end, our focus is on the tasks at hand and not on extraneous activities (i.e. texting, chatting, reading a newspaper, making phone calls, web surfing, etc).

Threatening Behavior Policy

The UA Threatening Behavior by Students Policy prohibits threats of physical harm to any member of the University community, including to one's self. See: http://policy.arizona.edu/education-and-student-affairs/threatening-behavior-students .

Accessibility and Accommodations

Our goal in this classroom is that learning experiences be as accessible as possible. If you anticipate or experience physical or academic barriers based on disability, please let me know immediately so that we can discuss options. You are also welcome to contact Disability Resources (520-621-3268) to establish reasonable accommodations. For additional information on Disability Resources and reasonable accommodations, please visit http://drc.arizona.edu/ .

If you have reasonable accommodations, please plan to meet with me by appointment or during office hours to discuss accommodations and how my course requirements and activities may impact your ability to fully participate. Please be aware that the accessible table and chairs in this room should remain available for students who find that standard classroom seating is not usable. Code of Academic Integrity Required language: Students are encouraged to share intellectual views and discuss freely the principles and applications of course materials. However, graded work/exercises must be the product of independent effort unless otherwise instructed. Students are expected to adhere to the UA Code of Academic Integrity as described in the UA General Catalog. See: http://deanofstudents.arizona.edu/academic-integrity/students/academic-integrity http://deanofstudents.arizona.edu/codeofacademicintegrity .

UA Nondiscrimination and Anti-harassment Policy

The University is committed to creating and maintaining an environment free of discrimination, http://policy.arizona.edu/human-resources/nondiscrimination-and-anti-harassment-policy . Our classroom is a place where everyone is encouraged to express well-formed opinions and their reasons for those opinions. We also want to create a tolerant and open environment where such opinions can be expressed without resorting to bullying or discrimination of others.

Additional Resources for Students

UA Academic policies and procedures are available at: http://catalog.arizona.edu/2015-16/policies/aaindex.html Student Assistance and Advocacy information is available at: http://deanofstudents.arizona.edu/student-assistance/students/student-assistance

Confidentiality of Student Records

http://www.registrar.arizona.edu/ferpa/default.htm .

Subject to Change Statement

Information contained in the course syllabus, other than the grade and absence policy, may be subject to change with advance notice, as deemed appropriate by the instructor.

Significant Dates (from the Registrar's Website)

Undergraduate

      Dates: 01/12/2022 - 05/04/2022
Date	Standard Class Dates: Spring 2022 - Undergraduate - Regular Academic Session
10/1/2021	Shopping Cart available
1/11/2022	Last day to file Undergraduate Leave of Absence
1/11/2022	Last day for students to add to or drop from a waitlist
1/12/2022	FIRST DAY OF SPRING CLASSES

    UAccess still available for registration
    First day to file for the Grade Replacement Opportunity (GRO)
    First day to add classes for audit and instructor signature is required 

1/17/2022	Martin Luther King Day, no classes
1/19/2022	Last day to use UAccess for adding classes, changing classes, or changing sections
1/20/2022	Instructor approval required on a Change of Schedule form to ADD or CHANGE classes
1/25/2022	

    Last day to drop without a grade of W (withdraw)
    Classes dropped on or before this date will remain on your UAccess academic record with a status of dropped, but will not appear on your transcript
    Last day to change from credit to audit, or vice versa, with only an instructor's signature

1/25/2022	Last day for a refund
1/26/2022	Beginning today, students may completely withdraw from all classes in the term
1/26/2022	

    W period begins A penalty grade of W will be awarded for each withdrawal and the class(es) will appear on your transcript
    Beginning today, a change from credit to audit requires instructor approval on a Change of Schedule form

2/1/2022	Last day to apply for Spring degree candidacy without a late fee After this date a $50 00 Late Candidacy Application fee will be assessed
2/9/2022	Last day to change from pass/fail to regular grading or vice versa with only instructor approval on a Change of Schedule form
2/10/2022	Instructor's and dean's signatures are required on a Change of Schedule form to change from pass/fail to regular grades or vice versa
3/7/2022	Spring recess begins
3/9/2022	Last day to make registration changes without the dean's signature
3/10/2022	Instructor's and dean's signatures are required on all Change of Schedule forms to ADD or CHANGE classes
3/13/2022	Spring recess ends
3/29/2022	Last day to file for Grade Replacement Opportunity (GRO)
3/29/2022	

    Last day for students to withdraw from a class online through UAccess
    Last day for students to change to/from audit with only instructor approval
    Last day for instructors to administratively drop students 

3/30/2022	

    Instructor and dean's signatures required on a Late Change Petition in order to withdraw from class and students must have an extraordinary reason for approval
    Instructor's and dean's permission required on a Change of Schedule form to change to/from audit

4/12/2022	Last day for students to submit a Late Change Petition to their college
5/4/2022	Last day to request a complete withdraw from all classes in the term
5/4/2022	Last day of class--no registration changes can be made after the last day of class and last day to file a Complete Withdraw
5/5/2022	Reading day, no classes
5/6/2022	Final exams begin
5/12/2022	

    Final exams end
    Final grades are available in UAccess as soon as the instructor posts them
    Per Faculty Senate Policy, grades should be submitted within two business days after the final exam 

5/13/2022	Degree award date

    

Graduate

      Date	Standard Class Dates: Spring 2022 - Graduate - Regular Academic Session
10/1/2021	Shopping Cart available
1/11/2022	Last day for students to add to or drop from a waitlist
1/12/2022	FIRST DAY OF CLASS

    UAccess still available for registration
    First day to add classes for audit; instructor signature is required 

1/17/2022	Martin Luther King Day, no classes
1/19/2022	Last day to use UAccess for adding classes, changing classes, or changing sections
1/20/2022	Instructor approval required on a Change of Schedule form to ADD or CHANGE classes
1/25/2022	Last day for a refund
1/26/2022	Beginning today, students may completely withdraw from all classes in the term
2/8/2022	

    Last day to drop without a grade of W (withdraw)
    Classes dropped on or before this date will remain on your UAccess academic record with a status of dropped, but will not appear on your transcript
    Last day to change from credit to audit, or vice versa, with only an instructor's signature

2/9/2022	Last day for department staff to add or drop in UAccess
2/9/2022	Last day to change from pass/fail to regular grading or vice versa with only instructor approval on a Change of Schedule form
2/9/2022	

    W period begins A penalty grade of W will be awarded for each withdrawal and the class(es) will appear on your transcript
    Beginning today, a change from credit to audit requires instructor approval on a Change of Schedule form

2/10/2022	Instructor's and dean's signatures are required on a Change of Schedule form to change from pass/fail to regular grades or vice versa
3/7/2022	Spring recess begins
3/9/2022	Last day to make registration changes without the dean's signature
3/10/2022	Instructor's and dean's signatures are required on all Change of Schedule forms to ADD or CHANGE classes
3/13/2022	Spring recess ends
3/29/2022	

    Last day for students to withdraw from a class online through UAccess
    Last day for students to change to/from audit with only instructor approval
    Last day for instructors to administratively drop students 

3/30/2022	

    Instructor's and Graduate College dean's permission required on a Change of Schedule form to withdraw from a class--penalty grade of W will be awarded and the class will appear on your transcript
    Instructor's and dean's permission required on a Change of Schedule form to change to/from audit

5/4/2022	Last day to request a complete withdraw from all classes in the term
5/4/2022	Last day of class--no registration changes can be made after the last day of class and last day to file a Complete Withdraw
5/5/2022	Reading day, no classes
5/6/2022	Final exams begin
5/12/2022	

    Final exams end
    Final grades are available in UAccess as soon as the instructor posts them
    Per Faculty Senate Policy, grades should be submitted within two business days after the final exam 

5/13/2022	Degree award date

    

Material Covered

We will cover Chapters 1-4 of the book ("Stochastic Processes up to Margingales"). Here is the approximate schedule with approximate dates when the particular sections shall be covered.

Id	Chapter.Section.Subsection	Title	Page	Date
173		Markov Chains
174		Definitions and Examples	1
175		Multistep Transition Probabilities	9
176		Classification of States	13
177		Stationary Distributions	21
178	1.1	Doubly Stochastic Chains	26
179		Detailed Balance Condition	28
180	1.1	Reversibility	34
181	1.2	The Metropolis–Hastings Algorithm	35
182	1.3	Kolmogorow Cycle Condition	38
183		Limit Behavior	40
184		Returns to a Fixed State	46
185		Proof of the Convergence Theorem*	50
186		Exit Distributions	53
187		Exit Times	61
188		Infinite State Spaces*	68
189		Chapter Summary	74
190		Exercises	78
191		Poisson Processes
192		Exponential Distribution	95
193		Defining the Poisson Process	100
194	2.1	Constructing the Poisson Process	103
195	2.2	More Realistic Models	104
196		Compound Poisson Processes	106
197		Transformations	108
198	2.1	Thinning	108
199	2.2	Superposition	112
200	2.3	Conditioning	113
201		Chapter Summary	115
202		Exercises	116
203		Renewal Processes
204		Laws of Large Numbers	125
205		Applications to Queueing Theory	130
206	3.1	GI/G/1 Queue	130
207	3.2	Cost Equations	131
208	3.3	M/G/1 Queue	133
209		Age and Residual Life*	136
210	3.1	Discrete Case	137
211	3.2	General Case	139
212		Chapter Summary	141
213		Exercises	142
214		Continuous Time Markov Chains
215		Definitions and Examples	147
216		Computing the Transition Probability	152
217	4.1	Branching Processes	157
218		Limiting Behavior	162
219	4.1	Detailed Balance Condition	167
220		Exit Distributions and Exit Times	170
221	4.1	Exit Distribution	170
222	4.2	Exit Times	173
223		Markovian Queues	176
224	4.1	Single Server Queues	177
225	4.2	Multiple Servers	180
226	4.3	Departure Processes	182
227		Queueing Networks*	183
228		Chapter Summary	190
229		Exercises	192
230		Martingales
231		Conditional Expectation	201
232		Examples	204
233		Gambling Strategies, Stopping Times	207
234		Applications	211
235	5.1	Exit Distributions	212
236	5.2	Exit Times	214
237	5.3	Extinction and Ruin Probabilities	216
238	5.4	Positive Recurrence of the GI/G/1 Queue*	218
239		Exercises	220
240		Mathematical Finance
241		Two Simple Examples	223
242		Binomial Model	227
243	6.1	One Period Case	227
244	6.2	N Period Model	229
245		Concrete Examples	232
246		American Options	237
247		Black–Scholes Formula	241
248	6.1	The Black–Scholes Partial Differential Equation	244
249		Calls and Puts	245
250		Exercises	247
251		Review of Probability	251
252		Probabilities, Independence	251
253	A.1	Conditional Probability	253
254		Random Variables, Distributions	255
255		Expected Value, Moments	261
256		Integration to the Limit	267
257		References	269
258		Index	271