Вы находитесь на странице: 1из 2

Syllabus for Dynamic Programming Spring 2017

Course Instructor: Diego Klabjan

Course Time and Location: MW 11:00 - 12:15 p m; Tech M228

Best Way to Contact in Order of Preference :

skype dklabjan (do n ot add me as a contact)

hangout dklabjan

sms 847 691 1148

email d - klabjan@northwestern.edu

Response time directly correlated with the order of preference.

Textbook: Warren B. Powell; Approximate Dynamic Programming: Solving the Courses of Dimensionality; John Wil ey & Sons, 2007. ISBN 978 - 0 - 470 - 17155 - 4

Goal: This course will cover reinforcement learning aka dynamic programming, which is a modeling principle capturing dynamic environments and stochastic nature of events. The main goal is to learn dynamic programming and how to apply it to a variety of problems . The course will cover both theoretical and computational aspects.

Tentative list of topics:

1. Introduction to dynamic programming (Chapters 1 and 2)

2. Value and policy iterations (Chapter 3)

3. Stochastic gradient a lgorithm (Chapter 6)

4. Q - learning and temporal differences (Chapter 8)

5. Value function approximation and Monte - Carlo sampling (Chapter 4)

6. Linear and dynamic programming (time permitting, not in the text book)

Grading: There will be mandatory individual homewo rk assignments. In addition, each student will have two options: either performing a quarter long project, or having two take home exams. They will be weighed as follows:

Homework ass ignments: 3 0%

Midterm and final exam: 35 % each (if you choose the homewor k/exam route)

Project: 7 0% (if you choose the homework/project route) .

Homework Assignments: There will be a homework assignment every other week. You will not be allowed to use any literature except the textbook. You may discuss homework assignments but the final homework solution that you turn in may not be collaborative. The grading will be bas ed on peer grading.

Exams: The two exams will be seven - day take home exams. Only the textbook will be allowable. These are individual exams and thus absolu tely no collaboration is allowed .

Project: There will be a major project of y ou r choice. Teams of a t most three (preferably two) students are allowed. The project must be selected during the first two weeks. During the quarter, bi - weekly status reports must be posted on Canvas under discussions. These reports must be approximately half a page long and t hey must state the progress made in the previous two weeks. At the end of the course, an in - class presentation will be made and a final report not exceeding five pages is expected.

Requirements: Basic mathematical knowledge is required. Exposure to fundam ental probability is requested. Only very basic mathematical programming knowledge is desired (basic linear programming duality, linear modeling concepts). The course is suitable also for first year students.