ISE 589: Stochastic Models in Industrial Engineering

Fall 2015
M/W: 10:15 11:30am
Location: 130 Park Shops
Instructor: Professor Julie Ivy
Phone: (919) 513 1683
Office Hours: T/Th: 10 11:30am and by appointment

Office: 380 Daniels

E-mail: jsivy@ncsu.edu

TA: Ms. Shadi Hassani Goodarzi

E-mail: shassan3@ncsu.edu
Office Hours: TBA

Office: 418 Daniels

Textbook: Modeling Random Processes for Engineers and Managers, Solberg, 2009. Publisher:
Wiley; and Online Reserve: Introduction to Operations Research, Ninth Edition, Hillier and
Lieberman, 2010. Chapters: 15, 19. Publisher: McGraw-Hill
Course Website: http://moodle.wolfware.ncsu.edu/
Course Description:

An introduction to mathematical modeling, analysis, and

uncertain (stochastic) service and production systems.
probability theory and stochastic processes. Applications
problems, capacity planning, inventory control, waiting
maintainability.

solution procedures applicable to

Methodologies covered include
relate to design and analysis of
lines, and system reliability and

Course Objectives:
This course has three components:
I. Probability Tools: characterizing uncertainty using probabilities and random variables.
II. Decision Modeling under Uncertainty: models for decision making under uncertainty:
decision tress, utility theory, value of information, Bayes rule.
III. Stochastic Modeling: characterizing uncertainty over time and space using:
A. Discrete Time Markov Processes: Markov Chains & Markov decision processes
B. Continuous Time Markov Processes: Birth & Death Processes, Poisson Processes and
Queueing Theory
C. Probabilistic Inventory Models
Upon the completion of this course, students will be able to use the tools of probability and
stochastic processes to develop models to improve decision-making in an uncertain environment.
Specifically:
Probability Objectives: Students will be able to:
Identify and Apply probability distributions appropriately to various applications
Understand probability distributions (unconditional and conditional) for single and
multivariate random variables
1

Decision Modeling Objectives: Students will be able to:

Formulate/Structure decision problems using tables and decision trees
Use various criteria to evaluate decision problems
Estimate the value of perfect and sample information
Demonstrate understanding of utility theory and calculate a utility
Formulate dynamic decision problems using a Markov decision process
Stochastic Modeling Objectives: Students will be able to:
Define and characterize a stochastic process
Identify, Define and Apply stochastic models particularly, Markov chains (discrete and
continuous), queueing models and inventory models
Develop Stochastic Models for various real world applications
Apply stochastic modeling theory to a real world problem.
General Description:
The main topics and reading assignments appear on the course outline. You will be expected to:
1. Complete the reading assignments (prior to class).
2. Try some of the problems to test your level of understanding (prior to class)
Homework: There will be approximately eight to ten homework assignments, assignments are
due on Mondays at 11:59pm, the assignment due dates are indicated on the course outline. You
will work in groups of two and turn in one assignment for your group with names of the group
members clearly indicated. Homework groups must remain the same throughout the semester.
Homework 0 is an individual assignment for everyone, each person should turn in their own
assignment.
Project: Working in your homework groups, you will develop a stochastic model for a real
world problem, prepare a written project report and a short presentation. More details on the
project will follow.
Assignment Groups: You will complete your homework assignments and projects in groups of
two students. Once you select your assignment group it should remain fixed for the entire
semester, i.e., all assignments must be completed with the same group members. Please select
your homework group and notify me using the googledoc by August 30th. If you are looking for
a homework group please indicate this on the googledoc by August 26th.
Grading:
Homework
Project
Quizzes
Final Exam
Total

20%
15%
30%
35%

No Late Homework Assignments will be accepted

Proposal: Oct. 16, Presentations: Nov. 30 & Dec. 2, Report: Dec. 4
Two Quizzes (15% each): Sept. 30 and Oct. 28
Dec. 14: 8 11am

100%

ISE 589 Living Course Outline:

Week
1

Class
Aug. 19

Aug. 24

Aug. 26
3

Sept. 16

Decision Trees: Sample Information and Bayes

Rule
Probability I: Joint Distributions, Conditional
Probability & Conditional Expectation
HOMEWORK 0: Probability Review DUE
Probability II: Conditional Expectation &
Moment Generating Functions
LABOR DAY
Introduction to Stochastic Processes and Markov
Chains I
Markov Chains II
HOMEWORK I: Decision Analysis &
Probability & Conditional Expectation DUE
Markov Chains III

Sept. 21

Markov Chains IV

Sept. 23

HOMEWORK II: Markov Chains I DUE

Markov Chains V

Aug. 31

Sept. 2
4

Topic
Course Introduction & Probability Review I:
Conditional Probability, Independence, Bayes
Formula, Random Variables, Expectation,
Variance, Distributions
Decision Making: Decision Criteria, Decision
Trees, Perfect Information

Sept. 7
Sept. 9
Sept. 14

Sept. 28

Sept. 30
Oct. 5

Quiz 1 Review
HOMEWORK III: Markov Chains II DUE
QUIZ I
Markov Reward Processes

Oct. 7

Markov Decision Processes I

Oct. 8/9
Oct. 12

Markov Decision Processes II

Oct. 14

Review: Exponential and Poisson Distributions

Oct. 16

PROJECT PROPOSAL DUE

Reading
Ch. 1.: 1.1 1.7, 1.10
1.13: Pg. 1 20, 20 - 28

Hillier &
Lieberman
(Online Reserve): Ch. 15:
Pg. 672 679, 686 690
H&L: Ch.15: Pg. 680
690
Ch. 1: 1.6 1.9, Pg. 11
20
Ch. 1: 1.7 1.9, Pg. 14 20
Ch. 2: 2.1 2.5, Pg. 32 44
Ch. 2: 2.6 2.12, Pg. 45 67
Ch. 3: 3.1 3.6: Pg. 72
83
Ch. 3: 3.7 3.13: Pg. 83
96
Ch. 3: 3.13 3.26: Pg. 96
115

Ch. 4: 4.1 4.11, Pg. 119

136
H&L: Ch. 19: Pg. 905
911, 915 928

FALL BREAK

H&L: Ch. 19: Pg. 905

911, 915 928
Ch. 1: 1.10 1.11 Pg.22,
25 26

ISE 589 Living Course Outline (cont.):

Week Class
10
Oct. 19

Oct. 21
11

12
13

Oct. 26

Oct. 28
Nov. 2
Nov. 4
Nov. 9
Nov. 11

14

Nov. 16

Nov. 18

15

16

18

Nov. 23
Nov. 25
29
Nov. 30
Dec. 2
Dec. 4
Dec. 14

Topic
Continuous Time Markov Chains - Queueing
HOMEWORK IV: Markov Reward
Processes, MDPs & Exponential & Poisson
Distributions DUE
CTMC: Continuous-time Markov Chain: Birth
and Death Processes - Poisson Processes
Quiz II Review

Reading
Ch. 5: 5.1 5.4: Pg. 140
152

Ch. 5: 5.5 5.12, Pg. 152

161

HOMEWORK V:, CTMC, Birth and Death

Processes, Poisson Processes DUE
QUIZ II
INFORMS: Project Day
INFORMS: Project Day
CTMC: Counting Processes - Poisson Processes Ch. 5: 5.13 5.19: Pg.
161 173
Queueing Theory I: Intro Queueing Systems
Ch. 6: 6.1 6.8: Pg. 179
200
Queueing Theory II: Queueing Systems
Ch. 6: 6. 10 6.14, Pg.
202 210
HOMEWORK VI: Poisson Processes &
Queueing I DUE
Queueing Theory III: Queueing Systems
Ch. 6: 6.15 6.17, 6.20
6.21, Pg. 210 214, 218 220
Queueing Theory IV: Queueing Systems
Ch. 7: 7.1 7.6, Pg. 225
233
HOMEWORK VII: Queueing II DUE
THANKSGIVING BREAK
Project Presentations
Project Presentations
PROJECT REPORT DUE by 5pm
FINAL EXAM: 8 11am

Academic Integrity
Students are required to comply with the university policy on academic integrity found in the
Code of Student Conduct found at
http://www.ncsu.edu/policies/student_services/student_discipline/POL11.35.1.php
Academic Honesty
See http://www.ncsu.edu/policies/student_services/student_discipline/POL11.35.1.php for a
detailed explanation of academic honesty.
Honor Pledge
Your signature on any test or assignment indicates "I have neither given nor received
unauthorized aid on this test or assignment."
Attendance Policy
Absences will be excused if approved in advance. Please refer to the university attendance
regulation for a list of excused absences: http://policies.ncsu.edu/regulation/reg-02-20-03.
Students will be allowed to make-up work missed due to an excused absence within a reasonable
amount of time as determined by the instructor.
Late Assignments
No late assignments will be accepted.
Accommodations for Disabilities
Reasonable accommodations will be made for students with verifiable disabilities. In order to
take advantage of available accommodations, student must register with the Disability Services
Office (http://www.ncsu.edu/dso) located at 1900 Student Health Center, Campus Box 7509,
919-515-7653. For more information on NC State's policy on working with students with
disabilities, please see the Academic Accommodations for Students with Disabilities Regulation
at http://www.ncsu.edu/policies/academic_affairs/courses_undergrad/REG02.20.1.php.
NC State University Policies, Regulations and Rules (PRR):
Students are responsible for reviewing the PRRs which pertain to their course rights and
responsibilities. These include: http://policies.ncsu.edu/policy/pol-04-25-05 (Equal Opportunity
and Non-Discrimination Policy Statement), http://oied.ncsu.edu/oied/policies.php (Office for
Institutional Equity and Diversity), http://policies.ncsu.edu/policy/pol-11- 35-01 (Code of
Student Conduct), and http://policies.ncsu.edu/regulation/reg-02-50-03 (Grades and Grade Point
Average).
Transportation
NA
Safety & Risk Assumptions
None