1

Queueing Theory Fundamentals

Required reading:

CSE 3214, Winter 2011 Instructor: N. Vlajic

Queueing Theory - Motivation

lots of systems have the following structure, and Why do we need naturally fall in the category of queueing systems: Queueing Theory? a) jobs / customers arrive according to some discipline b) if server busy jobs wait in waiting area (buffer) c) jobs receive service d) jobs depart
example: banks and grocery-store checkout lines,

computer OSs, web servers, etc.

Queueing Theory deals with analysis of such system

and attempts to predict their behavior

example: given some info about customer arrival

patterns and average service time per customer, how long the waiting line will be?

Queueing Theory - Motivation (cont.)

Example [ packet switched networks = complex queueing systems ]
In packet switched networks events are random:
packets arrive at random times to switches job arrival times are random; packets are of random lengths service times per job are random; each packet passes through multiple switches on the way to its destination

Packet Network

M A

packets = jobs packet switches = servers

PS

PS

PS PS

PS

L P

http://media.pearsoncmg.com/aw/aw_kurose_network_2/applets/queuing/queuing.html

Basic Assumptions and Notation

Queueing Model
arrivals T waiting line departure T

FIFO Server

- average
arrival rate

w - # of waiting items Tw - average waiting

time

Ts - mean service time

- average
departure rate

arrivals:

average job arrival rate: mean inter-arrival time:

[ jobs / sec ]
1

typically known / given higher demand higher

T =

[ sec / job ]

server & departure:

average service time:

for each arrival

Ts [ sec / job ] =
1 [ job / sec ] Ts

typically known / given faster server smaller Ts

average departure rate:

if items in the queue

Basic Assumptions and Notation (cont.)

Queueing Model (cont.)
arrivals T waiting line departure T

FIFO Server

- average
arrival rate

w - # of waiting items Tw - average waiting

time

Ts - mean service time

- average
departure rate

waiting line:

average time spent in the queue: average # of jobs in the queue:

Tw [ sec ] w = * Tw [ jobs ]
Tw = w = w * T

Cannot be found directly from and Ts. Depends on the nature of arrival and departure process !!!

Littles Law

w = * Tw [ jobs ]

[ sec ]

The expected time that a newly arrived job will spent waiting in the queue is equal to the product of the average number of jobs seen in the queue and the average job inter-arrival time!

Basic Assumptions and Notation (cont.)

Example [ Littles Law ]

Intuition behind Littles Law: a departing customer, on the average, should leave behind the same number of customers in the system / lineup as he found in the system when she arrived.

w = Naverage = Nfound at arrival = Nleft at departure Nfound at arrival = ??? Nleft at departure = (average time spend in queue) * (average arrival rate to queue) = Tw *

w = Naverage = Tw *

arrival

departure

Basic Assumptions and Notation (cont.)

Example [ supermarket checkout line ]
Assume at the supermarket you notice that a checkout line has always 3 customers ( w = 3 [jobs] ), and customers arrive every 2 minutes ( T = 2 min ). How long is the average waiting time in the queue?

w=3

Tw = w * T = 3 * 2 [min] = 6 [min]

Can also be seen as average length of the queue in minutes.

Basic Assumptions and Notation (cont.)

Queueing Model (cont.)
arrivals T waiting line departure T

FIFO Server

- average
arrival rate

w - # of waiting items Tw - average waiting

time

Ts - mean service time

- average
departure rate

utilization:

aka average system throughput:

= Ts =

Ts = T

fraction of time that server is busy serving customers

Ts

(a) (b) (c)

< (T > Ts) < 1 - stable system = (T = Ts) = 1 - server 100% busy > (T < Ts) > 1 - queue grows without limit !!! unstable system

Basic Assumptions and Notation (cont.)

Example [ Web server ]
The average response time of a Web server is measured to be 11 [msec] when the request rate is 450 [req/sec]. What is the average number of requests that are buffered in the system?

Tw = 11 [msec]

and

= 450 [req/sec] w = 4.95 [requests]

w = *Tw = 450*11*10-3

Example [ computer processor ]

Computer processor receives 10 I/O jobs per second; average processor service time is 20 [msec]. On average, how utilized is the processor?

= 10 [jobs/sec]

and Ts = 20 [msec] = 0.2

= / = *Ts = 10*20*10-3