Er.

Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
Topic: Overview of scheduling problem, Different types of scheduling, Scheduling for
independent & dependent tasks, Static versus dynamic scheduling, Optimization
techniques for scheduling
Topic-1: Scheduling
Nowadays, many companies offering services to the customer based on the concept of
pay as a service, where each customer pays for the services obtained from the
provider. The cloud environment provides a different platform by creating a virtual
machine that assists users in accomplishing their jobs within a reasonable time and
cost-effectively without sacrificing the quality of the services. The huge growth in
virtualization and cloud computing technologies reflect the increasing number of jobs
that require the services of the virtual machine.
Scheduling is a process of allocating jobs onto available resources in time. Such
process has to respect constraints given by the jobs and the cloud.
Scheduling is a process of finding the efficient mapping of tasks to the suitable
resources so that execution can be completed with the satisfaction of objective
functions such as minimization of execution time as specified by customers.
The need for scheduling arises because efficiency of scheduling algorithm directly
affects the performance of the system with respect to delivered QoS, resource
utilization.
Task: Represents a computational unit to run on a node. A task is considered as an
indivisible schedulable unit. Tasks could be independent (or loosely coupled) or there
could be dependencies (cloud workflows).
Job: A job is a computational activity made up of several tasks that could require
different processing capabilities and could have different resource requirements (CPU,
number of nodes, memory, software libraries, etc.) and constraints, usually expressed
within the job description. Each job may have various parameters such as required
data, desired completion time often called the deadline, expected execution time, job
priority etc.
Resource: A resource is something that is required to carry out an operation, for
example: a processor for data processing, a data storage device, or a network link for
data transporting.
Cloud computing gives the illusion of infinite (virtual) resources. Actually there is a
finite amount of (physical) resources. We would like to efficiently share those
resources:
1. being able to distinguish high priority (serving customer now) from low priority (batch)
requests;
2. schedule accordingly.
Therefore, we should be able to plan ahead computations.
To efficiently increase the working of cloud computing environments, job scheduling is
one the tasks performed in order to gain maximum profit.
The goal of scheduling algorithms in distributed systems is spreading the load on
processors and maximizing their utilization while minimizing the total task execution
time.
Job scheduling, one of the most famous optimization problems, plays a key role to
improve flexible and reliable systems.
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
The main purpose is to schedule jobs to the adaptable resources in accordance with
adaptable time, which involves finding out a proper sequence in which jobs can be
executed under transaction logic constraints.

Benefits of scheduling:
To improve the quality of services in executing the jobs and provide the expected
output on time.
To maintain efficiency and fairness for all jobs.

What users want?

There are two types of users in cloud environment & each has its own requirements
1. Cloud Consumers
Execute jobs for solving problem of varying size and complexity
Benefit by selecting and aggregating resources wisely
Tradeoff time and cost

2. Cloud Providers
Contribute (idle) resource for executing consumer jobs
Benefit by maximizing resource utilization
Tradeoff local requirements & market opportunity
Strategy: maximize return on investment

Scheduling Process
Scheduling process in cloud can be generalized into three stages namely
1. Resource discovering and filtering Datacenter Broker discovers the resources
present in the network system and collects status information related to them.
Resource discovery may be described basically as the task in which the provider
should find appropriate resources in order to comply with incoming consumers
requests.
Considering that one of the key features of Cloud Computing is the capability of
acquiring and releasing resources on-demand, resource monitoring should be
continuous.
2. Resource selection Target resource is selected based on certain parameters of task
and resource. This is deciding stage.
After acquiring information about
available resources in the Cloud
(during the discovery phase), a set
of appropriate candidates is
highlighted.
The resource selection mechanism
elects the candidate solution that
fulfils all requirements and
optimizes the usage of the
infrastructure.
The resource selection may be done
using an optimization algorithm.
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
Many optimization strategies may be used, from simple and well-known
techniques such as simple metaheuristic algorithms are Genetic Algorithm, Ant
Colony, and Particle Swarm Optimization (PSO) for Clouds.
3. Task submission -Task is submitted to resource selected.

Optimization Criterion
Optimization criterion is used when making scheduling decision and represents the
goals of the scheduling process.
Typically, we want to e.g., increase resource usage, number of successfully completed
jobs or minimize the response time.
This criterion is expressed by the value of objective function which allows us to
measure the quality of computed solution and compare it with different solution.

Quality of Service (QoS)

Quality of Service (QoS) is the ability to provide different priority to different jobs and
users, or to guarantee a certain level of performance to a job.
If the QoS mechanism is supported it allows the users to specify desired performance
for their jobs such as
o completion before given deadline,
o dedicated access to resources during some time period (advanced reservation),
o requested amount of resources (CPUs, RAM, HDD, network bandwidth),etc.
Such request is formally established between the user and the resource manager
(scheduler) through negotiation which produces a formal description of the
guaranteed service called Service Level Agreement (SLA).
In systems with limited resources the QoS support results in additional cost which is
related to the complexity of QoS requests and the efficiency of the scheduler when
dealing with them.

Scheduling problem in cloud

The problem of mapping tasks on distributed services belongs to a class of problems
known as NP-hard problems. For such problems, no known algorithms are able to
generate the optimal solution within polynomial time.
Solutions based on exhaustive search are impractical as overhead of generating
schedules is very high.
In cloud environments, scheduling decisions must be made in the shortest time
possible, because there are many users competing for resources, and time.

SLA Monitor
When a customer first submits the service request, the SLA Monitor interprets the
submitted request for QoS requirements before determining whether to accept or
reject the request.
It is also responsible to monitor the progress of the submitted job.
If any violation is observed from SLA it has to act immediately for corrective action.

Task Scheduling
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
The input of task scheduling algorithms is normally an abstract model which defines
tasks without specifying the physical location of resources on which the tasks are
executed.
Reschedule: When a task cant be completed due to processor failure or, a disk
failure, or other problems. The uncompleted tasks could be rescheduled in the next
computation.

Advance Resource Reservation Monitor

Provides QoS guarantees for accessing various resources across Data centres.
Users are able to secure resources required in the future which is important to ensure
successful completion of time-critical applications such as real time and workflow
applications or parallel applications requiring a number of processors to run.
Provider can predict future demand and usage more accurately.
Topic-2: Types of Scheduling
1. Static vs. Dynamic scheduling
2. Centralized, Hierarchical vs. Distributed Scheduling
3. Cooperative vs Non-Cooperative Scheduling
4. Independent vs. Workflow scheduling

Category-1: Static versus Dynamic

Static Scheduling:
Pre-schedule jobs.
Information regarding all resources as well as all the tasks in an application is
assumed to be available by the time the application is scheduled.
No job failure and resources are assumed available all the time.

Dynamic Scheduling:
It is more flexible than static scheduling where task allocation is done on the fly when
the application executes.

Category-2: Centralized, Hierarchical vs. Distributed Scheduling

Centralized and decentralized scheduling differs in the control of the resources and
knowledge of the overall system.

Centralized Scheduling:
In the case of centralized scheduling, there is more control on resources: the
scheduler has knowledge of the system by monitoring of the resource state.
Advantages: ease of implementation, efficiency and more control and monitoring on
resources.
Disadvantages: Lacks scalability, fault tolerance and efficient performance

Hierarchical Scheduling
Allows one to coordinate different schedulers at a certain level.
Schedulers at the lowest level in the hierarchy have knowledge of the resources.
Disadvantages: lack of scalability and fault tolerance, yet it scales better and is more
fault tolerant than centralized schedulers.
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi

Decentralized or Distributed Scheduling

No central entity controlling the resources.
Scheduling decisions are shared by multiple distributed schedulers.
Less efficient than centralized schedulers.

Category-3: Cooperative vs Non-Cooperative Scheduling:

If a decentralized scheduling algorithm is adopted, the next issue that can be
considered is whether the nodes involved in job scheduling are working cooperatively
or independently (non-cooperatively).
In the case of non-cooperative scheduling, individual schedulers act alone as
autonomous entities and arrive at decisions regarding their own optimum objects
independent of the effects of the decision on the rest of system.
In the cooperative case, each scheduler has the responsibility to carry out its own
portion of the scheduling task, but all schedulers are working toward a common
system-wide goal.

Category-4: Independent vs. Workflow scheduling:

Independent Scheduling
Tasks can be run independently.

Workflow scheduling
Tasks are dependent on each other.
Dependency means there are precedence orders existing in tasks, that is, a task
cannot start until all its parent are done.

Category-5: Batch mode heuristic scheduling algorithms (BMHA) and online mode
heuristic algorithms

Batch Mode Heuristic Scheduling Algorithm

 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
In BMHA, Jobs are queued and collected into a set when they arrive in the system.
The scheduling algorithm will start after a fixed period of time.
The main examples of BMHA based algorithms are; First Come First Served
scheduling algorithm (FCFS), Round Robin scheduling algorithm (RR), MinMin
algorithm and MaxMin algorithm.

On-line mode heuristic scheduling algorithm

Jobs are scheduled when they arrive in the system.
Since the cloud environment is a heterogeneous system and the speed of each
processor varies quickly, the on-line mode heuristic scheduling algorithms are more
appropriate for a cloud environment.
Most fit task scheduling algorithm (MFTF) is suitable example of On-line mode
heuristic scheduling algorithm.

a. First Come First Serve Algorithm: Job in

the queue which come first is served. This
algorithm is simple and fast.
b. Round Robin algorithm : In the round robin
scheduling, processes are dispatched in a
FIFO manner but are given a limited amount
of CPU time called a time-slice or a quantum. If a process does not complete before its
CPU-time expires, the CPU is preempted and given to the next process waiting in a
queue. The preempted process is then placed at the back of the ready list.
c. MinMin algorithm: This algorithm chooses small tasks to be executed firstly, which
in turn large task delays for long time.
d. Max Min algorithm: This algorithm chooses large tasks to be executed firstly, which
in turn small task delays for long time.
e. Most fit task scheduling algorithm: In this algorithm task which fit best in queue
are executed first. This algorithm has high failure ratio.
f. Priority scheduling algorithm: The basic idea is straightforward: each process is
assigned a priority, and priority is allowed to run. Equal-Priority processes are
scheduled in FCFS order. The shortest-Job-First (SJF) algorithm is a special case of
general priority scheduling algorithm. An SJF algorithm is simply a priority algorithm
where the priority is the inverse of the (predicted) next CPU burst. That is, the longer
the CPU burst, the lower the priority and vice versa. Priority can be defined either
internally or externally. Internally defined priorities use some measurable quantities
or qualities to compute priority of a process.
g. Random Algorithm
The idea of random algorithm is to
randomly assign the selected jobs to
the available Virtual Machines (VM).
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
The algorithm does not take into considerations the status of the VM, which will either
be under heavy or low load.
Hence, this may result in the selection of a VM under heavy load and the job requires
a long waiting time before service is obtained.
The complexity of this algorithm is quite low as it does not need any overhead or pre-
processing.
h. Minimum Completion Time Algorithm
The Minimum Completion Time job scheduling algorithm attempts to allocate the
selected job to the available VM that
can offer the minimum completion
time taking into account its current
load.
The main criterion to determine the
VM in the minimum completion time
scheduling algorithm is the processor
speed and the current load on each
VM. The algorithm first scans the available VMs in order to determine the most
appropriate machine to perform the job.
Subsequently, it dispatches the job to the most suitable VM and starts execution
i. Opportunistic Load Balancing Algorithm
This algorithm attempts to dispatch the
selected job to the available VMs which has
the lowest load compared to the other VMs.
The idea is to scale the current loads for
each VM before sending the job. Then, the
VM that has the minimum load is selected to
run the job.

Metrics:
1. Makespan: The makespan represent the maximum finishing time among all received
jobs per time. This parameter shows the quality of job assignment to resources from
the execution time perspective.
2. Throughput: Each job is assumed to have hard dead-lines which represent the
finishing time. Therefore, the throughput is the number of executed jobs, which is
calculated to study its efficiency in satisfying the jobs dead-lines.
3. Total Cost: If the basic concept of Could computing is renting resources for
customers, then the total cost needed for executing the list of jobs is essential for
evaluating system performance. The total cost is calculated based on the processing
time and the amount of data transferred.

Topic-3: Scheduling Independent Tasks

1. Min-Min
2. Max-Min
3. Sufferage

Min-Min Heuristic
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
Static Task Scheduling Algorithm
Tasks are scheduled based on minimum completion time
For each task determine its minimum completion time over all machines
Over all tasks find the minimum completion time
Assign the task to the machine that gives this completion time
Iterate till all the tasks are scheduled

Max-Min Heuristic
Static Task Scheduling Algorithm
Two Phases
o First Phase : Set of all minimum expected completion time are found for every
task and resource
o Second phase : Task with the maximum value within the above set is selected
for execution
For each task determine its minimum completion time over all machines
Over all tasks find the maximum completion time
Assign the task to the machine that gives this completion time
Iterate till all the tasks are scheduled

Sufferage Heuristic
For each task determine the difference between its minimum and second minimum
completion time over all machines (sufferage)
Over all tasks find the maximum sufferage
Assign the task to the machine that has resulted in obtaining minimum completion
time
Iterate till all the tasks are scheduled

Topic-4: Dependent Task Scheduling

Tasks composing a job have precedence orders.
DAGs (Directed Acyclic Graphs) are used to represent these tasks.
In DAGs, a node represents a task and a directed edge denotes the precedence orders
between its two vertices.
In some cases, weights can be added to nodes and edges to express computational
costs and communicating costs respectively.

List Scheduling Heuristics

An ordered list of tasks is constructed by assigning priority to each task
Tasks are selected on priority order and scheduled in order to minimize a predefined
cost function
Tasks have to be in a topologically sorted order
The list scheduling algorithm is so-called because of its use of a ready list.
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
The algorithm iterates through machine cycles sequentially, and at each cycle it
populates the ready list with the set of all candidate instructions which could begin
execution in the current cycle.
It then selects the best instructions from the ready list to begin execution in the
current cycle, subject to resource constraints.
The algorithm also makes use of an execution list : whenever an instruction is issued,
it is placed on the execution list, a list of all instructions currently being executed.
When an instruction i finishes executing, any successor j of i becomes a candidate
instruction as long as all other predecessors of j have also finished executing.
The execution list is used to easily identify instructions that finish executing, so it can
quickly be determined if j may be added to the ready list.

Clustering Heuristics
Clustering is a mapping of the nodes of a DAG onto labeled clusters. A cluster
consists of a set of tasks; a task is an indivisible unit of computation. All tasks in a
cluster must execute in the same processor.
The clustering problem has been shown to be NP-complete for a general task graph
and for several cost functions.
The tasks are convex ,i.e., once the task starts its execution it can run to completion
without interruption.
Clustering is a mapping of the tasks of a DAG onto clusters. A cluster is a set of tasks
which will execute on the same processor. Clustering is also known as processor
assignment in the case of an unbounded number of processors.
A clustering is called nonlinear if two independent tasks are mapped in the same
cluster; otherwise it is called linear.
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi

Topic-5: Optimization Techniques

1. Genetic Algorithms
2. Ant Colony Optimization

Genetic Algorithms
Genetic algorithms generate solutions to optimization problems using techniques
inspired by natural evolution, such as mutation, selection, and crossover.
The fundamental unit of GA is chromosome.
Each chromosome represents a solution to the problem and is composed of genes.
The fitness value is function or objective against which chromosome is tested for its
suitability to the problem in hand.
A typical genetic algorithm requires:
o a genetic representation of the solution domain,
o a fitness function to evaluate the solution domain
Steps:
Initialize a population of solutions,
Improve it through repetitive application of the mutation, crossover, selection
operators.

Algorithm
1. Generate an initial population.
2. Select pair of individuals based on the fitness function.
3. Produce next generation from the selected pairs by performing random changes on the
selected parents (by applying pre-selected genetic operators).
4. Test for stopping criterion:
a. return the solutions or individuals if satisfied, or
b. go to step 2. if not. Algorithm

Genetic algorithms are based on the principles that crossing two individuals can
result in offsprings that are better than both parents and slight mutation of an
individual can also generate a better individual.
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
The crossover takes two individuals of a population as input and generates two new
individuals, by crossing the parents' characteristics.
The offsprings keep some of the characteristics of the parents.

Representation of chromosome in Scheduling

Each gene is a pair of values (Tj, Pi), indicating that task Tj is assigned to processor
Pi.
The execution order of tasks allocated to the same processor is given by the positions
of the corresponding genes in the chromosome.
Tasks allocated to different processors can be executed in parallel.
The number of genes of each chromosome is equal to the number of the tasks.
The initial population is initialized by randomly placing each task on a processor.
To avoid the generation of non-optimal results, heuristic approach along with
precedence resolution can be applied to generate the initial population that gives
better results in terms of quality of solutions.
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi

Ant Colony Optimization (ACO)

ACO is a class of optimization algorithms modeled on the actions of an ant colony.
ACO methods are useful in problems that need to find paths to goals.
Artificial antssimulation agentslocate optimal solutions by moving through a
parameter space representing all possible solutions.
Natural ants lay down pheromones directing each other to resources while exploring
their environment.
The simulated ants similarly record their positions and the quality of their solutions,
so that in later simulation iterations more ants locate better solutions.

Topic-6: Energy management

Cost of powering and cooling accounts for 53% of the total operational expenditure
Goal is not only to cut down energy cost but also to meet govt. regulations and
environmental standards.

Two common ways

Server consolidation
Energy aware job scheduling

Server Consolidation
Effective approach to maximize resource utilization while minimizing energy
consumption.
Live VM migration is often used to consolidate VMs residing on multiple under-utilized
servers onto a single server, so that the remaining servers can be sent to an energy
saving state.
Problem here is optimal consolidation of servers.
Dependencies among VMs( communication requirements) can be considered.
Key challenge is trade off between energy savings and application performance
Using voluntary resources (resources donated by end users) or a mixture of voluntary
and dedicated resources for hosting non-profit cloud applications such as scientific
computing. Challenge is managing heterogeneous resources and frequent churn
events.
Building small data centers instead of big data centers as former are cheaper to build
and better geographically distributed. Desirable for response time-critical services.