UMBC - CMSC 331 – Principles of Programming Languages - Fall 2018

Disclaimer: This manuscript has been prepared for the course CMSC 331 at UMBC, department CSEE. The author
has made his best efforts in preparing this manuscript. However, he does not guarantee the accuracy of
information. Errors can be reported to the author.

Copyright: Since this is the first draft, there may be small parts directly copied from sources. The author does not
claim any right to those parts of text.

References: There are three main sources of material for this manuscript:

● The book “Concepts of Programming Languages” by Robert Sebesta.

● The book “Programming Language Pragmatics” by Michael Scott.
● https://docs.microsoft.com/en-us/windows/desktop/ProcThread/about-processes-and-threads
● Other sources are from Internet websites.
Principles of Programming Languages

Introduction to Concurrent Programming

Examples
A program is concurrent if it contains more than one execution context. Concurrent programs
are almost everywhere today.

The web browsers run in this way, because they need to take care of many things at the same
time. When we run Chrome, if we check the list of running processes, we can see clearly that
there are more than one processes running with this name. The browser needs to render the
application window while it is communicating with a web server to get the data. In the meantime,
the web page which is provided by the web server has scripts to run in the browser. This is all
with the fact that if the browser does not respond to the user actions in a timely manner, user
feels that browser is frozen.

Why Concurrent Programs

● Scalability, increasing the execution speed where more computation power is available,
e.g. availability of more CPUs. Google search needs to process 40,000 requests per
second on average. That is an example of where we need to scale.
● Modularity, separating execution of independent tasks and keeping track of every task
independently.

Parallel Programming
When more than a task are executed at the same time or almost at the same time, we call it
parallel computing. This can be performed with one or more CPUs. An example of such
program would be a flight simulator. A flight simulator needs to react in real time (hard real time)
or almost real time (soft real time).

Distributed Programming
When tasks are executed on more than one processors which are physically separated such as
on different computers at the same time, a distributed system is in effect. An example of such
program would be Computer Generated Imagery (CGI) in the film industry. Without distributed
UMBC - CMSC 331 – Principles of Programming Languages - Fall 2018

programming rendering a movie like Cars 2 from Pixar Studios would take more than 200 years.
According to reports on average it took 11 hours to render one frame. The company used
12,500 CPUs for the production.

Process vs Thread
Concurrent programs can be organized in the form of processes and threads. Processes and
threads can be scheduled to run at the discretion of programmer. The following presents the
definitions from Microsoft documentation.

Each process provides the resources needed to execute a program. A process has a virtual
address space, executable code, open handles to system objects, a security context, a unique
process identifier, environment variables, a priority class, minimum and maximum working set
sizes, and at least one thread of execution. Each process is started with a single thread, often
called the primary thread, but can create additional threads from any of its threads.

A thread is the entity within a process that can be scheduled for execution. All threads of a
process share its virtual address space and system resources. In addition, each thread
maintains exception handlers, a scheduling priority, thread local storage, a unique thread
identifier, and a set of structures the system will use to save the thread context until it is
scheduled. The thread context includes the thread's set of machine registers, the kernel stack, a
thread environment block, and a user stack in the address space of the thread's process.
Threads can also have their own security context, which can be used for impersonating clients.

Multitasking
A multitasking operating system divides the available processor time among the processes or
threads that need it. The system is designed for preemptive multitasking; it allocates a
processor time slice to each thread it executes. The currently executing thread is suspended
when it’s time slice elapses, allowing another thread to run. When the system switches from one
thread to another, it saves the context of the preempted thread and restores the saved context
of the next thread in the queue.

The length of the time slice depends on the operating system and the processor. Because each
time slice is small (approximately 20 milliseconds), multiple threads appear to be executing at
the same time. This is actually the case on multiprocessor systems, where the executable
threads are distributed among the available processors. However, you must use caution when
using multiple threads in an application, because system performance can decrease if there are
too many threads.

(https://docs.microsoft.com/en-us/windows/desktop/ProcThread/multitasking)
UMBC - CMSC 331 – Principles of Programming Languages - Fall 2018

Synchronization
Synchronization is a mechanism that controls the order in which tasks execute. Two kinds of
synchronization are required when tasks share data: cooperation and competition.

Cooperation synchronization is required between task A and task B when task A must wait for
task B to complete some specific activity before task A can begin or continue its execution.

Competition synchronization is required between two tasks when both require the use of some
resource that cannot be simultaneously used. Specifically, if task A needs to access shared
data location x while task B is accessing x, task A must wait for task B to complete its
processing of x.

Problems Requiring Synchronization

Producer-Consumer
One program unit produces some data value or resource and another uses it. Produced data
are usually placed in a storage buffer by the producing unit and removed from that buffer by the
consuming unit. The sequence of stores to and removals from the buffer must be synchronized.
UMBC - CMSC 331 – Principles of Programming Languages - Fall 2018

The consumer unit must not be allowed to take data from the buffer if the buffer is empty.
Likewise, the producer unit cannot be allowed to place new data in the buffer if the buffer is full.
This is a problem of cooperation synchronization because the users of the shared data structure
must cooperate if the buffer is to be used correctly.

Race Condition
Consider the following scenario, task A has the statement TOTAL += 1, where TOTAL is a
shared integer variable. Furthermore, task B has the statement TOTAL *= 2. Task A and task B
could try to change TOTAL at the same time. Four different values could be stored in TOTAL,
depending on the order of read and write by two different tasks. This is a problem of competition
synchronization. Two or more tasks are racing to use the shared resource and the behavior of
the program depends on which task wins the race.

Deadlock
Suppose task A and task B both need the shared resources X and Y to complete their work.
Furthermore, suppose that task A gains possession of X and task B gains possession of Y. After
some execution, task A needs resource Y to continue, so it requests Y but must wait until B
releases it. Likewise, task B requests X but must wait until A releases it. Neither relinquishes the
resource it possesses, and as a result, both lose their liveness, guaranteeing that execution of
the program will never complete normally. This particular kind of loss of liveness is called
deadlock.

Semaphores
a semaphore is a variable or abstract data type used to control access to a common resource
by multiple processes in a concurrent system such as a multitasking operating system, or trying
multiple concurrent access to a database. A semaphore is simply a variable.
Semaphore is a useful tool in the prevention of race conditions; however, their use is by no
means a guarantee that a program is free from these problems. Semaphores which allow an
arbitrary resource count are called counting semaphores, while semaphores which are
restricted to the values 0 and 1 (or locked/unlocked, unavailable/available) are called binary
semaphores and are used to implement locks.
Suppose a library has 10 identical study rooms, to be used by one student at a time. Students
must request a room from the front desk if they wish to use a study room. If no rooms are free,
students wait at the desk until someone relinquishes a room. When a student has finished using
a room, the student must return to the desk and indicate that one room has become free.
In the simplest implementation, the clerk at the front desk knows only the number of free rooms
available, which they only know correctly if all of the students actually use their room while
they've signed up for them and return them when they're done. When a student requests a
room, the clerk decreases this number. When a student releases a room, the clerk increases
this number. The room can be used for as long as desired, and so it is not possible to book
rooms ahead of time.
In this scenario the front desk count-holder represents a counting semaphore, the rooms are the
resource, and the students represent processes/threads. The value of the semaphore in this
UMBC - CMSC 331 – Principles of Programming Languages - Fall 2018

scenario is initially 10, with all rooms empty. When a student requests a room, they are granted
access, and the value of the semaphore is changed to 9. After the next student comes, it drops
to 8, then 7 and so on. If someone requests a room and the current value of the semaphore is 0,
they are forced to wait until a room is freed (when the count is increased from 0). If one of the
rooms was released, but there are several students waiting, then any method can be used to
select the one who will occupy the room (like first-in-first-out or random selection).
It is noteworthy that the thread that releases a semaphore need not be the same thread that
acquired it. Semaphore is a global object, then it is accessible by all threads.

For example, we can use semaphores to control the number of database accesses, since
every database has a limited number of concurrent access. This is an example of
competition synchronization.

The following example is runnable Python code which presents the use of Semaphore class
in Python for the synchronization.
UMBC - CMSC 331 – Principles of Programming Languages - Fall 2018

The following link provides a free book on the topic of synchronization:

https://greenteapress.com/wp/semaphores/
http://greenteapress.com/semaphores/LittleBookOfSemaphores.pdf

Monitors
A Monitor is an object designed to be accessed from multiple threads. The member functions or
methods of a monitor object will enforce mutual exclusion, so only one thread may be
performing any action on the object at a given time. If one thread is currently executing a
UMBC - CMSC 331 – Principles of Programming Languages - Fall 2018

member function of the object, then any other thread that tries to call a member function of that
object will have to wait until the first has finished.

A Monitor’s job is to encapsulate the shared data structure with its operations. One of the most
important features of monitors is that shared data is resident in the monitor rather than in any of
the client units. The programmer does not synchronize mutually exclusive access to shared
data through the use of semaphores or other mechanisms. Because the access mechanisms
are part of the monitor, implementation of a monitor can be made to guarantee synchronized
access by allowing only one access at a time. Calls to monitor procedures are implicitly blocked
and stored in a queue if the monitor is busy at the time of the call. The following graph presents
the communication between Monitor and the processes or threads.

We can use Monitors for cooperation synchronization. The Monitor concept in Python can be
implemented through Queue class. The following pseudocode presents the use of Queue class
in the context of producer and consumer problem.
(https://www.oreilly.com/learning/python-cookbook-concurrency)
UMBC - CMSC 331 – Principles of Programming Languages - Fall 2018

Message Passing

In Python the message passing concept is implemented using Condition variables. The
following example code presents a simple synchronization between in a producer-consumer
problem.
(https://www.bogotobogo.com/python/Multithread/python_multithreading_Synchronization_Cond
ition_Objects_Producer_Consumer.php)
UMBC - CMSC 331 – Principles of Programming Languages - Fall 2018

The following example presents a pseudocode in using Python Condition class for mutually
exclusive use of a resource.
(http://www.cs.cornell.edu/courses/cs4410/2012fa/mp1-faq.php)
UMBC - CMSC 331 – Principles of Programming Languages - Fall 2018