IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY

by

Franois Bergeon

A DISSERTATION

Submitted to

The University of Liverpool

in partial fulfillment of the requirements for the degree of

MASTER OF SCIENCE Information Technology (Software Engineering)

January 5, 2009

ABSTRACT

IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY by Franois Bergeon

The subject of this dissertation is the implementation of container classes in shared memory in order to allow concurrent high-level access to easily retrievable structured data by more than one process.

The project extends the sequence containers and associative containers offered by the C++ Standard Template Library (STL) in the form of a Shared Containers Library which purpose is to allocate both the containers and the objects they contain in shared memory to make them accessible by other processes running on the same machine.

Shared STL-like containers can be used for a variety of implementations that require high-level sharing of structured and easily retrievable data between processes. Applications of shared containers may include realtime monitoring, metrics gathering, dynamic configuration, distributed processing, process prioritization, parallel cache seeding, etc.

The Shared Container Library presented here consists of a binary library file, a suite of C++ header files, and the accompanying programmers

documentation. STL-type containers can be allocated in a shared memory segment thanks to a class factory mechanism that abstracts the complexity of the implementation. The Shared Containers Factory offers synchronization for concurrent access to containers in the form of a mechanism offering both exclusive and a non-exclusive locks.

The target platform of the Shared Container Library is the Microsoft implementation of the STL for the 32bit Microsoft Windows operating systems (Win32) as distributed with the Microsoft Visual Studio 2008 development environment and its Microsoft Visual C++ compiler. Further developments of this library may include porting to the Unix platform and designing a similar library for other languages such as Java and C#.

DECLARATION

I hereby certify that this dissertation constitutes my own product, that where the language of others is set forth, quotation marks so indicate, and that appropriate credit is given where I have used the language, ideas, expressions, or writings of another.

I declare that the dissertation describes original work that has not previously been presented for the award of any other degree of any institution.

Signed, Franois Bergeon

ACKNOWLEDGEMENTS

The author wishes to thank Martha McCormick, the dissertation advisor for the project, for her support, kind words and constructive comments. The author thanks the members of the EFSnet development team for their insight and support during the development of an online transaction processing system when the idea of implementing containers in shared memory for monitoring and dynamic configuration was born. Thanks to Eddy Luten for his high-resolution timer code that is used in the test scaffolding. Thanks to Gabriel Fleseriu and Andreas Masur for pointing out the adequacy of segregated storage in custom allocators. Thanks to Ion Gaztaaga for his work on the Boost.Interprocess library and to Dr. Marc Ronell for his prior work on shared allocators. A special thanks to Roy Soltoff, formerly of Logical Systems Inc, who authored LC (elsie), a simple C compiler for the TRS-80 with which the author learned the C language in 1982.

TABLE OF CONTENTS
Page

LIST OF TABLES LIST OF FIGURES LIST OF CODE LISTINGS Chapter 1. Introduction

1.1 1.2 1.3 1.4 1.5 Overview Scope Problem Statement Approach Outcome

ix x xi 1
1 1 2 4 4

Chapter 2. Background and review of literature

2.1 2.2 Related Work Literature

6
6 6

Chapter 3. Theory
3.1 3.2 3.3 3.4 Generic Programming and the Standard Template Library Interprocess communication and shared memory Memory allocation and segregated storage Object oriented approaches design patterns
3.4.1 3.4.2 3.4.3

8
8 8 9 11

Inheritance ............................................................................................................11 Abstraction, encapsulation, separation of concerns.............................................11 Design patterns .....................................................................................................12

Chapter 4. Analysis and Design

4.1 4.2 Introduction Functional requirements
4.2.1 4.2.2 4.2.3 4.2.4 4.2.5 4.2.6

13
13 14

Allocation of container objects in shared memory..............................................14 Allocation of containers contents in shared memory.........................................14 Coherence of internal data structures between processes ...................................15 Container attachment from a non-creating process .............................................15 Reference safety ...................................................................................................16 Concurrent access safety ......................................................................................16

4.3

Non-functional requirements

16

vi

4.4 4.5

Approach Detailed design

4.5.1 4.5.2 4.5.3 4.5.4

17 20

Overview ..............................................................................................................20 Low level use cases ..............................................................................................23 High-level classes shared containers and class factories..................................25 Process flows ........................................................................................................28

Chapter 5. Methods and Realization

5.1 5.2 5.3 Methodology Iterative design steps Shared containers library source code
5.3.1 5.3.2 5.3.3 5.3.4 5.3.5 5.3.6

31
31 31 32

Naming conventions.............................................................................................32 Structure of library and header files.....................................................................32 Interface to shared memory pointer consistency ..............................................33 Object instantiation and attachment map of shared containers ........................35 Synchronization....................................................................................................37 Reference safety ...................................................................................................38

5.4

Verification and validation

5.4.1 5.4.2

38

Unit testing ...........................................................................................................38 Test scaffolding ....................................................................................................39

Chapter 6. Results and Evaluation

6.1 6.2 6.3 Functional evaluation Performance evaluation Concurrency issues

42
42 43 45

Chapter 7. Conclusions
7.1 7.2 7.3 7.4 Potential applications Lessons Learned Future Activity Prospects for Further Work

47
47 48 49 50

REFRENCES CITED Appendix A. Shared Container Library Programmers Manual (excerpts)

A.1 Introduction to the Shared Containers Library A.2 Shared heap parameters A.3 sync A.4 shared_vector<T > A.5 shared_stack<T, Sequence > A.6 shared_map <Key, Data, Compare> A.7 shared_hash_set <Key, Compare> A.8 SharedVectorFactory<T >

51 54
54 55 56 58 59 61 63 64

vii

A.9 SharedStackFactory<T , Sequence> A.10 SharedMapFactory map<Key, Data, Compare> A.11 SharedHashSetFactory set<Key, Compare>

65 67 69

Appendix B. Shared Containers Library Source Code (Excerpts)

B.1 safe_counter.h B.2 sync.h B.3 sync.cpp B.4 heap_parameters.h B.5 shared_heap.h B.6 shared_heap.cpp B.7 shared_pool.h B.8 shared_pool.cpp B.9 shared_allocator.h B.10 container_factories.h B.11 shared_deque.h B.12 shared_stack.h B.13 shared_set.h B.14 shared_hash_map.h

71
71 72 74 79 79 81 93 94 102 105 113 114 116 117

Appendix C. Performance Evaluation Results Appendix D. Test Scaffolding Source Code

D.1 VectorTest.cpp D.2 MapTest.cpp

120 121
121 123

viii

LIST OF TABLES
Page Table 1: Functional requirements ................................................................... 14 Table 2: Performance comparison ................................................................. 44 Table 3: Raw performance evaluation results .............................................. 120

ix

LIST OF FIGURES
Page Figure 1: Top level use cases......................................................................... 13 Figure 2: Sample linked list............................................................................. 15 Figure 3: Layered design class diagram......................................................... 22 Figure 4: Low level use cases ........................................................................ 23 Figure 5: Shared container creation, attachment and release ....................... 24 Figure 6: Shared container classes and class factories ................................. 25 Figure 7: Container factories class hierarchy ................................................. 27 Figure 8: High-level use cases ....................................................................... 28 Figure 9: State chart diagram for the synchronization mechanism ................ 29 Figure 10: Sample shared container usage ................................................... 30 Figure 11: Header file inclusion hierarchy ...................................................... 34 Figure 12: Test scaffolding interactive flow .................................................... 40 Figure 13: Monitoring / configuring an application server............................... 47

LIST OF CODE EXCERPTS

Page Listing 1: The shared_object sub-class of shared_pool........................ 36 Listing 2: Declaration of the the shared container map .................................. 36 Listing 3: Shared heap header structure and the get_params method ....... 37 Listing 4: safe_counter.h................................................................................. 71 Listing 5: sync.h .............................................................................................. 74 Listing 6: sync.cpp .......................................................................................... 78 Listing 7: heap_parameters.h ......................................................................... 79 Listing 8: shared_heap.h ................................................................................ 81 Listing 9: shared_heap.cpp............................................................................. 92 Listing 10: shared_pool.h................................................................................ 94 Listing 11: shared_pool.cpp.......................................................................... 102 Listing 12: shared_allocator.h....................................................................... 104 Listing 13: container_factories.h ................................................................... 113 Listing 14: shared_deque.h .......................................................................... 114 Listing 15: shared_stack.h ............................................................................ 115 Listing 16: shared_set.h................................................................................ 117 Listing 17: shared_hash_map.h.................................................................... 119 Listing 18: VectorTest.cpp ............................................................................ 123 Listing 19: MapTest.cpp................................................................................ 125

xi

Chapter 1. INTRODUCTION
1.1 Overview

The aim of this project is to offer a practical programming interface that allows developers to implement generic container classes in shared memory. Such shared containers provide concurrent high-level access to easily retrievable structured data by more than one process.

This project extends the sequence containers and associative containers offered by the C++ Standard Template Library (STL) in the form of a Shared Containers Library which enables programmers to allocate both the containers and the objects they contain in shared memory in order to make them accessible by other processes running on the same machine.

Container classes can be found in most object oriented programming languages such as C++ (vectors, deques, sets, maps, etc.) Java (Maps, Sets, Lists, etc.) and C# (Hashtables, Queues, Stacks, etc.). Well-known data structures and algorithms are used to store and retrieve data with flexibility and optimal performance. Inter-process communication, on the other hand, offers low-level mechanisms to share data among processes. A gap has been identified between the high-level access offered by container classes and the low-level nature or typical inter-process communication. By implementing container classes in shared memory, the gap can be filled with a mechanism that makes high-level structured data concurrently accessible by various processes.

1.2

Scope

The main focus of the project is to extend the sequence containers and associative containers offered by the C++ Standard Template Library (STL) so that the containers and the objects they contain are allocated in a shared memory segment in order to be accessible by other processes. Care was taken to preserve the properties of the container classes when implemented in shared memory. In particular, the aim was to

maintain the genericity of the containers and to ensure that the different retrieval mechanisms (e.g. iterators) and sorting algorithms offered by the Standard Template Library still remain transparently usable when containers are allocated in shared memory.

This project concentrated on the Microsoft implementation of the STL for the 32bit Windows operating system platform as distributed with Microsoft Visual Studio 2008. Implementation of container classes in other languages such as Java and C# may be discussed but no specific research for these platforms has been included in the scope of this project. Access to shared memory was concentrated on the primitives offered by the Win32 platform and no effort was made to research other operating system implementations.

The following functional requirements have specifically been excluded of this project and no effort was be made to address them:

Data encryption or persistence Data access security Support for shared memory segment re-sizing Garbage collection

1.3

Problem Statement

Container classes are classes whose purpose is to contain other objects. These classes, parameterized to contain any class of object (SGI 2006), constitute a cornerstone of generic programming, a programming paradigm popularized by the C++ Standard Template Library (Stroustrup, as quoted by Buchanan 2008).

Since their introduction in the C++ Standard Template Library, container classes have gained in popularity and similar implementations can now be found in most object oriented programming languages. C++ containers like vectors, deques, sets and maps, Java collections such as Maps, Sets and Lists and C# collections such as Hashtables, Queues and Stacks all use well-known data structures and algorithms to

efficiently store and retrieve data in memory with great flexibility and performance. Furthermore, the generic and parameterized approach of these container classes fosters code reuse by abstracting the complexity of retrieval and sorting algorithms into standardized libraries (Baum & Becker 2000). The corresponding library classes that implement these containers are widely used in a large number of applications, but these well-known software tools have some intrinsic limitations.

By default, both data and indexing structures are meant to be stored in the contiguous memory space of the running process. If it is possible for different threads of a same process to access data in these containers provided thread-safe code and adequate synchronization are used, none of the aforementioned libraries currently offer a mechanism that allows these structures to be concurrently accessible by more than one process.

Mechanisms offered by operating systems for inter-process communication, on the other hand, tend to only offer a low-level access to data. Shared memory, for example, consists of a finite segment of memory, mapped to a file, and accessible concurrently and randomly by various processes. But this mechanism only supports a low-level access to a common contiguous memory region and requires a somewhat large amount of non-trivial work to be usable. Like in the case of a raw buffer, it is up to the developer to structure the data stored in a shared memory segment so that various processes can efficiently access it.

By coupling the usability, performance and code reuse benefits of container classes along with the inter-process communication nature of shared memory, the aim of this project is to offer developers a practical high-level access to easily retrievable data shared between programs while abstracting the inherent complexity of the implementation. Embracing code reuse and logical encapsulation, the approach selected is to extend the container classes offered by the C++ Standard Template Library (STL) so that both the containers and the objects they contain can be located in a segment of shared memory, and are therefore made concurrently accessible to various processes.

1.4

Approach

The main focus of the project is the C++ programming language on a 32-bit Microsoft Windows platform. This language and this platform have been chosen for the availability of a standard set of container classes (the STL) and by the well-documented nature of the shared memory primitives offered by the Windows operating system. In addition, the C++ language allows for a straightforward implementation of low-level operating system calls, an option that may not be directly available with other object oriented languages if the needed primitives are not exposed in their standard library which is usually not the case.

The implemented design follows a layered approach consisting of a low level interface managing a segment of shared memory, a higher-level shared memory pool responsible for allocation and memory management, and a highest level programming interface composed of classes derived from STL containers and their corresponding class factories. Good practices of object-oriented analysis and design are embraced in the form of code reuse, and abstraction of complexity through encapsulation and separation of concerns.

1.5

Outcome

This project offers a practical method to instantiate STL containers in shared memory on the Win32 platform. The approach and design selected provide a class library that allows programmers to instantiate and access STL-like shared containers with little knowledge or concern about the complexity of the underlying architecture.

Functional evaluation demonstrated that containers can be created and populated by one process and concurrently accessed and modified by others. Performance evaluation showed that, although containers located in shared memory may present degradation in performance when simple data types are being considered, the response time for populating and accessing a container composed of complex data type items such as strings is not significantly longer than for similar operations in the process main

memory. Although the code developed for this project lacks some important features such as solid exception handling, strong runtime type safety and garbage collection, it can serve as the basis of a wider-scoped effort to standardize allocation of STL containers in a shared memory on the Windows 32bit operating system or on Unix or Linux platforms.

Chapter 2. BACKGROUND AND REVIEW OF LITERATURE

2.1 Related Work

Ion Gaztaaga has extended the Boost initative with the Boost.Interprocess library, a C++ class library that offers a suite of containers similar to the STLs and that can be instantiated in shared memory (Gaztaaga 2008). However, the aim of this project differs from Gaztaagas approach and one could argue that modifying the code of the STL classes or designing all new classes may be considered contrary to object orientation and code reuse principles.

In a paper entitled A C++ pooled, shared memory allocator for the Standard Template Library, Dr. Ronell proposes an architecture for sharing STL containers between processes (Ronell 2003). After contacting the author who maintains a SourceForge project for a shared allocator it appears that the initiative suffered from a roadblock in the form of an impossibility for some versions of the GNU C++ compiler (GCC) to adequately instantiate shareable objects with the placement new directive. Dr. Ronell went as far as submitting a bug report to the GCC committee before putting the project on indefinite hold pending resolution of the issue and due to conflicting work schedule (Ronell 2006).

Other initiatives range from various flavors or STL custom allocators to libraries implementing standalone data structures in shared memory with various degrees of success. One project of interest is the STLshm library which published aim is to create and use STL containers in [] shared memory (anonymous n.d.), but according to the apparently rarely updated SourceForge website, the project seems to be stalled and no contact is provided for its author or caretaker to verify its progress.

2.2

Literature

Numerous books on the C++ language exist and the most commonly referenced is certainly The C++ Programming Language authored by its creator (Stroustrup 2000).

The standard documentation of the STL provides some basic insights as to container classes and their usage (SGI 2006). Additional literature provides extensive coverage of implementation details of the STL (e.q. Austern 1999).

In his book Effective STL Scott Meyers (2001) details some of the idiosyncrasies of the STL. In particular, this reading gives an understanding of some important limitations placed upon custom allocators.

A simple implementation of a custom allocator using segregated storage (Stroustrup 2000, p.570-573) is the subject of an article containing sample pseudo-code by Fleseriu & Masur (2004).

Charles Petzolds Programming Windows (1999) covers Win32 mechanisms that can be used for synchronizing access to shared memory between concurrent processes such as mutexes and semaphores.

Academic papers and public websites present a number of initiatives related to containers implemented in shared memory. Given gthe potential usefulness of the subject, it comes as no surprise that such an implementaton has been attempted before.

In addition to Dr. Ronells paper mentioned above (Ronell 2003), a 2003 C++ Users Group Journal article reprinted by DrDobbs magazine offers a special-purpose alocator to make STL containers available in shared memory (Ketema 2003). But one may argue that the author only describes a sample high-level architecture that falls short on concrete details for an implementation and lacks coverage of some functional requirements like, for example, the issue of pointer compatibility between processes.

Chapter 3. THEORY
3.1 Generic Programming and the Standard Template Library

Containers are an integral part of the Standard Template Library and some may argue that generic programming represent a defining feature of the C++ language. In fact, the STL draws its origins from the Ada Generic Library that pioneered the concepts of generic programming as described by David R. Musser and Alexander A. Stepanov: Generic programming centers around the idea of abstracting from concrete, efficient algorithms to obtain generic algorithms that can be combined with different data representations to produce a wide variety of useful software (Musser & Stepanov, 1988). The authors went on to create the Ada Generic Library (Musser & Stepanov 1989) followed a few years later by the C++ Standard Template Library (Stepanov & Lee 1995), which was in turn incorporated in the standard C++ library (Austern 1999).

It is a commonly accepted idea that the abstraction mechanisms offered by the STL provide great flexibility for data storage and retrieval using parameterized container classes. One example of such a beneficial abstraction mechanism of particular interest to this projects aim is the availability of custom allocators. Custom STL allocators were originally developed as an abstraction for memory models [] to ignore the distinction between near and far pointers (Meyers 2001). This same mechanism can be used to develop a custom memory allocation mechanism that differs from the default heap allocation offered by the STL. In addition to changing the memory allocation strategy, it can also be used in to redirect memory allocation to a managed segment of shared memory (Ketema 2003).

3.2

Interprocess communication and shared memory

Mechanisms are offered by operating systems to allow processes to communicate between each-other. Built-in primitives exist, for example, to create and manage mutexes and semaphores, giving access to flags and counters meant to be used for synchronization purposes (Downey 2008). Other mechanisms exist to transfer data

between processes sequentially such as with named pipes on the Unix operating system. Shared memory is a more flexible mechanism that can be used to allow multiple processes to share virtual memory space (Gray 1997).

On the Microsoft Windows platform, shared memory had a humble start by relying at first on a specific data segment declared in a Dynamic Linked Library (DLL). The programmer could allocate character strings in that segment and access them from the various processes having loaded the DLL (Petzold 1999). With the introduction of the Win32 Application Programming Interface (API) first in Windows NT, then in Windows 95 true shared memory capability was added to the operating system in the form of memory-mapped files: Memory-mapped files (MMFs) offer a unique memory management feature that allows applications to access files on disk in the same way they access dynamic memorythrough pointers (Kath 1993).

To access a segment of shared memory, a target memory-mapped file is first created for this very purpose. A Windows File-Mapping object is created or accessed by a call to CreateFileMapping or OpenFileMapping with the target memory-mapped file handle as a parameter. Then, using Win32 APIs MapViewOfFileEx function, a view into the File-Mapping object can be mapped into the process address space.

3.3

Memory allocation and segregated storage

If it can be said that memory management is one of the most fundamental areas of computer programming (Bartlett 2004), one can also argue that there is no memory allocation strategy that addresses all aspects of performance and resource utilization efficiency in one simple solution. One of the fundamental components of this project is the implementation of a low-level shared memory pool able to satisfy the allocation needs of a custom STL allocator. Given the diverse choice of available allocation algorithms, the determination was made to implement a simple segregated storage memory pool to manage a segment of shared memory.

Segregated storage revolves around a first-fit allocation of memory chunks of fixed size rather than allocation of variable size, dynamic memory sub-segments. When the memory pool is first initialized, every fixed-size chunk is available and points to the next contiguous chunk of memory in a manner similar to a linked list.

When an allocation request is processed, the fist available chunk is considered and the next available chunk becomes the first available chunk for the next request. If the required allocation size is smaller than one allocation chunk, no effort is made to reuse the unemployed memory. If the required size is larger than one chunk, on the other hand, a search for contiguous chunks with a total size at least that of the required allocation size is performed in the memory pool. If enough contiguous chunks are found, they are removed from the available pool by having the previous free chunk point to the next free chunk and the address of the first chunk is returned.

When previously allocated memory is released into the pool, the released allocation chunks are inserted at the beginning of the free chunk list. This algorithm presents the advantage of simplicity and rapid allocation and deallocaiton response time (Purdom et al. 1970). But numerous allocation and deallocation of various size blocks tend to fragment the memory pool and may lead to its inability to satisfy subsequent allocation requests of more than one chunk. A hardened segregated storage algorithm would therefore not be complete without an adequate garbage collection mechanism that guarantees that released memory chunks are returned to the pool in the most efficient way while preserving the highest ratio of contiguous free space. It is not, however, part of the scope of this project to design such a garbage collection mechanism and the segregated storage memory pool developed for the shared memory segment of this project does not make any effort at optimizing subsequent allocations. For this reason, chunk size has to be chosen with great care: too large a chunk size and a lot of memory is wasted for the most frequent allocations, too small a chunk size and

allocation/deallocation of multiple-chunk buffers may eventually fail because of high fragmentation.

10

3.4

Object oriented approaches design patterns

Of particular interest of this project, the following principles of Object-Oriented Programming have been embraced during the design and implementation phases:

3.4.1 Inheritance
According to Peter Frohlch (2002) inheritance is an incremental modification mechanism that transforms an ancestor class into a descendent class by augmenting it in various ways. In the spirit of this definition, inheritance allows for the reuse of the STL container code into derived classes tailored to be allocated in shared memory. In addition, the multiple inheritance feature offered by the C++ language allows for each STL container class to be derived into a shared version that also derives from a synchronization class.

Generalization and specialization through object inheritance is also present in the hierarchy of shared container class factories. During the normal iterative refactoring process for this project, a number of commonalities have been identified between the constructors of the various STL containers. If a one-to-one relationship exists between the type of shared container and its corresponding class factory, containers nevertheless fall into simple sequence containers, associative containers and hash-based containers. The class factory hierarchy is built along these same lines.

3.4.2 Abstraction, encapsulation, separation of concerns

As defined by Bennett et al. (2006) encapsulation is a kind of abstraction that [] focuses on the external behaviour of something and ignores the internal details of how the behaviour is produced. One of the aims of this project is to mask the internal complexity of allocating containers in shared memory. Abstraction is used to isolate the inherent complexity of interfacing with shared memory from the programmer by encapsulating complex tasks into the lowest levels of the implementation and exposing a simplified interface in the higher levels.

11

To use a term coined by Edsger W. Dijkstra, separation of concerns was used during the analysis phase to ensure that responsibilities of the different objects falls along the lines corresponding to a logically layered approach (Dijkstra 1974).

3.4.3 Design patterns

Each pattern describes a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice (Alexander et al., 1977). With the observation that different people sometimes solved similar problems in similar ways, some well-known creational design patterns have found their way in the architecture defined for this project.

In particular, two well-known design patterns were used during the design phase of this project, the singleton and the abstract class factory.

The singleton pattern is applied in order to ensure that a only one shared heap object is instantiated so as to provide a single access point of access for all shared memory allocators (Gamma et al. 2002, p.127). This behavior is required by the fact that STL custom allocators are not permitted to have non-static members (Meyer 2001, p 54), and that a single segment of shared memory is targeted by the Shared Containers Library.

The abstract class factory pattern was used to design the mechanism responsible for instantiating shared containers or connecting to existing ones. Class factories are defined as providing an interface for creating families of related objects while abstracting the complexity related to object instantiation (Gamma et al. 2002, p.87). Shared containers factories used in the design fall into this definition by allowing creation, attachment and release of shared containers.

12

Chapter 4. ANALYSIS AND DESIGN

4.1 Introduction

From a high level approach, implementing container classes in shared memory needs to satisfy a finite number of functional requirements. As it is the case for non-shared containers, a process should be able to create a shared container, access it and release its resources when it is no longer needed. But, in the case of shared containers, processes need also to attach to an existing container created by another process and safely access it concurrently. Figure 1 details the top level use cases for an implementation of a container object in shared memory.

Figure 1: Top level use cases

For various processes to create new container objects, attach to an existing ones, access and modify them concurrently, and then release their resources, the shared container objects needs to be located in a segment of memory that is commonly accessible between processes. In order to locate a container object in a segment of shared memory, not only the object itself needs to be residing in shared memory, but all the objects it contains along with its internal data structures have to be consistently accessible and modifiable from all processes as well.

13

4.2

Functional requirements

At a lower level, the requirement of creating, attaching to and accessing a shared container can be divided into items related to allocation of the container and items that provide access to it (table 1).

Allocation requirements Allocation of container objects in shared memory Allocation of containers contents in shared memory Coherence of internal data structures between processes

Access requirements Container attachment from a non-creating process Reference safety Concurrent access safety Table 1: Functional requirements

4.2.1 Allocation of container objects in shared memory

For a container object to be accessible by more than one process, it needs to be located inside a segment of shared memory. A mechanism is therefore needed to allocate a buffer of shared memory and build a container object in this buffer. Considering a lower level of requirements, it becomes clear that a mechanism is required to create and access a segment of shared memory and manage variable size memory allocation within this segment.

4.2.2 Allocation of containers contents in shared memory

Along its lifespan, a container object allocates and deallocates memory for objects it contains as they are inserted or removed. It also allocates memory for its own internal data strucutres and storage needs. A mechanism is therefore needed to allow the

14

container object to interface with the above mentioned mechanism in order to allocate variable size buffers in shared memory as needed.

4.2.3 Coherence of internal data structures between processes

It is the authors contention that the internal data structure coherence requirement is less trivial than the other functional requirements and may easily be overseen when first considering this project. The inherent complexity of this requiement resides in ensuring proper function of container objects used by concurrent processes. Specifically, the internal data structures of shared containers need to consistently point to the valid memory locations regardless of the process in which they are accessed.

To illustrate the discussion, consider the case of a simple linked list where each node contains a pointer to the next element (figure 2). If the nodes are located in shared memory, the pointers they contain need to consistently address the same memory locations regardless of the process in which they are accessed or the linked list is invalid. Without a mechanism devised to address this issue, a linked list built by process A would contain pointers to elements which are valid in process A but point to invalid memory addresses in process B.

Figure 2: Sample linked list

4.2.4 Container attachment from a non-creating process

Creating a container object located with its contents in shared memory and providing a mechanism that ensures consistency of its internal data structures between concurrent

15

processes would be useless without a way for a process to gain access to a container object created by another process. The requirement of container attachment from a noncreating process ensures that a process that did not create a shared container can nevertheless obtain access to it.

4.2.5 Reference safety

Once a process no longer has the need to access a shared container object, it is desirable to release memory allocated to that object. But other processes may still require access to it, and it cannot be destroyed without first verifying that it is no longer in use. A reference tracking mechanism is required to identify the last process that uses a particular shared container in order to destroy the container and release the memory it occupies once that process no longer needs it.

4.2.6 Concurrent access safety

Two or more processes could be granted read access to a shared container (without attempting to modify its contents) without penalty, but if a process attempts to access the contents of a shared container while another one modifies it, the internal data structures of the container may be in an intermediary state that would lead to unexpected results or access violations. An access mechanism is therefore needed that allows any number of concurrent reader processes, but ensures exclusive access to a process before it is allowed to modify a shared container. This mechanism should guarantee that the writing process is the only one accessing the container and that no other process is accessing the same container at the time.

4.3

Non-functional requirements

No limitation should be imposed on the shared memory segment in terms of number of shared containers it holds, overall size or individual size of containers, or number of concurrent client processes.

16

The programming interface presented to users/developers should be consistent with that of the STL and laid out in a clear, concise and logical way. Care should be taken to mask undue complexity and favor practicality whenever possible.

4.4

Approach

To satisfy the requirements set forth above, the necessity of implementing a mechanism to manage a memory pool in shared memory was considered. The aim of this memory management mechanism is multiple. First, it should create and map a segment of shared memory to the calling process address space and facilitate consistent access among processes. Secondly, it is responsible for implementing a memory allocation scheme that allows variable length buffers to be allocated, and to ensure that memory subsequently released returns to the pool for possible reallocation. Finally, in its quality of focal point for communication among the different processes, the memory management mechanism can also be used to help satisfy the requirement of container attachment from a non-creating process. In this regard, it can offer a centralized mechanism to link container objects to unique identifiers that can be used by non-creating processes to obtain access to existing containers.

Once a memory pool mechanism that allocates and manages a segment of shared memory exists, creating a container object in shared memory can be performed by invoking the placement new C++ instruction to instantiate the desired object in this buffer. This language extension allows the new keyword to target a specific memory address instead of allocating memory from the process heap (Stroustrup 2000, p.255256).

Note that, with the use of the placement new operator, final destruction of the shared object cannot be performed by invoking the C++ delete instruction but rather should be done first by calling the objects destructor method and then by releasing the memory buffer from the pool.

17

The functional requirement of allocating the containers contents in shared memory is addressed by using a mechanism provided by the STL in the form of a custom allocator. A custom allocator is a class used as a parameter in the creation of a STL container class that exposes allocation methods used by the container to allocate memory in a custom location chosen by the programmer. Such a custom allocator can be designed to use the above mentioned shared memory pool for its allocation need. It is outside the scope of this discussion to compare the merits of different memory allocation strategies. In this regards, the decision was made to use a form of segregated storage as documented by Stroustrup (2000, pp.570-573).

The diverse responsibilities are given to the shared memory management mechanism and points to a layered implementation in order to ensure a proper separation of concerns. The lower level of this mechanism is in charge of interfacing directly with the shared memory segment, whereas the higher level manages memory allocation within the segment and facilitates low level communication between like-mechanisms in concurrent processes. The highest level interfaces directly with the STL and offers a practical programming interface.

The issue of consistency of the containers internal data structures among processes is of particular concern for the success of this project. After a superficial analysis, it would be tempting to conclude that virtualization of memory references between processes can simply be achieved by wrapping the containers contents in relative pointers adjusted within each process for the offset where the shared memory segment is mapped. If this approach would indeed ensure consistency of pointers within each process address space, this idea does not stand further scrutiny as it could solve the issue of virtualization of the contents but it would not address the need to virtualize the containers internal data structures as illustrated by the linked-list example (figure 2).

In this regards, one could envision the need for a modification of the container classes so that they use a relative pointer for their internal data structures as well. But this approach requires a customized behavior on the part of the container classes that can only be achieved with a modification of their source code. Changing the code of the Standard

18

Template Library clearly falls outside the scope of this project, and one may argue that it goes against the core principles of reuse and encapsulation defined as some tenets of object oriented programming. For these reason, other options were researched to satisfy this requirement.

Although seemingly restrictive and apparently difficult to achieve, the approach selected is in fact to ensure that the segment of shared memory is mapped at the very same offset in each process. By forcing mapping of the shared memory segment to the same address in each process space, each byte of shared memory is guaranteed to be located exactly at the same offset in every process, and this solves the pointer consistency issue altogether. This behavior guarantees consistency of the shared containers internal data structures between the processes while at the same time reducing the complexity of the implementation.

The requirement for concurrent access safety is satisfied by implementing a locking mechanism. But, unlike the simplest existing forms of locking solutions, it was desirable to design a mechanism that allowed multiple read-only processes to obtain a non-exclusive lock ensuring that no writers were currently accessing the container, while allowing a writer process to obtain an exclusive lock that ensure that no other writer and no readers concurrently access the container.

To achieve the non-functional requirements of simplicity and accessibility of the programming interface, the approach selected is to use a class factory mechanism that can be invoked to create a new container, attach to an existing one or detach from a container. The class factory approach masks the complexity of the implementation and provides a high-level interface for accessing a shared container by a non-creating process. It interfaces with the lower levels of the implementation to allocate memory in the shared segment, obtain a pointer to a shared container object, and release unused resources thanks to a built-in reference safety mechanism.

19

4.5

Detailed design

4.5.1 Overview
Problem domain analysis for the functional requirements set forth led to the selection of a layered architecture approach. Interface to an allocated segment of shared memory is layered between a low-level interface implemented by the shared_heap class and a higher level interface implemented by the shared_pool class. The shared heap manages the shared memory segment and generic allocation whereas the shared pool manages allocation of objects and containers. The highest level of implementation is materialized at the shared container factory level with the help of the

shared_allocator custom allocator class (figure 3).

The shared heap is implemented as a singleton class that offers a low-level interface to an allocated segment of shared memory. The shared heap object implements the mechanism for segregated storage allocation within the shared memory segment.

Other responsibilities of the shared heap include connecting client processes to the same segment of shared memory while ensuring consistency between processes address space and offering a way for the higher level interface to access common parameters. A mechanism is also provided to detect when the last process releases the shared memory segment in order to perform resource deallocation and cleanup.

The shared heap uses a discrete header allocated at the beginning of the shared memory segment. This header allows the shared heap to map the shared memory segment to the same address space in all processes in order to ensure pointer consistency between processes. The header also holds thread safe variables used by the memory allocation algorithm for synchronization, along with custom parameters common to all process and used by higher level objects through its getparams method.

The shared pool is a higher level interface that implements access to the segment of shared memory that is tailored to be used by shared containers. As a static class, it enforces the singleton nature of the shared heap by forwarding allocation and

20

deallocation requests to a single static shared heap object. The shared_pool class maintains a map of shared containers allocated in the segment of shared memory in order to offer a mechanism for all class factories to create, attach to, and detach from identifiable instances of shared containers. The shared map resides in shared memory and is indexed by a unique container object name (string). It contains instances of shared_object, a simple class defined to hold parameters and references to each shared container. The shared pool uses the ptrparam parameter of the shared heaps header exposed through the getparams method to connect concurrent processes to the shared container map.

The highest level of the interface consists of class factories deriving from a generalized _SharedContainerFactory class. Class factories are used to instantiate and attach to shared container object. A custom allocator class, shared_allocator, is used as a template parameter for STL containers to target the segment of shared memory for their allocation needs.

21

Figure 3: Layered design class diagram

22

4.5.2 Low level use cases

The different low level use cases of the system are depicted in figure 4.

The shared allocator is a STL custom allocator that uses the shared pool to interface to the shared memory segment. Shared container factories also use the shared pool to build new container objects in shared memory and insert newly created shared containers or find existing ones in the container map that is kept in shared memory. The shared pool keeps a reference counter updated for each shared container. When no more processes hold any reference to a particular shared container, it is destroyed and the shared memory it utilizes is freed by the class factory. The shared pool interface is then called upon to remove the corresponding entry in the container map.

Figure 4: Low level use cases

The sequence diagram in figure 5 details a typical shared container creation, attachment by another process and release.

23

Figure 5: Shared container creation, attachment and release

24

4.5.3 High-level classes shared containers and class factories 4.5.3.1 Overview
Instantiation of shared containers is performed by using derived shared container classes and their corresponding class factories.

For each container class defined in the STL, a corresponding shared container class and a class factory are defined. The shared container class derives from the base STL container class and from the sync class which provides a synchronization mechanism for concurrent access. The corresponding class factory allows the programmer to create a new shared container, attach to an existing one, or detach from a previously attached instance. Each newly created shared container is identified with a user-assigned alphanumeric name so that other processes can identify it and attach to it.

As an illustration, the shared_vector class derives from the STL vector class and uses the SharedVectorFactory for instance creation and access. Similarly, the shared_map class derives from the STL map class and is exposed through SharedMapFactory (figure 6).

Figure 6: Shared container classes and class factories

25

The shared container factory classes have been generalized along the lines of the three generic types of containers defined in the STL: sequential, associative and hash-based. Each container type offers specific constructors that are exposed by the corresponding class factories. Each shared container-specific class factory derives from the corresponding type-generic container factory class. For example, as the vector class is an STL sequence container, the SharedVectoryFactory class derives from the _SharedSequenceContainerFactory class. This class, in turn, derives from the generalized _SharedContainerFactory class. Figure 7 details the generalization and specialization relationships between container factory classes. Common constructors are implemented in the base _SharedContainerFactoryClass. A set of constructors used by sequence containers are class. implemented Another set in of the derived for the

_SharedSequenceContainerFactory associative containers are

constructors in

implemented

_SharedAssociativeContainerFactory class. Finally, a set of constructors specific to hash-based containers are implemented in the _SharedHashContainerFactory class. Container-specific class factories derive from one of these base classes according to the type of container considered.

26

Figure 7: Container factories class hierarchy

27

4.5.3.2 High level use cases

The high level use cases of the system are depicted in figure 8.

Figure 8: High-level use cases

A client process can create a new shared container or attach to an existing one created by another process. Before accessing the container it needs to ensure no other process is modifying it by obtaining a non-exclusive read lock. Before modifying a shared container, it needs to ensure that no other process is accessing it by obtaining an exclusive write lock. The client process also needs to be able to release the locks. When its use of the shared container is over, it releases it back, thereby signaling that no more access will be performed on this object.

4.5.4 Process flows

Figure 9 is a state chart diagram detailing the different states and transitions of a sync object.

28

Figure 9: State chart diagram for the synchronization mechanism

A process can obtain an exclusive or non-exclusive lock on the object, it can attempt to upgrade an existing non-exclusive lock to an exclusive on, or it can downgrade an exclusive lock into a non-exclusive one.

The following UML sequence diagram (figure 10) details a simple shared container creation by one process and access by another one.

The first process creates the shared vector and obtains a write lock on it. The second process attaches to the shared vector by name and attempts to get a read lock on it. Once the first process releases the write lock, the second process obtains the read lock.

The first process releases the shared container. The second process releases the read lock and releases the shared container. Because the second process was the last user, the shared container is then destroyed and all shared memory occupied by it is released.

29

Figure 10: Sample shared container usage

30

Chapter 5. METHODS AND REALIZATION

5.1 Methodology

Due to the very nature of this project, an iterative approach similar to Boehms spiral model was selected for analysis, design, implementation and testing (Boehm 1986). This approach allowed for incremental discovery of the functional requirements as a better understanding of the different issues was obtained. This approach proved valuable when addressing the issue of pointer consistency, for example, and when the concurrency issue linked to multi-core processors was unveiled.

5.2

Iterative design steps

Original analysis identified the functional requirements for this project and the issue of pointer consistency among processes was researched in depth. In an attempt to address the requirements in a most elegant way, a preliminary draft of the design evolved from a class wrapper around STL containers to a parameterized class in the form:

shContainer<class T, class shAlloc>

where T was an STL container class and shAlloc was an implementation of a custom STL allocator managing a segment of shared memory. With this approach, a declaration for a shared vector of integers would have consisted of:

typedef shContainer<vector<int,shAlloc>,shAlloc> shINTVECTOR;

In the end, it was determined that this approach would have fallen short of implementing some of the requirements (e.g. access by non-creating process) and may have proven to be less intuitive to use by the programmer than the class factories.

The issue of pointer consistency among processes was researched and initial design experimented with the concept of virtual pointers similar in concept to the auto_prt STL class. Literature research showed that similar efforts to virtualize pointers in each process were not being successful or required rewrite or modification of the STL

31

container classes and the decision was made to align the shared memory segment on the same offset in all the processes.

5.3

Shared containers library source code

5.3.1 Naming conventions

The following naming conventions are used throughout the source code:

lowercase is used for most class names, attributes and operations, for consistency with the STL. BiCapitalisation is used for class factory names to differentiate them from regular classes. underscore_separated_words are used for class names and operations. _leading_underscores are used for class attributes and for class names meant to be derived (e.g. generalized class factories)

5.3.2 Structure of library and header files

The shared container library consists of a binary library file shared_containers.lib and a series of C++ header files. First-line header files match corresponding STL container header files and are meant to be included by programmers in their source code (shared_vector.h, shared_map.h, shared_hash_set.h, etc). Each shared container class is declared in its corresponding header file. Most header files include the corresponding STL header and the container_factories.h header file. One exception is the shared_stack and shared_queue classes that are declared in their respective headers but include the shared_deque.h header file due to their adaptor nature. Additional, lower-level header files are included by the container_factories.h header file (figure 11). Due to the parameterized nature of the shared container classes, most class instantiation logic is contained in the class_factories.h header files rather than a compiled object library file. Lower-level interface to the shared memory pool is implemented in the shared_pool.cpp and shared_heap.cpp files. The

32

synchronization class is implemented in the sync.cpp file. All cpp files are compiled into the binary library file for distribution along with the headers. In addition to the library and traditional header files, the heap_parameters.h header file contains a one-time declaration of heap variables that need to be present at link time (listing 7). This header can be included one time in any module of the C++ project. The external heap variables can be customized at compile time and are used by the shared_heap object to open and initialize the shared memory segment. These variables include the size and identifier of the segment of shared memory, the size of memory chunks used by the segregated storage algorithm and the target process base address for mapping the shared memory segment.

5.3.3 Interface to shared memory pointer consistency

To solve the issue of pointer consistency between processes it has been decided to map the shared memory segment at the same offset in each process. Consistent mapping is achieved by using the MapViewOfFileEx Win32 API calls that accepts a forced mapping offset as an optional parameter and is available in Microsoft Windows 2000 and newer operating systems, including Windows XP (Microsoft 2008 [1]).

To achieve consistent mapping among processes, the segment of shared memory is in fact mapped twice. First the shared header is mapped at an address determined automatically by the operating system so as not to conflcit with the rest of the process memory. The shared header contains, among other things, the offset at which the rest of the shared memory segment should be mapped in every process in order to ensure pointer coherence. This value, stored in the _beaseaddress member of the shared header (listing 3), is used for a second, separate mapping of the shared memory segment which extends for the remainder of the segment size.

33

Figure 11: Header file inclusion hierarchy

34

Selecting an appropriate offset for mapping the segment of shared memory at the same address in every process is non-trivial and no guarantee is made that mapping can be obtained without errors. To facilitate consistent mapping, the offset should be high enough in the process memory space that it does not interfere with the process heap, but low enough so that it falls in the allowable addressing space of the process, while at the same time providing enough headroom for a large enough segment of shared memory.

In Win32, a process memory space is limited to an addressable 2 Gb of memory for a highest 32-bit address of 0x7fffffff (Kath 1993). Any value corresponding to the size of the shared memory segment subtracted from the maximum address would be appropriate as the starting offset of the mapped segment of shared memory. In the current configuration, an arbitrary value of 0x01100000 has been selected for the 32-bit mapping offset of the shared memory segment along with a heap size of 64 Mb or 0x04000000 bytes (listing 7). Base offset (0x01100000) + heap size (0x04000000) = 0x051000000, which is a lower memory address than the process highest address of 0x7fffffff.

5.3.4 Object instantiation and attachment map of shared containers

The requirement of container attachment from a non-creating process is addressed by the existence of a master list of shared containers allocated in the segment of shared memory. This master list exists in the form of a STL map container that contains the name of each shared container along with a structure containing a pointer to the shared container object, its typeof size, and a reference counter. This structure is implemented in the form of shared_object, a sub-class of the shared_pool class which is responsible for managing the shared map (listing 1):

class shared_pool::shared_object { public: void* _object; // Pointer to object size_t _size; // Size of object safe_counter _refcount; // Usage count

35

shared_object(void* object, size_t size): _object(object), _size(size), _refcount(1) {}; shared_object() {}; ~shared_object() {}; }; Listing 1: The shared_object sub-class of shared_pool

The shared map is then defined as shared map of string and pointers to shared_object objects which is allocated in the segment of shared memory and uses the shared_allocator custom allocator for its allocation needs (listing 2).

template<class Key, class Data, class Compare=std::less<Key>> class shared_map : public sync, public std::map<Key, Data, Compare, shared_allocator<std::pair<const Key, Data>>> {}; typedef shared_map<std::string, shared_object*> container_map; static container_map* _containermap; Listing 2: Declaration of the the shared container map

The mechanism implemented in order to grant access to the shared map from all processes uses the get_params method exposed in the shared_heap class. Designed to use an approach similar to the parameters used in Windows messaging (Petzold 1999), the get_params method returns a pointer to user-defined parameters stored in the shared heap header (listing 3). The shared_pool class uses the _ptrparam void* header member (returned in the ptrparam parameter) to store the address of the shared map and make it accessible among processes.

__declspec(align(32)) volatile class shared_heap::shared_header { public: char _signature[sizeof(SHAREDHEAP_SIGNATURE)]; // Signature void* _baseaddress; // Common base address void* _head; // First available chunk size_t _heapsize, _chunksize; // Size of heap and chunk size long _longparam; // Client long parameter void* _ptrparam; // Client pointer parameter safe_counter _numprocesses; // Number of processes safe_counter _allocating; // 1 if allocation in progress char _filename[MAX_PATH]; // Temp filename }; void shared_heap::get_params(long** longparam, void*** ptrparam) { *longparam = &_header->_longparam;

36

*ptrparam = &_header->_ptrparam; } Listing 3: Shared heap header structure and the get_params method

In addition, the shared_pool class performs simple verification when attaching to an existing shared container. To this extent, the size of the container object returned by the sizeof operator is validated against the size stored in the corresponding member in the shared_object object in order to prevent mapping a pointer to a container of one type to an existing shared container of a different type. This weak verification is far from perfect and does not guarantee the denial of attachment of an incorrect shared container type, but it is designed as a first line of defense in a more complex mechanism that can be subsequently developed.

5.3.5 Synchronization

Interprocess synchronization is provided in the form of a low-level sync class from which every shared container class derives using C++s multiple inheritance mechanism (listings 5 & 6). The sync class uses safe_counter members that are designed as a wrapper aroung a simple, thread-safe counter using Win32s interlocked primitives (listing 4). Note that the safe_counter type is also used in the shared heaps header to facilitate synchronization of allocation operations.

The sync class maintains a safe_counter for read locks and a separate one for write locks. It provides a mechanism for obtaining a non-exclusive read lock or an exclusive write lock, along with a mechanism for upgrading from a non-exclusive read lock to an exclusive write lock, and for downgrading from an exclusive write lock to a non-exclusive read lock. Downgrade is a guaranteed operation, but upgrade is not, and it does not support an infinite timeout to avoid possible dead-lock scenarios (e.g. two threads attempting to upgrade a non-exclusive lock simultaneously).

In addition, each sync object maintains a reference of the process and thread number that owns an exclusive write lock in order to accept subsequent write lock and unlock

37

requests from the same thread/process without error. A fail-safe mechanism is provided to prevent any lock counters to attain a negative value.

5.3.6 Reference safety

The shared_object class used in the shared container attachment mechanism contains a safe_counter object used by the shared_pool for reference counting. The counter is set to one when the shared container is created and incremented when a process attaches to it. The counter is decremented each time a process detaches from the container. When the counter reaches zero the containers destructor is called and the memory it occupied is released by the shared_pool.

5.4

Verification and validation

Due to the unavailability of an industry sponsor for this project, validation of the Shared Container Library was limited to the authors perception based on prior experience with STL containers. Some key qualitative properties were confirmed during the

implementation of the test scaffolding software, in particular in terms of ease of deployment and abstraction of complexity for the programmer-user. Unit testing was performed on each component as an initial quality assurance step and a test scaffolding program was designed to verify that the key functional requirements of the project were met. Regression testing was performed with the help of the test scaffolding at each incremental phase of the development effort.

5.4.1 Unit testing

Each component of the library was unit tested with an emphasis on structural coverage and in particular branch coverage analysis. Due to the low-level nature of the produced code, the decision was made to rely on the step-by-step feature of the development environments built-in debugger to conduct most of the initial testing instead of using repeatable unit testing tools such as Cunit which would have required an extensive simulated running environment.

38

5.4.2 Test scaffolding

For the purpose of evaluating the functionalities and the performance of the produced library, a test scaffolding program was developed to exercise a variety of STL containers operations and types. The test scaffolding program was designed to be instantiated in more than one process in order to test concurrency behavior.

In its interactive flow (figure 12), the test scaffolding first offers the choice to consider containers allocated in the default process heap or in a shared memory segment. Then the user is asked to choose what type of container is to be exercised among the following:

- vector - map - queue

The user is then given a choice of operation to perform on the selected container:

create/attach : create a new container or attach to an existing one populate : populate the container with discrete values verify : verify that discrete values populate the container destroy/detach : destroy the container or detach if still in use by another process

Each operation is timed with a millisecond timer in order to measure performance. A typical functional evaluation consists in create, populate, verify and destroy operations. Multi-process functional evaluations consists in create and populate operations in the main process followed by concurrent attach, verify and detach operations in additional processes.

Queue functional testing varies from other containers for populates makes one process push values into the queue while verify makes another process pop and verify values from the queue. Timeouts are adjusted for queue testing in order for the popping process

39

to wait for the pushing process without generating a verification error if the queue is empty.

Figure 12: Test scaffolding interactive flow

In order to populate the containers with non-repetitive discrete values, the following approach was used:

Non-repetitive numeric values are obtained by adding the order of the value to the previous result to obtain a suite in the form ni = n(i-1) + i for i > 0 where n0 = 0 (listing 18).Similarly, non-repetitive string values are obtained by considering an ASCII character value varying with the index number modulo 26, and building a string consisting of the obtained character followed by a numeric representation of the index value (listing 19). The pseudocode form of this algorithm would be:

40

Char(A + i%26) & i for i 0

which would provides the following suite of values:

"A0", "B1", "C2", , "Z25", "A26", "B27", etc.

41

Chapter 6. RESULTS AND EVALUATION

6.1 Functional evaluation

Functional evaluation of the shared containers was performed by instantiating a container, populating it with non-repetitive but predictable discrete values and then verifying the presence and correctness of the values in the container.

The test scaffolding was used for testing a vector-type container of integer values and a map-type container of variable-length string keys and integer values. In each case, the number of iteration was variable with final evaluation performed on 1,000,000 items.

Evaluation results were positive with the successful verification of the contents of a container by one process while it had been populated by a different process, both for a shared vector of integers and a shared map of string and integers.

The test scaffolding was also used to perform concurrent testing on a queue adapter to a shared deque. One process pushed 10,000 integer values into the queue while another process popped values from it. It was noted that push and pop operations on a shared deque (or a stack or queue adapter) required exclusive access (write lock) due to the fact that the first and last element of the deque were constantly being modified and internal iterators were thus being invalidated.

The inefficiency of the segregated storage algorithm in terms of reallocation of multiplechunk buffers was evidenced by the original failure of a test consisting in creating and populating a large map of strings and integers, destroying it, and then re-creating it. The fact that the simple segregated storage algorithm does not offer any garbage collection mechanism led to high fragmentation of the shared memory pool, and, more specifically, to the deallocation of multiple chunks of memory in random order. After destroying the map object, the shared pool consisted in free chunks of memory, few of which were consolidated in the correct order with their immediate neighbors. This resulted in the inability of the algorithm to allocate multiple-chunk buffers and led to a failure of the test

42

program. After analysis of the cause of the failure, it was decided to double the size of the allocation chunk in the shared heap mechanism from its initial testing value of 32 bytes thereby greatly reducing the number of multiple-chunk allocations (listing 7). This change solved the reallocation error issue and multiple creation and destruction of large maps of strings was subsequently successful.

6.2

Performance evaluation

For performance evaluation, the response time of standard operations on shared containers was compared to the performance delivered by pure STL containers during the same allocation, insertion of records, retrieval of records and release. Test was performed allocating 1 million integer items into a vector and 1 million string and integer pairs into a map. The comparison was made using the following parameters:

Local memory:

Instantiation of pure STL vector and map containers

Shared memory with create:

Instantiation of shared_vector and shared_map containers. First instantiation was not measured because it triggers creation and initialization of the shared memory segment and associated segregated storage pool

Shared memory with attach:

Creation of shared_vector and shared_map containers by one process followed by attach, populate, verify and release by another process, then release of the container by the creating process for deallocation.

No synchronization delays (successive, not concurrent access)

Table 2 details the results of this comparison.

43

Local memory vector<int> - create / attach - populate - verify - destroy / detach map<string,int> - create / attach - populate - verify - destroy / detach

Shared memory w/create < 1ms 18ms 1ms < 1ms < 1ms 1,683ms 1,261ms 193ms

Shared memory w/attach < 1ms 17ms 1ms < 1ms < 1ms 1,752ms 1,317ms < 1ms

< 1ms 13ms 1ms < 1ms < 1ms 1,699ms 1,107ms 393ms

Table 2: Performance comparison

Local memory results were obtained by instantiating native STL classes. The Shared memory w/creation column corresponds to standalone shared container classes. The creation operation does not include initialization of the shared memory segment and its corresponding segregated storage memory pool which was independently measured as adding less than 100ms for a 64Mb shared pool with a chunk size of 64 bytes. The Shared memory w/attach results were obtained by creating a container in one process and then measuring attach, populate, verify and destroy operations from another process. In each case, the tests have been run several times and the average results are being presented here (see appendix C for complete results).

Creation time of the container objects in the Shared memory w/creation case is on par with that of other cases. Response time for attach in the Shared memory w/attach case reflects simple attachment to an existing shared container.

Once the shared memory segment is initialized, creation of a shared container is performed in a similar time than the time required to create a local container. To this duration, the time O(log n) required to insert a new container into the shared container maps balanced binary tree structure needs to be taken into account (where n is the number of shared containers present in the container map). Similarly, the time required to attach to an existing container is O(log n), the time required to retrieve the container by name from the shared container map (Cormen et al. 2001).

44

One important finding is that, except when considering very simple data types, response time to populate and verify a container in shared memory does not show significant degradation compared to locally allocated containers. Performance related to populating a container and verifying its contents does seem to suffer a 30-40% degradation when simple data types are used (e.g. a vector of integers) but does not significantly differ for complex types (e.g. a map of strings) whether the container is located on the process heap or in shared memory. Of course, possible delays due to high disk activity or concurrency locking requests are not taken into account in the above measured results.

Destruction of a shared memory allocated container proves to be faster than that of a local one, a fact that may be explained by the fact that the segregated storage algorithm used in this implementation does not make any attempt at garbage collection whereas the default heap allocator may perform cleanup tasks when memory is deallocated. In addition, the short time observed for detach in the Shared memory w/attach case (<1ms) can be explained by the fact that the shared container is not actually erased as the creating process still holds a reference to the object. In this case, the reference counter of the shared container is simply decremented.

6.3

Concurrency issues

The postulate that multithreaded programs are deterministic by nature [] many serious bugs will not happen all the time, or even with any regularity (Cohen & Woodring 1998) was verified during the testing phase of this project.

Testing was conducted on two machines both running the same version of Windows XP professional with the latest Microsoft updates. A desktop computer with a dual-core Pentium 6400 processor running at 2.13 Ghz was used for development and testing, and a laptop with a Pentium M processor at 1.6 Ghz was used for further testing and text editing. Although concurrent multi-process testing on the laptop did not show any issue, initial multi-process testing on the desktop computer with a debug version of the code terminated with a fatal exception: DEBUG_ERROR("ITERATOR LIST CORRUPTED!")

45

The error was traced to the xutility header of the Dinkumware implementation of the STL (based on code written by P.J. Plauger) distributed with Microsoft Visual C++ 2008. Further research showed that the error was in fact linked to an iterator debugging feature specific to this implementation of the STL. Because the error only occurred on the desktop and not on the laptop, this issue may only exist on multi-core processors capable of true multitasking as opposed to single core variants that only simulate multitasking by the use of time-sharing techniques.

A solution was found in the form of defining the macros _HAS_ITERATOR_DEBUGGING and _SECURE_SCL both with a value of zero to turn these additional features off at compile time (Microsoft 2008 [2], Microsoft 2008 [3]). Once these macros were applied and the test code recompiled, no additional faults were noted during subsequent multi process concurrent testing on the desktop computer.

46

Chapter 7. CONCLUSIONS
7.1 Potential applications

Shared STL-like containers can be used for a variety of implementations that require high-level access to structured data between processes.

Figure 13: Monitoring / configuring an application server

47

Such needs are frequently encountered in concurrent interactive applications where users are allowed to exchange data. Other applications include monitoring of online transaction processing system. Databases and local files can be used in lieu of shared containers for these applications, but these solutions may add overhead, additional failure points and/or increased code complexity compared to shared containers.

One practical example of shared container could be for a transaction processing system where one could envision each worker thread of an application server updating a given set of metrics residing in a shared map that can be read by another process. This other process could, for example, expose these metrics via the Simple Network Management (SNMP) protocol. The same SNMP interface could use another shared map to dynamically modify the internal parameters representing the configuration used by the worker threads, thereby providing a path to dynamic configuration (figure 13). One could also envision a separate process using the metrics harvested by worker threads in the shared container to dynamically change the configuration of the working system in order to react to connection errors or system-wide congestion (e.g. automated database failover, etc).

7.2

Lessons Learned

The subject of implementing STL containers in shared memory has attracted quite a bit of attention over time, possibly because it may look easily feasible to the casual observer, but proves to be in fact more challenging than originally thought. It would be an oversimplification to think that implementing a custom allocator targeting a shared memory segment would be sufficient to reach the goals of this project. In fact, allocating the data held by the containers in shared memory is only one of several functional requirements and maybe one of the easiest to satisfy. Other requirements include placing the internal indexing structures of the containers in shared memory and making them consistently accessible by various processes. In addition, such an implementation would not be complete without an adequate synchronization and reference counting mechanism that guarantees safe concurrent access to the containers and their contents

48

but releases resources used by objects that are no longer in use. One could therefore argue that the challenges presented by a project like this one may remain unrecognized until one starts performing a detailed analysis of the implementations requirements.

Functional evaluation on the shared containers library led to a surprising finding in the form of satisfactory performance on a computer equipped with a single core processor but an unexpected behavior on a double-core equipped machine. The lesson here is that multithreaded or multi-process applications should be tested on architectures where true concurrency can occur in this case a dual-core Pentium processor to guarantee that all potential synchronizing issues have been addressed. Processors with a high degree of compatibility can still have significant architectural differences that may cause issues in software programs to remain hidden for a long time.

7.3

Future Activity

The code implemented for this project lacks strong exception handling and this should be added before the library is used in real-life applications. The simple segregated storage algorithm does not currently offer a garbage collection mechanism and one could argue that its usefulness would consequently be limited if numerous allocation/deallocation of large memory buffers take place. A garbage collection mechanism would be a welcome addition to the library.

In addition, the current library implementation only performs very limited runtime type checking when attaching to an existing shared container. Only the memory footprint of the container object is taken into account (its size returned by sizeof) to verify that the container expected from a call to ContainerFactory::attach is of the correct type. One could envision using C++s runtime type information (RTTI) mechanism using the typeid operator to verify the containers type upon attachment by a non-creating process.

49

Finally, no real-life testing of the shared container library has been performed at this time and it would stand to reason to think that early practical usage of the library may unveil issues that were not uncovered during initial testing.

7.4

Prospects for Further Work

The current implementation of the shared container library is limited to the Win32 operating system and the STL library distributed with Microsoft Visual Studio 2008. Porting the shared containers library to other operating systems such as Unix or Linux should prove feasible by developing platform-specific versions of the shared heap and the safe counter classes. Developing a similar library for other languages such as Java and C# would also be an interesting endeavor, albeit one that would pose its own set of challenges and may lead to a different approach. For example, if parameterized classes can be declared in newer versions of Java and C# by using generics similar in concept and syntax to C++s templates (Bracha 2004 & Microsoft 2008 [4]), neither languages supports multiple inheritance which is extensively used in this implementation.

Another direction for further work could be to widen the features of the Shared Containers Library. Additional desirable functionalities can be envisioned such as the possibility to use more than one memory mapped file a current limitation of this implementation the possibility to persist the memory mapped file on disk once all processes have terminated execution and the addition of security and data encryption for the shared memory segments and the containers they hold.

50

REFRENCES CITED
Alexander C. et al. (1977) - "A Pattern Language" - Oxford University Press - ISBN 9780195019193 p.x anonymous (n.d.) - "STLshm" - SourceForge.net - [Online] Available at: http://stlshm.sourceforge.net/Introduction.html (Accessed November 25, 2008) Austern, M. (1999) - "Generic Programming and the STL" - Addison Wesley - ISBN 0201-30956-4 Bartlett, J. (November 16, 2004) - "Inside memory management - The choices, tradeoffs, and implementations of dynamic allocation" - IBM developerWorks - [Online] Available at: http://www.ibm.com/developerworks/linux/library/l-memory/ (Accessed December 14, 2008) Baum, L. & Becker, M. (2000) - "Generic components to foster reuse" - Proceedings of the 37th International Conference on Technology of Object-Oriented Languages and Systems - ISBN 0-7695-0918-5 pp.266-277 Bennet S. et al. (2006) - "Object-Oriented Systems Analysis And Design Using UML" p.226 - McGraw Hill - ISBN 13-978-0-07-711000-0 Boehm, B.W. (May 5, 1986) - "A spiral model of software development and enhancement " - IEEE Computer - [Online] Available at: http://ieeexplore.ieee.org.ezproxy.liv.ac.uk/iel1/2/6/00000059.pdf?tp=&arnumber=59&isnu mber=6 (Accessed December 18, 2008) Bracha G. (July 5, 2004) - "Generics in the Java Programming Language" - Sun Microsystems - [Online] Available: http://java.sun.com/j2se/1.5/pdf/generics-tutorial.pdf (Accessed December 18, 2008) Buchanan, J (March 27, 2008) - "An Interview with Bjarne Stroustrup" - Dr. Dobb's Journal - [Online] Available at: http://www.ddj.com/cpp/207000124 (Accessed November 17, 2008) Cohen, A. & Woodring, M. (1998) - "Win32 Multithreaded Programming" - O'Reilly - ISBN 1-56592-296-4 p.570 Cormen, T. et al. (2001) - "Introduction to Algorithms" - McGraw Hill - ISBN 978-0-26203293-3 p.253 Dijkstra E.W. (1974) - "On the role of scientific thought - Selected writings on Computing: A Personal Perspective" - Springer-Verlag New York, Inc. - ISBN ISBN 0-387-90652-5 Downey, A.B. (2008) - "The Little Book of Semaphores" - Green Tea Press - [Online] Available at: http://greenteapress.com/semaphores/downey08semaphores.pdf (Accessed December 17, 2008) Fleseriu G. & Masur, A. (February 10, 2004) - "Allocators (STL)" - CodeGuru - [Online] Available at: http://www.codeguru.com/Cpp/Cpp/cpp_mfc/stl/article.php/c4079/ (Accessed October 13, 2008) Frohlch, P.H. (2002) - "Inheritance Decomposed" - Department of Information and Computer Science University of California, Irvine - [Online] Available: http://www.cs.jyu.fi/~sakkinen/inhws/papers/Froehlich.pdf (Accessed December 10, 2008)

51

Gamma E. et al. (2002) - "Design Patterns - Elements of Reusable Object-Oriented Software" - Pearson Education - ISBN 81-7808-135-0 Gaztaaga , I. (March 28, 2008) - "Boost.Interprocess" boost C++ libraries - [Online] Available at: https://svn.zib.de/lenne3d/lib/boost/1.35.0/doc/html/interprocess.html (Accessed November 25, 2008) Gray, J.S. (1997) - "Interprocess Communications in Unix" - Prentice Hall - ISBN 0-13186891-8 p.193 Kath, R. (February 9, 1993) - "Managing Memory-Mapped Files in Win32" - Microsoft Developer Network Technology Group - [Online] Available at: http://msdn.microsoft.com/en-us/library/ms810613.aspx (Accessed October 7, 2008) Ketema, G. (April 1, 2003) - "Creating STL Containers in Shared Memory" - Dr.Dobb's [Online] Available at: http://www.ddj.com/cpp/184401639 (Accessed October 7, 2008) Meyers, S. (2001) - "Effective STL" - Addison Wesley - ISBN 0-201-74962-9 pp.48-58 Microsoft (November 6, 2008 [1]) - "MapViewOfFileEx Function" - Microsoft Corp. [Online] Available at: http://msdn.microsoft.com/en-us/library/aa366763(VS.85).aspx (Accessed November 25, 2008) Microsoft (2008 [2]) - "Debug Iterator Support" - Microsoft Corp. - [Online] Available at: http://msdn.microsoft.com/en-us/library/aa985982(VS.80).aspx (Accessed December 10, 2008) Microsoft (2008 [3]) - "Checked Iterators" - Microsoft Corp. - [Online] Available at: http://msdn.microsoft.com/en-us/library/aa985965(VS.80).aspx (Accessed December 10, 2008) Microsoft (2008 [4]) - "Introduction to Generics (C# Programming Guide" - Microsoft Corp. - [Online] Available at: http://msdn.microsoft.com/enus/library/0x6a29h6(VS.80).aspx (Accessed December 18, 2008) Musser, D.R. & Stepanov A.A. (1988) - "Generic programming" - Proceeds of the First International Conference of ISSAC-88 and AAECC-6 - [Online] Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.51.1780&rep=rep1&type=pdf (Accessed December 8, 2008) Musser, D.R. & Stepanov A.A. (1989) - "The Ada Generic Library: Linear List Processing Packages - Springer Compass International ISBN: 0387971335 Petzold, C. (1999) - "Programming Windows" - Microsoft Press - ISBN 1-57231-995-X Purdom, P.W. et al. (August 1, 1970) - "Statistical Investigation of Three Storage Allocation Algorithms" - Computer Science Dept. - The University of Wisconsin Madison [Online] Available at: http://www.cs.wisc.edu/techreports/1970/TR98.pdf (Accessed December 14, 2008) Ronell, M. (2003) - "A C++ Pooled, Shared Memory Allocator For The Standard Template Library" - Proceedings of the Fifth Real Time Linux Workshop - [Online] Available at: http://allocator.sourceforge.net/ (Accessed November 21, 2008) Ronell, M. (May 29, 2006) - "GCC Bugzilla Bug 21251 - Placement into shared memory" GNU - [Online] Available at: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21251 (Accessed November 21, 2008)

52

SGI (2006) - "Standard Template Library Programmer's Guide - Introduction to the Standard Template Library" - Silicon Graphics, Inc. - Hewlett-Packard Company - [Online] Available at: http://www.sgi.com/tech/stl/stl_introduction.html (Accessed September 25, 2008) Stepanov A. & Lee M. (1995) - "The standard template library" - Hewlett-Packard Company - [Online] Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.108.7776&rep=rep1&type=pdf (Accessed December 8, 2008) Stroustrup, B. (2000) - "The C++ Programming Language" - Addison Wesley - ISBN 0201-70073-5

53

Appendix A. SHARED CONTAINER LIBRARY PROGRAMMERS MANUAL (EXCERPTS)

The latest version of the programmers documentation for the shared container library is available online at: http://bergeon.francois.perso.neuf.fr/containers/doc.html.

A.1 Introduction to the Shared Containers Library The Shared Containers Library is a C++ library of container classes implemented in shared memory. Containers offered by the Shared Containers Library are derived from the containers offered in the Standard Template Library (STL). With the Shared Containers Library, programmers are able to instantiate STL containers and the objects they contain in a segment of shared memory accessible by concurrent processes. Programmers are responsible for using the provided synchronization mechanisms to avoid access and update conflicts between concurrent processes accessing a shared container. The Shared Containers Library was written by Franois Bergeon for his dissertation for a Master of Science in Information Technology (Software Engineering) at the University of Liverpool, UK (Martha McCormick Dissertation Advisor). Permission to use, copy, modify, and distribute this software and its documentation for any non-commercial purpose is hereby granted without fee, provided that the below copyright notice appears in all copies and that both the copyright notice and this permission notice appear in supporting documentation. Please contact the author at fbergeon@gmail.com to inquire about possible commercial use of this library. The author makes no representations about the suitability of this software for any purpose. It is provided "as is" without express or implied warranty. Copyright 2008 Franois Bergeon

54

A.2 Shared heap parameters

Description
Shared heap parameters need to be defined in the code in order to be present at link time. Default values are defined in the header heap_paramters.h that can be included in the source code or modified to suit.

char* _heapname
_heapname is an arbitrary name assigned to the shared heap. The default value is SHARED_CONTAINERS.

size_t _heapsize
_heapsize is the size of the shared heap in bytes. The default value is 10,485,760 bytes or 10 Mbytes.

size_t _chunksize
_chunksize is the size of each allocation chunk in bytes. The default value is 32 bytes.

void* _baseaddress
_baseaddress is the address in the process address space where the shared heap is to be mapped. The default value is 0x01100000, an arbitrary high offset in the process space.

Definition
These values are defined in the header heap_parameters.h.

55

A.3 sync Description

sync is a synchronization class designed to be used as-is or derived by classes that require synchronization. sync offers both exclusive (write) locks and non-exclusive (read) locks. Lock behavior follows the following matrix: Request\Existing lock read_lock write_lock upgrade_lock downgrade_lock is_locking_thread none success success n/a n/a false read lock success wait for release of all read locks wait for release of all other read locks n/a false write lock wait for release of write lock wait for release of write lock n/a success true if thread owns the lock

For A corresponding call to read_unlock is required for each call to read_lock. If the same thread calls write_lock more than once, the operation succeeds. A corresponding call to write_unlock is required for each successful call to write_lock. A call to upgrade_lock is functionally equivalent to a thread-safe call to read_unlock followed by a call to write_lock. Upgrade from a non-exclusive lock to an exclusive lock cannot be guaranteed. A call to downgrade_lock is functionally equivalent to a thread-safe call to write_unlock followed by a call to read_lock. Downgrade from an exclusive lock to non-exclusive lock is always guaranteed. A call to is_locking_thread returns true if the calling thread and process is the current owner of the exclusive lock. The timeout parameter is rounded down to the closest increment of 10ms. A value of less than 10 returns immediately. A value of -1 is equivalent to an infinite timeout. An infinite timeout should never be used in a call to upgrade due to the risk of a possible race condition.

Example
sync s; s.read_lock(); if (s.upgrade_lock()) s.write_unlock(); else s.read_unlock();

Definition
Defined in the header sync.h.

56

Members
Member bool read_lock(long timeout = -1) Description Obtain non-exclusive read lock within timeout milliseconds, 0 for immediate or -1 for infinite. Returns true if lock obtained. Release read lock

Void read_unlock()

bool write_lock(long timeout = -1) Obtain exclusive write lock within timeout milliseconds, 0 for immediate or -1 for infinite. Returns true if lock obtained. void write_unlock() bool upgrade_lock(long timeout = 0) Release write lock Attempt to upgrade an existing non-exclusive read lock to an exclusive write lock within timeout milliseconds or 0 for immediate. Returns true if the lock was successfully upgraded. If the upgrade fails, the non-exclusive read lock is not released. Downgrade from an existing exclusive write lock to a nonexclusive read lock.

void downgrade_lock()

57

A.4 shared_vector<T > Description

A shared_vector is a shared memory implementation of a STL vector container. shared_vector offers the same functionalities and properties than vector. Please refer to the STL documentation for information on the vector class. shared_vector also offers synchronization methods that should be used if other processes are susceptible to access it concurrently. shared_vector objects are be created, attached to and released from by calling the static methods of the SharedVectorFactory class.

Example
shared_vector<int> *V = SharedVectorFactory<int>::create(name); V->write_lock(); V->insert(V->begin(), 3); V->write_unlock(); V->read_lock(); assert(V->size() == 1 && V->capacity() >= 1 && *V[0] == 3); V->read_unlock();

Definition
Defined in the header shared_vector.h.

Template parameters
Parameter T Description The shared vector's value type: the type of object that is stored in the vector. Default

Type requirements

Same requirements than for vector.

Public base classes

sync

Members

All public members of the vector class and of the sync class are exposed.

58

A.5 shared_stack<T, Sequence > Description

A shared_stack is a shared memory implementation of a STL stack adapter. A shared_stack offers the same functionalities and properties than the stack. Please refer to the STL documentation for information on the stack class. shared_stack also offers synchronization methods that should be used before accessing the container if other processes are susceptible to access it concurrently. shared_stack objects are created, attached to and released from by calling the static methods of the SharedStackFactory class.

Example
int main() { shared_stack<int> *S = SharedStackFactory<int>::create(name); S->write_lock(); S->push(8); S->push(7); S->push(4); assert(S->size() == 3); assert(S->top() == 4); S->pop(); assert(S->top() == 7); S->pop(); assert(S->top() == 8); S->pop(); assert(S->empty()); S->write_unlock(); SharedStackFactory<int>::detach(name); }

Definition
Defined in the header shared_stack.h.

Template parameters
Parameter T Description The shared deque's value type: the type of object that is stored in the deque. shared_deque<T> Default

Sequence The type of the underlying container used to implement the stack.

Type requirements

59

Same requirements than for stack.

Public base classes

sync

Members

All public members of the stack class and of the sync class are exposed.

60

A.6 shared_map <Key, Data, Compare> Description

A shared_map is a shared memory implementation of a STL map container. A shared_map offers the same functionalities and properties than the map. Please refer to the STL documentation for information on the map class. shared_map also offers synchronization methods that should be used before accessing the container if other processes are susceptible to access it concurrently. shared_map objects are created, attached to and released from by calling the static methods of the SharedMapFactory class.

Example
struct ltstr { bool operator()(const char* s1, const char* s2) const { return strcmp(s1, s2) < 0; } }; int main() { shared_map<const char*, int, ltstr> *months = SharedMapFactory<const char*, int, ltstr>.create(my map); months->write_lock(); *months["january"] = 31; *months["february"] = 28; *months["march"] = 31; *months["april"] = 30; *months["may"] = 31; *months["june"] = 30; *months["july"] = 31; *months["august"] = 31; *months["september"] = 30; *months["october"] = 31; *months["november"] = 30; *months["december"] = 31; months->write_unlock(); months->read_lock(); cout << "june -> " << months["june"] << endl; shared_map<const char*, int, ltstr>::iterator cur = months.find("june"); shared_map<const char*, int, ltstr>::iterator prev = cur; shared_map<const char*, int, ltstr>::iterator next = cur; ++next; --prev; cout << "Previous (in alphabetical order) is " << (*prev).first << endl; cout << "Next (in alphabetical order) is " << (*next).first << endl; months->read_unlock(); }

Definition

61

Defined in the header shared_map.h.

Template parameters
Parameter Key Data Compare Description The map's key type and value type. This is also defined as shared _map::key_type The map's data type. This is also defined as shared_map::data_type. The key comparison function, a Strict Weak Ordering whose argument type is key_type; it returns true if its first argument is less than its second argument, and false otherwise. This is also defined as shared_map::key_compare less<Key> Default

Type requirements

Same requirements than for map.

Public base classes

sync

Members

All public members of the map class and of the sync class are exposed.

62

A.7 shared_hash_set <Key, Compare> Description

A shared_hash_set is a shared memory implementation of a STL hash_set container. A shared_hash_set offers the same functionalities and properties than the hash_set. Please refer to the STL documentation for information on the hash_set class. shared_hash_set also offers synchronization methods that should be used before accessing the container if other processes are susceptible to access it concurrently. shared_hash_set objects are created, attached to and released from by calling the static methods of the SharedHash_setFactory class.

Definition
Defined in the header shared_hash_set.h.

Template parameters
Parameter Key Description The set's key type and value type. This is also defined as shared _hash_set::key_type and shared_hash_set::value_type Default

Compare The key comparison function, a Strict Weak Ordering whose argument less<Key> type is key_type; it returns true if its first argument is less than its second argument, and false otherwise. This is also defined as shared_hash_set::key_compare and shared_hash_set::value_compare.

Type requirements

Same requirements than for hash_set.

Public base classes

sync

Members

All public members of the hash_set class and of the sync class are exposed.

63

A.8 SharedVectorFactory<T > Description

SharedVectorFactory is a static class used to create, attach to or detach from a shared_vector. Shared vectors are named entities to be identifiable by other processes.

Example
Process 1: shared_vector<int> *V = SharedVectorFactory<int>::create(name); Process 2: shared_vector<int> *V = SharedVectorFactory<int>::attach(name); ... SharedVectorFactory<int>::detach(name); Process 1: SharedVectorFactory<int>::detach(name);

Definition
Defined in the header shared_vector.h.

Template parameters
Parameter T Description The shared vector's value type: the type of object that is stored in the vector.

Type requirements

Same requirements than for vector.

Members
Member shared_vector* attach(char* name) shared_vector* create(char* name) shared_vector* create(char* name, shared_vector&) shared_vector* create(char* name, size_t n) shared_vector* create(char* name, size_t n, const T& t) void detach(char* name) Description Attach to an existing shared_vector named name Create new shared_vector named name Copy constructor Create new shared_vector named name with n elements Create new shared_vector named name with n copies of t Detach from shared_vector named name

64

A.9 SharedStackFactory<T , Sequence> Description

SharedStackFactory is a static class used to create, attach to or detach from a shared_stack. Shared stacks are named entities to be identifiable by other processes.

Example
Process 1: shared_stack<int> *S = SharedStackFactory<int>::create(name); Process 2: shared_stack<int> *S = SharedStackFactory<int>::attach(name); ... SharedStackFactory<int>::detach(name); Process 1: SharedStackFactory<int>::detach(name);

Definition
Defined in the header shared_stack.h.

Template parameters
Parameter T Description The shared stack's value type: the type of object that is stored in the stack. shared_deque<T> Default

Sequence The type of the underlying container used to implement the stack.

Type requirements

Same requirements than for stack.

Members
Member shared_stack* attach(char* name) shared_stack* create(char* name) shared_stack* create(char* name, shared_stack&) shared_stack* create(char* name, Description Attach to an existing shared_stack named name Create new shared_stack named name Copy constructor Create new shared_stack named

65

size_t n) shared_stack* create(char* name, size_t n, const T& t) void detach(char* name)

name with n elements Create new shared_stack named name with n copies of t Detach from shared_stack named name

66

A.10 SharedMapFactory map<Key, Data, Compare> Description

SharedMapFactory is a static class used to create, attach to or detach from a shared_map. Shared maps are named entities to be identifiable by other processes.

Example
Process 1: shared_map<std::string, int> *S = SharedMapFactory<std::string, int>::create(name); Process 2: shared_map<std::string, int> *S = SharedMapFactory<std::string, int>::attach(name); ... SharedMapFactory<std::string, int>::detach(name); Process 1: SharedMapFactory<std::string, int>::detach(name);

Definition
Defined in the header shared_map.h.

Template parameters
Parameter Key Data Description The map's key type and value type. This is also defined as shared _map::key_type and shared_map::value_type The map's data type. This is also defined as shared_map::data_type. Default

Compare The key comparison function, a Strict Weak Ordering whose argument less<Key> type is key_type; it returns true if its first argument is less than its second argument, and false otherwise. This is also defined as shared_map::key_compare and shared_map::value_compare.

Type requirements

Same requirements than for map.

Members
Member shared_map* attach(char* name) shared_map* create(char* name) Description Attach to an existing shared_map named name Create new shared_map named name

67

shared_map* create(char* name, const Creates an empty shared_map named key_compare& comp) name, using comp as the key_compare object. template <class InputIterator> shared_map* create(char* name, InputIterator f, InputIterator l) template <class InputIterator> shared_map* create(char* name, InputIterator f, InputIterator l, const key_compare& comp) shared_map* create(char* name, shared_map&) void detach(char* name) Creates a shared_map named name with a copy of a range. Creates a shared_map named name with a copy of a range, using comp as the key_compare object. Copy constructor Detach from shared_map named name

68

A.11 SharedHashSetFactory set<Key, Compare> Description

SharedHashSetFactory is a static class used to create, attach to or detach from a shared_hash_set. Shared hash sets are named entities to be identifiable by other processes.

Example
Process 1: shared_hash_set<int> *S = SharedHashSetFactory<int>::create(name); Process 2: shared_hash_set<int> *S = SharedHashSetFactory<int>::attach(name); ... SharedHashSetFactory<int>::detach(name); Process 1: SharedHashSetFactory<int>::detach(name);

Definition
Defined in the header shared_hash_set.h.

Template parameters
Parameter Key Description The set's key type and value type. This is also defined as shared _hash_set::key_type and shared_hash_set::value_type Default

Compare The key comparison function, a Strict Weak Ordering whose argument less<Key> type is key_type; it returns true if its first argument is less than its second argument, and false otherwise. This is also defined as shared_hash_set::key_compare and shared_hash_set::value_compare.

Type requirements

Same requirements than for hash_set.

Members
Member shared_hash_set* attach(char* name) shared_hash_set* create(char* name) shared_hash_set* create(char* name, const key_compare& comp) Description Attach to an existing shared_hash_set named name Create new shared_hash_set named name Creates an empty shared_hash_set named name, using comp as the

69

key_compare object. shared_hash_set* create(char* name, size_t n) Create new shared_ash_set named name with at least n buckets

shared_hash_set* create(char* Creates an empty shared_hash_set name, size_t n, const key_compare& named name, with at least n buckets, using comp) comp as the key_compare object. template <class InputIterator> shared_hash_set* create(char* name, InputIterator f, InputIterator l) template <class InputIterator> shared_hash_set* create(char* name, InputIterator f, InputIterator l, size_t n) template <class InputIterator> shared_hash_set* create(char* name, InputIterator f, InputIterator l, size_t n, const key_compare& comp) shared_hash_set* create(char* name, shared_hash_set&) void detach(char* name) Creates a shared_hash_set named name with a copy of a range.

Creates a shared_hash_set named name with a copy of a range and a bucket count of at least n. Creates a shared_hash_set named name with a copy of a range and a bucket count of at least n, using comp as the key_compare object. Copy constructor Detach from shared_hash_set named name

70

Appendix B. SHARED CONTAINERS LIBRARY SOURCE CODE (EXCERPTS)

The latest version of the source code for the shared container library is available online at: http://bergeon.francois.perso.neuf.fr/containers/src.zip

B.1 safe_counter.h
/* * safe_counter.h - Thread-safe counters * * --(c) Francois Bergeon 2008 --* ---University of Liverpool ---* * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include <windows.h> /* * Wrapper class around an interlocked counter * - Class is platform dependent. */ class safe_counter { private: // Counter is volatile and needs to be aligned // on a 32bit boundary for interlocked operations _declspec(align(32)) volatile LONG _counter; public: // Constructor - no need for interlocked access at this time inline safe_counter(long n = 0) : _counter(n) {}; // Assignment operator inline safe_counter& operator=(long n) { InterlockedExchange(&_counter, n); return *this; }; // Cast operator to const long inline operator const long() { return _counter; }; // Increment and decrement operators inline long operator++() { return InterlockedIncrement(&_counter); }; inline long operator--() { return InterlockedDecrement(&_counter); }; }; Listing 4: safe_counter.h

71

B.2 sync.h
/* * sync.h - Simple read/write locking mechanism * * --(c) Francois Bergeon 2008 --* ---University of Liverpool ---* * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include "safe_counter.h" /* * sync class - implements a synchronization for shared containers * - Class is platform dependent. */ class sync { private: // Internal read/write sempahores safe_counter _readlock; safe_counter _writelock; long _lockingthreadid; long _lockingprocessid; // Standard loop delay is 10ms static const unsigned DELAY = 10;

72

public: // Obtain non-exclusive read lock with timeout // - More than one read lock is allowed at any single time // timeout - time out in milliseconds, -1 for infinite (default), 0 for no wait // bool read_lock(long timeout = -1); // Release non-exclusive read lock void read_unlock(); // Obtain exclusive write lock with timeout // - No read locks and only one write lock // is allowed at any single time // timeout - time out in milliseconds, -1 for infinite (default), 0 for no wait // inline bool write_lock(long timeout = -1) { return write_lock(timeout, 0); }; // Check if the current thread owns the write lock bool is_locking_thread(); // Release exclusive write lock void write_unlock(); // Upgrade from a non-exclusive read lock to an // exclusive write lock // - Thread must already own a read lock // - If another thread is already waiting for a wite lock it will be preempted // timeout - time out in milliseconds, 0 for no wait (default) // bool upgrade_lock(long timeout = 0); // Downgrade from an exclusive write lock to a non-exclusive read lock // - Thread must already own a write lock // - Read lock is guaranteed // void downgrade_lock(); private:

73

// Internal write lock with acceptable number of existing locks bool write_lock(long timeout, long existing); // Get thread and process id inline long get_threadid() { return GetCurrentThreadId(); } inline long get_processid() { return GetCurrentProcessId(); } };

Listing 5: sync.h

B.3 sync.cpp
/* * sync.cpp - Simple read/write locking mechanism * * --(c) Francois Bergeon 2008 --* ---University of Liverpool ---* * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #include "sync.h" // Obtain non-exclusive read lock with timeout // - More than one read lock is allowed at any single time // timeout - time out in milliseconds, 0 for infinite, 1 for no delay //

74

bool sync::read_lock(long timeout) { // Obtain maximum number of iterations to run long n = (timeout / DELAY) + 1; for (;;) { // Check that no write lock has been requested if (_writelock == 0) { // Obtain read lock ++_readlock; // Has no write lock been requested in the meantime? if (_writelock == 0) return true; // Release read lock and try again --_readlock; } // Timeout expired? if (timeout >= 0 && --n == 0) return false; // Wait Sleep(DELAY); } };

// Release non-exclusive read lock void sync::read_unlock() { // Safety if (--_readlock == -1) ++_readlock; }

75

// Obtain exclusive write lock with timeout // - No read locks and only one write lock is allowed at any single time // timeout - time out in milliseconds, 0 for infinite, 1 for no delay // existing - acceptable number of existing locks (0:normal, 1:upgrade) // bool sync::write_lock(long timeout, long existing) { // Has this thread already obtained an exclusive write lock? if (is_locking_thread()) { // Increment lock counter so that the next call to // write_unlock() does not release the lock ++_writelock; return true; } // Obtain maximum number of iterations to run long n = (timeout / DELAY) + 1; for (;;) { // Try to obtain an exclusive write lock // or preempt other threads if we are upgrading // from a non-exclusive one while (++_writelock > 1 && existing == 0) { --_writelock; // Timeout expired? if (timeout >= 0 && --n == 0) return false; // Wait Sleep(DELAY); } // Then wait for all readers to release their non-exlusive read locks while (_readlock > existing)

76

{ // Timeout expired? if (timeout >= 0 && --n == 0) { // Release write lock --_writelock; return false; } // Wait Sleep(DELAY); } // Write lock obtained, store locking thread id _lockingthreadid = get_threadid(); _lockingprocessid = get_processid(); return true; } }; // Check if the current thread owns the write lock bool sync::is_locking_thread() { return (_writelock > 0 && _lockingthreadid == get_threadid() && _lockingprocessid == get_processid()); } // Release exclusive write lock void sync::write_unlock() { long n = --_writelock; // Safety if (n == -1) ++_writelock; // Reset thread and process id if (n == 0) _lockingthreadid = _lockingprocessid = -1; }

77

// Upgrade from non-exclusive read lock to exclusive write lock // - Thread must already own a read lock // - If another thread is already waiting for a wite lock it will be preempted // - An infinite timeout may lead to a running condition // timeout - time out in milliseconds, 0 for no wait // bool sync::upgrade_lock(long timeout) { // Obtain a write lock with one existing read lock (ours) if (!write_lock(timeout, 1)) return false; // Release our read lock read_unlock(); // Lock upgraded return true; } // Downgrade from an exclusive write lock to a non-exclusive read lock // - Thread must already own a write lock // - Read lock is guaranteed // void sync::downgrade_lock() { ++_readlock; write_unlock(); } Listing 6: sync.cpp

78

B.4 heap_parameters.h
/* * heap_parameters.h - Parameters for shared heap * * --(c) Francois Bergeon 2008 --* ---University of Liverpool ---* * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #include <stddef.h> // These variables can be modified to suit specific needs char* _heapname = "SHARED_CONTAINERS"; // Arbitrary heap name size_t _heapsize = 67108864; // 64Mb size_t _chunksize = 64; // 64 bytes void* _baseaddress = (void*)0x01100000; // Arbitrary high for debug Listing 7: heap_parameters.h

B.5 shared_heap.h
/* * shared_heap.h - Heap implementation in shared memory * * --(c) Francois Bergeon 2008 --* ---University of Liverpool ----

79

* * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #ifdef WIN32 #include <windows.h> #endif /* * shared_heap class - implements a heap in shared memory * * This class is platform dependent. * */ class shared_heap { private: class shared_header; // Forward class declaration // Class members shared_header* _header; void* _heap; long _lasterror; #ifdef WIN32 HANDLE _mapping; #endif

// Pointer to shared header // Address of shared heap // Last error caught // Handle of Win32 mapping object

// Standard loop delay is 20ms static const unsigned DELAY = 20;

80

public: // Public methods shared_heap(const char* heapname, size_t heapsize = 0x100000, size_t chunksize = 0x1000, void* baseaddress = NULL); ~shared_heap(void); void* allocate(size_t size); void deallocate(void* head, size_t size); void get_params(long** longparam, void*** ptrparam); #ifdef _DEBUG void dump(); #endif private: // Private methods void cleanup(); void lock(); void unlock(); }; Listing 8: shared_heap.h

B.6 shared_heap.cpp
/* * shared_heap.cpp - Heap implementation in shared memory * * --(c) Francois Bergeon 2008 --* ---University of Liverpool ---* * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a

81

* Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * * This class implements a heap of segregated storage in shared memory. * It uses WIN32 primitives and is platform dependent. * */ #include "shared_heap.h" #include "safe_counter.h" #ifndef BYTE #define BYTE unsigned char #endif // The signature is used to confirm that an existing memory // mapping has been created by this same code #define SHAREDHEAP_SIGNATURE "**FBSH**" // Indicator of last chunk in chunk list #define LAST_CHUNK ((LPVOID)~0) // Macro to roundup value #define ROUNDUP(a,b) ((a)%(b)==0?(a):((((a)/(b))+1)*(b))) // // The shared header is located at the beginning of the shared memory mapped segment // All processes accessing shared containers rely on the shared header to map // shared memory to the same virtualized memory space and to access existing containers // __declspec(align(32)) volatile class shared_heap::shared_header { public: char _signature[sizeof(SHAREDHEAP_SIGNATURE)]; // Signature that identifies view as a shared heap void* _baseaddress; // Common base address to be used by all processes void* _head; // Pointer to first available chunk size_t _heapsize, _chunksize; // Size of heap and chunk size

82

long _longparam; void* _ptrparam; safe_counter _numprocesses; safe_counter _allocating; char _filename[MAX_PATH]; };

// // // // //

Client long parameter Client pointer parameter Number of processes currently accessing shared memory 1 if allocation in progress, 0 otherwise Temp filename

// // Shared heap constructor // // heapname - name of shared heap // heapsize - size of shared heap in bytes (defaults to 0x100000 = 1Mb) // chnksize - size of segregated storage chunks (defauts to 0x1000 = 4Kb) // baseaddress - address of shared mempory mapping of NULL if default // shared_heap::shared_heap(const char* heapname, size_t heapsize, size_t chunksize, void* baseaddress) { // Flag for creation of new shared heap BOOL bCreateNew = FALSE; // Size of shared header is rounded up to proper allocation granularity SYSTEM_INFO SystemInfo; GetSystemInfo(&SystemInfo); DWORD dwHeaderSize = ROUNDUP(sizeof(shared_header), SystemInfo.dwAllocationGranularity); // Buffer for temp filename and wide char heap name TCHAR szTmpFile[MAX_PATH]; TCHAR szHeapName[MAX_PATH]; // Convert heap name to wide char for Win32 API calls MultiByteToWideChar(CP_ACP, 0, heapname, -1, szHeapName, MAX_PATH); // Initialize members _lasterror = 0; _mapping = NULL; _header = NULL; _heap = NULL;

83

try { // Open file mapping _mapping = OpenFileMapping(FILE_MAP_WRITE, // RW access FALSE, // Handle cannot be inherited szHeapName); // Name of file mapping object // Obtain last system error _lasterror = GetLastError(); // Unable to open file mapping, attempt to create temp file if (_mapping == NULL && _lasterror == ERROR_FILE_NOT_FOUND) { // Create file mapping bCreateNew = TRUE; // Check requested heap size if (heapsize == NULL) { SetLastError(ERROR_BAD_LENGTH); throw; } // Create temporary file for mapping TCHAR szTmpPath[MAX_PATH-14]; // Get temp dir & file GetTempPath(MAX_PATH-14, szTmpPath); GetTempFileName(szTmpPath, szHeapName, 0, szTmpFile); // Create temp file HANDLE hFile = CreateFile(szTmpFile, GENERIC_WRITE | GENERIC_READ, FILE_SHARE_WRITE, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_TEMPORARY, NULL); // Obtain last system error _lasterror = GetLastError();

// // // // // // //

Temp file name RW Access Share mode Optional security attributes Create file Temporary file Flags & attributes

84

// Exit if we could not create temp file if (hFile == INVALID_HANDLE_VALUE) throw; // Add header size to desired size __int64 qwMaxSize = heapsize + dwHeaderSize; DWORD dwSizeHigh = (qwMaxSize & 0xFFFFFFFF00000000) >> 32; DWORD dwSizeLow = (qwMaxSize & 0x00000000FFFFFFFF); // Create new file mapping object _mapping = CreateFileMapping(hFile, NULL, PAGE_READWRITE, dwSizeHigh, dwSizeLow, szHeapName); // Obtain last system error _lasterror = GetLastError(); // Close underlying temp file CloseHandle(hFile); } // Exit function if OpenFileMapping or CreateFileMapping failed if (_mapping == NULL) throw; // Map view of file to obtain/create header _header = (shared_header *)MapViewOfFileEx(_mapping, // File mapping object FILE_MAP_WRITE, // RW access 0, 0, // Offset 0 dwHeaderSize, // Header size NULL); // Any base address // Unable to map view of file: cleanup & exit if (_header == NULL) { _lasterror = GetLastError(); throw;

// Hi dword // Low dword

// // // // // //

Handle to temp file Optional security attributes Page protection Hi-order DWORD of size Low-order DWORD of size Name of file mapping object

85

} if (bCreateNew) { // Map view of file to target base address _heap = MapViewOfFileEx(_mapping, // FILE_MAP_WRITE, // 0, dwHeaderSize, // 0, // baseaddress); //

File mapping object RW access Offset (must be aligned to allocation granularity) Full size Target base address or NULL

// Save last error if failed _lasterror = GetLastError(); if (_heap == NULL) throw; // Populate header _header->_baseaddress = _heap; // Now common base address _header->_head = LAST_CHUNK; // Storage not initialized _header->_heapsize = heapsize; // Heap size _header->_chunksize = max(chunksize, sizeof(void**)); // Chunk size _header->_longparam = 0; // Set client DWORD parameter _header->_ptrparam = NULL; // Set client LPVOID parameter _header->_numprocesses = 1; // Currently 1 process _header->_allocating = 0; // No allocation currently in progress WideCharToMultiByte(CP_ACP, 0, szTmpFile, -1, _header->_filename, MAX_PATH, NULL, NULL); // Convert & store temp filename // Initialize segregated storage deallocate(_heap, heapsize); // Insert signature last to prevent race conditions with other processes strcpy_s(_header->_signature, sizeof(SHAREDHEAP_SIGNATURE), SHAREDHEAP_SIGNATURE); // Signature } else { // Check signature or fail if (strcmp(_header->_signature, SHAREDHEAP_SIGNATURE) != 0) { _lasterror = ERROR_BAD_FORMAT; throw;

86

} // Increment number of client processes if (++_header->_numprocesses == 1) { // A process is currently closing the shared memory _lasterror = ERROR_FILE_NOT_FOUND; throw; } // Map view of file to target base address _heap = MapViewOfFileEx(_mapping, // File mapping object FILE_MAP_WRITE, // RW access 0, dwHeaderSize, // Offset (must be aligned to allocation granularity) 0, // Full size _header->_baseaddress); // Common base address // Save last error if failed _lasterror = GetLastError(); if (_heap == NULL) throw; } } catch (...) { // Cleanup cleanup(); // Re-set error code SetLastError(_lasterror); } } // Standard destructor shared_heap::~shared_heap(void) { cleanup(); } // // Cleanup is called by destructor or

87

// if an exception is caught in the constructor // void shared_heap::cleanup() { // Buffer for temp filename TCHAR szTmpFile[MAX_PATH]; // Initialize as an empty string szTmpFile[0] = '\0'; // Unmap heap if (_heap != NULL) UnmapViewOfFile(_heap); // Unmap header if (_header != NULL) { // Are we the last process to use this shared memory file? if (--_header->_numprocesses == 0) { // If so, convert temp filename so we can delete it MultiByteToWideChar(CP_ACP, 0, _header->_filename, -1, szTmpFile, MAX_PATH); } // Unmap header UnmapViewOfFile((LPVOID)_header); } // Close file mapping if (_mapping != NULL) CloseHandle(_mapping); // Reset member variables _heap = NULL; _header = NULL; _mapping = NULL; // Delete temp file if we are the last user process

88

if (szTmpFile[0] != '\0') DeleteFile(szTmpFile); } // // Allocate memory // // size - number of bytes to allocate // void* shared_heap::allocate(size_t size) { // Synchronization lock(); // Out of memory? if (_header->_head == LAST_CHUNK) { unlock(); return NULL; } // Roundup allocation size to a multiple of chunk size size = ROUNDUP(size, _header->_chunksize); // Special case: allocation of a single chunk if (size == _header->_chunksize) { // Obtain head chunk void** p = (void**)_header->_head; // Move head to point to next chunk _header->_head = *p; // Return former head chunk unlock(); return (void*)p; } // Number of contiguous chunks to allocate if > 1

89

unsigned chunks = size / _header->_chunksize; // Start at head chunk and look for contiguous block of chunks void* head = _header->_head; // Start of contiguous block void** previous = &_header->_head; // Pointer to previous chunk // Loop do { // Loop variables void** p; // Index pointer to chunks unsigned n; // Counter for contiguous chunks // Loop through contiguous chunks for (p = (void**)head, n = 1; *p == ((BYTE*)p + _header->_chunksize); p = (void**)*p, ++n) { // Return if we found enough contiguous chunks if (n == chunks) { // Set chunk before block to point to next chunk *previous = *p; // Return start of chunk block unlock(); return head; } } // Continue searching from next chunk forward previous = p; head = *p; // Stop when we reach the end of the heap } while (head != LAST_CHUNK); // No contiguous chunks found, allocation failed unlock(); return NULL; } // // Deallocate buffer - can also be used to initialize segregated

90

// storage for the entire heap // // head - pointer to buffer // size - buffer size // void shared_heap::deallocate(void* head, size_t size) { // Synchronization lock(); // Roundup size to an integer number of chunks size = ROUNDUP(size, _header->_chunksize); // Calculate address of last chunk to be deallocated void** toe = (void**)((BYTE*)head + size - _header->_chunksize); // Loop variables void** previous = NULL; void* p;

// Address of previous chunk // Index pointer to chunks

// Go through all chunks from head to toe for (p = head; p <= toe; p = ((BYTE*)p + _header->_chunksize)) { // Update previous chunk so it points to current chunk if (previous != NULL) *previous = p; // Store address of new previous chunk previous = (void**)p; } // Make last chunk point to previous head *toe = _header->_head; // Make head point to first chunk _header->_head = head; unlock(); } //

91

// Obtain client parameters from shared header // // longparam - pointer to long parameter // ptrparam - pointer to pointer parameter // void shared_heap::get_params(long** longparam, void*** ptrparam) { *longparam = &_header->_longparam; *ptrparam = &_header->_ptrparam; } // Lock shared heap. Wait if busy. No timeout void shared_heap::lock() { if (_header != NULL) { while (++_header->_allocating > 1) { if (--_header->_allocating == 0) continue; Sleep(DELAY); } } } // Unlock shared heap void shared_heap::unlock() { if (_header != NULL) --_header->_allocating; } Listing 9: shared_heap.cpp

92

B.7 shared_pool.h
/* * shared_pool.h - Object allocation in shared memory * * --(c) Francois Bergeon 2008 --* ---University of Liverpool ---* * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include #include #include #include #include <map> <string> "shared_heap.h" "shared_allocator.h" "sync.h"

/* * shared_map class * - derives from the map and sync classes * - implements the map class with the shared_allocator template parameter */ template<class Key, class Data, class Compare=std::less<Key>> class shared_map : public sync, public std::map<Key, Data, Compare, shared_allocator<std::pair<const Key, Data>>> {}; /* * Shared pool class - Provides singleton allocation interface * to shared heap and manages map of shared objects * */ class shared_pool

93

{ private: // shared_object class describes each shared objects class shared_object; // Map of objects stored in shared memory typedef shared_map<std::string, shared_object*> container_map; // Shared heap and map of shared objects static shared_heap* _heap; static container_map* _containermap; public: // Allocator interface to shared heap static void* allocate(size_t size); static void deallocate(void* buffer, size_t size); static size_t max_size(); // Insert, retrieve and release objects in shared map static bool insert(char *name, void* object, size_t size); static void* retrieve(char *name, size_t size); static void* release(char *name); private: // Lazy initialization static void init(); }; Listing 10: shared_pool.h

B.8 shared_pool.cpp
/* * shared_pool.cpp - Object allocation in shared memory *

94

* * * * * * * * * * */

------

(c) Francois Bergeon 2008 University of Liverpool

------

IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY A dissertation for the completion of a Master of Science in IT - Software Engineering Martha McCormick Dissertation Advisor

// Force Microsoft VC++ 6 to turn off iterator checking #ifdef WIN32 #define _HAS_ITERATOR_DEBUGGING 0 #define _SECURE_SCL 0 #endif #include "shared_pool.h" // External parameters used to construct a shared heap // These variables can be modified to suit specific needs extern char* _heapname; // Arbitrary heap name extern size_t _heapsize; // 10Mb extern size_t _chunksize; // 32 bytes extern void* _baseaddress; // Arbitrary high for debug

// Class for shared object stored in container map class shared_pool::shared_object { public: void* _object; // Pointer to object size_t _size; // Size of object safe_counter _refcount; // Usage count shared_object(void* object, size_t size): _object(object), _size(size), _refcount(1) {};

95

shared_object() {}; ~shared_object() {}; };

// Initialize static members shared_heap* shared_pool::_heap = NULL; shared_pool::container_map* shared_pool::_containermap = NULL; // // Allocate buffer from shared heap // // size - number of bytes to allocate // void* shared_pool::allocate(size_t size) { // Perform lazy intialization if shared heap is not initialized if (_heap == NULL) init(); // Allocate memory and return return _heap->allocate(size); } // // Deallocate buffer // // head - pointer to buffer // size - buffer size // void shared_pool::deallocate(void* buffer, size_t size) { // Shared heap must be initialized if (_heap == NULL) return; // Deallocate buffer _heap->deallocate(buffer, size);

96

}; // // Return heap size // size_t shared_pool::max_size() { return _heapsize; } // // Insert shared object in container map // // name - name of object // object - pointer to object // size - size of object // bool shared_pool::insert(char *name, void* object, size_t size) { // Perform lazy intialization if shared heap is not initialized if (_heap == NULL) init(); // Look for entry - obtain non-exclusive lock first // Return failure if a container with the same name is already present in the shared container map std::string s(name); _containermap->read_lock(); if (_containermap->find(s) != _containermap->end()) { // Release read lock and return failure _containermap->read_unlock(); return false; } // Allocate and build new shared object void* p = allocate(sizeof(shared_object)); if (p == NULL)

97

{ // Release read lock and return failure _containermap->read_unlock(); return false; } shared_object* so = new(p) shared_object(object, size); // Upgrade to an exclusive lock if (!_containermap->upgrade_lock()) { // If upgrade failed we have to do it manually // First release the read lock - this may give another process a write lock _containermap->read_unlock(); // Then obtain a write lock the old fashioned way _containermap->write_lock(); // Check for name again in case it was inserted concurrently if (_containermap->find(s) != _containermap->end()) { // Release write lock and deallocate object, return failure _containermap->write_unlock(); deallocate(p, sizeof(shared_object)); return false; } } // Insert shared object and release lock (*_containermap)[s] = so; _containermap->write_unlock(); // Return success return true; }

// // Retrieve shared object from container map //

98

// name - name of object // size - size of object // void* shared_pool::retrieve(char *name, size_t size) { // Perform lazy intialization if shared heap is not initialized if (_heap == NULL) init(); // Look for entry - obtain non-exclusive lock _containermap->read_lock(); std::string s(name); container_map::iterator it = _containermap->find(s); // Return NULL if no container with that name is present in the shared container map if (it == _containermap->end()) { // Release read lock and return failure _containermap->read_unlock(); return NULL; } // Obtain pointer to shared object and release read lock shared_object* so = it->second; _containermap->read_unlock(); // Check shared object and return NULL if size does not match if (so == NULL || so->_size != size) return NULL; // Increment refcount and return pointer to object ++so->_refcount; return so->_object; } // // Release object by decrementing its refcount. If refcount reaches zero,

99

// return pointer to object so its destructor can be called. // // name - name of object // void* shared_pool::release(char *name) { // Shared heap must be initialized if (_heap == NULL) return NULL; // Look for entry - obtain non-exclusive lock first std::string s(name); _containermap->read_lock(); container_map::iterator it = _containermap->find(s); // Return NULL if no container with that name is present in the shared container map if (it == _containermap->end()) { // Release read lock and return failure _containermap->read_unlock(); return NULL; } // Obtain pointer to shared object and decrement refcount shared_object* so = it->second; if (so == NULL || --so->_refcount > 0) { // Release read lock and return NULL if object still in use _containermap->read_unlock(); return NULL; } // // refcount == 0 - discard object // // Retain pointer to object

100

void* p = so->_object; // Upgrade to an exclusive lock if (!_containermap->upgrade_lock()) { // If upgrade failed we have to do it manually // First release the read lock - this may give another process a write lock _containermap->read_unlock(); // Then obtain a write lock the old fashioned way _containermap->write_lock(); // Get shared object again in case it was removed or attached concurrently if (_containermap->find(s) == _containermap->end() || so->_refcount > 0) { // Return NULL if object not found or still in use _containermap->write_unlock(); return NULL; } } // Erase map entry and release lock _containermap->erase(it); _containermap->write_unlock(); // Deallocate object deallocate(so, sizeof(shared_object)); // Return pointer to object for destruction return p; }

// // Lazy initialization of shared heap // void shared_pool::init() { // Pointers to shared client parameters

101

long* longparam; void** ptrparam; // Build new shared heap from external parameters _heap = new shared_heap(_heapname, _heapsize, _chunksize, _baseaddress); // Obtain pointers to client parameters in shared header _heap->get_params(&longparam, &ptrparam); // Create container map if not initialized if (*ptrparam == NULL) { _containermap = new(_heap->allocate(sizeof(container_map))) container_map; // Store pointer to map in pParam *ptrparam = _containermap; } else { // Or attach to existing container map _containermap = (container_map*)*ptrparam; } } Listing 11: shared_pool.cpp

B.9 shared_allocator.h
/* * shared_allocator.h - STL allocator in shared memory * * --(c) Francois Bergeon 2008 --* ---University of Liverpool ---* * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY

102

* * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include "shared_pool.h" class shared_pool; template<class T> class shared_allocator { public: typedef T value_type; typedef unsigned int size_type; typedef ptrdiff_t difference_type; typedef T* pointer; typedef const T* const_pointer; typedef T& reference; typedef const T& const_reference; pointer address(reference r) const { return &r; } const_pointer address(const_reference r) const {return &r;} shared_allocator() throw() {}; template<class U> shared_allocator(const shared_allocator<U>& t) throw() {}; ~shared_allocator() throw() {}; // space for n Ts pointer allocate(size_t n, const void* hint=0) { return(static_cast<pointer> (shared_pool::allocate(n*sizeof(T)))); }

103

// deallocate n Ts, don't destroy void deallocate(pointer p, size_type n) { shared_pool::deallocate((LPVOID)p, n*sizeof(T)); return; } // initialize *p by val void construct(pointer p, const T& val) { new(p) T(val); } // destroy *p but don't deallocate void destroy(pointer p) { p->~T(); } size_type max_size() const throw() { return shared_pool::max_size(); } template<class U> struct rebind { typedef shared_allocator<U> other; }; }; template<class T> bool operator ==(const shared_allocator<T>& a, const shared_allocator<T>& b) throw() { return(TRUE); } template<class T> bool operator !=(const shared_allocator<T>& a, const shared_allocator<T>& b) throw() { return(FALSE); } Listing 12: shared_allocator.h

104

B.10 container_factories.h
/* * containers_factories.h - STL containers in shared memory * * --(c) Francois Bergeon 2008 --* ---University of Liverpool ---* * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include "shared_pool.h" // Placement new in the shared pool #define SHARED_NEW new(shared_pool::allocate(sizeof(container_type))) /* * SharedContainerFactory - parameterized abstract factory class * - Used to create new shared containers or attach to existing ones. * - Methods of this class are static and call static methods of shared_pool. * - Class is meant to be derived into specific shared container factory classes. */ template<class C> class _SharedContainerFactory { public: typedef C container_type; // Container's type - this type is available to all derived classes // Create new shared container and assign specified name // name - name of shared container // static container_type* create(char* name)

105

{ // Allocate new container in shared memory container_type* c = SHARED_NEW container_type(); // Add container to shared pool and return return insert_container(name, c); }; // Create new shared container with copy constructor // name - name of shared container // cont - container to copy from // static container_type* create(char* name, const container_type& cont) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type(cont); // Add container to shared pool and return return insert_container(name, c); }; // Attach to existing shared container by name // name - name of shared container // static container_type *attach(char* name) { // Retrieve container from shared pool return (container_type*)shared_pool::retrieve(name, sizeof(container_type)); }; // Detach from shared container - release if last user // name - name of shared container // static void detach(char* name) { // Detach from container in sahred pool container_type* c = (container_type*)shared_pool::release(name); // shared_pool returns pointer to the container if we are the last user, NULL otherwise if (c != NULL)

106

destroy_container(c); } protected: // Attempt to insert container in shared pool // name - name of shared container // c - pointer to shared container // static container_type* insert_container(char* name, container_type *c) { // Add container to shared pool - destroy it and return NULL if insertion failed if (!shared_pool::insert(name, c, sizeof(container_type))) { destroy_container(c); return NULL; } // Returned newly created container return c; } // Destroy container and deallocate // c - pointer to shared container // static void destroy_container(container_type* c) { // Call destructor on shared container c->~container_type(); // Deallocate memory used by shared container object shared_pool::deallocate(c, sizeof(container_type)); }; }; /* * SharedSequenceContainerFactory - parameterized abstract factory class * - Specialization of the SharedContainerFactory class for sequence containers. * - Implements the pre-allocation and replication constructors offered * by sequence containers.

107

*/ template<class C> class _SharedSequenceContainerFactory : public _SharedContainerFactory<C> { public: typedef typename container_type::value_type value_type; typedef typename container_type::size_type size_type;

// Container's value type // Container's size_type

// Redefinition of base class static create methods static container_type* create(char* name) { return _SharedContainerFactory<container_type>::create(name); }; static container_type* create(char* name, const container_type& c) { return _SharedContainerFactory<container_type>::create(name, c); }; // Create new shared container with pre-allocation // name - name of shared container // n - number of objects to pre-allocate // static container_type* create(char* name, size_type n) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type(n); // Add container to shared pool and return return insert_container(name, c); }; // Create new shared container with replication // name - name of shared container // n - number of objects to replicate // t - object to replicate // static container_type* create(char* name, size_type n, value_type& t) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type(n, t); // Add container to shared pool and return return insert_container(name, c);

108

}; }; /* * SharedAssociativeContainerFactory - parameterized abstract factory class * - Specialization of the SharedContainerFactory class for associative containers. * - Implements constructors with copy of range and specified key_compare function * offered by associative containers. */ template<class C> class _SharedAssociativeContainerFactory : public _SharedContainerFactory<C> { public: typedef typename container_type::key_compare key_compare; // Container's key_compare function // Redefinition of base class static create methods static container_type* create(char* name) { return _SharedContainerFactory<container_type>::create(name); }; static container_type* create(char* name, container_type& c) { return _SharedContainerFactory<container_type>::create(name, c); }; // Create new shared container with key_compare object // name - name of shared container // comp - key_compare object // static container_type* create(char* name, const key_compare& comp) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type(comp); // Add container to shared pool and return return insert_container(name, c); } // Create new shared container with copy of range // name - name of shared container // f - first iterator // l - last iterator

109

// template<class InputIterator> static container_type* create(char* name, InputIterator f, InputIterator l) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type<InputIterator>(f, l); // Add container to shared pool and return return insert_container(name, c); }; // Create new shared container with copy of range and key_compare object // name - name of shared container // f - first iterator // l - last iterator // comp - key_compare object // template<class InputIterator> static container_type* create(char* name, InputIterator f, InputIterator l, const key_compare& comp) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type<InputIterator>(f, l, comp); // Add container to shared pool and return return insert_container(name, c); }; };

/* * SharedHashContainerFactory - parameterized abstract factory class * - Specialization of the SharedContainerFactory class for hash-based associative containers. * - Implements constructors with copy of range and specified number of buckets, * hash and key_equal functions offered by hash-based associative containers. */ template<class C> class _SharedHashContainerFactory : public _SharedContainerFactory<C> {

110

public: typedef typename container_type::size_type size_type; typedef typename container_type::key_compare key_compare;

// Container's size_type // Container's hash function

// Redefinition of base class static create methods static container_type* create(char* name) { return _SharedContainerFactory<container_type>::create(name); }; static container_type* create(char* name, container_type& c) { return _SharedContainerFactory<container_type>::create(name, c); }; // Create new shared container with a number of buckets // name - name of shared container // n - number of buckets // static container_type* create(char* name, size_type n) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type(n); // Add container to shared pool and return return insert_container(name, c); } // Create new shared container with a number of buckets and a hash function // name - name of shared container // n - number of buckets // comp - key comparison function // static container_type* create(char* name, size_type n, const key_compare& comp) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type(n, comp); // Add container to shared pool and return return insert_container(name, c); } // Create new shared container with copy of range // name - name of shared container

111

// f - first iterator // l - last iterator // template<class InputIterator> static container_type* create(char* name, InputIterator f, InputIterator l) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type<InputIterator>(f, l); // Add container to shared pool and return return insert_container(name, c); }; // Create new shared container with copy of range and number of buckets // name - name of shared container // f - first iterator // l - last iterator // n - number of buckets // template<class InputIterator> static container_type* create(char* name, InputIterator f, InputIterator l, { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type<InputIterator>(f, l, n); // Add container to shared pool and return return insert_container(name, c); }; // Create new shared container with copy of range, number of buckets // and hash function // name - name of shared container // f - first iterator // l - last iterator // n - number of buckets // comp - key comparison function // template<class InputIterator> static container_type* create(char* name, InputIterator f, InputIterator l, size_type n, const key_compare& comp)

size_type n)

112

{ // Allocate new container in shared memory container_type* c = SHARED_NEW container_type<InputIterator>(f, l, n, comp); // Add container to shared pool and return return insert_container(name, c); }; }; Listing 13: container_factories.h

B.11 shared_deque.h
/* * shared_deque.h - STL containers in shared memory * * --(c) Francois Bergeon 2008 --* ---University of Liverpool ---* * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include <deque> #include "container_factories.h" /* * shared_deque class * - derives from the deque and sync classes

113

* - implements the deque class with the shared_allocator template parameter */ template<class T> class shared_deque : public sync, public std::deque<T, shared_allocator<T>> {};

/* * ShareddequeFactory - factory class to create shared deques * - Used to create new shared deques or attach to existing ones. * - Derives from the _SharedSequenceContainerFactory to implement * pre-allocation and replication constructors */ template<class T> class SharedDequeFactory : public _SharedSequenceContainerFactory<shared_deque<T>> {};

Listing 14: shared_deque.h

B.12 shared_stack.h
/* * shared_stack.h - STL containers in shared memory * * --(c) Francois Bergeon 2008 --* ---University of Liverpool ---* * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */

114

#pragma once #include <stack> #include "shared_deque.h" /* * shared_stack class * - derives from the stack and sync classes * - default sequence class is shared_deque */ template<class T, class Sequence=shared_deque<T>> class shared_stack : public sync, public std::stack<T, Sequence> {};

/* * SharedStackFactory - factory class to create shared stacks * - Used to create new shared stacks or attach to existing ones. * - Derives from the _SharedSequenceContainerFactory to implement * pre-allocation and replication constructors * - Default sequence class is shared_deque */ template<class T, class Sequence=shared_deque<T>> class SharedStackFactory : public _SharedSequenceContainerFactory<shared_stack<T, Sequence>> {}; Listing 15: shared_stack.h

115

B.13 shared_set.h
/* * shared_set.h - STL containers in shared memory * * --(c) Francois Bergeon 2008 --* ---University of Liverpool ---* * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include <set> #include "container_factories.h" /* * shared_set class * - derives from the set and sync classes * - implements the set class with the shared_allocator template parameter */ template<class Key, class Compare=std::less<Key>> class shared_set : public sync, public std::set<Key, Compare, shared_allocator<Key>> {}; /* * shared_multiset class * - derives from the multiset and sync classes * - implements the multiset class with the shared_allocator template parameter */

116

template<class Key, class Compare=std::less<Key>> class shared_multiset : public sync, public std::multiset<Key, Compare, shared_allocator<Key>> {};

/* * SharedSetFactory - factory class to create shared sets * - Used to create new shared sets or attach to existing ones. * - Derives from the _SharedAssociativeContainerFactory to implement * constructors with copy of range and specified key_compare function * offered by associative containers. */ template<class Key, class Compare=std::less<Key>> class SharedSetFactory : public _SharedAssociativeContainerFactory<shared_set<Key, Compare>> {}; /* * SharedMultisetFactory - factory class to create shared multisets * - Used to create new shared multisets or attach to existing ones. * - Derives from the _SharedAssociativeContainerFactory to implement * constructors with copy of range and specified key_compare function * offered by associative containers. */ template<class Key, class Compare=std::less<Key>> class SharedMultisetFactory : public _SharedAssociativeContainerFactory<shared_multiset<Key, Compare>> {}; Listing 16: shared_set.h

B.14 shared_hash_map.h
/* * shared_hash_map.h - STL containers in shared memory * * --(c) Francois Bergeon 2008 --* ---University of Liverpool ---*

117

* IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include <hash_map> #include "container_factories.h" /* * shared_hash_map class * - derives from the hash_map and sync classes * - implements the hash_map class with the shared_allocator template parameter */ template<class Key, class Data, class Compare=stdext::hash_compare<Key, std::less<Key>>> class shared_hash_map : public sync, public stdext::hash_map<Key, Data, Compare, shared_allocator<std::pair<const Key, Data>>> {}; /* * shared_hash_multimap class * - derives from the hash_multimap and sync classes * - implements the hash_multimap class with the shared_allocator template parameter */ template<class Key, class Data, class Compare=stdext::hash_compare<Key, std::less<Key>>> class shared_hash_multimap : public sync, public stdext::hash_multimap<Key, Data, Compare, shared_allocator<std::pair<const Key, Data>>> {}; /* * SharedHashMapFactory - factory class to create shared hash maps * - Used to create new shared hash maps or attach to existing ones. * - Derives from the _SharedHashContainerFactory to implement * constructors with copy of range and specified key_compare function * offered by associative containers.

118

*/ template<class Key, class Data, class Compare=stdext::hash_compare<Key, std::less<Key>>> class SharedHashMapFactory : public _SharedHashContainerFactory<shared_hash_map<Key, Data, Compare>> {}; /* * SharedHashMultimapFactory - factory class to create shared hash multimaps * - Used to create new shared hash multimaps or attach to existing ones. * - Derives from the _SharedHashContainerFactory to implement * constructors with copy of range and specified key_compare function * offered by associative containers. */ template<class Key, class Data, class Compare=stdext::hash_compare<Key, std::less<Key>>> class SharedHashMultimapFactory : public _SharedHashContainerFactory<shared_hash_multimap<Key, Data, Compare>> {}; Listing 17: shared_hash_map.h

119

Appendix C. PERFORMANCE EVALUATION RESULTS

Table 3 presents the raw performance evaluation results discussed above (averages are in bold). Local memory populate verify 17 12 12 12 12 12 14 12 12 12 13 12 13 13 1670 1685 1726 1702 1713 1699 1 1 1 1 3 1 1 1 1 1 1 1 1 1 1091 1091 1091 1123 1138 1107 Shared memory (create) Shared memory (attach) erase create populate verify erase attach populate verify detach vector<int> 1,000,000 non-repetitive elements 0 0 17 1 0 0 14 1 0 0 0 21 1 0 0 18 1 0 0 0 18 1 0 0 15 1 0 0 0 22 1 0 0 21 1 0 0 0 29 1 0 0 4 1 0 0 0 5 1 0 0 31 1 0 0 0 5 1 0 0 5 1 0 0 0 40 1 0 0 40 1 0 0 0 5 1 0 0 4 1 0 0 0 4 1 0 0 4 1 0 0 0 5 1 0 0 5 1 0 0 0 57 1 0 0 60 1 0 0 0 4 1 0 0 5 1 0 0 0 18 1 0 0 17 1 0 map<string,int> 1,000,000 non-repetitive elements 394 0 1512 1107 187 0 1777 1306 0 394 0 1657 1261 194 0 1729 1307 0 394 0 1754 1301 198 0 1790 1317 0 376 0 1758 1327 194 0 1734 1314 0 407 0 1736 1307 193 0 1730 1341 0 393 0 1683 1261 193 0 1752 1317 0 Table 3: Raw performance evaluation results

create 1 2 3 4 5 6 7 8 9 10 11 12 13 Avg 1 2 3 4 5 Avg 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

120

Appendix D. TEST SCAFFOLDING SOURCE CODE

The latest version of the shared container library test scaffolding is available online at: http://bergeon.francois.perso.neuf.fr/containers/test.zip

D.1 VectorTest.cpp
/* * VectorTest.cpp - Test scaffolding vector test * * --(c) Francois Bergeon 2008 --* ---University of Liverpool ---* * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * * This class implements a vector test * */ #include "VectorTest.h" // Shared container global name static char* _name = "my vector"; // Create/attach vector double VectorTest::create() { // In shared memory if (_shared) { // Instantiate a shared_vector<int> object start_timer(); _s = SharedVectorFactory<int>::create(_name); // Attempt attach if creation failed if (_s == NULL) { start_timer(); _s = SharedVectorFactory<int>::attach(_name); } // Map to a STL vector for testing _c = (std::vector<int>*)_s; } // In local memory else { // Instantiate a vector<int> object start_timer(); _c = new std::vector<int>; } // Error checking

121

if (_c == NULL) return -1.0; return end_timer(); } // Populate vector double VectorTest::populate() { int i; int n = 0; // Start start_timer(); // Exclusive lock if (_shared) _s->write_lock(); // Populate for (i = 0; i < ITER; i++) { // ni = n(i-1) + i n += i; if (_shared) _c->push_back(n); else _c->push_back(n); } // Release exclusive lock if (_shared) _s->write_unlock(); return end_timer(); } // Verify contents double VectorTest::verify() { int i; int n = 0; // Start start_timer(); // Non-exclusive lock if (_shared) _s->read_lock(); // Verify for (i = 0; i < ITER; i++) { // ni = n(i-1) + i n += i; // Fail if no match if ((*_c)[i] != n) { if (_shared) _s->read_unlock(); return -1.0; } } // Release non-exclusive locks if (_shared) _s->read_unlock(); return end_timer(); }

122

// Destroy/detach vector double VectorTest::destroy() { start_timer(); if (_shared) SharedVectorFactory<int>::detach(_name); else delete _c; return end_timer(); } Listing 18: VectorTest.cpp

D.2 MapTest.cpp
/* * MapTest.cpp - Test scaffolding map test * * --(c) Francois Bergeon 2008 --* ---University of Liverpool ---* * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * * This class implements a vector test * */ #include "MapTest.h" // Shared container global name static char* _name = "my map"; // Create/attach map double MapTest::create() { // In shared memory if (_shared) { // Instantiate a shared_map<string,int> object start_timer(); _s = SharedMapFactory<std::string,int>::create(_name); if (_s == NULL) { start_timer(); _s = SharedMapFactory<std::string,int>::attach(_name); } // Map to a STL map for testing _c = (std::map<std::string,int>*)_s; } else { // Instantiate a map<string,int> object

123

start_timer(); _c = new std::map<std::string,int>; } // Error checking if (_c == NULL) return -1.0; return end_timer(); } // Populate map double MapTest::populate() { int i; char buffer[BUFSIZ]; std::string str; int n = 0; // Start start_timer(); // Exclusive lock if (_shared) _s->write_lock(); // Populate for (i = 0; i < ITER; i++) { // Char(A + i%26) & i char c = 'A' + (i%26); sprintf_s(buffer, BUFSIZ, "%c%d", c, i); str = std::string(buffer); n += i; if (_shared) (*_s)[str] = n; else (*_c)[str] = n; } // Release exclusive lock if (_shared) _s->write_unlock(); return end_timer(); } // Verify contents double MapTest::verify() { int i; char buffer[BUFSIZ]; std::string str; int n1 = 0, n2; // Start start_timer(); // Non-exclusive lock if (_shared) _s->read_lock(); for (i = 0; i < ITER; i++) { // Char(A + i%26) & i char c = 'A' + (i%26); sprintf_s(buffer, BUFSIZ, "%c%d", c, i); str = std::string(buffer); n1 += i;

124

if (_shared) n2 = (*_s)[str]; else n2 = (*_c)[str]; // Fail if no match if (n1 != n2) { if (_shared) _s->read_unlock(); return -1.0; } } // Release non-exclusive locks if (_shared) _s->read_unlock(); return end_timer(); } // Destroy/detach map double MapTest::destroy() { start_timer(); if (_shared) SharedMapFactory<std::string,int>::detach(_name); else delete _c; return end_timer(); }

Listing 19: MapTest.cpp

125