Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Deadlock Immunity: Enabling Systems To Defend Against

Deadlocks
Authors: Horatiu Jula1 , Daniel Tralamazza1 ,
Cristian Zamfir1 ,George Candea1
Presenter: Mihaela-Andreea Vasile2
1

Dependable Systems Laboratory

EPFL
Switzerland

Computer Science Department, Faculty of Automatic Control and Computers

University Politehnica of Bucharest
Bucharest, Romania

December 4, 2014

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

1 Introduction

2 Previous Work

3 Proposed Solution

4 Evaluation

5 Conclusions

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Deadlocks

Parallel applications
- behaviour of concurrently running threads
- great number of bugs

Deadlocks
- synchronization mutex locks
- incorrect coordination in using locks

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Deadlocks

Sources for deadlocks

- bugs in the source codes
- threads scheduling at runtime
- using third party code

Deadlock immunity
- develop resistance against deadlocks
- automatically
- at runtime

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Techniques for avoiding deadlocks

Language-level approaches
- new type systems: simplify
writing concurrent code
- + no runtime overhead
- - new languages/constructs

Transactional memory
- threads execute transactions
instead of synchronized blocks
- locking thread scheduling
- improve performance use
locks

Ownership types for safe programming: preventing data races and

deadlocks (Boyapati, Lee, Rinard)

Static analysis tools

- detect deadlocks at compile
time
- programmers involvement

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Transactional memory: Architectural support for lock-free data

structures (Herlihy, Moss)

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Techniques for avoiding deadlocks

Toward dynamic approaches

- detect deadlocks at runtime gate locks
- serialize the access to locks that could cause deadlocks (JVM)
- large amount of false positives

Pure dynamic approaches

-

create checkpoints and revert execution

re-execute in modified environment
program does not improve
performance overhead

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

System overview
RAG
- allocation graph
- built by Dimmunix based on
runtime events

Avoidance code
- intercept lock/unlock
- decide if GO/YIELD based on
cached RAG
- place events in queue for
Dimmunix

Monitor Thread
- build RAG
- update cached RAG for
Avoidance code
- cycle detection
- save deadlock/starvation
signatures
H. Jula, D. Tralamazza, C. Zamfir, G. Candea
Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Figure: Dimmunix architecture

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

RAG

Built based on runtime vents provided by Avoidance component

Updated every miliseconds
Vertices
- threads
- locks

Edges
- request - T wants to acquire L
(T L)
- allow - T waits for L (T L)
- hold - T acquired L (L T)
- yield - T cannot acquire L
because of T1(T T1)

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Figure: RAG example

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Detect Deadlocks/Starvation

Detect cycles in RAG

Deadlock cycles
- T in cycle with only hold, allow, request edges

Starvation cycles
- yield cycles
- T1 is reachable from T through yield
edges
- every T1 can reach T
Figure: Starvation cycles

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Recover from Deadlocks/Starvation

Deadlocks
- not part of Dimmunix
- program restart
- define recovery handlers, called after saving the signature

Starvation
- T - starved, holds most locks
- cancel yield for T
- go for the most recent requested lock

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Avoid Previous Patterns

Save signatures
- for detected deadlock/starvation cycles
- stack labels for all hold/yield edges
- general patterns - only call flow

Avoid Deadlocks
- request lock
-

add allow edge in cached RAG

check signatures in history
potential deadlock: yield T (cached RAG, event, response)
no deadlock: allow T (cached RAG, event, response)

- acquire lock
- allow edge hold edge
- acquired event to monitor

- release lock
- remove hold edge
- release event to monitor

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Avoid Previous Patterns

Avoid Deadlocks
- save a matching depth
- importance: miss deadlock/false positives
- fixed value/ heuristic

Avoid Starvation
- weak immunity
-

default
break starvation
ignore Dimmunix
miss some deadlocks

- strong immunity
- restart program

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Properties and Limitations

Deadlock-free programs
- not affected by Dimmunix
- signatures saved on deadlock occurrences

Correct outputs
- alter thread scheduling
- non-real-time systems: the output does not depend on the scheduler

Lose functionality
- all execution paths are deadlocked
- user-disabled signatures
- automatically disable frequent patterns

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Implementations and Evaluation

Implementations
- Java - byte code instrumentation
- POSIX threads: FreeBSD, LinuxNPTL - modified libraries
- no need to modify source code nor programming language

Experiments conditions
- real-systems integration: MySQL, Apache Active MQ, Java JDK
- strong immunity
- = 100 msec

Considered Topics
- Real deadlocks efficiency
- Performance Overhead
- False Positives

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Real Deadlocks Efficiency

- Deadlock bugs: test cases to reproduce reported bugs

- Third party libraries: Java JDK - synchronized classes

Figure: Reported bugs avoided by

Dimmunix

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Figure: Java JDK 1.6 deadlocks avoided

by Dimmunix

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Performance Overhead

- real-applications: various history

size
- micro benchmarks
- Nt threads, Nl locks
- held time, and request time
in, out
- synthetic deadlock history

Figure: Overhead as function of history

size

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Figure: Overhead as function of no

threads

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Performance Overhead

Figure: Overhead as function of history

size, pthreads

Figure: Overhead as function of in,

out, Java
H. Jula, D. Tralamazza, C. Zamfir, G. Candea
Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Figure: Breakdown overhead, Java

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

False Positives and Resource Utilization

- False Positives
- too shallow signature match
- the patterns depend on inputs
- the patterns depend on the program state

- Resource utilization
- 2-1024 threads, 8-32 locks, 64 signatures
- pthreads: 6-25 MB
- Java: 79-127 MB

Figure: False positives overhead

H. Jula, D. Tralamazza, C. Zamfir, G. Candea
Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Further work

Optimizations for Dimmunix

- deadlock detection: performed offline
- deadlock avoidance
- monitor statements previously envolved in deadlocks
- reduce the number of false positives
- reduce yielding time

Communix
-

collaborative immunity framework for Java

complementary for Dimmunix
sends signatures to a server
select new signatures appropriate to the Java application

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Dependable Systems Laboratory EPFL

Introduction

Previous Work

Proposed Solution

Evaluation

Conclusions

Conclusions

- programs: develop resistance against deadlocks

- complementary tool for software systems
- may be used when fixing is expensive/risky
- may trigger latent bugs
- Dimmunix has been improved and extended

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Dependable Systems Laboratory EPFL

Introduction

Previous Work

H. Jula, D. Tralamazza, C. Zamfir, G. Candea

Deadlock Immunity: Enabling Systems To Defend Against Deadlocks

Proposed Solution

Evaluation

Conclusions

Dependable Systems Laboratory EPFL