CALIFORNIA PATH PROGRAM INSTITUTE OF TRANSPORTATION STUDIES UNIVERSITY OF CALIFORNIA, BERKELEY

Hierarchical, Hybrid Control of Large Scale Systems

John Lygeros

California PATH Research Report

UCB-ITS-PRR-96-23

This work was performed as part of the California PATH Program of the University of California, in cooperation with the State of California Business, Transportation, and Housing Agency, Department of Transportation; and the United States Department of Transportation, Federal Highway Administration. The contents of this report reflect the views of the authors who are responsible for the facts and the accuracy of the data presented herein. The contents do not necessarily reflect the official views or policies of the State of California. This report does not constitute a standard, specification, or regulation.

September 1996 ISSN 1055-1425

Hierarchical, Hybrid Control of Large Scale Systems

by John Lygeros B.Eng. Imperial College of Science Technology and Medicine, London 1990 M.Sc. Imperial College of Science Technology and Medicine, London 1991 A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Engineering-Electrical Engineering and Computer Sciences in the GRADUATE DIVISION of the UNIVERSITY of CALIFORNIA at BERKELEY Committee in charge: Professor Shankar Sastry, Chair Professor Pravin Varaiya Professor Roberto Horowitz 1996

by John Lygeros Doctor of Philosophy in Engineering-Electrical Engineering and Computer Sciences University of California at Berkeley Professor Shankar Sastry, Chair The control of large scale dynamical systems is one of the biggest challenges facing control engineers today. Large scale systems are common in applications such as chemical process control, power generation and distribution and highway and air tra c control, among others. Of particular interest are multi-agent, scarce resource problems, where a large number of agents, equipped with communication and control capabilities have to make e cient use of a scarce resource. The size and complexity of these systems makes it di cult to approach them using tools from the classical, central control literature. To manage the system complexity designers are often forced to use hybrid control schemes, i.e. designs that make use of both discrete and continuous controllers. In this dissertation we try to approach the problem of large scale systems from such a hierarchical, hybrid control view point. Our analysis is based on a new hybrid dynamical system formulation, that allows us to model large scale systems in a modular fashion. Three problems are addressed: controller design, closed loop performance veri cation and the extension of system autonomy. First, a control scheme based on semi-autonomous agent operation is proposed. Our scheme naturally leads to hierarchical, hybrid designs, with continuous controllers trying to optimize each agent's resource utilization at a lower level and discrete controllers resolving inter-agent con icts at a higher level. An algorithm is presented to produce the continuous controllers, as well as abstractions of their performance in terms of the discrete level. The algorithm makes use of ideas from game theory, treating the design process as a two player, zero sum game, between the controller of an agent and the disturbance generated by the actions of other agents. The resulting abstractions can be thought of as guidelines for the design of the discrete layer. The implication is that if the resulting continuous controllers are used and the discrete controller satis es the guidelines, the closed loop hybrid system is, by design, guaranteed to exhibit the desired behavior. This approach also extends to the veri cation of closed loop performance requirements for a given control scheme. The problem can be approached from an optimal control view point, as one of the players in the game namely the controller has its strategy xed by the given control scheme. Finally, some discussion of the design issues involved in extending the controller autonomy in presented. The emphasis is on hybrid e ects such as instantaneous faults that qualitatively change the plant dynamics. We investigate the hierarchical and hybrid issues involved in the design of a fault tolerant" control scheme. The algorithms are illustrated by means of examples throughout the dissertation. In addition, a case study on the application of the proposed techniques to the control of an automated highway system is presented in the concluding chapters.

Hierarchical, Hybrid Control of Large Scale Systems

Abstract

Shankar Sastry Chair 2

Contents
1 Introduction
1.1 Control of Large Scale Systems : : : 1.2 Issues to be Addressed : : : : : : : : 1.2.1 Controller Design : : : : : : : 1.2.2 Veri cation : : : : : : : : : : 1.2.3 Extending System Autonomy 1.3 Outline : : : : : : : : : : : : : : : :

: : : : : :

: : : : : :

: : : : : :

: : : : : :

: : : : : :

: : : : : :

: : : : : :

: : : : : :

: : : : : :

: : : : : :

: : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

1 3 3 5 7 7

2 Modeling & Mathematical Tools

2.1 Hybrid Dynamical Systems : : : : : : : : : : : : : : : : 2.1.1 The Elements : : : : : : : : : : : : : : : : : : : : 2.1.2 The De nitions : : : : : : : : : : : : : : : : : : : 2.1.3 Special E ects : : : : : : : : : : : : : : : : : : : 2.1.4 Graphical Representation : : : : : : : : : : : : : 2.1.5 Operations on Hybrid Dynamical Systems : : : : 2.2 Agent Model : : : : : : : : : : : : : : : : : : : : : : : : 2.2.1 Hierarchy of subsystems : : : : : : : : : : : : : : 2.2.2 Hybrid Automaton Model : : : : : : : : : : : : : 2.2.3 Multiagent Design and Veri cation Environment 2.3 Mathematical Tools : : : : : : : : : : : : : : : : : : : : 2.3.1 Game Theory : : : : : : : : : : : : : : : : : : : : 2.3.2 Optimal Control : : : : : : : : : : : : : : : : : : 2.3.3 Dynamical Systems and Topology: : : : : : : : : 3.1 Discrete Layer : : : : : : : : : : : : : : : : : : : : 3.1.1 Design Phases : : : : : : : : : : : : : : : : 3.1.2 Discrete Layer Abstraction : : : : : : : : : 3.2 Continuous Layer : : : : : : : : : : : : : : : : : : : 3.2.1 Multiobjective Controller Design Algorithm 3.2.2 Controller Automaton : : : : : : : : : : : : 3.2.3 Technical Issues : : : : : : : : : : : : : : : : 3.2.4 Interface and Discrete Design Revisited : : 3.3 The Train-Gate Example : : : : : : : : : : : : : : 3.3.1 Problem Statement : : : : : : : : : : : : : : 3.3.2 Game Theoretic Formulation : : : : : : : : 3.3.3 Design for Safety : : : : : : : : : : : : : : : iii

9 9 11 12 14 15 17 17 18 19 21 21 22 23

3 Controller Design

: : : : : : : : : : : :

: : : : : : : : : : : :

: : : : : : : : : : : :

24

24 24 25 25 26 27 28 28 29 29 31 32

3.3.4 Design for Throughput : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 34 3.3.5 Controller Design : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 35 3.4 Summary of Key Points : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 37

4 Veri cation

4.1 Invariant Veri cation : : : : : : : : : 4.2 Multi Agent System Veri cation : : 4.3 Example: The Leaking Gas Burner : 4.3.1 Problem Statement : : : : : : 4.3.2 Optimal Control Formulation 4.3.3 Veri cation Process : : : : :

: : : : : :

: : : : : :

: : : : : :

: : : : : :

: : : : : :

: : : : : :

: : : : : :

: : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

38

39 39 41 41 41 42 45 46 47 48 48 48 49 50 50 50 51 53 55 55 56 58 63 65 67 67 68 70 71 72 73 73 78 78 79 79 81 84 86

5 System Autonomy

5.1 Adaptation and Learning : : : : : : : : : : : : : : 5.2 Hybrid Issues : : : : : : : : : : : : : : : : : : : : : 5.2.1 Fault Modeling : : : : : : : : : : : : : : : : 5.2.2 Modeling of Extreme Parameter Variation : 5.3 Fault Tolerant Controller Design : : : : : : : : : : 5.3.1 Fault Detection : : : : : : : : : : : : : : : : 5.3.2 Fault Handling : : : : : : : : : : : : : : : : 5.3.3 Dealing with Extreme Parameter Variation 5.4 Hierarchical Issues : : : : : : : : : : : : : : : : : : 5.4.1 Information Hierarchy : : : : : : : : : : : : 5.4.2 Control Hierarchy : : : : : : : : : : : : : : 6.1 The Platooning Concept : : : : : : : : : 6.2 Problem Formulation : : : : : : : : : : : 6.2.1 Discrete Level Abstraction : : : : 6.2.2 Overview of the AHS Research E 6.2.3 Hybrid Vehicle Model : : : : : : 6.2.4 Design Requirements : : : : : : : 6.2.5 Assumptions : : : : : : : : : : : 6.3 Single Lane Safety Theorem : : : : : : : 6.3.1 Background Lemmas : : : : : : : 6.3.2 Safety Theorem : : : : : : : : : : 6.4 Multi Lane Safety Theorem : : : : : : : 6.4.1 Background Lemmas : : : : : : : 6.4.2 Safety Theorem : : : : : : : : : : 6.5 The Leader Control Law : : : : : : : : : 6.5.1 Design for safety : : : : : : : : : 6.5.2 Design for comfort : : : : : : : : 6.5.3 Design for e ciency : : : : : : : 6.5.4 Proof of Leader Safety Lemma : 6.6 The Join Control Law : : : : : : : : : : 6.6.1 Design for Safety : : : : : : : : : 6.6.2 Numerical Investigation : : : : : 6.6.3 Completing the Design : : : : : :

45

6 Automated Highway Systems: Hybrid Design

:::: :::: :::: ort : :::: :::: :::: :::: :::: :::: :::: :::: :::: :::: :::: :::: :::: :::: :::: :::: :::: ::::
iv

: : : : : : : : : : : : : : : : : : : : : :

: : : : : : : : : : : : : : : : : : : : : :

53

6.6.4 The Split Design : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 87 6.7 The Lane Change Laws : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 87 6.8 Key Points and Discussion : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 90

7 Automated Highway Systems: Fault Tolerant Design

7.1 Extended Information Structure : : : : : 7.1.1 Capability Monitor : : : : : : : : : 7.1.2 Performance Monitor : : : : : : : : 7.2 Extended Control Structure : : : : : : : : 7.2.1 Coordination Supervisor Strategies 7.2.2 Atomic Maneuvers : : : : : : : : : 7.2.3 Veri cation : : : : : : : : : : : : : 7.3 Key Points and Discussion : : : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

: : : : : : : :

94 94 98 100 100 103 104 105

94

8 Concluding Remarks A Sensor & Actuator Ranges

A.1 Vehicle Capabilities : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 116 A.2 Relative velocity at impact : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 117 A.3 Sensor Ranges : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 117

107 116

Chapter 1

Introduction
1.1 Control of Large Scale Systems
The demand for increased levels of automation and system integration have forced control engineers to deal with increasingly larger and more complex systems. Among the reasons that have contributed to this push for higher levels of automation are nancial bene ts and environmental considerations. Preliminary studies indicate that increased automation in air tra c management systems, highway systems, chemical process control, power generation and distribution etc. can lead to performance improvement in terms of fuel consumption, and e ciency. The size of these businesses is such that even minor changes in performance translate to large amounts of money gained or lost and or considerable impact to the environment. At the same time, recent technological advances, such as faster computers, cheaper and more reliable sensors and the integration of control considerations in the product design and manufacturing process, have made it possible to extend the practical applications of control to systems that were impossible to deal with in the past.

The need for Hybrid Control

To deal with large, complex systems engineers are usually inclined to use a combination of continuous and discrete controllers. The reasons why continuous controllers are used are many: Interaction with the physical plant, through sensors and actuators, is essentially analog, i.e. continuous, from the engineering point of view. Continuous models have been developed, used and validated extensively in the past in most areas that interest control engineers e.g. electrical and mechanical systems, electromagnetic systems, etc. Powerful control techniques have already been developed for many classes of continuous systems. Moreover, in conjunction with the reliable continuous models, proofs of guaranteed performance can be obtained for these techniques. An equally compelling case can be made in favor of discrete controllers, however: Discrete abstractions make it easier to manage the complexity of the system. It is not an accident that most of the work on discrete controllers started in the area of manufacturing, where complex systems were rst encountered and modeled. Discrete models are easier to compute with, as all computers and algorithms are essentially discrete. 1

Discrete abstractions make it easy to introduce linguistic and qualitative information in the controller design. In the literature, the term hybrid systems is used to describe systems that incorporate both continuous and discrete dynamics. Design and analysis of such systems is particularly challenging. An obvious challenge arises from the size of the problem and the diverse requirements that the design needs to satisfy. Even though the ultimate goal of a design may be nancial bene t, an underlying constraint is that this should be achieved without an adverse impact on the environment, the safety of the users, etc. These considerations often make contradictory demands on the design and a compromise between them needs to be sought. This may require the fusion of expertize from many scienti c disciplines: computer science, control, chemical engineering, operations research, etc. The problem becomes even more di cult for systems where a partly satisfactory design already exists. The momentum of the users, designers and agencies often makes the implementation of new ideas problematic. For such systems it may be di cult to distinguish between the actual issues that need to be addressed by a new design and issues that arise because of adherence to current practices. The biggest challenge, however, is the lack of formal mathematical tools for the analysis and design of such systems. Through the years a number of tools have been developed to deal with purely continuous or purely discrete systems. None of these tools however is capable of fully addressing the issues arising in hybrid systems. The development of specialized hybrid system tools has recently been the focus of a number of research groups 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 . This dissertation will summarize one such e ort 12, 13, 14, 15, 16, 17, 18, 19, 20, 21 .

Multi-Agent Scarce Resource Systems

A very interesting class of systems that are naturally suited for hybrid control are multiagent, scarce resource systems. Their common characteristic is that many agents are trying to make optimum use of a congested, common resource. Examples of such systems are highway systems, where vehicles compete for scarce highway space-time, air tra c management systems where aircraft compete for air space and runway space, power generation and distribution systems where producers and consumers of power make use of the common distribution grid, computer networks, etc. Because of the challenging problems associated with the control of multi-agent, scarce resource systems, this area has attracted considerable attention both theoretically and in application 22, 23, 24, 17, 25 . The design of controllers for multiagent scarce resource problems presents a number of di culties: Even though the individual agent state space may be small, the possible coupling between the agents leads to a very large number of interacting states. The number of agents may be dynamically changing planes take o and land, vehicles get on and o the highway, etc.. Faults in one agent may adversely e ect everyone else. The optimum policy for each agent may not coincide with the common good". Therefore, compromises need to be made. The last comment indicates that the information structure plays a crucial role in the design. To achieve the common optimum one should ideally have a centralized control scheme, i.e. one or all the agents have access to all the information. The centralized controller computes the global optimum and commands the agents accordingly. Such centralized schemes have been used successfully in certain applications. They seem to work best in cases where the environment is highly structured for example in train scheduling. In most cases, however, a solution like this may be undesirable for a number of reasons: 2

It is likely to be very computationally intensive, as a large centralized computer will probably be needed to make all the decisions. It may be less reliable, as the consequences are likely to be catastrophic if the centralized controller is disabled. The information that needs to be exchanged may be too large for the available communication capabilities. As the number of agents may be large and or dynamically changing, the optimal strategy may be hard to obtain and may have to be constantly updated. Another extreme alternative would be a completely decentralized scheme. Here, each agent takes control decisions individually based on local information for example information obtained through sensors about its own state and the state of neighboring agents. The advantage of such a scheme is that it requires no information exchange between the agents. Hence the calculations involved in the controller design are likely can be a lot simpler. On the other hand it is likely to be very ine cient compared to the global optimum. Moreover, proofs of performance claims are going to be very hard, as the global behavior may be di cult to infer from the local calculations. If a completely decentralized solution is unacceptable and a completely centralized solution is prohibitingly complex or expensive, an in-between compromise will have to be sought. Such a compromise will feature semiautonomous agent operation. In this case each agent is trying to optimize its own usage of the resource and coordinates with neighboring agents if there is a con ict of objectives. Clearly such a solution is likely to be less e cient than a centralized scheme and harder to implement than a decentralized scheme; however, it may be the only feasible choice. It should be noted that semiautonomous agent control is naturally suited for hybrid designs. The controller can be arranged in a multi level hierarchy. At the lower levels each agent chooses its own optimal strategy, which will be in the form of a continuous control law. At the higher levels discrete coordination is used to resolve con icts. The hybrid dynamics arise from the interaction between continuous single agent optimal strategies and discrete con ict resolution protocols. This dissertation will be primarily concerned with this approach to multi agent hybrid control design. We will address three problems that arize in large scale systems: controller design, veri cation and extending system autonomy. For the rst two problems the treatment will be formal, while for the third it will primarily be based on intuition and examples. Our work aims at designing hybrid controllers so that certain aspects of the closed loop system performance are a-priori guaranteed, without having to carry out any veri cation. This approach is also taken by other researchers in the eld see for example 8, 6, 23, 26, 27, 18, 28 . The advantage is that it eases the requirements on veri cation somewhat as a large part of the complexity can be absorbed by careful design.

1.2 Issues to be Addressed

1.2.1 Controller Design

Framework
The plan is to start by modeling the systems dynamics at the continuous level. Each agent will be modeled independently by a state space model with relatively few states. Two factors a ect the system evolution 3

at this level. The rst is the control, that the designer has to determine. The second is the disturbances that enter the system, over which we assume no control. We will distinguish three classes of disturbances: Class 1: Exogenous signals, such as unmodeled forces and torques in mechanical systems, sensor noise, etc. Class 2: Unmodeled dynamics Class 3: The actions of other agents, in a multiagent setting. Disturbances of Class 1 and 2 are standard in classical control theory. Class 3 will be the most interesting one from the point of view of hybrid control. Recall that at this stage we are merely modeling the plant, therefore we assume no cooperation between the agents. As a result, each agent views the actions of its neighbors as uncontrollable disturbances. In this setting speci cations about the closed loop system can be encoded in terms of cost functions. A number of cost functions may need to be considered to encode di erent requirements for example safety, e ciency, etc.. It will be assumed that the cost functions can be ranked in terms of importance for example safety will be more important than e ciency. Acceptable performance can be encoded by means of thresholds on the nal costs.

Continuous Design
The rst objective is to derive a continuous design for the control inputs that guarantees performance despite the disturbances. The design of the continuous laws will be optimal with respect to the closed loop system requirements. An ideal tool for this kind of set up is game theory. In the game theoretic framework the control and the disturbances are viewed as adversaries in a game. The control seeks to improve system performance while the disturbance seeks to make it worse. Games like these do not necessarily have winners and losers. If, however, we set thresholds on the cost functions to distinguish acceptable from unacceptable performance we can say that the control wins the game if the requirements are satis ed for any allowable disturbance, while the disturbance wins otherwise. The principles involved in game theoretic design are very similar to the ones for optimal control. Roughly speaking, the designer has to nd the best possible control and the worst possible disturbance. If the requirements are met for this pair, it is possible to obtain a satisfactory design one such design is the best possible" control. If the requirements are not satis ed the problem can not be solved as is, since there exists a choice of disturbance for which, no matter what the controller does, the closed loop system will fail to satisfy the requirements. Game theoretic ideas have already been applied in this context to problems with disturbances of Class 1 and 2 and quadratic cost functions. The resulting controllers are the so called H1 or L2 optimal controllers see for example 29, 30 . We will try to extend these ideas to the multiagent, hybrid setting and focus on disturbances of Class 3.

Discrete Design
The solution to the game theoretic problem will produce a continuous control law and a set of initial conditions for which performance is guaranteed for any disturbance. If discrete switching between control laws is required, the set of initial conditions can be used as guidelines for the switching a form of interface between the continuous and discrete domains. If it turns out that the disturbance is such that the speci cations can not be met for any controller the design fails. The only way to salvage the situation is to somehow limit the disturbance. For disturbances of Class 3 this may be possible by means of communication and coordination between the agents. The 4

objective then is to come up with a discrete design that limits the disturbance so that a continuous design is feasible. Unfortunately, no formal technique exists for determining the coordination necessary to solve such a problem. Typically a discrete design is carried out based on heuristics and insight gained from the continuous level design. Then the system is analyzed to determine whether the coordination provides enough disturbance reduction for the problem to have a solution. In this dissertation we will limit our attention to the design of continuous control laws and interfaces between these laws and the discrete world. In the examples we consider the requirements on the discrete design will intuitively follow. Summarizing, our approach to hybrid controller design consists of determining continuous control laws and conditions under which they satisfy the closed loop requirements. Then a discrete design is constructed to ensure that these conditions are met. This process eliminates the need for automatic veri cation as the hybrid closed loop system is guaranteed to satisfy the speci cations by design. Another standard approach to the design of hybrid controllers involves independently coming up with a reasonable design for both the discrete and continuous parts. The combined hybrid controller is then put together by means of interfaces, and veri cation is carried out to ensure that it satis es certain properties. Because of the complexity of the system, veri cation is usually done automatically, using some specialized computer program.

1.2.2 Veri cation

Computer Aided Veri cation

This approach has been motivated by the success of veri cation techniques for nite state systems. Veri cation algorithms for nite state machines have been in use for years and e cient programs exist to implement them COSPAN 31 , HSIS 32 , STATEMATE 33 , etc.. They have proved very successful in discrete problems such as communication protocols 34 and software algorithms. The push towards stronger veri cation techniques has been in the direction of extending the standard nite state machine results to incorporate progressively more complicated continuous dynamics. The rst extension has been for systems with clocks 35, 36 and multi-rate clocks 5 . Theoretical results have established conditions under which problems like these can be solved computationally and algorithms have been developed to implement the veri cation process for example timed COSPAN 37 , KRONOS 38 and UUPAAL 39 . Veri cation of timed systems has proved useful in applications such as digital circuit veri cation 40 and real-time software 41 . Recently the theory has been extended to systems where the dynamics can be modeled by rectangular di erential inclusions. The results indicate that, under certain conditions, automatic veri cation should also be possible for such systems 42 , but most applications have been to academic examples rather than actual systems. To our knowledge the only computer package capable of dealing with di erential inclusions, HYTECH, is still under development 43 . Progress in the direction of automatic veri cation has been impeded by two fundamental obstacles. The rst is undecidability. To guarantee that automatic veri cation algorithm will terminate in nite number of steps with an answer, the system needs to satisfy very stringent technical requirements. It can be shown 10 that relaxing any of these requirements makes the problem undecidable. The second problem is computational complexity. Even relatively simple hybrid systems lead to very large numbers of discrete states when looked at from the point of view of automatic veri cation. Even though e cient algorithms that make use of formal reduction techniques as well as heuristics and user input to facilitate the search exist, the problem may still be prohibitively large for current computers 41 . Computational complexity is a fundamental issue for veri cation and can, in principle, cause problems to any veri cation algorithm. To address the issue of undecidability the concept of abstraction may be 5

used. A hybrid system that fails to satisfy the decidability requirements is replaced by an abstraction, i.e. a hybrid system that can produce at least as many runs as the original system and possibly more. The abstraction can be chosen to be a decidable hybrid system; formal results exist that determine the quality of abstraction that can be expected 9 . Veri cation can then be carried out on the abstraction. If it turns out that the abstraction satis es the necessary requirements then the original system is also guaranteed to satisfy them. If the abstraction fails to satisfy the requirements however, no conclusion can be drawn about the original system. The failure may be due to a trajectory of the original system or to a spurious trajectory generated by overabstraction". The abstraction will then have to be re ned and the veri cation repeated. Besides this technical problem, the main drawback of abstraction techniques is that they exasperate the complexity situation. Even a simple dynamical system may require an immensely complicated abstraction automaton to reproduce its trajectories to within reasonable accuracy. Optimal control and gaming ideas may also prove useful for veri cation. Standard automatic veri cation techniques involve some form of exhaustive search, to verify that all possible runs of the systems satisfy a certain property. As discussed above this leads to undecidability and complexity problems. An optimal control approach to veri cation involves solving an optimal control problem to obtain the worst possible run with respect to a given requirement and then verifying that the speci cation holds for this run. If this is the case, it will also hold for all other runs. Veri cation of closed loop hybrid systems is better suited for optimal control, rather than game theory, as one of the two players the controller has its strategy xed a-priori. Therefore only the disturbances, trying to do their worst to upset the design, enter the picture. An interesting class of disturbances that need to be considered in this context is: Class 4: Commands from the discrete controller From the point of view of the continuous system where the optimal control problem is to be solved these commands can be viewed as signals that make the continuous system switch between control laws, xed points etc. Optimal control can be used to determine the discrete command sequences that force the continuous system to violate the performance speci cations. If the discrete design is such that these command sequences are excluded then the hybrid design is veri ed. 44 discusses the application of these ideas to the automated highway example. The main advantage of the optimal control point of view to veri cation is that, by removing the requirement for an exhaustive search, the limitations of complexity and undecidability disappear. It is quite likely that many hybrid systems that are undecidable from the automatic veri cation point of view will be amenable to optimal control veri cation. On the down side, veri cation using optimal control requires a lot of input from the user. Moreover, optimal control problems will, in general, be very hard to solve analytically. It is therefore unlikely that this approach can be applied, at least by hand, to systems with more than a few discrete states. For the time being veri cation using optimal control is no match for automatic veri cation techniques for systems to which the latter can be applied.

Veri cation using Optimal Control

Generation of Abstractions

Optimal control ideas can also be used to generate abstractions of continuous behavior in terms of discrete languages. For example optimal control can be used to obtain the minimum and maximum times that a hybrid system spends in each discrete state. These bounds can then be used as a rudimentary timed abstraction of the hybrid system. Similarly, the sets of initial conditions for which the performance requirements are satis ed, that are generated by solving the game theory problems, can be used as a di erent 6

discrete abstraction  safe" vs unsafe" states for example. As in the case of veri cation, the requirement for designer input limits the complexity of the problems that can be handled analytically. The techniques discussed above will primarily be applicable for system operation under normal conditions. The problem becomes a lot more complicated if system faults are to be considered. Part of our e ort will be to set down guidelines for increasing the system autonomy and producing a fault tolerant design. Roughly speaking, these guidelines can be thought of as the equivalent of adaptation and learning for a hierarchically organized, hybrid system. Our approach will be illustrated by means of an example from highway automation. Conventional adaptation and learning techniques focus on quantitatively tuning the controller by observing the input output performance of the closed loop system. The only information needed for this purpose relates to the state, inputs and outputs of the system and can be obtained through sensors. A fault tolerant design on the other hand will have to adapt to qualitative changes in the system dynamics loss of capabilities, etc.. As a consequence, in addition to state information, the fault tolerant controller will need information about the system capabilities. For a hybrid design like the ones discussed so far this information will have to be presented in a form compatible with the level of abstraction of each controller discrete information for discrete controllers and continuous information for continuous ones. Extending the information structure in this way will essentially amount to adding a new hierarchy, that mimics the control hierarchy, to propagate the capability information.

1.2.3 Extending System Autonomy

Extended Information Structure

Extended Control Structure

The requirement for autonomy also enhances the hybrid nature of the design, as it forces the designer to make an explicit distinction between strategic planning and control execution. Our design procedure will involve splitting the controller into two layers, a supervisor and a regulator. The supervisor makes use of the capability information propagated by the extended information structure to determine strategy, i.e. a sequence of control actions compatible with the system capability. The strategy may need to change only when the system capability changes. The regulator on the other hand is responsible for implementing this strategy.

1.3 Outline
The material of the dissertation is arranged in eight chapters. In Chapter 2 a framework for modeling multiagent systems by means of interacting hybrid automata and their input output connections is presented. Some background material from game theory and optimal control is also given, along with references where a more thorough presentation can be found. In Chapter 3 the design methodology outlined above is presented. An algorithm for reconciling the often contradictory demands made by the various design requirements is proposed. The result is an optimal controller and a set of initial conditions for which performance is guaranteed. The requirements that the discrete controller inherits from the continuous design are derived as corollaries of the algorithm. The application of our approach is illustrated by means of the train-gate controller", a simple example from the timed automata veri cation literature. 7

In Chapter 4 veri cation using optimal control ideas is discussed. Our approach is illustrated by means of another example from timed automata veri cation, the leaking gas burner". In Chapter 5 the principles involved in extending the system autonomy to make it capable of dealing with degraded conditions of operation are presented. The discussion will rely heavily on intuition derived from the application of these principles to the automated highway example. In Chapters 6 and 7 possible applications of our approach are discussed. The focus is the automated highway problem. This problem displays all the characteristics that make large scale system design challenging large, dynamically changing number of agents, con icting demands for e ciency, safety and comfort, possibility of faults, etc.. In Chapter 6 a design approach for a hybrid controller that guarantees safety, e ciency and passenger comfort is presented1 . Even though the design focuses on only one highway automation concept, most of the calculations are general enough to apply to any other concept verbatim. In Chapter 7 the design is extended along the lines of Chapter 5. The resulting fault tolerant design is outlined and proofs of certain aspects of its performance will be given. Finally, Chapter 8 contains some concluding remarks and directions of future work.

1 A similar design is presented in 28 under di erent assumptions

Chapter 2

Modeling & Mathematical Tools

In this chapter, we present a rather general model for multiagent systems. Each agent is modeled as a hybrid dynamical system. The modeling framework is modular, in that a hybrid system can be speci ed as a composition of subsystems. An agent model can include subsystems representing the plant dynamics, sensors, actuators, communication devices and controllers. In other words, each agent speci cation is a tree structured hierarchy of subsystems. In some cases, agents may coordinate their actions with each other, while in others, the actions of one agent may appear as disturbance to other agents. Interactions between agents can be captured via sensing and communication.

2.1 Hybrid Dynamical Systems

The basic entity of our models will be the hybrid dynamical system or hybrid automaton the terms will be used interchangeably. Hybrid automata are convenient abstractions of systems with phased operation and they appear extensively in the literature in various forms  8, 5 . The model we consider will be similar to models used primarily in computer science in particular the ones in 8 and 9 . However, because we are interested in modeling di erent agents and their interaction we will take a more input output approach, along the lines of the reactive module paradigm 45 . For an excellent overview of hybrid models from the dynamical systems point of view see 7 .

2.1.1 The Elements

A hybrid automaton is a dynamical system which determines the evolution and interaction of a nite collection of variables. We will consider two distinct kinds of variables, discrete and continuous. De nition 1 A variable is called discrete if it takes values in a countable set and it is called continuous otherwise. By a countable set we mean any set homeomorphic to a subset of the integers under the discrete topology. We will assume no special algebraic structure for the values of the discrete variables. The only operations we will allow are assigning a value to a variable and checking weather the value of a variable and a member of the value set or the values of two variables that take values in the same set are equal. Because we will be interested in di erentiating continuous variables with respect to time we will assume that they take values in sets that are di eomorphic to subsets of Rn for some value of n1 .
1 The de nitions generalize to weaker conditions, for example continuous variables lying on a manifold.

Variables

The variables in our model will be split into three classes: Inputs external, Outputs interface and State private2 . We will denote the input space set where the input variables take values by: U = UD  UC the output space by: and the state space by:

Y = YD  YC X = XD  XC

The subscripts D and C indicate whether the variable is discrete or continuous. To avoid unnecessary subscripts we will denote by u an element of U , by y an element of Y and by q; x an element of X . Under the above assumptions X can be shown see Section 2.3 to be a metric space.

Time
We are interested in capturing the dynamic evolution of these variables, so we need some notion of time". Let T denote the set of times of interest. Our models will evolve in continuous time, so we will assume that T = ti ; tf R. The de nitions should easily extend to other sets T with appropriate topological and algebraic structure. The variables will evolve either continuously as a function of time or in instantaneous jumps. Therefore the evolution of the system will be over sets of the form: 0 T = f 00 ; 1 10 ; 2 ; : : : n,1; n g 2.1 with i 2 T for all i, 00 = ti ; n = tf and i = i0  i+1 for all i = 1; 2; : : : ; n , 1. The implication is that i are the times where the discrete jumps of the state or input occur. We will use to denote an element of T . For visualization purposes, for every interval i0,1 ; i with i0,1 i we will denote the evolution of the continuous state as a continuous function of t 2 i0,1 ; i and the evolution of the discrete state as a constant function. At the jump times the state is not a function as it can assume many values, jumping from one to the other in the order indicated by the i 's.

Dynamics
The variable evolution will be determined by four quantities:

I X f : X  U ,! TXC E X U X h : X  U ,! Y Here TXC represents the tangent space of the space XC . The interpretation is the following: I is the set of possible values of the initial states, i.e.: q 00 ; x 00  2 I The vector eld, f , determines the evolution of the continuous state in intervals of the form i0,1 ; 0 i . For every t 2 i0,1 ; i : i,1 xt = f qt; xt; ut _
They can be used interchangeably, though we stick to the terms in bold most of the time.

2.2 2.3 2.4 2.5

with

2 The terms in bold come from the system theory literature while the terms in brackets are more computer science oriented.

10

Without loss of generality we will assume that f is time invariant3 . We will make the standard assumptions on f for existence and uniqueness of solutions to the ordinary di erential equation. The set E determines the discrete evolution of the state. The interpretation is that the state can keep evolving continuously as long as: qt; xt; ut; qt; xt 2 E The state can take a discrete jump at time i from q i ; x i  2 X to q i0 ; x i0  2 X if: q i ; x i ; u i ; q i0 ; x i0  2 E Finally, h determines the output evolution. For all t 2 :

yt = hqt; xt; ut

2.1.2 The De nitions

Collecting the above elements we give the following de nitions: De nition 2 A hybrid dynamical system, H , is a collection X; U; Y; I; f; E; h, with:

X U Y I f E h

= XD  XC = UD  UC = YD  YC

X : X  U ,! TXC X U X : X  U ,! Y

where XC ; UC ; YC are respectively di eomorphic to open subsets of Rn ; Rm ; Rp , for some nite values of n; m; p and XD ; UD ; YD are countable sets.

 ; q; x; y; u:

De nition 3 A run of the hybrid dynamical system H over an interval T = ti; tf consists of a collection
q x y u
which satis es the following properties:
0 0 1. Initial Condition: q 0 ; x 0  2 I . 2. Discrete Evolution: q i ; x i ; u i ; q i0 ; x i0  2 E , for all i.

2 T
: : : :

,! XD ,! XC ,! Y ,! U

advantage to this would be that the de nition would directly extend to other cases, such as discrete time systems.

3 With some additional notation overhead the same de nition can be given in terms of the ow of the vector eld. The

11

3. Continuous Evolution: for all i with i0

i+1 and for all

t 2 i0 ;

i+1 :

xt = f qt; xt; ut _ qt = q i0  qt; xt; ut; qt; xt 2 E
For every pair of state-input values q; x; u consider the quantity4 : q; x; u = x + f q; x; u

2.6 2.7 2.8

De nition 4 A hybrid automaton, H , is non-blocking if for all q; x; u 2 X  U one of the following
is true: 1. fq0 ; x0 jq0 ; x0  2 X n fq; xg; q; x; u; q0 ; x0  2 E g 6= ; 2. There exist

0 such that for all 2 0; : q; q; x; u; u; q; q; x; u 2 E

Roughly speaking a hybrid automaton is non-blocking if a run can continue from every state, either by a discrete jump 1 or by owing along the vector eld 2.

2.1.3 Special E ects

We brie y investigate some of the behaviors that our hybrid model is capable of generating. First note that the model is nondeterministic. Even if the continuous evolution is uniquely determined once u is speci ed, there is still nondeterminism in the discrete evolution. For a given choice of q; x; u both discrete and continuous evolution may be possible. Moreover, for the discrete evolution, many destinations q0 ; x0  may be possible. For convenience we will distinguish two special kinds of variables, events and parameters. De nition 5 An event, , is a discrete variable whose transitions rather than its actual value are important. We say occurred at i if  i  6=  i0 .

De nition 6 A parameter, p 2 XC , is a continuous variable which remains constant during the continuous evolution, that is: for all q; x; u 2 X  U . fp stands for the vector eld projected onto the coordinate p 2 XC .

fpq; x; u  0

A simple construction allows us to associate an event to a transition of the discrete state. Construction 1 Event Generation: Given a hybrid automaton, H , assume that an event, is to occur if and only if the discrete state transitions from q1 2 XD to q2 2 XD . Construct a new automaton, H 0 such that: X 0 = XD  f0; 1g  XC U0 = U
4 By abuse of notation, we identify the tangent space of XC = R n with XC itself.

12

Y0 =Y I 0 = I  f0g f 0q; ; x; u = f q; x; u for 2 f0; 1g E 0 X 0  U 0  X 0 , with e0 = q; ; x; u; q0 ; 0 ; x0  2 E 0 if and only if:
q; x; u; q0 ; x0  2 E q; q0  6= q1 ; q2  and = 0 q; q0  = q1 ; q2  and 6= 0 2.9 2.10 2.11

h0 q; ; x; u = hq; x; u

In some cases the system dynamics may have to depend not only on the current value of the state but also on the value at the time of the last discrete jump see for example the hybrid models of 2 and 7 . This e ect can be modeled in this framework using parameters.

Construction 2 Memory Parameters As usual let q 2 XD denote the discrete state, x 2 XC denote the continuous state and u 2 U denote the
input. Let x denote the value of the continuous state right after the last discrete jump. We would like to model continuous dynamics of the form:

x = F q; x; x ; u _ q0 2 ID x0 2 IC

Consider the hybrid automaton given by: X = XD  XC  XC

f q; x; p; u = F q; x; p; u 0 T E  fq; x; p; u; x; pg S  fq; x; p; u; x; xjq; x 6= q0 ; x0 g hq; x; p; u = q; x; p The e ect of x on the discrete evolution is also easy to model by further restricting E . 2 Finally, by appropriately de ning the boundary of E , one is able to model autonomous jumps i.e.

U Y =X I = ID  Sx2IC x; x

situations where the state has to jump to a new value and controlled jumps i.e. situations where the system can be forced to jump by the input u5 .
switching i.e. situations where the di erential equation can be forced to change by the input are special cases of autonomous and controlled jumps respectively
5 In 7 it is shown that autonomous switching i.e. situations where the di erential equation has to change and controlled

13

Remark: note that it is easy to model continuous control systems in this framework. For example,
consider a standard control system on real variables, with n states m inputs and p outputs, de ned by:

xt = F xt; ut _ x0 = x0 yt = H xt; ut

It can be modeled by a hybrid automaton with a single discrete state call it Q de ned by:

X U Y I f

Rm Rp Q; x0  X  U ,! Rn f Q; x; u = F x; u E = Q; x; U; Q; x h : X  U ,! Y hQ; x; u = H x; u

x2Rn

= = = = :

fQg  Rn

The set E is essentially the graph of the identity map on X for each values of u. In the sequel we will omit the discrete state when writing hybrid automata corresponding to standard control systems. This is done to simplify the notation somewhat. For more complicated examples of modeling using hybrid automata the reader is referred to Chapters 5 and 6. If the discrete state can assume a nite number of values it is very convenient to represent the hybrid automaton by a nite graph. We can associate a nite graph to a given hybrid automaton H using the following construction.

2.1.4 Graphical Representation

Construction 3 Graph Generation: Nodes: the number of nodes in the graph will be equal to the number of possible values of the discrete state. The nodes will be indexed by a corresponding discrete state value, q 2 XD . Continuous Evolution: to each node, q, we associate a vector eld, fq : fq : XC  U ,! TXC 2.12 x; u 7,! f q; x; u
The implication is that while in the node q the continuous state evolves according to fq . Node Invariants: To each node, q, we associate an invariant: Invq = fx; ujx 2 XC ; u 2 U; q; x; u; q; x 2 E g XC  U The interpretation is that the system can remain in node q if and only if x; u 2 Invq . 14 2.13

(x,u) Enqq

qq

q .
x = fq (x,u) (x,u) Inv q

....

.
x = fq (x,u) (x,u) Inv q

x := x Re (x,u) qq

....

Figure 2.1: Hybrid automaton graph nodes

Transition Guards: To the transition from node q to node q0 we associate a guard: Enqq0 = fx; ujx; x0 2 XC ; u 2 U; q; x; u; q0 ; x0  2 E g XC  U 2.14 The interpretation is that the transition can take place if and only if x; u 2 Enqq0 . Transition Event: To the transition from node q to node q0 we associate an event qq0 see construction

1. The interpretation is that the event occurs whenever the transition takes place. Transition Reset: To the transition from node q to node q0 we associate a set valued map: Resqq0 x; u = fx0 jx0 2 XC ; q; x; u; q0 ; x0  2 E g XC 2.15 The interpretation is that if the transition takes place from x; u then after the transition the state nds itself in q0 ; x0  with x0 2 Resqq0 x; u. 2 The above construction allows us to represent a hybrid automaton graphically as shown in Figure 2.1. Note that there is no requirement that q 6= q0 , i.e. loops to the same node are allowed. In ensuing chapters some of the elements of the graph may be omitted to simplify the gures. The interpretation will be that a missing invariant or guard is equal to XC  U while a missing reset map is the identity in x, i.e. for all u 2 U: Resqq0 x; u = fxg We will not worry too much about missing events.

2.1.5 Operations on Hybrid Dynamical Systems

We will only de ne three operations on hybrid dynamical systems: interconnection, renaming and hiding. Interconnection will allow us to form new hybrid systems out of collections of existing ones, renaming will allow us to connect systems with their subsystems while hiding will hide some outputs of a given hybrid system from the rest of the world. Let fHi gN be a collection of hybrid automata, Hi = fXi ; Ui ; Yi ; Ii ; fi ; Ei ; hi g. We can write the inputs i=1 and outputs in vector form as:
2

ui = 6 ... 7 2 Ui yi = 6 4 5 4 ui;mi
Let:

ui;1

yi;1 yi;pi
.. .

3 7 5

2 Yi

^ U = f1; 1; 1; 2; : : : ; 1; m1 ; 2; 1; : : : ; 2; m2  : : : ; N; 1; : : : ; N; mN g ^ Y = f1; 1; 1; 2; : : : ; 1; p1 ; 2; 1; : : : ; 2; p2 ; : : : ; N; 1; : : : ; N; pN g De nition 7 An interconnection, I , of a collection of nite automata, fHigN , is a partial map: i=1 ^ ^ I : U ,! Y 15

An interconnection of hybrid automata can be though of as a pairing ui;j ; yk;l  of inputs and outputs. An interconnection is only a partial map i.e. some inputs may be left free, need not be surjective i.e. some outputs may be left free and need not be injective i.e. an output may be paired with more than one input. ^ Let PreI  be the subset of U for which the partial map I is de ned. Also let  denote the projection of a vector valued quantity to the element with index . De nition 8 Given a collection of hybrid automata fHigN and an interconnection I , the symbolic oper1 ation substitution, denoted by ;, assigns to each input, ui;j a map on X1  : : :  XN  U1  : : :  UN , according to:

ui;j ;

ui;j if i; j  62 PreI  hI i;j : X1 I i;j   U1 I i;j  ! Y1 I i;j  if i; j  2 PreI 

Operation ; can be repeatedly applied to the right hand side by appropriate map compositions. The construction terminates for each ui;j if the right hand side either contains ui;j itself or contains only uk;l 62 PreI . The resulting map will be denoted by ui;j ;.

Because there are a nite number of inputs, the construction of ui;j ;  terminates in a nite number of steps. To ensure that an interconnection is well de ned as an operation between hybrid automata we impose the following technical conditions: De nition 9 An interconnection, I , of a collection of hybrid dynamical systems, fHigN , is well posed i=1 if: 1. For all i; j  2 PreI , YI i;j  Ui;j , 2. There are no algebraic loops, i.e. for all i; j  2 PreI  the map ui;j ;  does not involve ui;j . These requirements imply that ui;j ;  is well de ned as a map between the following spaces: ui;j ;  : X1  : : :  XN  U nPreI  U1  : : :  UN  ,! i;j U1  : : :  UN  ^

Lemma 1 The posed interconnection, I , of a collection of hybrid dynamical systems, fHi gN , de nes a i=1
new hybrid dynamical system.

Proof: Let H = fX; U; Y; I; f; E; hg denote the interconnection automaton, de ned by: X = X1  : : :  XN U = U nPreI U1  : : :  UN  ^ Y = Y1  : : :  YN I = I1  : : :  IN
f = fi ui ;  N i=1 E X  U  X with e = q; x; u; q0 ; x0  2 E if and only if for all i = 1; : : : ; N :
qi ; xi ; ui ; q; x; u; qi0 ; x0i  2 Ei 16

h = hi ui ;  N i=1

The expression ui ;  denotes the map generated by applying ; to the elements ui;1 ; : : : ; ui;mi . The terms in square brackets have the obvious interpretation as vectors. The symbol denotes composition of maps. By a slight abuse of notation, we will refer to H itself as the interconnection of fHi gN . i=1

Y = Y1  : : :  Yp and variables ui 2 Ui ^ ^

with the same state and output spaces, input space: ^ ^ U = U1  : : : Ui,1  Ui  Ui+1  : : :  Um

De nition 10 Consider a hybrid automaton H with input space U = U1  : : :  Um and output space Ui and yj 2 Yj Yj . Renameui !ui H  is a new automaton ^ ^ ^

^ ^ ^ the same initial condition set and f; E and h equal to the restrictions of the corresponding quantities of ^ H to U . Similarly, Renameyi !yi H  is a new automaton with the same state and input spaces, output ^ space: ^ ^ Y = Y1  : : : Yj ,1  Yj  Yj+1  : : :  Yp and the same dynamics. This operation can be used when a hierarchy of hybrid automata is constructed to identify inputs and outputs of systems and their subsystems. De nition 11 Given a hybrid automaton H with output space Y = Y1  : : :  Yp and an output yi taking values in Yi the operation Hideyi H  produces a new hybrid automaton, with the same state and input spaces, output space:

Y1  : : :  Yi,1  Yi+1  : : : Yp the same dynamics as H and output map h1 : : : hi,1 hi+1 : : : hm T .

This operation can be used when hybrid automata are interconnected to form agents to hide certain local features of an agent from the rest of the world.

2.2.1 Hierarchy of subsystems

2.2 Agent Model

Each agent will be modeled as an interconnection of hybrid dynamical systems. The intuition is that each subsystem will be used to describe a distinct functionality of the agent. Typically an agent will contain subsystems modeling the plant dynamics, the sensors, the actuators, the continuous and discrete controllers and the communication devices. For the purpose of modeling multiagent systems we will distinguish three kinds of inputs for each subsystem: 1. Control inputs, that can be speci ed locally i.e. within the agent by interconnections to the outputs of other subsystems. They re ect the actions that the agent may decide to take. 2. Environmental inputs or disturbances, that can not be locally speci ed and re ect actions of the environment such as sensor noise or the e ect of unmodeled dynamics or actions of other agents. 3. Coordination inputs, that are used for interagent cooperation, for example through communication protocols. 17

Coordination Inputs

Local Inputs -State (Continuous, Discrete) -Hybrid Dynamics -Subystems Interconnections Environment Inputs

Outputs

Figure 2.2: An agent subsystem A typical subsystem input-output arrangement is shown in Figure 2.2. This classi cation of inputs is motivated by our approach to the control of multiagent systems which is based on semiautonomous agent operation. This approach, described in Chapter 3, results in a controller hierarchy. At the low levels control actions are taken locally by each agent. At this level the actions of all other agents can be viewed as environment disturbances over which the agent has no control. The agents coordinate at the higher levels. Here each agent can in uence the actions of other agents, although, in general, no agent will be in full control of another. As a result the input output interactions at this level are neither in the form of pure control inputs, nor in the form of pure disturbances. It should be noted that this partitioning of the inputs is purely an artifact of our attempt to capture the multi agent nature of the system; the inputs are indistinguishable from the hybrid automata point of view. The agent model will itself be a dynamical system of the form shown in Figure 2.2. Within it, it may contain an entire hierarchy of interconnected subsystems. The agent inputs and outputs will be connected to the subsystem inputs and outputs according to a set of rules: system input and subsystem local input system output and subsystem output subsystem local input and subsystem output system parameter and subsystem input subsystem parameter and subsystem input The couplings between subsystems can be implemented with the interconnection operation. The coupling between system and subsystem inputs and outputs can be implemented by renaming. It is assumed that any redundant subsystem outputs are hidden once the agent is formed. The dynamic evolution and the subsystem interconnection rules are de ned so that the agent model can be attened" into a single equivalent hybrid system. As before, let q; x 2 X denote the state of the agent discrete and continuous. Also let u 2 UD  UC denote the control inputs and d 2 DD  DC denote the environmental inputs. Because in subsequent chapters we will concentrate more on the design of the low level controllers we will not use special symbols for the coordination inputs. 18

2.2.2 Hybrid Automaton Model

The dynamics of the attened agent model over a set of times of interest T = ti ; tf will be determined by a set of relations of the form: qti ; xti  = q0 ; x0  2 I  xt = f qt; xt; ut; dt _ qt; xt; ut; dt; qt; xt 2 E q; x; u; q0 ; x0  2 E for jumps q; x ! q0 ; x0  yt = hxt; ut; dt Initial Condition continuous evolution discrete evolution output evolution

Physical considerations such as actuator saturation impose certain restrictions on the system evolution. We encode these restrictions in terms of constraints on the state, input and disturbance trajectories. q; x 2 Q  X PC T ; XD   PC 1 T ; XC  u 2 U PC T ; U  d 2 D PC T ; D 2.16 2.17 2.18

PC T ;  denotes the set of piecewise continuous maps from T to , whereas PC 1T ;  the set of piecewise

di erentiable maps. The de nitions are straightforward generalizations of the corresponding de nitions for T = ti ; tf . We will use the symbols PC and PC 1 to denote these sets whenever there is no ambiguity about the domain and range. Of particular interest for continuous variables are constraints that can be encoded by requiring that the variable lies in a certain set for all times, i.e. for all t 2 T :

xt 2 X Rn ut 2 U Rm dt 2 D Rp

It should be noted that this class of constraints excludes certain important cases such as non-holonomic and isoperimetric" constraints. In its most general form a hybrid design formulation will also have to account for constraints like these. For discrete variables pure value" constraints play less of a role. A discrete variable can be though of as a piecewise constant function of time multi valued at the transition points. Two aspects of such a function are of interest: The order in which the values are observed. The times at which the function jumps from one value to the next. The constraint set will impose limits on these two aspects of the evolution of the discrete variables. It should be noted that both these constraints can be encoded by the requirement that the discrete variable sequence is generated accepted by a timed automaton 35 . We envision a design, simulation and veri cation environment for multi-agent hierarchical hybrid control systems as shown in Figure 2.3. The speci cations are described by the desired emergent behavior of the collection of agents. These are simple requirements usually described linguistically: increased throughput and safety and reduced emissions for Automated Highway Systems, increased frequency of landings and takeo s and optimum utilization of air space for Air Tra c Management Systems, etc. 19

2.2.3 Multiagent Design and Veri cation Environment

Experiments

Description Language for Multiple Agent Hybrid Systems

DESIGN SPECS

DESIRED EMERGENT BEHAVIOR

DESIGN Air Traffic System (ATMS) SIMULATION ATMS

Preprocessor - Coordination Protocols - Game Theoretic Design Tools

ATMS
Architectures: -Current ATC -Free Flight

VERIFICATION

- MacroSimulation -Adaptation & Learning -Operation under Degraded Conditions -Robustness to Faults

DECISION Module

VALIDATION Automated Highway System (AHS)

AHS
Architectures: -Platooning -Autonomous

SAFETY EFFICIENCY COMFORT COST ENV IMPACT

AHS
Preprocessor

-Modularity -Reasoning with Uncertainty

Physical Systems

Simulation Visualization Validation

Preprocessing/ Customization

System Design

Architecture Concepts

Evaluation Module

Figure 2.3: Multi-Agent Hybrid System Design Environment The requirements have to get parsed into a system architecture. Designing a control architecture involves decomposing the system into a subsystem hierarchy, specifying the subsystem interconnections and determining the limits of the environmental inputs. We will concentrate on partially decentralized architectures, with coordinating semi-autonomous agent operation. The controller of each agent is described by a multi layer hierarchy. The higher layers are typically modeled by discrete event systems and take strategic decisions in coordination with other agents. The lower layers, on the other hand, typically involve continuous dynamics and performing path planning and regulation tasks. The control laws at di erent layers along with the inter-agent coordination schemes are to be designed using design tools, in order to satisfy the speci ed properties such as safety, productivity, e cient resource utilization, etc. The design tools may be conventional discrete and continuous tools as well as specialized tools for hybrid control. We will present examples of specialized tools in the next chapters. Once the control laws for individual agents are designed, we would like to be able to abstract their detailed behavior so that the collective emergent characteristics can be evaluated using tools such as macro-simulation. The process can be customized to a speci c application, such as Automated Highway Systems AHS or Air Tra c Management Systems ATMS. As an example, consider the case of AHS. The hybrid system description language introduced above can be customized to the speci c application domain as shown in 46 . Then controllers for the various maneuvers required on the AHS can be designed to satisfy safety, comfort and e ciency requirements using the tools presented in Section 3. To get to the emergent behavior, the various maneuver control laws of an automated vehicle can then be abstracted into spacetime requirements. This information can be used in a tra c ow model to evaluate the throughput of the highway. The detailed designs can also be converted into a generic multi-agent hybrid system simulation language and simulated using a micro-simulator. One such microsimulator SmartAHS is being built for AHS applications at Berkeley. Note that the simulation program should be able to handle dynamically 20

changing inter-connections between di erent agents. A language for describing such a dynamically changing network of hybrid systems, called SHIFT, is described in 47 . SHIFT adds the functionality to dynamically create and destroy new components and change input output connections as part of the reset actions. Speci cations in SHIFT can be veri ed or validated using hardware-in-the-loop experiments.

2.3 Mathematical Tools

The analysis presented in subsequent chapters will require concepts from the areas of game theory, optimal control, dynamical systems and topology. We provide the minimum material necessary to follow the arguments. For more details the reader will be referred to standard references for each eld.

2.3.1 Game Theory

We will be dealing the following type of two player, zero sum games: De nition 12 A two player, zero sum dynamic game involves: 1. A time interval 0; T . 2. A trajectory space, X with some topological structure. The elements of X , denoted fxt; 0  t  T g, are the permissible state trajectories of the game. 3. An input space, U , with some topological structure. The elements of U , denoted fut; 0  t  T g, are the permissible inputs of player 1. 4. An disturbance space, D, with some topological structure. The elements of D, denoted fdt; 0  t  T g, are the permissible inputs of player 2. 5. A di erential equation:

xt = f t; xt; ut; dt _ x0 = x0

whose solution determines the state trajectory for a given selection of u and d. 6. A cost function:

2.19 2.20

J : X  U  D ,! R+

2.21

To be consistent with the notation we develop for the multiagent models we will give the following interpretation: the reward" of player 1 for a given play is ,J x; u; d while the reward of player 2 is J x; u; d hence a zero sum game. In other words, player 1 is trying to minimize J while player 2 is trying to maximize it. For our games we will assume the so called closed-loop, perfect state information structure. This means that, when called upon to decide their strategy at time t, both players have access to the entire state fxs; 0  s  tg up to that point. For our purposes we will assume that xt 2 X Rn , ut 2 U Rni and dt 2 D Rnd for all t 2 0; T and some n; ni and nd . We will also assume that X will be a subset of the set of piecewise di erentiable and U and D subsets of the set of piecewise continuous functions of t. Note that, if the di erential equation 2.19 de nes a unique state trajectory for a given choice of u and d,

21

we can write the cost function as a function of the initial condition, rather than the whole state trajectory, i.e.

J : X  U  D ,! R x0 ; u; d 7,! J x0 ; u; d

We will be interested in saddle solutions to these games: De nition 13 A saddle solution to the two player, zero sum game is a pair of input trajectories u ; d  2 U  D such that for any u 2 U and any d 2 D:

J x0 ; u ; d  J x0 ; u ; d   J x0 ; u; d 

In other words, a saddle solution will be such that any unilateral deviation from it will leave the player who decided to deviate worse o . The games considered in the examples of this dissertation will turn out to have unique saddle solutions. Existence and uniqueness of solutions can not be guaranteed for general games, however. For results in this direction the reader is referred to 48, 49, 50 . The saddle" is not the only solution concept of interest in dynamic games. Other types of solution can be de ned Nash, Stackelberg, etc.. The solution concept will depend, among other things, on the nature of the game discrete vs. continuous, deterministic vs. stochastic, the number of players and the information structure. The saddle solution used here is the simplest solution concept and applies only to two player games. For a thorough treatment of dynamic games, solution concepts and applications the reader is referred to 49, 50, 51 .

2.3.2 Optimal Control

From optimal control we will make use of the Maximum Principle 52 . The following statement comes from 53 , with a slight modi cation in notation. Given the dynamical system on Rn :

x = f x; u _
consider the problem of steering it, using a piecewise continuous ut, from the point P at time t1 to the point Q at time t2 along a trajectory that minimizes:
Z

J x0 ; u =
function pt such that:

t2 t1

Lx; udt

Theorem 1 6 A necessary condition for a trajectory xt and the corresponding control ut, 0  t  T to be optimal is that there exist a constant p0  0 and a vector valued, piecewise continuously di erentiable
1. pt and p0 not both 0 for any t 2. At t; xt; pt; p0 , the function of u:

~ H x; p; p0 ; u = p0 L + p0 f
examples.
6 In fact the theorem holds under more general conditions e.g. time varying dynamics but this version is su cient for our

assumes its maximum, H t; x; p; p0 , for u = u t.

22

3. For the arguments t; xt; pt; p0 ; u t the Euler equation holds: ~ ~ x = @H p = , @H _ _

@p

@x

4. The end points satisfy the transversality condition:

where x is a variation in state and t is a variation in time about the optimal trajectory. In the examples we will only need to consider a special case of the transversality condition. In particular, assume that the initial point P = x0 and time t1 = 0 are xed and that the nal point and time can vary, with the restriction that xt2  2  where  is a hyperplane. In this setting the last condition of the theorem becomes: H = 0 at time T 2.22 pt2  normal to  2.23

p x,H tQ =0 P

We will also need some very basic concepts from dynamical systems and topology: invariant sets, interior and boundary of a set, induced topology of a subset, etc. These concepts are covered in the standard textbooks e.g. 54, 55, 56 . Here we will only show that the state space X of a hybrid automaton is a metric space. Recall that X = XD  XC . Let: mC : XC  XC ,! R be the metric induced on XC by the Euclidean metric in Rn . mC satis es the standard metric axioms, i.e. for all x; y; z; 2 XC : mC x; y  0 and mC x; y = 0  x = y mC x; y = mC y; x mC x; y  mC x; z + mC z; y For the discrete states we will use the metric that induces the discrete topology, namely: mD : XD  XD ,! R  0 if q1 = q2 q1 ; q2  7,! 1 if q1 6= q2 Clearly mD satis es the metric axioms. Together with mC they de ne a metric on X by: m : X  X ,! R q1 ; x1 ; q2 ; x2  7,! mD q1 ; q2  + mC x1 ; x2  It is easy to check that m also satis es the metric axioms. The metric can be thought of as the distance between two points in X . This notion can be extended to sets M1 ; M2 X by de ning: mM1 ; M2  = inf inf mq1 ; x1 ; q2 ; x2 
q1 ;x1 2M1 q2 ;x2 2M2

2.3.3 Dynamical Systems and Topology:

23

Chapter 3

Controller Design
The design of hybrid controllers can be very subtle. The di culties that arise at the interaction between the discrete and continuous components are mainly due to inadequate abstraction of the continuous layer performance at the discrete level. This may result in the discrete controller issuing commands that are incompatible with the state of the continuous system. The results may be catastrophic see Chapter 6 for examples. We will try to deal with this di culty by using game theory to generate continuous controllers and consistent discrete abstractions for the resulting closed loop system.

3.1 Discrete Layer

Most of this dissertation will be devoted to the design of the continuous part of the controller and the subsequent extraction of appropriate discrete abstractions. In this section we give a brief outline of the issues involved in the discrete design and state the assumptions we make about it. The overall goal of the design is to produce a control scheme that will produce a certain desired behavior, given a model of the process and a description of the desired behavior. Our work indicates that two phases can be distinguished in the design process: a top down phase and a bottom up phase. In the topdown phase, the desired emergent behavior is parsed to a discrete design and from there to performance requirements on the continuous controllers. The technique presented in this chapter can then be used to design the low level controllers. The design process will indicate further requirements that the discrete layer needs to satisfy. In the bottom-up phase the discrete design is modi ed to address these requirements and then analysis is carried out to determine the emergent behavior of the combined system and compare it to the original speci cations. Roughly speaking, the top-down phase aims at squeezing as much as possible out of the system while the bottom-up phase acts as a reality check on the original requirements. In this chapter we will only address the problems associated with turning the corner", i.e. with the last part of the top down and the rst part of the bottom-up phases. Given a preliminary discrete design we produce continuous controllers and suggestions on how the discrete design needs to be modi ed to ensure that the closed loop hybrid system satis es the performance speci cations. The questions of obtaining the preliminary discrete design from the desired emergent behavior, re ning it so that the suggestions emerging from the continuous design are met and evaluating the closed loop system performance are topics of further research. Here we only note that, with the discrete abstractions that result from our algorithm the re nement of the discrete design is a purely discrete problem. This step could therefore be carried out by any of the standard discrete tools 8, 57 . Moreover, if a discrete design is given it is easy to check that it satis es the requirements made by the continuous level automatically. 24

3.1.1 Design Phases

Within each agent, the discrete design will be abstracted for the bene t of the lower levels in terms of three quantities: d A sequence of desired way points yj , j = 1; 2; : : :, that should be tracked. For each way point, a set of design speci cations Ji ; Ci , i = 1; : : : ; N . These are pairs of cost functions: Ji : PC  PC 1  PC  PC ,! R 3.1 mapping a run q; x; u; d of the agent automaton to a real number, and thresholds Ci 2 R. An acceptable trajectory must be such that Ji q; x; u; d  Ci for all i = 1; : : : ; N We assume that the design speci cations are ordered in the order of decreasing importance. Qualitatively, the most important cost functions encode performance aspects such as safety, while the least important ones encode performance aspects such as resource utilization. The design should be such that the most important constraints are not violated in favor of the less important ones, in other words the design should lead to Ji  Ci whenever possible, even if this means that Jj Cj for some j i. The cost functions can penalize" various aspects of the system runs. A typical cost function will involve a combination of the following elements: 1. Continuous evolution costs, i.e. costs associated with the evolution of the continuous state under the vector eld f . These are the kinds of costs usually encountered in optimal control 52, 53 . A special case of particular interest is when each pair of inputs u; d generates a unique state trajectory for a given initial condition q0 ; x0 . Then the cost function can be thought of as a map: Ji : I  PC  PC ,! R 3.2 When the discrete evolution of hybrid automaton may be nondeterministic, it may not be possible to obtain Ji in the form of equation 3.2. 2. Discrete evolution costs, i.e. costs associated with jumps of the states, inputs and disturbances from one value to another. For example, if a jump in the state takes place at time the cost associated with it may be thought of as a function: Ji : X  U  D  X ,! R that maps q ; x ; u ; d ; q 0 ; x 0  2 E to a real number. Similar costs can be associated to input jumps: Ji : U  U ! R and disturbance jumps: Ji : D  D ! R As discussed above, the question of how emergent behavior objectives, which are usually given linguistically, get parsed to way points, cost functions and thresholds is the topic of further research.

3.1.2 Discrete Layer Abstraction

3.2 Continuous Layer

We present a technique for systematically constructing controllers which carry out the objectives set by the discrete layer and are optimal with respect to the given cost functions. In this section we restrict our attention to cost functions associated with the continuous state evolution and in particular functions of the form 3.2. This restricted class of functions will su ce for all the examples considered in our work. We will also assume that the entire state of the plant is available for feedback. 25

3.2.1 Multiobjective Controller Design Algorithm

At the rst stage we treat the design process as a two player, zero sum, dynamic game with cost J1 . One player, the control u, is trying to minimize the cost, while the other, the disturbance d, is trying to maximize it. Assume that the game has a saddle point solution, i.e. there exist input and disturbance trajectories, u and d such that: 1 1

J1 q0 ; x0  = max min J1 q0 ; x0 ; u; d d2D u2U = min max J1 q0 ; x0 ; u; d u2U d2D = J1 q0 ; x0 ; u ; d  1 1
Consider the set: This is the set of all initial conditions for which there exists a control such that the objective on J1 is satis ed for the worst possible allowable disturbance and hence for any allowable disturbance. u can now be used as a control law. It will guarantee that J1 is minimized for the worst possible 1 disturbance. Moreover if the initial state is in V1 it will also guarantee that the performance requirement on J1 is satis ed. u however does not take into account the requirements on the remaining Ji 's. To include 1 them in the design let: U1 q0; x0  = fu 2 UjJ1 q0 ; x0 ; u; d   C1g 3.3 1 Clearly:  0 ; x0  = ; for q0 ; x0  62 V1 U1 q 6= ; for q0; x0  2 V ; as u 2 U q0 ; x0 
1 1 1

V1 = fq; x 2 X jJ1 q; x  C1 g

The set U1 q0 ; x0  is the subset of admissible controls which guarantee that the requirement on J1 is satis ed, whenever possible. Within this class of controls we would like to select the one that minimizes the cost function J2 . Again we pose the problem as a zero sum dynamic game between control and disturbance. Assume that a saddle solution exists, i.e. there exist u and d such that: 2 2

J2q0 ; x0  = max u2Umin;x0 J2 q0 ; x0 ; u; d d2D 1 q0 = min max J q0 ; x0 ; u; d u2U q0 ;x0  d2D 2
= J2 q0 ; x0 ; u ; d  2 2
1

Consider the set:

V2 = fq; x 2 X jJ2 q; x  C2 g As the minimax problem only makes sense when U1 q0 ; x0  6= ; we assume that V2

the set of initial conditions for which there exists a control such that for any allowable disturbance the requirements on both J1 and J2 are satis ed. To introduce the remaining cost functions to the design we again de ne: U2 q0; x0  = fu 2 U1q0 ; x0 jJ2 q0 ; x0 ; u; d   C2g 3.4 2 i.e. the subset of admissible controls that satisfy the requirements on both J1 and J2 for any disturbance. The process can be repeated for the remaining cost functions. At the i + 1st step we are given a set of admissible controls Ui q0 ; x0  and a set of initial conditions Vi such that for all q0 ; x0  2 Vi there exists 26

V1 . V2 represents

(q,x) VN-1\V

u = u*

2 (q,x) V (q,x) VN-1\V (q,x) V

u = u*

(q,x)

(q,x) X\VN-1

(q,x) X\VN-1

u = u*

Figure 3.1: Controller automaton for switching between control objectives

u 2 Ui q0 ; x0  that for all d 2 D and for all j = 1; : : : ; i leads to Jj q0 ; x0 ; u; d  Cj . Assume the two i i player, zero sum dynamic game for Ji+1 has a saddle solution, u+1 ; d+1 : i i Ji+1 q0 ; x0  = max u2Umin;x0  Ji+1 q0 ; x0 ; u; d d2D i q0 = min max J q0 ; x0 ; u; d u2U q0 ;x0  d2D i+1
= Ji+1 q0 ; x0 ; u+1 ; d+1  i i
i

De ne: and:

Vi+1 = fq; x 2 X jJi+1 q; x  Ci+1g

3.5 The process can be repeated until the last cost function. The result is a control law u and a set of initial N conditions VN = V such that for all q0 ; x0  2 V , for all d 2 D and for all j = 1; : : : ; N , Jj x0 ; u ; d  Cj . N

Ui+1q0 ; x0  = fu 2 Uiq0 ; x0jJi+1 q0; x0 ; u; d+1   Ci+1 g i

3.2.2 Controller Automaton

The controller can be extended to values of the state in the complement of V using the following switching scheme: 8 u q; x q; x 2 V N  q; x = u ,1 q; x q; x 2 VN ,1 n V N u 3.6 ::: ::: : u q; x q; x 2 X n V2 1 Therefore, even for a single set point, the resulting controller given by equation 3.6 involves switching due to multiple objective functions Ji . It therefore has to be implemented by a hybrid automaton. An example of such an automaton in a three cost function situation is shown in Figure 3.1. This procedure has to be repeated for each set point provided by the discrete layer.

27

3.2.3 Technical Issues

The above algorithm is sound in theory but can easily run into technical di culties when applied in practice: 1. There is no guarantee that the dynamic games will have a saddle solution. 2. There is no straight-forward way of computing Ui x0  3. There is no guarantee that the sets Vi and consequently Ui x0  will be non-empty. Fortunately none of these issues will turn out to be a problem in the examples considered here. In fact, even in the most complicated cases of Chapter 6, a solution can be obtained analytically, or using simple numerical calculations. We can not hope for such luck in general however. New and sophisticated optimal control tools 58 will hopefully make the solution of more general problems feasible, at least numerically.

3.2.4 Interface and Discrete Design Revisited

The sets V are such that for all initial conditions in them all requirements on system performance are guaranteed. These sets impose conditions that the discrete switching scheme needs to satisfy. The discrete layer should not issue a new command encoded by a way point if the current state does not lie in the set V for the associated controller. Essentially, these sets o er a way of consistently abstracting performance of the continuous layer. It should be noted that, by construction, the sets Vi are nested. Therefore there is a possibility that an initial condition lies in Vi for some i = 1; : : : ; N 0 N but not in V . This implies that certain requirements on the system performance e.g. safety can be satis ed, while others e.g. e cient resource utilization can not. This allows the discrete design some more freedom. For example, a new command may be issued if it is dictated by safety, even though it violates the requirements of e ciency. This construction provides a convenient way of modeling gradual performance degradation, where lower priority performance requirements are abandoned in favor of higher priority ones. It can be particularly useful for operation under degraded conditions, for example in the presence of faults. The overall continuous design including the interface is shown in Figure 3.2. Switching of continuous controllers takes place at two levels. Assume that the discrete layer speci es two set points and two objectives, for example safety and e ciency. For each set point, the game theoretic framework will produce optimal controllers for safety u and e ciency u along with the safe sets of initial conditions1 V1 and V2 . 1 2 As illustrated above, switching between these two controllers will be carried out depending on the current value of the state represented in the gure as an input u3 to the switching scheme. After receiving the new set point command from the discrete layer, the interface will switch to a new controller if the system state belongs to the corresponding safe set. If this requirement is not satis ed, and the discrete controller insists on the new way point, the discrete layer needs to coordinate with other agents. This type of coordination is what makes the operation of the agents semi-autonomous as opposed to completely autonomous. It can be viewed as a way of restricting the domain, D, of the disturbances. This biases" the game in favor of the control input, hence enlarges the set of safe" states, V , and hopefully makes the transition to the new way point feasible. A more abstract view of the e ect of coordination in this setting is as a way of turning the zero sum, non-cooperative game to a game with partial cooperation between the players. In extreme cases for example presence of faults a fully cooperative game may be needed to salvage the situation as will be seen in the algorithms proposed in Chapter 7.
1 The controllers and safe sets may actually depend on the set point, as will be illustrated in Chapter 6. To avoid additional

subscripts we will ignore this dependency in the notation.

28

DISCRETE LAYER Command B Set Point A Set Point B

Command A

Coordination with other agents

CONTINUOUS LAYER

Safety * u1

Performance Objective Switching u1 u3 Discrete Commands

Efficiency * u2

u2 Plant Dynamics Set point switching

Controller for set point A u1 u3 u2 Safety * u1 Performance Objective Switching u1 u3 u2 Safe Sets VA VB u y=x

INTERFACE PLANT

Efficiency * u2

Controller for set point B

Figure 3.2: Hybrid controller for a single agent

3.3 The Train-Gate Example

To illustrate how this design methodology can be useful in applications we consider a classic example from the timed automata veri cation literature, the train gate controller". This example is ideal for our purposes because it can be easily cast into the game theoretic framework and the equations are simple enough for the complete analysis to be carried out by hand. The train gate set up is shown in Figure 3.3. We will work on the problem formulation of 59 . For simplicity we assume that the train is going around on a circular track of length L, where L is large enough to ensure adequate separation between consecutive train appearances at the gate. This assumption will be discussed further in Section 3.3.3. We will also assume that the train can be approximated by a point. It is easy to extend the analysis to trains of nite length.

3.3.1 Problem Statement

The Train: The train moves clockwise around the track. Let:
L; L x2 2 , 2 2 x2 2 v1 ; v2 _
29


denote the position of the train, with the implicit assumption that x2 wraps around at L=2. The details of the train dynamics are abstracted away by assuming that the train velocity is bounded, i.e.:

X S

1 Gate

1 S Road 2

2 Train

Track -L/2 L/2

Figure 3.3: The train-gate set up From the analysis it will become apparent that in order to guarantee that the problem is well de ned we need to assume that: 0  v1  v2 1

The Gate: The crossing is located at position x2 = 0 on the train track. It is guarded by a gate that,
when lowered, prevents the cars from crossing the tracks. Let:

x1 2 0; 90
denote the angle of the gate in degrees. Assume that the gate dynamics are described by the rst order di erential equation: 1 x1 = , 2 x1 + u _ where u is the input to be chosen by the designer of the gate controller.

The Sensor: The design of 59 is based on discrete sensor measurements. We will assume that there are

two sensors located at distances S1 and S2 respectively on the track. The sensor at S1 detects when the train is approaching the crossing, while the one at S2 detects when it has moved away. From the analysis it will become apparent that in order for the control problem to have a solution we will need to assume that: , L S1 0 S2 L 2 2

Speci cations: Two requirements are imposed on the design: safety and throughput. For safety it is
required that the gate must be lowered below a certain threshold whenever the train reaches the crossing. This can be encoded as: x2t = 0  x1 C1 30

The value C1 = 10 will be used in subsequent analysis. For throughput it is required that the gate should be opened whenever it is safe to do so. This is done to maximize the number of cars that get to cross the tracks. The problem can immediately be cast in the game theoretic framework. The plant can be modeled by a hybrid automaton2 with one discrete state omitted in the notation and two continuous states x = x1 x2 T 2 R2 . The two players inputs are the gate controller, u and the train speed disturbance, d. The continuous dynamics are linear in the state and a ne in the two inputs:

3.3.2 Game Theoretic Formulation

x = _

x0 2 I = fx0 g
The state is constrained to lie in the set X:

,1=2 0 x + 1 u + 0 d 0 0 0 1

3.7 3.8

 x 2 X = fx1 ; x2  2 R2 jx1 2 0; 90 ; x2 2 , L ; L g R2 2 2 with the understanding that x2 wraps around at L=2. The input and disturbance are constrained to lie in

U and D respectively:

u 2 U = 0; 45 R d 2 D = v1 ; v2 R
With a slight abuse of notation bringing the second copy of X to the front the set E is:

3.9 3.10 3.11

E=

x2X

x; x  U  D

The analysis will reveal that the dynamics and input constraints automatically guarantee the state constraints. The two players compete over two cost functions J1 and J2 . J1 encodes the requirement for safety. Let:

T x0  = minft  0jx2 t = 0g

3.12

be the rst time that the train reaches the crossing. Then the requirement for safety can be encoded by the cost function: J1 x0 ; u; d = x1 T x0   C1 3.13 The requirement for throughput can be encoded by a number of cost functions. A simple one is:

J2 x0 ; u; d =

Minimizing J2 implies that the gate is open for as long as possible. We need not set an explicit bound on J2 for acceptable performance, so we assume C2 = 1.
and one for the gate. The system is simple enough that we do not need to get into the details of this exercise.
2 The automaton can be thought of as the attened version of the interconnection of two agent automata, one for the train

90 , x1 t2 dt

31

The system dynamics given in equation 3.7 are simple enough to allow us to write an analytic expression for the value of the state at time t:

x1 t = e,t=2 x0 + 1 x2 t = x0 + 2

Z

Lemma 2 If x0 2 X and the input constraints of 3.9 are satis ed, xt 2 X for all t  0. Proof: Under the wrap around assumption on x2 the only thing we need to show is that x1 t 2 0; 90 for all t  0. Indeed:
Z ,t=2 x0 + t e,t, =2 u d x1 t = e 1 0Z t  e,t=2 x0 + 45 e,t, =2 d 1 0 ,t=2 x0 + 901 , e,t=2  = e 1 ,t=2 + 901 , e,t=2   90e

d d

t ,t, =2 e u

d

3.14 3.15

x1 t  e,t=2 x0 1

= 90

 0

2
Our rst goal is to nd the safe set of initial conditions and the control that makes them safe. In other words we are looking for x0 2 X and u with u t 2 U such that for all d satisfying dt 2 D, J1 x0 ; u ; d  C1 . 1 1 1 Fortunately the dynamics are simple enough to allow us to guess a candidate saddle solution:

3.3.3 Design for Safety

u t  0 1 d t  v2 1
The state trajectories for the candidate saddle strategy are:

3.16 3.17

xt = e,t=2 x0 1 1 xt = x0 + v2 t 2 2

For v2 = 0, the system is unsafe if x0 = 0 and x0 10 and safe otherwise. We will concentrate on the case 2 1 v2 6= 0. From 3.12 x T x0  = 0, hence T x0  = ,x0 =v2 . Therefore, from 3.13: 2 2
2 J1 x0  = J1 x0 ; u ; d  = ex0=2v2  x0 1 1 1

Lemma 3 u ; d  is globally a saddle solution. 1 1

32

Proof: First x d = d and vary u. Then: 1

x1 t = e,t=2 x0 + 1 x2 t = x0 + v2 t 2
Hence T x0  = ,x0 =v2 again. Therefore: 2
Z

t ,t, =2 e u

d

J1 x0; u; d  = 1
=


since:

Z ,x0 =v2 2 x0 =2v2  x0 + 2 2 e ex0 =v2 + =2 u 1 0 Z ,x0 =v2 2 J1 x0  + 2 ex0 =v2 + =2 u d 0 J1 x0  Z

d

ut 2 0; 45 

Now x u = u and vary d. Then: 1

,x0 =v2 x0 =v + =2 2 e 2 2 u d

0

x1 t = e,t=2 x0 1 Z t x2 t = x0 + d d 2

0

Let T 0 x0  be the time that the train reaches the crossing.
Z 0 0 0 x0  = 0  T x  d d = ,x0 x2 T 2 0

But dt 2 v1 ; v2 therefore:

d d  v2 d = v2 t 0 0  T 0x0   ,x0=v2 = T 2 2  J1 x0 ; u ; d = e,T 0 x0=2 x0  ex0 =2v2 x0 1 1 1 0 ; u ; d  J  x0   J1 x 1 1

Summarizing, for any admissible u; d:

J1 x0 ; u ; d  J1 x0   J1 x0 ; u; d  1 1

Therefore, by de nition, u ; d  is a global saddle solution to the game with cost function J1 . 2 1 1 The saddle solution can now be used to calculate the safe set of initial conditions: Lemma 4 The set of safe initial conditions is:
1 V1 = x0 2 Xjx0 0 or x0  2v2 ln C0  2 2 x 1

3.18

3.19

33

10

x2

Unsafe

10

15

Safe

20 0

10

20

30

40 x1

50

60

70

80

90

Figure 3.4: Safe Set of Initial Conditions

Proof: For x0  0 safety is equivalent to: 2

2 J1 x0  = ex0=2v2  x0  C1 1

Since v2 and x0 are positive and the exponential is monotone this is equivalent to: 1
1 x0  2v2 ln C0  2 x 1

For su ciently large values of L, all initial conditions with x0 0 should be safe the train has 2 already passed the crossing. In particular, the fact that the logarithmic function is monotone leads to the requirement that: L  ,4v2 ln C1  90

The safe set for C1 = 10 and v2 = 3m=s is shown in Figure 3.4. The safe initial conditions are above the horizontal axis, on or below the curve or to the left of x1 = 10.

Lemma 5 The class of safe controls is:

8

U1x0  =

u if x0 = 2v2 ln C01  1 2 x1

; if x0 62 V1

U otherwise

The proof follows from the above discussion.

3.3.4 Design for Throughput

As:

J2 x0 ; u; d =

1
0

90 , x1 t2 dt

34

Sensor Train Near

x2

Controller Raise

Actuator

Raise => u=45 u

x2=S2 Train Far

x2=S1

Train Near

Train Far Lower => u=0

Lower

Figure 3.5: Discrete Control Scheme and, by Lemma 2, x1 t  90, maximizing throughput minimizing J2  is equivalent to maximizing x1 t. From the proof of Lemma 2 this is equivalent to setting:

u t  45 2

3.20

This choice is vacuously a saddle solution for any d, as the disturbance does not enter the dynamics of x1 and therefore does not a ect the cost.

3.3.5 Controller Design

The optimal controller can be obtained by combining the designs for safe and e cient operation. Let S denote the interior of the set V1 . As safety takes precedence over e ciency the resulting controller will be:
c 0 u = 45 x 2 S x2S


Optimal" Controller

3.21

^ By design, the controller of 3.21 will be safe. Moreover any controller which uses S S in the place of S will also be safe, but not as e cient in terms of throughput. These observations are summarized below: ^ Theorem 2 A switching controller of the form of 3.21 with switching taking place at S will be safe if 0 2V. ^ S S and x ^ Proof: The theorem follows directly from Lemmas 2, 3, 4 and the fact that S S V1 . 2

Discrete Controller

Due to its bang-bang nature, the optimal controller can easily be implemented by a discrete scheme, using the discrete sensor and an appropriate actuator as an interface. The resulting controller automaton is shown in Figure 3.5. The interpretation of inputs, outputs and states should be clear from the gure. Theorem 3 The discrete control scheme will be safe if:

S1  2v2 ln C1  90 S2 0
35

Proof: When viewed as an input-output system in the continuous domain, the combination of the discrete
controller and the interface looks like: 0  x2 u = 45 S1otherwiseS2


3.22

^ ^ Let S = fx 2 X jS1  x2 S2 g. By Theorem 2 the controller will be safe if S S , i.e. S2 0 and C0  for all x0 2 0; 90 . As the logarithm function is monotone, the above conditions are the 1 S1  2v2 ln x1 1 same as the ones in the theorem statement. 2 Note that the above calculations gives us the optimum placement for the discrete sensors for free. In particular, for the most e cient but still safe operation the sensors should be placed at:

S1 = 2v2 ln C1  90 S2 = 0
What if the sensors were placed improperly, for example S1 2v2 ln C1 . The above analysis can not 90 guarantee a safe controller. One solution may be to modify the hardware and provide additional sensors. An alternative would be to obtain a promise from the train that it will slow down once it enters the sensor 0 0 0 range, i.e. a promise that x2 2 v1 ; v2 with v2 v2 when x2 2 S1 ; S2 . For appropriate choices of v2 _ 0 in particular v2  lnCS11=90  safe operation with a discrete controller will still be possible. This example indicates how inter-agent communication which can be used to provide such promises can bias the game in the controllers' favor and help the designer produce an acceptable design.

Extensions
The discrete design can easily be modi ed to deal with multiple trains, by introducing a counter in the controller. The counter is incremented whenever a train crosses S1 and is decremented whenever one crosses S2 . The state Train Far is interpreted as the counter reading 0. Provided that the conditions of Theorem 2 are met and all initial conditions are chosen appropriately the resulting closed loop design will be safe. If discontinuous controls are undesirable, the controller can be made smooth by applying:

u = 1 , gx45
where f is any smooth function satisfying:

g : R2 ,! R  ^ 2S x 7,! 0 xx2 S 0c ^ 1
^ ^ with S S 0 . Such functions are easy to construct see for example 60 . According to Theorem 2, the ^ resulting controller will be safe if S 0 S . With small changes in the analysis, safe designs can also be obtained for systems with delays e.g. between the occurrence of an event and sensing it, and between command and execution. With a bit more work designs with the additional constraint y 2 a1 ; a2 can also be obtained. We will forgo these calculations in favor of the more interesting and challenging vehicle following calculation, coming up in Chapter 6. 36

3.4 Summary of Key Points

The design algorithm presented here provides a formal way of producing hybrid controllers that guarantee certain properties of the closed loop system by design, eliminating the need for veri cation. Of course there is always the possibility that for a given realistic problem the calculations will be too di cult to carry out, at least analytically. The results obtained however, in cases where the calculations are feasible, are very powerful and can lead to interesting design alternatives. The conditions obtained by solving the gaming problem are, in some sense, necessary and su cient. They are su cient from the point of view of design. Any controller that satis es the derived conditions is guaranteed to lead to acceptable performance for any disturbance. On the other hand, the conditions are necessary from the point of view of veri cation, in the sense of being tight". Given a design that fails to satisfy these conditions, there exists a disturbance trajectory, d, and an initial condition, q0 ; x0 , allowed by the design, such that the trajectory generated by starting at q0 ; x0  and applying the given controller and d violates at least one of the performance requirements. An advantage of the approach presented here is that the calculations are exible enough to be used in various contexts, other than controller design and veri cation. For example, as indicated by the train gate problem, they can be used to produce technological requirements on the physical plant in particular the sensors and actuators in order to guarantee certain levels of performance. This point of view will be particularly evident after the automated vehicle calculations. There the solution to the game theory problems will be used to derive minimum sensor ranges, limits on the braking capability etc. needed to guarantee safe operation under certain requirements on throughput. The big problem that needs to be addressed in this context is the design of the discrete layer. The approach proposed here can be used to provide guidelines for the discrete design for semi-autonomous agent operation. In case of con icting agent objectives, the design process presented above will indicate which disturbance ranges should be reduced and by how much, in order to achieve the desired performance. If these disturbances are due to the action of other agents, this suggests which agents should cooperate and what demands they need to make on each other. Unfortunately, the way in which the discrete controllers should be designed in order to achieve the required reduction in disturbance is by no means clear. However, once the calculations presented here have been carried out, the modi cation of the discrete design becomes a purely discrete problem that may be attacked using the standard discrete design and veri cation tools. The problem of parsing requirements on the emergent behavior that are usually given linguistically into sequences of way points, cost functions and thresholds also needs to be addressed. We will discuss this process in the context of automated highways in Chapter 6. In that example the parsing relied heavily on engineering intuition, experimental results on test vehicles, statistical data collected from real highways, simulation results and, above all, common sense. Clearly, this process is very problem speci c, requires a tremendous amount of e ort and is by no means formal. All these design issues are very interesting and deserve to be the topic of further research.

37

Chapter 4

Veri cation
The methodology presented in Chapter 3 can be used to approach the design of the lower level controllers for multiagent hybrid systems. The result is a controller that can guarantee the closed loop speci cations by design, without the need of any further veri cation. Even in this setting however veri cation is bound to play an important role for a number of reasons: Approximate Solutions: the design is likely to be di cult to obtain analytically, therefore approximate solutions may have to be used. The resulting approximate controller will have to be veri ed to possess the same properties as the exact controller. Discrete Design: the techniques of Chapter 3 produce guidelines for the high level controllers but not a complete design. Even though these guidelines will in general be easy to follow, veri cation may be needed after the design is complete. Emergent Behavior: once a successful design has been implemented the designer may wish to analyze it further for properties that were not considered originally as design speci cations. Modi cations: the algorithm of Chapter 3 would dictate full redesign if small changes e.g. in the information structure are needed. Existing Designs: for many examples controllers have already been implemented in practice. One would like to be able to analyze and modify these controllers without having to redesign the system from the beginning. In this chapter we will brie y discuss the issue of veri cation for hybrid automata. The most common approach to the problem is algorithmic veri cation 37, 11 . A solid background of theoretical results exist to support algorithmic veri cation and a number of e cient algorithms have been implemented. The bottom line, roughly speaking, is an exhaustive search of the state space or a nite quotient thereof to determine whether all runs of the system satisfy a certain property. Here we introduce an alternative approach that makes use of optimal control techniques. We attempt to determine the worst possible run with respect to the given property and verify that the property is satis ed only for this one run. The discussion in this chapter indicates that even though no bene ts can be directly derived for discrete problems there may be some bene ts in terms of the complexity of the computation and circumventing undecidability for problems with continuous state spaces.

38

4.1 Invariant Veri cation

We start by showing how the invariant veri cation problem can be cast in an optimal control framework. We address the following problem: De nition 14 Given a hybrid automaton, H and a set F X , we say that F is reachable by H if there exists a run,  ; q; x; y; u of H and a time t 2 such that qt; xt 2 F . For a given pair H; F  we would like to solve the reachability problem, i.e. determine if F is reachable by H . By a simple construction the reachability problem can be shown to be equivalent to the invariant veri cation problem on state predicates boolean expressions over X  45 . The last observation can be used to formulate the reachability problem as an optimization problem. Consider the predicate:  P q; x = 1 if q; x 2 F 0 otherwise For every run  ; q; x; y; u of the automaton P induces a cost function:

J  ; q; x; y; u = max P qt; xt t2

The veri cation process involves maximizing J over all possible runs. The set F is reachable if and only if the optimal cost: J  = max J  ; q; x; y; u is greater than 0. A similar cost function can be derived using the notion of distance between sets, induced by the metric m on X . In either case the result of the optimization will be the same. Theorem 4 The set F is reachable by H if and only if:

J  =  ;q;x;y;u max mqt; xt; F c  0 max t2

4.2 Multi Agent System Veri cation

The above observations are not very instructive in this general form. In particular, for problems where the state space is purely discrete, solving for J  simply amounts to searching the reachable state space, therefore no improvement over algorithmic veri cation techniques can be expected. For problems with non-trivial continuous dynamics however and certain choices of F , considerable bene ts may be derived by applying optimal control techniques. In such cases, algorithmic veri cation would involve some form of discretization" of the state space followed by an exhaustive" search1 . If the optimal control problem has a solution, on the other hand, the optimal run is the only run that needs to be considered. Even if the optimal control problem can not be solved, there may still be some bene ts in guiding" the algorithmic search using optimization ideas. To illustrate this point we specialize the above procedure to the multiagent setting discussed in Chapter 3. Consider an agent hybrid automaton, H = X; U  D; Y; I; f; E; h and assume a controller has been designed for the inputs. If we model the controller as a hybrid automaton H 0 = X 0 ; Y; U; I 0 ; f 0 ; E 0 ; h0 , ^ ^ ^ ^ ^ ^ ^^ the closed loop system can also be described by an automaton H = X; D; Y ; I ; f; E; h, the attened 0 . To simplify matters we will take Y = Y and assume that h is the same as h ^ ^ interconnection of H and H
1 The terms discretization and exhaustive are used roughly.

39

^ applied to the appropriate sets. As for any interconnection, X = X  X 0 will contain both the plant and ^ will have two components, generated by f and f 0, augmented to account for the closed controller states. f ^ loop inputs. The set I will contain the initial states of the controller and the initial states of the plant allowable by the controller. For example, if the guidelines of the design process of Chapter 3 are followed ^ the part of I corresponding to the plant states will be equal to V I . Similar remarks can be made for ^ ^ ^ the set E . We will assume that the parts of I and E corresponding to the plant are subsets of I and E respectively, augmented to account for the closed loop inputs. This re ects the fact that the sets I and E represent physical restrictions on the plant dynamics, therefore can not increase by the presence of the controller. Note that the discrete controller may be nondeterministic, even if the plant is deterministic. Therefore, for the closed loop system the disturbances can be classi ed as: 1. External disturbances, dE , including the actions of other agents. 2. The high level commands issued, dI , in case the high level controller is non deterministic. ^ We will use d = dE ; dI  2 D to denote the overall disturbance. Note that some correlation is expected between these two classes of disturbance. For one, part of the nondeterminism of the high level design is likely to arise because of the demands actions of other agents. Moreover, for semi-autonomous agent operation, interagent coordination at the discrete level can be used by one agent to in uence the trajectory ^ of another. Overall, the set D in which the disturbances are limited is primarily determined by the high level controller. Again we assume that the set D encodes physical restrictions on the plant and therefore ^ the part of D corresponding to the plant must be a subset of D. For the design criteria introduced in Chapter 3 the set F in the reachability problem is already encoded by means of cost functions. For the closed loop system the number of arguments of the cost functions decreases, as the inputs u have been speci ed by the controller. In other words, each design speci cation can be encoded by pair J; C : J : PC  PC 1  PC ,! R and the requirement that J q; x; d  C for all runs  ; q; x; y; d. Veri ng that a design speci cation J; C  is satis ed can now be viewed as an optimal control problem. We will again assume deterministic evolution for the plant, in which case J simpli es to: ^ J : I  PC ,! R recall that any nondeterminism of the controller has been absorbed in d. Assume that an optimum disturbance, d , exists and let:

J  q0 ; x0  = sup J q0 ; x0 ; d = J q0 ; x0 ; d 

^ d2D

^ If J  q0 ; x0   C for all q0 ; x0  2 I then the requirement is satis ed by the closed loop design. Otherwise 0 ; x0  2 I and d 2 D such that the run starting at q0 ; x0  under d violates the speci cation. ^ ^ there exist q The above discussion can be summarized as: Theorem 5 The closed loop design satis es speci cation J; C  if and only if:
^ ^ q0 ;x0 2I d2D

sup sup J q0 ; x0 ; d  C

40

Normal x1=0 x2=0 x3=D . x1=1 . x2=0 . x3=1

x3:=0

Leaking . x1=1 . x2=1 . x3=1 x3:=0 x3< D 1

D < x3
2

Figure 4.1: The leaking gas burner hybrid automaton

4.3 Example: The Leaking Gas Burner

To illustrate how optimal control ideas can be used to carry out veri cation consider the leaking gas burner" example of 5 . Even though this problem is rather simple and amenable to other techniques for veri cation it highlights some of the advantages of verifying using optimal control. The gas burner has two states, normal" and leaking". There are two rules governing the leaking process: 1. Leaks are detected and repaired within D1 seconds 2. No leak occurs within D2 seconds of the last leak being xed. The gas burner can be modeled by the hybrid automaton of Figure 4.1. The automaton has two discrete states Normal and Leaking and three continuous states, x1 that keeps track of time, x2 that keeps track of leaking time and x3 that is used to guarantee the constraints. The set E can be constructed from the node invariants and the transition guards and reset maps shown in the gure. The initial state is Normal; 0; 0; 0. The requirement is that the accumulated time of leakage does not exceed a fraction in any interval larger than T 0 seconds, i.e.

4.3.1 Problem Statement

x1  T  x2

x1

To cast this problem as an optimal control problem, assume that the leaking process is controlled by a malicious opponent. Let: d : a; b ,! f0; 1g be the leaking times, with dt = 0 being normal" and dt = 1 leaking". A typical d is shown in Figure 4.2. We will use d = fN0 ; L1 ; N1 ; L2 ; : : :g to denote this sequence. The leaking rules limit the set of acceptable d's to: D = fd : a; b ! f0; 1gjLi  D1 ; Ni  D2 for all ig 4.1 We will treat the leaking process as an optimization problem. De ne: J  a; b ; d = L1 + L2 + : : : + Lk 2 0; 1 accumulated leaking time can now be posed as: J  a; b ; d  for all 0  a b with b , a  T
from the cost function.

4.3.2 Optimal Control Formulation

b,a to be the percentage of leaking time in the interval a; b R with 0  a

b2 . The requirement on the

4.2

2 As there is a unique initial condition for the leaking gas burner automaton we drop the dependence on the initial condition

41

0 N0 L1 N1 L2 N2 t

Figure 4.2: Typical leaking pattern We will try to verify that this requirement is satis ed by solving: max J  a; b ; d = max L1 + L2 + : : : + Lk
d2D d2D

b,a

First consider the case where a = 0; b 0, which trivially generalizes to any a; b. The problem is simple enough to allow us to guess a maximizing d:

4.3.3 Veri cation Process

Lemma 6 A global maximizer of J  a; b ; d is d = f0; D1 ; D2 ; D1 ; : : : ; L; N g 2 D with 0 L  D1 , 0  N  D2 and L = D1 if N 0. Proof: Consider an arbitrary d = fN0 ; L1 ; N1 ; L2 ; : : :g 2 D. Apply the following algorithm: Step 0: If N0 0 slide" L1 left by N0 , lling up with 0 on the right. The resulting d will be: d0 = f0; L1 ; N10 ; L2 ; : : :g with N10 = N0 + N1 . Note that d0 2 D as d 2 D and N10  D2 . Step 1: If L1 D1 slide" part of L2 left to ll up" L1: d1 = f0; D1 ; N10 ; L02 ; : : :g with L02 = L2 , D1 , L1 . If L2 is not enough use some of L3 and so on. Note that d1 2 D as d0 2 D and L02  D1 . Step 2: If N10 D2 slide" L02 left, until: d01 = f0; D1 ; D2 ; L02 ; N20 ; : : :g with N20 = N2 + N1 , D2 . Note that d01 2 D as d1 2 D and N20  D2 .

^ Steps 1 and 2 are repeated for the remaining L and N . Let d denote the resulting d: ^ d = f0; D1 ; D2 ; D1 ; D2 ; : : : ; L; N g 42

where 0 L  D1 and N  0. Note that all the steps of the algorithm preserve the leaking times therefore: ^ J  a; b ; d = J  a; b ; d To get the maximizer, if L D1 and N 0, ll up" L until L = D1 or N = 0. The resulting d will be: ^ d0 = f0; D1 ; D2 ; D1 ; D2 ; : : : ; D1 ; N 0 g or 0 = f0; D1 ; D2 ; D1 ; D2 ; : : : ; L0 ; 0g ^ d In either case: ^ ^ J  a; b ; d0   J  a; b ; d If N 0 D2 make N 0 = D2 and add a new interval of leaking time. Again this will lead to a higher value of J . The process can be repeated until: d = f0; D1 ; D2 ; D1 ; D2 ; : : : ; L; N g is obtained with 0 L  D1 and 0  N  D2 and the restriction that L = D1 if N 0. d is a global maximizer as by construction J  a; b ; d   J  a; b ; d for all d 2 D. It should be noted that the maximizer is not unique if N 0. 2 The above calculation trivially generalizes to any a and b. We will therefore restrict out attention to t intervals of the form 0; t . Let k be the integer part of D1 +D2 . Then the worst case accumulated leaking time is given by:  k+1D1 if t , kD + D   D   = max x 0; t  = 1 2 1 t J  0; t ; d 4.3 t,kD2 d2D if t , kD1 + D2   D1 t In order to answer the veri cation question we need to determine:  JT = max J  0; t ; d  tT for any given T . Note that the derivative with respect to t of the rst term in J  0; t ; d  is negative while that of the second term is positive. Therefore, for kD1 + D2   t  k + 1D1 + D2  the maximum with respect to t will occur at t = kD1 + D2  + D1 . Moreover:

J  0; kD1 + D2 + D1 ; d  =

 which decreases as k increases. Hence, given any T 0, JT must occur for T  t  kT + 1D1 + D2  T where kT is the integer part of D1 +D2 . A simple calculation based on this observation shows that:
8

k D1 + k+1 D2

D1

 JT =
:

Theorem 6 The accumulated leaking time speci cation is satis ed if and only if D1; D2 ; T are such that  JT  .

D1 k D1 + kTT D2 +1 kT +1D1 T D1 kT D1 + kT +1 D2 +2

if kT D1 + D2   T  kT D1 + D2  + D1

if kT D1 + D2  + D1  T  kT + 1D1 + kkTT+1 D2 +2 kT +12 D  T  k + 1D + D  if kT + 1D1 + kT +2 2 T 1 2

4.4

 Figure 4.3 shows a graph of JT for various values of D1 ; D2  and T = 60 seconds as in 5 . The dividing line between acceptable and unacceptable combinations is a horizontal plane at height .

43

T=60

0.8

0.6
J*

0.4

0.2 10 0 40 5 20 D2 10 0 0

30

D1

Figure 4.3: Worst case leaking percentage for D1 ; D2

44

Chapter 5

System Autonomy
The rst step towards controlling a given system is typically to design a controller to provide performance guarantees under some nominal conditions plant model, environmental inputs, etc.. However, in most cases one would like the control scheme to be capable of providing performance guarantees for a range of conditions, wider than just the nominal ones. The extension of the domain of applicability of a control scheme is common in control theory and special techniques have been developed to carry it out. They include robust control, adaptive control, learning algorithms, soft computing techniques fuzzy and neural controllers, genetic algorithms etc. and techniques from arti cial intelligence for example Bayesean belief networks. The common theme in all these methods is an attempt to increase the autonomy of the controller and make it capable of operating under some degree of uncertainty. For most of these algorithms some analysis of the overall system performance can be carried out. Usually, the higher the structure imposed on the uncertainty the stronger the results that can be obtained. In this chapter we investigate the problem of increasing the autonomy of a control scheme for large scale systems. Even though the above mentioned techniques can be used to solve some of the problems, the hierarchical and hybrid questions that arise in the process are di cult to approach using existing techniques. This indicates that there may be a need for further theoretical development in this direction. Unlike the previous chapters the discussion here will be informal. It will mostly be based on intuition gained from the design of fault tolerant control scheme for an Automated Highway System1 . The issues raised in this chapter should however be thought of as work in progress.

5.1 Adaptation and Learning

To highlight the need for a new paradigm, we give a brief overview of adaptation techniques from the point of view of hybrid systems. The algorithms in the adaptive control literature try to increase the automy of a controller by making it capable of dealing with a parametrized class of systems, rather than a single, nominal system. Typically, all systems in the class are qualitatively the same, but quantitatively di erent. The real plant is assumed to be an unknown but constant or at least slowly varying member of the class. The controller observes the closed loop plant response and tunes itself until the response becomes the desired. In the process the plant is identi ed", either implicitly or explicitly. The classic framework for this kind of adaptive control is linear, time invariant systems, with an a-priori known number of states:

1 The outline of the scheme is given in Chapter 7.

x = Ax + Bu _ y = Cx
45

possibly in discrete time or with some noise a ecting the dynamics. The class of plants is parametrized in this case by the unknown matrices, A; B and C , that determine the system dynamics. The number of parameters is constant once the number of states is xed. Because the only uncertainty that the controller has to cope with enters through the matrix parameters adaptive control for this class of systems is sometimes referred to as parametric adaptive control. The system uncertainty is highly structured and, as a consequence, very powerful results can be obtained for this class of systems, in terms of stability, parameter convergence etc. 61 . Parametric adaptive control has also been successfully applied to some classes of nonlinear systems 62 . A di erent approach for dealing with system uncertainty comes from the Arti cial Intelligence and Soft Computing communities. Here the emphasis is usually the lack of explicit assumptions about the model of the process to be controlled. The controller is assumed to belong to a parametrized class, e.g. a neural network with a xed topology. As before the system evolution is observed and the controller parameters are tuned accordingly. Because the number of parameters can be very large for example all the weights in the neural network the tuned controller can exhibit a very complex, nonlinear behavior. This large class of controllers makes it possible to apply the scheme to a wide range of situations. The lack of a nominal plant model, on the other hand, makes it very di cult to formulate proofs of performance for such systems. Typically, the only claim that can be made is that the scheme works in simulation or experiment. If the plant is also restricted to belong to a certain class e.g. fuzzy systems, stronger theoretical results may be obtained 63 . The above methodologies are based on a central control paradigm, where the information is collected from the entire plant and is processed centrally. We would like to extend these ideas to the decentralized, semiautonomous agent setting considered here. We will focus on the hierarchical and hybrid issues that arize in the process.

5.2 Hybrid Issues

The hybrid nature of the problem manifests itself in instantaneous transitions where the closed loop system behavior changes drastically, in the sense that the dynamics before and after the transition are qualitatively di erent. In particular we will be concerned with two sources of hybrid dynamics: 1. Faults, that take place instantaneously and cause a qualitative change in the dynamics of the plant itself. In terms of the preceding discussion, the class in which the plant belongs can be though of as changing. We will try to address two problems in this setting, how to detect the change in the dynamics fault detection and what to do to salvage the situation fault handling. 2. Extreme parameter variations, where even though the class of systems in which the plant belongs to does not change, the parameter values become such that the plant may no longer be e ectively controlled by the same controller. Here the discrete change is in the controller rather than the plant and it is dictated by the performance requirements. There is a qualitative di erence between the two cases. In the rst case, the discrete changes emerge from the lower level the plant while in the second case, the discrete changes are dictated by the high level control objectives. However, both these cases exhibit an inherently hybrid structure. In the rst case, the continuous evolution is disturbed by a discrete change in the plant, caused by the fault. In the second case, the discrete change is imposed by the fact that the performance speci cations can not be met by the current controller and therefore a switch in the control scheme is needed. An attempt to address switching issues like these in the central control paradigm can be found in 64 . Even though the context is quite di erent, the ideas of 64 could generalize to this framework. Here we will take a di erent approach, that will try to work along side the hierarchical structure of a distributed control scheme. 46

5.2.1 Fault Modeling

In the hybrid automaton model of Chapter 2 the e ect of faults can easily be captured by discrete transitions. Consider a single fault and assume that both normal and faulted operation is modeled by hybrid automata, HN = XN ; UN ; YN ; IN ; fN ; EN ; hN  and HF = XF ; UF ; YF ; IF ; fF ; EF ; hF  respectively. We assume that XN = XF ; UN = UF and YN = YF , i.e. the fault a ects the system dynamics but not the input, output and state spaces2 . To make the model as general as possible we introduce an extra transition set: The interpretation is that the set de ned by: ResNF q; x = fq0 ; x0 jq0 ; x0  2 X; q; x; u; q0 ; x0  2 E 0 g X is the set of states in which the system can end up, if the fault occurs while the state is q; x 2 X and the input is u 2 U . Finally, we assume that: ResNF q; x IF that is after the fault the system always nds itself in an initial state of HF . To model the occurrence of the fault we de ne a new discrete environmental input, Fault 2 f0; 1g, where Faultt = 1 if the system operation is faulted at time t and Faultt = 0 otherwise. To simplify matters it is usual to assume that the fault is irreversible and that the system starts in normal operation. This implies that the signal Fault
 satis es: Faultti  = 0 Faultt = 1  Faultt0  = 1 for all t0  t Such a signal can easily be generated by hybrid automaton with two discrete states, one discrete output and trivial continuous dynamics  nite state machine. The discussion in this section easily generalizes to reversible faults. A new automaton, H = X; U; Y; I; f; E; h, can now be constructed to capture the e ect of the fault: X = XN  fN; F g U = UN  f0; 1g Y = YN I = IN  fN g
q;x2X;u2U

E0 X  U  X

f q; N ; x; u; 0 = f q; N ; x; u; 1 = fN q; x; u f q; F ; x; u; 0 = f q; F ; x; u; 1 = fF q; x; u E = fN g  EN  f0g  fN g fF g  EF  f1g  fF g , fN g  E 0  f1g  fF g

47

2 This can be done without loss of generality, by appropriate modi cations to the automata HN and HF .

hq; N ; x; u; 0 = hq; N ; x; u; 1 = hN q; x; u hq; F ; x; u; 0 = hq; F ; x; u; 1 = hF q; x; u
The assumption that the fault is irreversible allows us to be a bit sloppy in the de nition of the set E . Note that Fault is essentially an event input, as only its transition from 0 to 1 a ects the system dynamics.

5.2.2 Modeling of Extreme Parameter Variation

The e ect of extreme parameter variations is more di cult to model in this framework, as the performance speci cations need to be taken into account. An intuitive way of modeling the process is by letting the set where all the performance requirements can be met denoted by V in Chapter 3 become empty. The worst case design approach of Chapter 3 implies that the size of V will implicitly depend on the input and state constraints as well as the state dynamics, all of which re ect the parameters of the system. For certain combinations of these parameters the set V may become empty. This implies that a controller satisfying all the closed loop speci cations can not be designed in this case.

5.3 Fault Tolerant Controller Design

In principle the tools developed in Chapter 3 could be used to design controllers for the overall automaton H . This is likely to be an unproductive approach, however. The deterministic, worst case approach taken by these tools will probably lead to an overly conservative design. The worst case scenario will involve combinations of many faults that lead to situations for which the system performance is extremely degraded. A better way to approach the problem is to note that faults are likely to be rare events. Therefore, even though the worst case fault combinations are possible they are very improbable. This observation indicates the need for stochastic design. Such techniques are commonly used in practice, usually based on experimental data or Monte Carlo simulations. Deterministic tools like the ones developed in Chapter 3 can still be used in this setting if we split the design process into two stages. The rst stage is fault detection, where the designer has to determine whether a certain fault has occurred or not. The second stage is fault handling, where special controllers are implemented to ensure that the impact of the fault on the system performance is minimized. The ^ overall fault tolerant control scheme is shown in Figure 5.1. The discrete signal F is the estimate of the environmental input Fault generated by the fault detection module.

5.3.1 Fault Detection

In the process of fault detection the designer is asked to infer whether a transition from the normal to a faulted state has occurred. In most cases the signal Fault will be unmeasurable3 so the decision will have to be based on the measurable inputs and outputs of the system. Therefore, the fault detection process has to infer the occurrence of the fault by its e ect on the system dynamics. Fault detection is inherently a hybrid process: a discrete transition has to be inferred from the continuous input output measurements. A number of algorithms have been developed to carry out fault detection in both continuous and discrete domains. The continuous algorithms address the case where the discrete dynamics are trivial and make use of information only about the continuous state. To infer the occurrence of a fault these algorithms make use of redundancy" in the physical process. This can be either hardware redundancy for example many
3 Fault is more a conceptual than an actual signal.

48

Fault

H
u

Fault Handling

^ F

Fault Detection

Figure 5.1: Fault Tolerant Controller Structure sensors measuring the same state or analytic redundancy where di erent measured states are connected through the dynamics fN and fF . One standard approach is to construct lters to estimate" the measured states, assuming that the system evolves according to fN . The estimated values are compared with the measured values to produce residues. Faults are declared when the residues of the lters exceed certain thresholds. The interpretation is that the discrepancy between estimated and measured values is due to the fact that the real state evolves according to fF while the estimated state evolves according to fN . Techniques have been developed to optimize the lter design and the residue processing with respect to missed detections and false positives. Algorithms implementing these techniques have successfully been applied to many systems. For example, fault detection on vehicles has been carried out using linear 65, 66 and nonlinear 67 lters, as well as lters based on probabilistic models 68 . Fault detection for discrete systems makes use of information only about the discrete state. The goal here is to infer the occurrence of the unobservable transition from N to F through the occurrence of observable transitions. In the nite state machine setting where the continuous dynamics are trivial this problem was solved in 69 . The techniques developed there establish conditions that the system needs to satisfy to allow the detection of the fault event within a certain number of transitions. If these conditions are satis ed a nite state machine design for the fault detection module called a diagnoser in 69  is also given. Clearly both the continuous and the discrete approaches do not make use of all the available information when applied to the hybrid setting. For example, discrete transitions may not be observable directly but may have to be inferred from continuous measurements4 and vice versa. Clearly an extension of these techniques to the hybrid setting is needed to complete the picture.

5.3.2 Fault Handling

6.

If the change in dynamics caused by the fault is signi cant it is likely that the normal mode controller will be incapable of producing satisfactory closed loop performance. Therefore, once the fault has been detected,
4 This will be the case, for example, with collisions between vehicles in the Automated Highway System described in Chapter

49

specialized controllers will have to be invoked to deal with the situation. Possibly some of the lower priority performance requirements will have to be dropped in the process, leading to degraded performance. The task of the specialized controllers is essentially to regulate the faulted" automaton HF ; therefore, their design should be amenable to the techniques of Chapter 3. The problem with this head-on approach is that the complexity of the design process increases exponentially with the number of faults that need to be considered. As each fault a ects the system dynamics di erently, to produce a control scheme that is capable of coping with combinations of up to i faults the designer will have to produce controllers for 2i di erent hybrid automata. For a complex, multiagent system this blow up in complexity makes the design process prohibitively expensive". A hierarchy can be introduced to manage the complexity. For example, faults and combinations of faults can be grouped in classes, depending on their e ect on the system dynamics. Each class contains faulted systems which are in some sense similar. Controllers can then be designed for the entire class, rather than for individual faults. As the number of classes is likely to be a lot smaller than the number of fault combinations, the design process may be a lot simpler. Clearly the resulting scheme will be suboptimal, as each class controller will re ect the worst situation in the given class and therefore some of the capabilities remaining in other situations may be wasted.

5.3.3 Dealing with Extreme Parameter Variation

The hybrid dynamics associated with extreme changes in parameters are closely linked to the high level performance objectives. This makes both detection and handling di cult. In some cases the values of the parameters may be estimated using conventional identi cation techniques for linear and nonlinear systems. In other cases, this may not be possible. For example the only way of estimating the maximum deceleration of a vehicle seems to be hard braking. However, considerations of safety and passenger comfort may prevent the designer from carrying out this identi cation. The design of specialized controllers is also tricky. In some cases, the controller for the nominal parameters may still be used, after appropriate tuning. This situation is similar to conventional adaptive control. However, in situations where the set V for the saddle design of Chapter 3 becomes empty, a satisfactory controller simply does not exist. In this case low priority performance requirements may have to be dropped in favor of high priority ones. The design process resembles the fault handling process outlined in the previous section.

5.4 Hierarchical Issues

As discussed above, the complexity of the design process for a fault tolerant controller increases dramatically with the number of faults that need to be considered. Therefore, the need for a hierarchical controller to manage the complexity of the design is even more pronouned in this case. A fault tolerant control hierarchy will have some special features that are worth pointing out. The controller for normal operation can assume that the capability of the plant is xed, i.e. the model of the plant belongs to a relatively narrow class. Under this assumption, the only information that the controller needs relates to the state of the physical process. Such information can be collected through sensors. Because the control hierarchy may involve di erent modeling languages at each layer, the state information needs to be processed and presented to the controller at the appropriate level of abstraction. It is convenient to think of this processing as being carried out by a sensor hierarchy, a separate hierarchical arrangement that operates along side the control hierarchy. Sensor information is collected at the bottom 50

5.4.1 Information Hierarchy

of the sensor hierarchy and is abstracted as it moves up. At each level this information is fed to the corresponding level of the control hierarchy. The constant capability assumption is no longer valid in the presence of faults. A fault tolerant design has to deal with situations where the process model changes quite drastically. Information about these changes may need to be propagated to all the levels of the control hierarchy. This information also has to be presented at the appropriate level of abstraction. It is convenient to think of this task as being carried out by a capability monitor, a hierarchical arrangement that collects the fault detection information at the bottom and abstracts it appropriately. At each level this information is fed to the corresponding level of the control hierarchy. Similar considerations are needed for the case of extreme parameter variations. The information processing in this case is more closely coupled to the controller structure. None the less it may still be convenient to think of a special hierarchical structure, the performance monitor, that collects the parameter information from the physical process, determines how it a ects each level of the control hierarchy and feeds this data to the appropriate level. Under the constant capability assumption the strategy" of the control scheme is xed. This strategy is determined by the emergent behavior speci cations and the design process of Chapter 3. It may involve switching between various controllers and control objectives, however, the switching patterns are xed and depend only on the state of the physical process. Once the system capabilities start changing though the strategy may also have to be modi ed: control objectives may have to be dropped, certain controllers may become inoperable or ine cient, etc. Switching between strategies takes place at a higher" level than switching for a xed strategy. It is convenient therefore to think of this meta-switching as being controlled by distinct layers of the control hierarchy. Each one of the original levels can be split into two layers. The top layer, the supervisor, receives the system capability information and switches between strategies accordingly. The lower layer, the regulator, is responsible for implementing the chosen strategy. If these guidelines are followed the resulting fault tolerant hierarchy is shown in Figure 5.2 At this stage there is not a lot more we can say in general about the design of autonomous control schemes for large scale systems. Further progress in this direction is through applications. In Chapter 7 a fault tolerant design for an automated highway system will be presented. The design will feature all the elements discussed here and will highlight the di culties associated with the problem. Even though the results are speci c to the example, they provide useful insight on how similar problems can be approached in di erent settings.

5.4.2 Control Hierarchy

51

Sensor Hierarchy

Control Hierarchy
Sup. Reg.

Capability Monitor

Plant

Sup. Reg.

Sup. Reg.

Performance Monitor

Figure 5.2: Fault Tolerant Controller Hierarchy

52

Chapter 6

Automated Highway Systems: Hybrid Design

A system that displays the characteristics discussed so far is the Automated Highway System AHS. The objective of an AHS design is to increase the safety and throughput of highways by full or partial automation of the tra c. In a design developed by researchers of the California PATH project, a full automation scheme is proposed based on the concept of platooning. Here we apply the techniques introduced previously to obtain hybrid controllers that guarantee safe and e cient operation. Even though the nal theorems are specialized to the platooning concept the details of the calculations are general enough to apply to any vehicle control problem.

6.1 The Platooning Concept

Highway travel has become an indispensable part of everyday life. The current highway system is being pushed to the limit however, as the ever increasing congestion and the consequent delays testify. Studies indicate that this trend is unlikely to be reversed in the near future. Building new highways does not seem like a viable, long term solution to the problem, because of the prohibitively large cost of construction especially in and near urban areas where real estate is expensive. One alternative is to improve the e ciency of current highways by automating their operation1 . The goal is essentially to funnel more vehicles over the same piece of real estate. An underlying constraint in this process is safety. One would like to achieve increased throughput with at least the level of safety of current highways in terms of accidents, injuries and vehicle damage per vehicle-mile traveled. At a rst glance the demands for throughput and safety seem contradictory: enhanced safety means large spacings and low speeds2, so that if needed the vehicles have enough time to stop before they collide, while enhanced throughput means exactly the opposite, tight spacing and high speed. Clearly any acceptable AHS design should seek a compromise between these two opposing trends. The concept of platooning 70 o ers such a compromise. Key to this concept is the observation that collisions are not dangerous in terms of passenger injuries and hardware damage unless there is a large velocity di erential between the colliding vehicles 71 . A simple calculation shows that, in the presence of actuation and sensing delays, di erences in deceleration capabilities etc., low relative velocity collisions can be guaranteed in two situations: If the vehicles are initially far apart, in which case they can come to a stop before they collide.
1 Another alternative is to improve public transportation. Sadly, this alternative has so far been ignored. 2 The safest highway is an empty highway.

53

If the vehicles are initially very close, in which case the collision happens almost instantaneously and therefore the relative velocity is low. The platooning concept is based on this observation. The vehicles travel in platoons, following each other very closely, with spacings of the order of 1 meter. Platoons are separated from each other by large spacings, of the order of 60 meters. In addition to safety this large spacing guarantees that disturbances are attenuated as they propagate down a long string of platoons. This property of string stability is achieved within the platoon by communicating the state information of the rst vehicle to all the vehicles in the platoon 72 . In terms of capacity, platooning clearly leads to an enormous improvement. Static" calculations 73 indicate that, with reasonable size platoons, a fourfold increase in throughput can be expected over a manual driving scheme with the same level of safety. In practice the improvement will probably be less, due to the transient e ects of forming and dissipating platoons, but it is still likely to be substantial. Unfortunately, human drivers are not fast and reliable enough to produce the steering, throttle and braking inputs required for such close following. Therefore fully automated vehicle operation is needed if the platooning concept is to be implemented. Macroscopically, a fully automated highway is a control problem of gigantic proportions, with many thousands of interacting agents. A centralized solution to this problem is clearly infeasible. Moreover, the calculations presented here will indicate that completely autonomous operation can not provide the guarantees necessary for safe formation of platoons. The problem is therefore ideally suited for a semiautonomous agent design. Such a design was proposed in 74 . Each vehicle is equipped with a two layer controller. The lower layer, called the regulation layer is responsible for trajectory tracking and regulation of each agent. The higher level, called the coordination layer is responsible for the communication and cooperation between the agents. A design for the coordination layer is given in detail in 34 . We will apply the design methodology developed here to come up with a controller for the regulation layer that can guarantee safe, comfortable and e cient operation under minimal assumptions on the coordination layer. These assumptions that naturally emerge from the calculations will be used as guidelines for forming an interface between the two layers and tuning the coordination layer. Our design methodology is primarily geared towards addressing the issue of safety. Even though the calculations can be used to quantify the tradeo between safety and throughput, the throughput of the resulting highway system will not be investigated. Throughput maximization is a macroscopic issue, an emergent behavior of our design, and is di cult to quantify at the microscopic vehicle level considered here. In the control architecture proposed in 74 , throughput maximization is the goal of two additional levels, called the network layer and the link layer. The overall control hierarchy is shown in Figure 6.1. The top two layers reside on the roadside, monitor the ow of tra c in entire sections of highway and broadcast commands to all the vehicles in the section. A number of designs have been proposed for the roadside controller, especially for the link layer. They make use of uid" ow models 75, 76 , the concept of highway work 77 or the concept of highway space-time 78 to model the tra c. The coordination design of 34 takes into account the demands made by the higher layers, assuming they have been ltered by a suitable interface. Summarizing, the controller presented here will not address the early part of the top-down design phase, i.e. the parsing of the requirements for increased throughput down the network, link and coordination layers. We will be concerned more with turning the corner", i.e. the impact of the throughput requirements on the regulation layer, the safety constraints that the latter imposes and the e ect of these constraints on the coordination layer. The last part of the bottom-up phase of the design process, i.e. the impact of the safety constraints on throughput is the topic of ongoing research.

54

NETWORK LAYER

SUGGESTED ROUTE

HIGHWAY TRAVEL TIMES

LINK LAYER Roadside

DESIRED SPEED, LANE CHANGE PROPORTIONS, PLATOON SIZE AGGREGATE TRAFFIC FLOW DATA

Vehicle

COORDINATION LAYER

MANEUVER REQUEST

FLAGS & AGGREGATE SENSOR DATA

REGULATION LAYER

CONTROL INPUT

RAW SENSOR DATA

PHYSICAL LAYER
(plant)

Figure 6.1: PATH Architecture

6.2 Problem Formulation

6.2.1 Discrete Level Abstraction

In the platooning concept a vehicle on the automated highway operates either as a leader of a platoon, a follower within a platoon or a free agent one vehicle platoon. The discussion of Section 6.1 indicates that two distinct modes of operation can be distinguished: the follower mode and the leader mode a free agent is a special case of a leader. These two modes di er in their set points small vs. large spacing and the information available for feedback control sensory data as well as information propagated through communication vs. only sensory information. In particular, in the leader mode, we assume that a vehicle is equipped with sensors to measure its own velocity and acceleration, as well as the relative position and velocity with respect to the vehicle ahead. The follower mode, on the other hand, assumes that infrared communication devices are used to provide measurements of the acceleration of the preceding vehicle and the state of the leader of the platoon if other than the preceding vehicle, in addition to the information provided by the sensors. The two modes can be though of as a form of discrete" state of the automated vehicle. Another discrete aspect of the vehicle state is the lane it is in. Assume that the automated lanes are indexed by numbers, increasing from right to left, with 1 representing the right most lane and let lane number 0 denote the situation where the vehicle is o the AHS. Then the discrete state of an automated vehicle can be summarized at any time by a pair: mode; lane 55

Lane Change Right lane:=lane-1

Lane Change Left lane:=lane+1

Join

LEADER Mode
Split

FOLLOWER Mode

Entry lane:=1

Exit lane:=0

OFF HIGHWAY Operation

Figure 6.2: Automated Vehicle Operation with mode 2 fleader, followerg and lane = 0; 1; : : :3 . This is a somewhat coarse description of the discrete state of the vehicle in particular we do not keep track of the exact position of a vehicle in the platoon, it will however be su cient for our purposes. Transitions between the discrete states are achieved by means of ve maneuvers: join, split, lane change, entry and exit. The join maneuver is used to join two platoons in the single lane to form a bigger platoon. The trailing platoon accelerates to catch up with the leading platoon. The split maneuver is used to break a platoon into two smaller platoons. The trailing platoon decelerates to safe inter-platoon distance. The lane change maneuver is used to move a vehicle from one lane to another. The entry and exit maneuvers can be considered as a special case of lane change maneuver that allow the vehicle to get on o the automated highway. In 34 communication protocols were designed to implement these maneuvers. Some assumptions were made to simplify the design process: a platoon can only be engaged in one maneuver at a time4 in particular all maneuvers are coordinated by the leader and only free agents can change lane, enter and exit. The logical correctness of the proposed protocols i.e. their e ectiveness as transitions between the discrete states was veri ed in 34 using automatic veri cation tools. The veri cation was based on assumptions about the behavior of the continuous state of the vehicles. Here we re ne these assumptions and determine if and how they can be met. The evolution of the discrete state of an automated vehicle can be represented by the automaton of Figure 6.2. Our goal here is to design control laws for tracking the set points and for executing the maneuvers such that the overall AHS is safe.

6.2.2 Overview of the AHS Research E ort

The possibility of automating highway tra c has attracted attention for many years. A number of solutions have been proposed for this problem, involving various degrees of automation 24, 79, 74, 80, 81 . The platooning architecture considered here was introduced in 24, 74 . Since then, considerable research has been carried out for all the levels of the hierarchical controller. In the link and network layers controllers have been designed based on tra c ow models 75, 76 , vehicle activity models 78 and highway work
of lanes is nite. 4 By de nition all cooperating platoons are engaged in the maneuver. For example while one platoon joins another they are both engaged in a join maneuver. Similarly a lane change may involve in addition to the vehicle changing lane vehicles in the target lane, as well as two lanes away.
3 Even though this is in theory an in nite state space, in practice there are a nite number of discrete states as the number

56

models 77 . At the coordination layer, a controller based on nite state machine communication protocols was designed and veri ed in 34 . Finally, at the regulation layer, controllers based on continuous vehicle models have been designed to regulate the longitudinal motion both for the platoon leaders 79, 60, 80, 28 and for the platoon followers 82, 83, 84, 85, 72 . For lateral vehicle motion controllers have been designed both for maintaining position in a lane 86 and for changing lanes 87 . Work at the physical layer on sensors 68, 88 , actuators 89 and communication devices 90 has allowed researchers to investigate the performance of some of these controllers in experiments 72, 66 . The main concern in the design of most of these controllers has been safety. Here we focus on the safety issue from the point of view of the platoon leader. In particular we are concerned with safety problems that might arize by the hybrid interaction between the coordination and regulation layers. In 12 , Godbole et.al. highlighted examples of such problems. The discussion in 12 was based on controllers designed in 60 for the various actions of the leader. These controllers were based on a third order, feedback linearizable model for the vehicle and were observed to perform well when simulated individually. However, when they were interfaced 91 with the coordination layer design of 34 and simulated in the SmartPath micro simulator 92 unsafe, high relative velocity collisions were observed under extreme tra c conditions. To remedy the situation, Frankel et.al. proposed leader mode controllers that explicitly incorporate safety considerations 27 . These controllers were based on a linear, second order vehicle model with actuation delay. Under the underlying assumption that the worst possible action of the preceding vehicle is to decelerate as fast as possible, it was shown that these controllers can guarantee low relative velocity collisions5. The proof was based on the construction of a safe set" that was rendered invariant if the proposed controller was applied and the preceding vehicle decelerated as fast as possible. In 44 , Puri-Varaiya proposed to use optimal control to determine the worst possible action of the preceding platoon, once a controller for the trailing platoon has been designed. It was also proposed to use the solution to the optimal control problem to obtain sets of initial conditions for which safety is guaranteed and hence establish guidelines for a discrete scheme to switch between the various controllers. The feasibility of the approach was illustrated by determining conditions under which the safety of some of the control laws designed in 60 can be guaranteed. In 18 , Lygeros et.al. introduced ideas from game theory to show that maximum deceleration by both leading and trailing vehicles is indeed a saddle solution, when the objective of the trailing vehicle is to avoid collision altogether6 . The results of 18 were based on the third order model used in 60 and made no assumptions about the relative deceleration capabilities of the vehicles. The calculations were used to determine safe" sets of initial conditions and classify the safe" controllers. Using the second order with delay model, Li et.al. were able to show that maximum deceleration by both leading and trailing vehicles is also a saddle solution when the objective of the trailing vehicle is to minimize the relative velocity at impact 28 . The calculations were based on the assumption that the deceleration capability of all vehicles is uniform. In 28 complete control schemes were designed based on these results and their performance in terms of stability and safety was proved. In 20 the assumption that all vehicles have uniform deceleration capability was relaxed. For the third order vehicle model it was shown that maximum deceleration by both leading and trailing vehicles is not necessarily a saddle solution in this case; in fact the saddle solution may initially involve maximum acceleration of both the leading and the trailing vehicles. Finally, in 21 , Lygeros et.al. proposed to introduce the e ect of collisions as a discrete disturbance to the system. Even though low relative velocity collisions were allowed by previous designs, their e ect on the system dynamics was never investigated. While this omission is insigni cant in the case of two vehicles, it may lead to catastrophic results in general, as it allows the relative velocity of collisions to increase geometrically down a long string of vehicles. In 21 ideas from game theory were used to derive saddle solutions for the third order vehicle model in the presence of collisions. It was shown that in this
5 The assumption was shown to be valid in this case in 28 . 6 The design algorithm presented in Chapter 3 was also introduced in 18 .

57

xB xA LB

A
x A
D

B
x B

Figure 6.3: Vehicle Following case formation and dissipation of platoons can not be carried out in safety without the cooperation of the coordination layer. Su cient conditions were established for the discrete design and it was shown that under these conditions safe operation can be guaranteed by controllers utilizing the saddle solutions. The work presented here is based on the results of 18, 20, 21 . The various elements are brought together and conditions on both the regulation and coordination layers are given under which safe operation of the AHS can be guaranteed. The analysis provides considerable insight into the physical restrictions of the system and suggests design modi cations that can be used to improve the closed loop system performance. Consider three vehicles labeled A, B and C moving along a single lane highway Figure 6.37 . We will primarily be interested in the interaction between vehicles A and B, vehicle C will be used only in certain cases, to isolate the system A-B from the rest of the highway. Assume that vehicles A and B have lengths LA and LB and let xA and xB denote their positions with respect to a xed reference on the road. Assume that vehicle B is leading while vehicle C comes last, i.e. xB xA xC 0. The problem we are interested in is the vehicle following problem: we assume no control over vehicle B and try to control vehicle A. The dynamics of the trailing vehicle will be approximated by a third order ordinary di erential equation:

6.2.3 Hybrid Vehicle Model

xA = bAxA; xA  + aAxAvA _ _

aA and bA are complicated nonlinear functions of the state with aA xA  6= 0. The rst two derivatives _ on xA arise from the laws of motion. The third has to do with the operation of the brakes and engine, which can be modeled by the throttle and brake angles acting through an integrator on some nonlinear dynamics involving engine time constants, aerodynamic drag, etc. 72, 89 . For our purposes the details of the nonlinear functions bA and aA are not important. Following the design of 60 , we will assume that feedback linearization has already been carried out, i.e. input: x x vAt = ,bAa_ A;x A + u 6.1 A _A 6.2
has been applied to yield:




7 Under the assumptions of Sections 6.2.5, A, B and C can in fact be entire platoons.

xA = u
58

6.3

We will design controllers for the resulting linear dynamics. We assume that the dynamics are constrained by the engine, tire and road conditions. More speci cally it is required that: A A xA 2 vmin ; vmax _
A For highway operation it is assumed that vehicles will not be allowed to go backwards, therefore vmin = 0 A is imposed by engine limitations. One of the objectives of the controllers we design will will be used. vmax be fuel e ciency. As a consequence the engine will not have to be pushed to its limits for maximum speed A A and therefore vmax will not feature in the safety calculations. We will assume vmax = 1 to simplify the A , etc. are given in the appendix. analysis. Typical values of amin

xA 2 aA ; aA min max ut 2 jmin ; jmax

Control Objectives
The objective is to design a controller for vehicle A. The requirements we impose on the design are safety, passenger comfort and e ciency. It is assumed that safety takes precedence over the other two requirements. Comfort will be assumed to be more important than e ciency. Quantitative de nitions of these design requirements will be given in Section 6.2.4.

The Leader" Mode

In the leader mode we can abstract the behavior of vehicle B by treating its unmeasured acceleration as a disturbance. Both the design speci cations and the system dynamics are independent of the absolute vehicle position. To remove the absolute position from the problem we introduce a new variable to measure the spacing between vehicles A and B: D = xB , xA , LB All pertinent information can now be encoded by the state vector: 2 3 2 3 xA _ x1 6 7 6 7 x = 6 xA 7 = 6 x2 7 2 R4 6 7 6 7 4 D 5 4 x3 5 _ x4 D From Section 6.2.3 the dynamics are: 3 3 2 3 2 2 0 0 0 1 0 0 7 7 6 7 6 6 x = 6 0 0 0 0 7 x + 6 1 7 u + 6 0 7 xB _ 7 6 7 6 6 5 4 0 0 0 15 405 407 1 0 0 ,1 0 0 = Ax + Bu + DxB 6.4 0 x0 = x For the vehicle following problem we are interested in regulating the spacing and relative velocity to a desired xed point. In addition, whenever possible, a vehicle may be required to track a certain velocity, vH , calculated by the roadside controllers in order to maximize throughput. These requirements can be encoded by means of three outputs: 2 3 2 3 D 0 0 1 0 y = 6 D 7 = 6 0 0 0 1 7x 4 _ 5 4 5 1 0 0 0 xA _ = Cx 59

Note that the assumed sensor arrangement can provide full state measurements. To complete the picture we also need to encode the constraints in the new coordinates. From the discussion in Section 6.2.3 it is required that, for all t:
A A B B xt 2 XC = x 2 R4 jx1 2 vmin ; vmax ; x2 2 aA ; aA ; x4 + x1 2 vmin ; vmax min max ut 2 U = jmin ; jmax xB t 2 D = aB ; aB min max
n o

The analysis can be greatly simpli ed if the input constraints are modi ed somewhat to guarantee that the state constraints are satis ed. We will assume that: 8 jmin ; jmax if x2 t 2 aA ; aA  min max ut 2 uminxt; umax xt = 0; jmax if x2 t = aA 6.5 min : j ;0 if x2 t = aA min max 8 h i B ; aB B B a if x4 t + x1 t 2 vmin ; vmax  h min max i B if x4 t + x1 t = vmin 6.6 xB t 2 dmin xt; dmax xt = h0; aB i max B ;0 B : a if x4 t + x1 t = vmax min This will ensure that the constraints on x2 and x1 + x4 will never be violated if x0 2 XC . A A The only state constraint that we have to worry about is x1 2 vmin ; vmax . For the reasons discussed above we will not be concerned with the upper bound too much. We can work around the lower bound by A A assuming that xA = x2 becomes zero when x1 = vmin is reached note that when x1 reaches vmin , x2  0. This modi cation will not have a drastic e ect on our analysis. The reason is that, what happens after x1 A reaches vmin = 0 is not very important, at least from the safety point of view. Recall that the constraint B 6.6 guarantees that xB  vmin = 0. Therefore, if no collision occurs until x1 = 0, none is going to occur from then on, unless vehicle A starts accelerating again. There are four kinds of exogenous inputs that in uence the system evolution: 1. the jerk input of vehicle A, ut, 2. the trajectory of xB t, 3. collisions of vehicle B with its preceding vehicle that a ect xB t and hence x4 , _ 4. collisions of vehicle A with vehicle C that a ect x1 and hence x4 . ut is assumed to be under the control of the designer. The remaining inputs, however, will have to be treated as disturbances to be guarded against. The acceleration disturbance xB t can be modeled as a piecewise continuous function of time:

xB : 0; 1 ,! dmin ; dmax
In the hybrid automaton terminology this is a continuous disturbance. The collision disturbances on the other hand can be modeled by an instantaneous jump in the state of the system. Assuming that the vehicle collisions are elastic, a collision of vehicle B at time TB with relative velocity vB will result in a reset of the state: 0 x4 TB  := x4 TB  + vB 60

Similarly an elastic collision between vehicles A and C at time TC and with relative velocity vC will result in a reset of the state:8
0 x1 TC  := x1 TC  , vC 0 x4 TC  := x4 TC  + vC

This model will be quite accurate if the relative velocity at impact is low, in which case the collision is more or less elastic 93 . We will assume that the relative velocity of all collisions is bounded in an interval va ; 0 . In hybrid system terminology the collisions are discrete disturbances. To model the general situation assume that vehicle B experiences nB collisions and that the ith collision takes place at TBi with relative velocity vBi similarly for vehicle C. Then the overall disturbance experienced by vehicle A can be encoded by:
B C d = fxB ; fTBi ; vBi gn=1 fTCi ; vCi gn=1 g i i

6.7

Physical considerations restrict the set of allowable disturbances. In particular:

d 2 D = fd j xB t 2 dmin ; dmax ; 0  TB1  TB2  : : : ; vBi 2 va ; 0 for all i; 0  TC1  TC2  : : : ; vCi 2 va ; 0 for all ig
In the ensuing discussion we will be interested in situations where vehicle B and vehicle C can experience at most one collision nC = nB = 1. To remove some of the indices we will use TB and TC to denote the collision times in this case. The requirement that vehicle B does not go backward after the collision can be encoded by restricting the set of allowable disturbances:

D = fd j xB t 2 dmin ; dmax ; 0  TB ; vB 2 maxfva ; x4 TB  + x1 TB g; 0 0  TC ; vC 2 va ; 0 g

6.8

Collisions between vehicles C and A can not lead to a violation of the state constraints, since we assume A vmax = 1. By abuse of notation we will use the same representation when vehicles B and C experience no collision. We will simply assume that vB = vC = 0. Under these assumptions, the leader evolution is determined by a hybrid dynamical system. It is shown pictorially in Figure 6.4. The discrete state can take one of ve values, 00, 01, 10, 11 and CRASH. The rst four keep track of whether vehicles B and C have experienced a collision 1 or not 0. The discrete state CRASH is introduced to model the situation where vehicle A crashes into vehicle B. It will be used to translate the safety problem into a reachability question on hybrid automata. There are six continuous states, two keeping track of when the collision occurred and four keeping track of the state of vehicle A. The initial conditions, node invariants, transition guards, events and reset maps should be obvious from the gure. The automaton runs can be obtained using the variation of constants formula 94 . De ne the step function:  1T t = 0 if t  T 1 if t T
8 Note that in the coordinate system de ned above, the collisions imply that

vB  0 and vC  0.

61

T B :=0

T C :=0

x(0):=x0

x = Ax + Bu + DxB T B= 1

. . .

00

..

T C= 1 u [u min , u max ] .. x [d min , d max ]

B

x3 - 0 > Coll_B, x := x + v
4 4

v [max{v , -x4 -x1 }, 0] B a

Crash, x3 - 0 <

Coll_C, x := x - v 1 1 C x := x + v 4 4 C v [va , 0] C

x = Ax + Bu + DxB T B= 0

. . .

10

..

CRASH
x = 0 Crash, x3 - 0 < T B {0,1} T C {0,1}

x = Ax + Bu + DxB T B= 1

T C= 1 u [u min , u max ]

. .

. . .

01

..

T C= 0 Crash, x3 - 0 < u [u min , u max ] .. x [d min , d max ]

B

..

x B [d min , d max ] x3 - 0 >

x3 - 0 >

Crash, x3 - 0 < Coll_C, x := x - v 1 1 C x := x + v 4 4 C v [va , 0] C

Coll_B, x := x + v
4 4

11 . .. x = Ax + Bu + Dx B . T = 0 .B
T = 0 C u e [u min, u max] .. x e [d min, d max] B x3 - 0 >

v [max{v , -x4 -x1 }, 0] B a

Figure 6.4: Hybrid Automaton for the Platoon Leader Then:

Z 20 6 6 6 4

xt = eAt x0

+ +

t At,  e Bu

7 1TB t vB + t ,0T 7 1TC t vC 7 t , TB C 5 1 1 7 7 7 5 6 6 6 4

0 0

d +

t At,  e DxB 
2

,1

d

Using the fact that A is nilpotent A3 = 0 we obtain: 1 t 6 6 0 eAt = 6 0 ,t1 =2 2 4 0 ,t 62

2

0 0 1 0

0 0 1

3 7 7 7 5

This leads to:

2 6 6 6 4

xt =
+

x0 + tx0 t, 1 2 7 Z t6 7 x0 1 7 6 7 2 7+ 6 5 ,t2x0 =2 + x0 + tx0 5 0 4 ,t , 2 =2 7 u d 2 3 4 ,tx0 + x0 ,t + 2 4

2 6 4

t6 6

t,
1

0 0

3 7 7 7 5

2 6

xB  d + 6 t , T 7 1TB t vB + 6 t ,0T 7 1TC t vC 6 7 6 7 4 4 B 5 C 5

7 6 7

0 0 1

,1
1

The Follower" Mode

The objective of the followers of a platoon is to maintain a small constant separation independent of vehicle speed from the preceding vehicle. The design of feedback control laws for safe follower operation has received considerable attention in recent years 95, 72 . There are three problems that make the design particularly challenging. The more obvious one is regulating to the follower xed spacing typically 1 meter without crashing into the preceding vehicle. In 72 it was shown that, in order to achieve this objective, the vehicle controller needs information about the acceleration of the preceding vehicle. Unlike the spacing and relative velocity information, preceding vehicle acceleration is not readily available through sensors and will have to be obtained through intervehicle communications. Because of the tight spacings communication between adjacent followers can easily be established by a point-to-point infrared link. The second problem associated with follower operation is attenuation of disturbances along the platoon. Unless special care is taken during the design, a small disturbance of the platoon leader e.g. a transient change in velocity may get ampli ed as it propagates from one follower to the next, possibly resulting in collisions down the string. In 72 it was shown that, in order to avoid this slinky e ect", followers need information about the state of the platoon leader in addition to the information about the vehicle immediately ahead of them. Because the accumulated delays caused by point-to-point communication increase with the size of the platoon, the leader information may have to be broadcast by the leader using a radio link for example. Finally, in 72 it was also shown that, under the information structure dictated by the previous observations, the control law will invariably result in ampli cation of the control e ort exerted by followers further down the platoon. This fact imposes restrictions on the operation of the leader. For example, assume that the braking force is ampli ed by at most a factor of  1, in a platoon of identical vehicles with deceleration capability amin . Then the maximum deceleration of the lead vehicle should be amin in order to avoid actuator saturation and the possibility of collisions within the platoon. This gure should be used as a deceleration constraint on the leader of the platoon, rather than amin itself. The above discussion indicates that the information structure required for safe operation within a platoon is very di erent from the one for the platoon leader. Here we will concentrate primarily on the leader operation. We will avoid dealing with the complications of the follower design by introducing the assumption that the follower operation is always safe Assumption 7. The ensuing discussion will provide some justi cation for this assumption. For more details the reader is referred to 72 . In the leader mode the problem can be viewed as a two player zero sum game, one player being the action of vehicle A and the other the disturbances action of vehicle B and possible collisions. The two players compete over a number of cost functions, Ji , encoding various desirable properties of the system. All cost 63

6.2.4 Design Requirements

functions will be independent of the discrete state of the system q 2 f00; 01; 10; 11; CRASH g, therefore only the continuous state x will appear in the expressions. 1. Safety No Collision: J1 x0 ; u; d = , tinf x3 t 6.9 0 A safe maneuver is one where: J1 x0 ; u; d  C1 = 0 m Allowing J1 = 0 meters makes the limiting case where the vehicles just touch with zero relative velocity acceptable. 2. Safety Low Relative Velocity Collision: Assume vehicles A and B collide at time T . J10 x0 ; u; d = jx4 T j 6.10 A safe collision is one where the relative velocity is less than a certain threshold: 0 J10 x0 ; u; d  C1 = ,va In the subsequent analysis va = ,3ms,1 will be used. 3. Comfort Bounded Jerk: J2 x0 ; u; d = sup jutj 6.11 A comfortable maneuver is one where:
t0

4. E ciency Fast Convergence:

The value C2 = 2:5ms,3 is suggested in the transportation studies literature.

J2 x0 ; u; d  C2
1
0

J3 x0 ; u; d =

y  , yd T P y  , yd d

6.12

where yd is the desired xed point for a given maneuver and P  0. The xed points and the choice of P will depend on the maneuver in question. For steady state leader operation and for the split maneuver, the xed point is: 2 3 v vH + p 7 L 4 0 yd = 6 5 for some v ; p 0 refer to 60 . For the join maneuver on the other hand the xed point is: 2 3 1 J 4 5 yd = 6 0 7 0 with P having zeros in its third row and column. This re ects the fact that tracking the optimum velocity vH is not important for the join maneuver, the objective is to catch up with the platoon ahead. Other cost functions may be used to encode e ciency, for example, convergence in minimum time. The choice will not in uence the safety of the design. The discussion of Section 6.2.3 indicates that, for the follower mode, the problem is harder to formulate as a two person game, as the lead vehicle information should also be taken into account. However, the design requirements on the control laws can still be expressed by similar cost functions. 64

vH

The basic underlying assumption is what constitutes a safe" system. Ideally we would like no collisions to occur on the AHS, under normal operation. Our analysis will indicate that this requirement may be too restrictive in terms of e ciency, as it does not allow platoons to join or break up. Previous studies Hitchcock 71  indicate that collisions with small relative velocities are not likely to result in serious damage. We will therefore assume that: Assumption 1 Low Relative Velocity Collisions Collisions with relative velocity smaller than a threshold va will not result in structural or functional damage to the vehicles or injury to the passengers. Assumption 1 motivates the following de nition: De nition 15 A safe collision between two vehicles is one where the relative velocity at impact is less than va . An Automated Highway System where vehicles experience only safe collisions will be called a safe AHS. Motivated by the analysis of 71 we will use va = ,3ms,1 in the calculations. Producing a completely safe AHS design is likely to be impossible, as one can not exclude the occurrence of combinations of faults that may cause unsafe collisions at any moment for examples see 16 . The following assumption is therefore crucial to proving safety: Assumption 2 Normal Mode AHS Operation Operation on the AHS is normal": no faults occur and the environmental conditions are benign. By assuming normal conditions we exclude hardware and software faults and adverse environmental conditions such as snow that limit the capabilities of the system. As a consequence of Assumption 2: The boundaries of sensor and actuator ranges e.g. aA , etc. can be assumed to be feasible. min Vehicles do not exhibit abnormal" or erratic" behavior, for example, do not change lane without warning due to a tire burst. Our analysis can be extended to a more fault tolerant" design 96 , if appropriate extensions to the de nitions of safety are given 16 . The outline of this extension will be given in Chapter 7.

6.2.5 Assumptions

Assumption 3 Known Model All vehicle model parameters are known.

This implies that linearization by state feedback equation 6.3 is feasible. Assumption 3 can be relaxed by an adaptive or robust design.

Assumption 4 Perfect Sensors Sensors can be used to provide noiseless, continuous measurements
of velocity and acceleration of the vehicle in question as well as distance and relative velocity with respect to the vehicle in front and in the adjacent lanes.

A consequence of Assumption 4 is that the leader mode controllers can utilize full state feedback without making use of an observer. We assume that the sensor ranges are limited. The limits given in the appendix and the analysis presented here can be used to infer limitations on the AHS operation dictated by safety e.g. limits on the speed of travel. These issues are discussed further in Section 6.8.

Assumption 5 Elastic Collisions Collisions are instantaneous, perfectly elastic and all vehicles have
equal masses.

65

This assumption is clearly unrealistic. Assuming perfectly elastic collisions is not unreasonable for low relative velocity collisions 93 . The assumption that all vehicles have equal mass can easily be relaxed. This assumption leads to: Fact 1 Under Assumption 5, a collision with relative velocity at most va induces an instantaneous change in velocity of at most va in the colliding vehicles. Fact 1 allows us to model collisions as instantaneous changes in velocity as required by the model. It will be needed as an implicit assumption in the proofs of the subsequent lemmas.

Assumption 6 Perfect Lateral Control Lateral operation is perfect

This assumption is not too unrealistic, as both simulation and experimental results indicate 97, 87 . If needed, the techniques presented here can easily be used to modify the lateral design and or obtain limits of guaranteed performance. As discussed in Section 6.2.3, designing control laws for the followers is a lot more complicated than for the leaders. The additional complication arises mostly because of the extra information that is available to the follower in a platoon and that is needed for safe operation. Fitting the follower scenario in this framework is the topic of ongoing work. Here we will bypass the problem by introducing an additional assumption. We will assume that a follower control law, uF , a safe set of initial conditions in the follower state space, VF , and a time, TF possibly dependent on the size of the platoon have been determined, such that: Assumption 7 String Stability under Front Rear Collision If uF is applied by all followers in a platoon during an interval ti ; tf  with ti tf and: 1. The acceleration of the leader is bounded in the interval dmin ; dmax , for all t 2 ti ; tf , 2. either the leader or the last vehicle of the platoon experiences at most one collision in ti ; tf , with relative velocity at most va , 3. x0 = xti  2 VF for all followers, then any collisions between vehicles in the platoon will be with relative velocity at most va . Moreover, no further collisions will take place TF seconds after the rst collision. Note that this assumption allows us to mostly ignore the internal dynamics of the platoon. As a consequence we will collapse the platoons for the purposes of safety and assume that they are all one vehicle long. The work of 72, 93 leads us to believe that this is not an unrealistic assumption. We are currently working on modifying the follower design to t it in this framework and subsequently prove Assumption 7 as a theorem. It seems likely that information about the last vehicle will need to be propagated to all vehicles in the platoon to guarantee Assumption 7. Finally we introduce one more assumption relating to the follower operation: Assumption 8 Join Split Termination After a safe collision, follower operation can be established between the colliding vehicles. This assumption is essentially a corollary" of Assumption 7. If Assumption 7 holds, the initial condition resulting from a safe collision should be within the safe set for follower operation. Moreover, at impact the spacing is de nitely small enough to establish communication. 66

6.3 Single Lane Safety Theorem

We will start by proposing a design for the discrete part of the hybrid control scheme and proving that it guarantees safe operation. The proof will be based on a series of lemmas concerning the continuous part of the hybrid control scheme. In subsequent sections we will derive conditions on the continuous controllers which guarantee that the lemmas indeed hold and provide possible designs that satisfy these conditions. The lemmas require the underlying Assumptions 2 6 not all assumptions are needed by all the lemmas though. The proofs are given in Sections 6.5 and 6.6.

Leader law: In Section 6.5 a continuous control law, uL, will be designed for a leader of a platoon. The
design process will also establish a safe" set of initial conditions, VL :

6.3.1 Background Lemmas

VL = fx 2 XC jJ1 x0 ; uL; d  C1 g

for all d. For this pair, uL ; VL , we will show that: Lemma 7 Leader Safety Lemma Assume that uL is applied during an interval ti; tf  with ti tf and that: 1. xB t 2 dmin ; dmax , for all t 2 ti ; tf , 2. B experiences at most one collision in ti ; tf , with relative velocity at most va , 3. Vehicles C and A collide at most once in ti ; tf , with relative velocity at most va , 4. x0 = xti  2 VL . Then vehicles A and B will not collide in ti ; tf . Moreover there exists a nite time TL such that if x leaves VL at time t, it is guaranteed to return to VL by at most t + TL .

Join and Split laws: In Section 6.6 the design of control laws for the leader of a platoon joining uJ 
and splitting uS  from the platoon ahead will be outlined. The design process will also produce safe" sets of initial conditions, VJ and VS for the two maneuvers:
0 VJ = fx 2 XC jJ10 x0; uJ ; d  C1 g 0 VS = fx 2 XC jJ10 x0; uS ; d  C1 g 0 for all d. It will be shown that, under the assumption that a saddle solution to the game with cost J1 can be obtained: Lemma 8 Join Safety Lemma Assume that uJ is applied during an interval ti; tf  with ti tf and that: 1. xB t 2 dmin ; dmax , for all t 2 ti ; tf , 2. B does not experience a collision in ti ; tf , 3. Vehicles C and A do not collide in ti ; tf , 4. x0 = xti  2 VJ .

67

Then vehicles A and B may collide at most once during ti ; tf , with relative velocity at most va .

Lemma 9 Split Safety Lemma Assume that uS is applied during an interval ti; tf  with ti tf and
that: 1. xB t 2 dmin ; dmax , for all t 2 ti ; tf , 2. B does not experience a collision in ti ; tf , 3. Vehicles C and A do not collide in ti ; tf , 4. x0 = xti  2 VS . Then vehicles A and B may collide at most once during ti ; tf , with relative velocity at most va .

The calculations will indicate that, without loss of generality, ti = 0 can be used in all the lemmas. The following theorem states conditions on the discrete design such that the hybrid control scheme is safe. Theorem 7 Single Lane Safety Theorem If: 1. The discrete controller may switch to ui at time t only if xt 2 Vi , where i 2 fL; J; S; F g. 2. The discrete controller allows at most one maneuver join or split per vehicle at a time9 . For a given vehicle the beginning of a new maneuver takes place at least TF + TL seconds after the end of the last maneuver. 3. After a collision between two vehicles follower operation is established. 4. Assumptions 2 8 are satis ed. 5. The AHS contains a nite number of vehicles. Then the single lane AHS is safe in the sense of De nition 15.

6.3.2 Safety Theorem

Comments: The rst assumption of the theorem imposes requirements on the design of the interface between the continuous and discrete controllers. In particular the safe sets, Vi , determine the safety checks and terminating conditions that need to be checked by the interface before starting and nishing a maneuver refer to 91 for more details. The one at a time" part of the second theorem assumption is already satis ed by the coordination level design of 34 . Some additions are needed to guarantee the timing constraints. Justi cation for the third and fourth theorem assumptions is given in Section 6.2.5. It should be noted that the third assumption can easily be implemented by simple modi cation of the join and split protocols, in particular the conditions under which the maneuvers get completed or aborted respectively. The last assumption is need to break the symmetry" of the problem and provides a starting point for the induction in the proof.
from cooperate and therefore are both involved in the maneuver.
9 Following 74 , it is assumed that during a maneuver the joining splitting vehicle and the vehicle being joined to split

68

Assumption 7 we will collapse all platoons to a single vehicle and ignore the case i = F . Note that if Lemmas 7 9 hold, then the conclusion of the theorem follows trivially. Therefore we only need to check that the assumptions of the lemmas hold. Assumption 1 of Lemmas 7 9 holds by theorem assumption 4. Assumption 4 of Lemmas 7 9 holds by theorem assumption 1. To check Assumptions 2 and 3 of the lemmas we will use an induction argument on the number of vehicles. The induction argument will start at 4 vehicles. The rst vehicle will be executing the leader controller and can not experience a collision with anyone ahead of it simply because there is no one there. The remaining three vehicles will be labeled B, A and C as in Figure 6.3. Let i; j; k denote the control laws executed by vehicle B, A and C respectively, where i; j; k 2 fL; J; S g u is suppressed to simplify the notation. There are 33 = 27 possible triplets. Theorem assumption 2 implies that some of them are impossible all the ones with S or J in adjacent locations. This eliminates 16 triplets. We enumerate the remaining: for L; L; L all assumptions of Lemma 7 are satis ed for vehicles A, B and C. for L; L; J  and L; L; S  all assumptions of Lemma 7 are satis ed for vehicles A and B. Hence all assumptions of Lemma 8 9 are satis ed for vehicle C. for L; S; L and L; J; L all assumptions of Lemma 7 are satis ed for vehicles B and C and all assumptions of Lemma 8 9 for vehicle A. for J; L; L and S; L; L all assumptions of Lemma 7 are satis ed for vehicles A and C and all assumptions of Lemma 8 9 for vehicle B. for J; L; J , J; L; S , S; L; J  and S; L; S  all assumptions of Lemma 7 are satis ed for vehicle A and all assumptions of Lemma 8 9 for vehicles B and C. Note that the 4 vehicle case contains the simpler 3, 2 and 1 vehicle cases they reduce to L; ; ; L; L;  and L; L; L respectively where  denotes a do not care" entry. The above enumeration indicates that for 4 vehicles: the system is safe, a vehicle never collides under L, any collision under S or J will be at relative velocity at most va Note that the above assertions will also hold for the rst 4 vehicles of any chain provided that the last vehicle is either executing J or S and is not hit from behind or is executing L and is hit from behind with relative velocity at most va . The last observation allows us to extend the argument to an arbitrary chain of vehicles of nite length. Assume that the 3 assertions hold for a chain of N vehicles. Consider a chain of N + 1 vehicles. The following situations are possible: Vehicle N + 1 executes J or S . In this case vehicle N will have to be executing L by Theorem assumption 2. By induction hypothesis vehicle N will not experience a front collision. By Lemma 8 9 vehicle N + 1 can collide with vehicle N with relative velocity at most va . This collision is safe for N + 1 and leaves the N chain safe as well. Vehicle N +1 executes L. Vehicle N can execute any control law. By induction hypothesis vehicle N can experience at most one front end collision with relative velocity at most va . By Lemma 7 vehicle N + 1 will not collide with vehicle N . 69

Proof: By design, any collision will occur while some ui, i 2 fL; J; S; F g, is being used. Following

Figure 6.5: Lane Change The conclusion follows by induction. 2 From the proof, it immediately follows that, under the theorem assumptions: Corollary 1 Leader Safety A vehicle under the leader control law, uL, will never collide.

Corollary 2 Join Split Safety The relative velocity of any collisions that take place during the join
or split maneuvers will be smaller than va . smaller than va .

Corollary 3 Follower Safety The relative velocity of any collisions experienced by a follower will be
The assumption that the chain of vehicles is nite is a bit disturbing, but it is needed to break the symmetry of the problem. It will be satis ed for example if we assume that the freeway is initially empty and is slowly populated by vehicles that follow the remaining assumptions of the theorem. Based on the theorem assumptions the task of designing the interface between the continuous and the discrete controllers becomes almost trivial. The complete interface design will be given after the more complicated safety theorem for the multi lane system is introduced in Section 6.4.

6.4 Multi Lane Safety Theorem

The only additional complication in a multi-lane AHS is that we have to guarantee that no unsafe collisions will occur during lane changes. Here we will assume the lane change scenario of 74 , where a vehicle is a free agent single car platoon both before and after the lane change. With minimal changes the same analysis can be applied to other lane change scenarios changing from to the end middle of a platoon, entire platoon lane change, etc.. A typical lane change situation is shown in Figure 6.5. Vehicle A wishes to move to the lane where vehicles D and E are moving. To avoid introducing additional notation let xAB denote the state of vehicle A with respect to vehicle B. Here state" has the same interpretation as in Section 6.2.3 velocity, acceleration, spacing and relative velocity. Also let uA denote the input applied by vehicle A and uAB the L input calculated by the lead controller of Section 6.5 for the state xAB similarly for the other subscripts and superscripts.

70

As for the single lane case, the safety theorem will be based on a sequence of lemmas about the individual maneuvers. The lemmas require the underlying Assumptions 2 8 not all assumptions are needed by all the lemmas though. The proofs are given in Section 6.7. In terms of continuous controllers, the lane change can be broken up into two parts. In the rst part vehicle A is aligned with an appropriate gap in the target lane. This is achieved by deceleration of either vehicle A or vehicle D. The continuous law applied at this stage is called the decelerate law. In Section 6.7 a decelerate law uD  and a safe set of initial conditions VD  will be designed. It will be shown that: Lemma 10 Decelerate Law Safety Lemma Assume that uD is applied by vehicle A during an interval ti; tf  with ti tf and that: 1. xB t 2 dmin ; dmax , for all t 2 ti ; tf , 2. B experiences at most one collision in ti ; tf , with relative velocity at most va , 3. Vehicles C and A do not collide in ti ; tf , 4. x0 = xAB ti  2 VD . AB Then vehicles A and B will not collide in ti ; tf . Moreover, if the state leaves VD at time t, it is guaranteed to return to VD by at most t + TL . For some initial conditions, the decelerate law will have to be applied by vehicle D, to allow vehicle A to move into the lane ahead of it. This decision is taken by the coordination layer. Lemma 10 also holds in this case for vehicles D and E. The second phase of the lane change is the actual movement of vehicle from lane to lane. Because of Assumption 6 we will only be concerned with the longitudinal aspects of this movement. In Section 6.7 longitudinal laws for vehicles A, C and D uMA ; uMC and uMD  and safe sets of initial conditions VMA ; VMC and VMD  will be designed. It will be shown that: Lemma 11 Change Lane Safety Lemma Assume that uMA; uMC and uMD are applied during an interval ti ; tf  with ti tf and that: 1. xB t 2 dmin ; dmax and xE t 2 dmin ; dmax for all t 2 ti ; tf , 2. Vehicles B and E experience at most one collision each in ti ; tf , with relative velocity at most va , 3. Vehicle D experiences at most one rear-end collision in ti ; tf , with relative velocity at most va , 4. x0 = xAB ti  2 VMA , x0 = xAE ti  2 VMA , x0 = xDA ti  2 VMD and x0 = xCA ti  2 VMC . AB AE DA CA Then no collision between vehicles A, B, C, D, E will occur during ti ; tf . Moreover, if xAB ; xAE ; xDA and xCA leave the corresponding safe sets at time t they are guaranteed to return by at most t + TL . The interval ti ; tf  will typically represent the entire time it takes to move from one lane to the next roughly 5 seconds under current lateral laws 87 . For the purposes of safety, entry and exit from the automated highway will be treated as a simpli ed version of a lane change. In particular, entry will be modeled as a lane change from the on-ramp to an automated lane, with vehicles C and B missing. Exit will be modeled as a lane change from an automated lane to an o ramp, with vehicles D, E and F missing. The entry exit scheme obtained in this way will be safe, but may be ine cient in terms of throughput. For a more thorough treatment of entry and exit scenarios the reader is referred to 98 . All the scenarios discussed therein are amenable to our analysis techniques. 71

6.4.1 Background Lemmas

The following theorem states conditions on the discrete design such that the hybrid control scheme is safe. Theorem 8 Multi Lane Safety Theorem If: 1. The discrete controller can switch to ui , i 2 fL; J; S; F; D; MA; MC; MDg, only if the state is in the corresponding safe set. 2. The discrete controller allows each vehicle to be involved in at most one maneuver entry, exit, decelerate, move, join or split at a time. For a given vehicle, the beginning of new join or split maneuver takes place at least TF + TL seconds after the end of the last join or split. 3. After any safe collision follower operation is established. 4. Assumptions 2 8 are satis ed. 5. The AHS contains a nite number of vehicles. Then the multi lane AHS is safe in the sense of De nition 15. Theorem Assumption 2 is to be interpreted according to the communication protocol design of 34 . A maneuver involves at least two cooperating platoons. For a join split both the joining splitting platoon and the platoon being joined to split from are involved. For a lane change entry exit the vehicle that is changing lane, a vehicle in the target lane either D or E and the vehicle in the lane adjacent to the target lane F are involved. The protocols do not require cooperation if some of these vehicles are missing. The proof of Theorem 8 follows by an induction argument similar to the one given for Theorem 7 slightly more complicated due to the presence of more maneuvers. The role of the lane changing vehicles in the proof is very similar to that of the lead vehicles in the proof of Theorem 7. In the multi-lane case we have, in addition to Corollaries 1 3: Corollary 4 Lane Change Safety A vehicle will never collide while executing a lane change Decelerate and Move control laws and Entry or an Exit. Assumption 1 of the theorem can be used to obtain an interface between the hybrid controller for the regulation layer and the nite state world of the coordination layer. The interface induced by the theorem is shown in Figure 6.6. It is essentially a nite state machine does not have a continuous state of its own, whose transitions contain: 1. Guards on the continuous state of the regulation layer. 2. An event issued by the coordination layer. The event represents the maneuver commanded by the coordination layer: Enter, Exit, Join, Split, Decelerate and Move. The transitions are taken when the coordination layer command is issued if the guard is satis ed. The interface of Figure 6.6 gives a compact representation of the necessary conditions that the guards on the edges need to satisfy. Transitions out of the states Move and Decelerate are automatically enabled if the entering transition guard is satis ed and the proposed control laws are applied refer to the proofs of Lemmas 10 and 11. The transitions may be enabled for quite some time possibly throughout the maneuver before they are taken. When the transitions takes place will be determined by other considerations coordination layer commands, criteria for aborting maneuvers or declaring them complete, etc.. Essentially each of the states of Figure 6.6 contains a small nite state machine that can be used to determine when transitions out of the state are executed in addition to the guards shown in the gure. For a more detailed discussion of this point refer to 91 . 72

6.4.2 Safety Theorem

MOVE
Move, (xAB V MA) (x V ) AE MA (x V ) CA MC (x V ) DA MD xV L

Enter

Join, xV J

JOIN

ENTRY

xV F

xV L

LEAD
Exit, xV L

FOLLOW

EXIT

xV L

Decelerate, (x V ) AB D

SPLIT

Split, xV S

DECEL.

Figure 6.6: Interface Design

6.5 The Leader Control Law

According to Lemma 7 the safety criterion used for the leader should be:

6.5.1 Design for safety

J1 x0 ; u; d = , tinf x3t t

i

ti = 0 can be assumed without loss of generality.

The design of a safe controller for the above criterion no collision can be posed as a reachability question on the hybrid automaton of Figure 6.4. To answer the safety question we would like to know under what conditions the discrete state CRASH is reachable. We will approach this problem from the point of view of dynamical games. In particular, we will seek a saddle solution u ; d  for the two player 1 1 zero sum game with cost function J1 . Because of the nature of the cost function it is impossible to tackle this problem with the conventional game theoretic techniques Hamilton-Jacobi-Isaacs equations, Maximum Principle, etc.. Instead, we will try to guess the saddle solution and then show that it satis es the de nition. Consider the candidate saddle strategy, u ; d , given by: 1 1

u t = 1 d t = 1
where:

jmin if t  T1 0 if t T1  ; T  ; v ; T  ; v g fxB B B C C


6.13 6.14

x t = B

 TB = ti = 0

dmin if t  T2 0 if t T2
73

6.15

 vB = maxfva ; x4 TB  + x1 TB g  TC = ti = 0  vC = va T1 is the time when the acceleration of vehicle A reaches aA under jmin and T2 the time when vehicle B min stops under dmin . Let T3 be the time when vehicle A stops. Then: A 0 T1 = amin , x2 6.16 jmin  0 0 T2 = , vB + x1 + x4 6.17 d
8 :

T3 =

,x0 , x0 2 ,2jmin x0 , vC  2 2 1  jmin aA ,x0 2 ,2jmin x0 , vC  1  min 2 A 2jmin amin

min p

if 0  T3  T1 if T1  T3

6.18

A simple calculation shows that, under the assumed input and disturbance constraints: Lemma 12 If x0 2 XC then xt 2 XC for all t 2 0; T3 .

Lemma 13 u ; d  is globally a saddle solution for cost J1 x0 ; u; d. 1 1

Proof: For u ; d  to be a saddle solution we need to show that a unilateral change in strategy leaves 1 1
the player who decided to change worse o . Let x t denote the state at time t under the inputs u ; d . 1 1 In particular: Z t Z t 2 h i xt = 0 ,t2=2 1 t x0 , t ,  u  d + t , x  d 1 B 3 2 0 0  + v  t + vB C Z Z h i  t = 0 ,t 0 1 x0 , t t , u  d + t x  d +  v + v  x4 1 B B C First, x d = d and let u vary. Let xt denote the state at time t under the inputs u; d . Then: 1 1
Z Z 0 , t t , u d + t x  d x4t = 0 ,t 0 1 x 0 0 B  + v  + vB C Z t  x4t , xt = t , u   , u d 1 4 0 h i

We need to distinguish two cases: 1. t  T1 . The bounds on u imply that:

u   u   1

 t , u   , u d  0 1 0  x4 t , x t  0 4

2. t  T1 . Recall that x t is piecewise constant, with a discontinuity at T2 which may be either B greater or less than T1 . Therefore x4 t and x t are piecewise di erentiable, with derivatives 4 ,x2t and ,xt respectively. By de nition of T1, xt = aA  x2t in the interval of interest. 2 2 min Therefore, as x4 0 = x 0, x t  x4 t. 4 4 74

In both cases x3 t = x4 t  x t = x t. Using the fact that x 0 = x3 0 = x0 and integrating: _ _3 4 3 3

x3 t  x t for all t 3  , tinf0 x3t  , tinf0 x t   3  J1 x0 ; u; d   J1 x0 ; u ; d  1 1 1 x4 t = 0 ,t 0 1 x0 , t , u  d + 1
0
h i Z

6.19

Now x u and allow d to vary. Let xt denote the state at time t under the inputs u ; d. Then: 1 1
t
Z

xB  d + vB 1TB t + vC 1TC t

Therefore:

x4 t , xt = 4
Note that:

  xB   , x  d +  vB 1TB t , vB  +  vC 1TC , vC  B

9

 vC = va =  vC  va = vC 1TC t , vC  0 va  0 ;    If vB = va , the same is true for the term  vB 1TB t , vB . If vB va , then x 0 + x 0 = 0 recall 4 1  = 0 and therefore x t + x t  0 once vehicle B stops it never starts moving again under d . that TB 4 1 1 But, x4 t + x1 t  0 by the assumed state constraint and x1 t = x t as x1 t does not depend on 1  vB . Therefore, if vB va , x4 t , x t  0 for all t  0. 4 Overall, under the assumed constraints on vB and vC , either x4 t  x t or the term outside the 4

integral is positive. For the integral term we need to distinguish two cases: 1. t  T2 . The bounds on xB imply that:
Z     t xB   , x  d xB    xB B 0 R

0

0 2. t  T2 . Then, by de nition of T2 , 0t x  d = ,x0 + xR . The state constraints imply that 1 4 B x1 t+ x4 t  0 vehicle B does not go backwards, therefore 0t xB  d  ,x0 + x0. Subtracting, 1 4 Rt  0 xB   , xB  d  0. The stopping time for B under d will always be greater than the stopping time under d . In both cases, we 1 are able to conclude that x3 t = x4 t  x t = x t. Integrating this inequality from 0 to t and using _ _3 4 the fact that x 0 = x3 0 = x0 : 3 3

x3 t  x t for all t 3  , tinf0 x3t  , tinf0 x t   3  J1 x0 ; u ; d  J1 x0 ; u ; d  1 1 1
Combining inequalities 6.20 and 6.19:

6.20 6.21

J1 x0 ; u ; d  J1 x0   J1 x0 ; u; d  1 1

for all d and u. By de nition, u ; d  is globally a saddle solution. 2 1 1 75

Safe set boundary for x3(0)=20, 30 and 40m

5
x4(0)

10

15

5 20 10 0 15 20 x1(0) 25 5 x2(0)

Figure 6.7: Safe set of initial conditions for x0 = 20; 30 and 40 meters 3

Safe Set of Initial Conditions.

Next we need to determine the set of initial conditions that can be safe under the lead control. Let:

J1 x0  = J1 x0 ; u ; d  1 1

Recall that x t is a continuously di erentiable function of t de ned on the compact interval 0; minfT3 ; tf g , 3 with derivative x t. Therefore: 4 ^ Lemma 14 There exist T 2 0; minfT3 ; tf g such that:

J1 x0  = ,x T  3 ^

 The calculations for analytically determining J1 x0  from this lemma are rather messy. However, as the set of times T where x T  = 0 is nite, we can easily carry out the calculation numerically. Indeed, 4  a simple computer program was written to calculate J1 x0  for various values of x0 . It should be noted 0 only enters the calculations as a constant o set on x3 t. Figure 6.7 shows the set of points where that x3 J10 x0  = 0 for some values of x0. Lemma 13 implies that the surfaces of Figure 6.7 are 2D slices of the 3 3D boundary of the safe set V1 for various values of x0 . Any initial condition on or above these surfaces 3 will not lead to a collision, provided u = u . The higher surfaces correspond to smaller initial spacings x0 1 3 the top to 20m, the next to 30m etc.. As expected the safe set shrinks as x0 decreases. Figure 6.7 can 3 be used to infer certain interesting system restrictions. For example, the graph indicates that steady state following at 25ms,1 can only take place safely at spacings of about 40m or greater otherwise the initial conditions corresponding to x0 = 0m=s become unsafe. 4

^ ^ Moreover, either T = 0 or T = minfT3 ; tf g or x T  = 0. 4 ^

76

It should be noted that the safe set depends on the discrete state:

q 2 f00; 01; 10; 11; CRASH g

Even though J1 is independent of q, the set of allowable disturbances is larger at q = 00, smaller for q 2 f01; 10g and even smaller for q = 1110 . In particular some collision disturbances are excluded in some discrete states. A simple corollary of the proof of Lemma 13 is:     Corollary 5 The pairs u ; fx ; t; 0; TC ; vC g, u ; fx ; TB ; vB ; t; 0g and u ; fx ; t; 0; t; 0g 1 B 1 B 1 B are globally saddle solutions for J1 on t; tf , if the discrete state becomes q = 10; 01; 11 at time t respectively. The e ect of the reduced disturbance is to move the boundary of the safe set down" refer to Figure 6.7 by jva j if q 2 f01; 10g and by 2jva j if q = 11. We will not investigate these re nements further. We will de ne the safe set V1 as the set of initial conditions for which the system is safe if q = 00, i.e.:

V1 = fx0 2 XCjJ1 x0   C1 = 0g

6.22

V1 still guarantees safety as it is contained in the safe sets for all q. It is not the best choice for a comfortable

ride. However, this choice simpli es the calculations somewhat and circumvents the technological di culty of how collisions of the preceding vehicle are to be detected. The above analysis also allows us to determine the class of safe controls U1 x0 , i.e. the controls for which any initial condition x0 2 V1 can not result in a crash. Let @V1 denote the boundary of V1 in the induced topology of XC as a subset of R4 . De ne the interior of V1 as: intV1  = V1 n @V1

The Class of Safe Controls.

Lemma 15 Class of Safe Controls for Leader Operation The class of safe controls for a given
initial condition x0 is given by:

U1 x0 =  if x0 2 XC n V1 ; u  u 2 U1 x0   ux 2 umin x; umax x if x 2 intV1intV  ux = 1 if x 2 XC n 1

where u is given by equation 6.13 with x0 being the acceleration of the vehicle at the time the boundary 1 2 is reached.

Lemma 15 follows as a corollary of Lemma 13. The class of safe controls can be fully speci ed in feedback form u is trivially a feedback controller. Notice that, if the discrete disturbances are removed, the 1 control u is such that if the continuous state starts on the boundary of V1 it will either slide" along 1 it or cross in the interior of V1 , depending on the acceleration of vehicle B. However, in the presence of discrete disturbances, V1 is not an invariant set under u . A collision of vehicle B and or C may push the 1 trajectory well outside V1 . The e ect of these collisions will be to change the discrete state to q = 01 or 10 or 11 refer to Figure 6.4. Corollary 5 indicates that, after the collisions, the continuous state will still be in the safe set of the new discrete state. Overall, even though the continuous state trajectory crosses outside V1 the system is still safe, as long as u is applied. 1
10 Clearly the safe set is empty if q = CRASH .

77

6.5.2 Design for comfort

Having established conditions for safety we can now improve the design by considering passenger comfort. We seek a saddle solution, u ; d : 2 2

J2 x0  = max u2U x0  J2 x0 ; u; d min d2D 1 = min 0 max J2 x0 ; u; d u2U x  d2D
= J2 x0 ; u ; d  2 2
t0
1

for the cost comfort cost function:

J2 x0 ; u; d = sup jutj u x = 2



Assume x0 2 V1 . Consider the candidate solution: 0

u if x 2 XC n intV1  1
otherwise

6.23

As neither d nor x enter the cost function J2 , u ; d will trivially be a saddle solution for every d. As 2 before we can determine the set of initial conditions, V2 , for which the requirement for comfort:

J2 x0   C2
can be satis ed. Assuming C2 jjmin j this set is: V2 = intV1  fx 2 @V1 jx2 = aA g min Moreover the class of comfortable controls for x0 2 V2 will be: 6.24
o

U2x0  = u 2 U1x0 jut 2 ,C2 ; C2

6.25

6.5.3 Design for e ciency

To complete the design the requirement for e ciency should also be addressed. We will not go into the details of the e cient design, as this problem can be approached in a number of ways and the solution does not a ect safety in anyway. The saddle solution, u ; d , for cost function J3 can be sought, for example: 3 3

= J3 x0 ; u ; d  3 3 The result will be some form of H1 optimal design for u . Other designs e.g. the one proposed in 60  3 are also acceptable. As we do not set a threshold for J3 the choice of u is not crucial. The requirement 3 that u 2 U2 x0  can be enforced a-posteriori, as will be shown in next section. To simplify the analysis, 3 L L we will assume that the leader xed point, yd satis es yd 2 V2 and that u has been designed so that the 3 vehicle velocity never exceeds the speci ed highway velocity vH . Such a design can be easily obtained, for example, by switching from tracking yd to tracking vH whenever x1 t exceeds vH and back to tracking 1 yd whenever the spacing becomes less than the desired x3 yd see 60 for more details. Such internal modi cations of u leave the safety of the system una ected. 3 78

J3 x0  = max u2U x0  J2 x0 ; u; d min d2D 2 = min 0 max J3 x0 ; u; d u2U x  d2D
2

For the complete leader controller consider the switched feedback law as the safe leader law: 8 u if x 2 V2 and u 2 ,C2 ; C2 3 3 C2 if x 2 V2 and u C2 uL x = ,C if x 2 V and u3 ,C  2 2 2 3 :  u1 if x 2 XC n V2 Let: VL = V1

6.5.4 Proof of Leader Safety Lemma

6.26 6.27

Proof of Lemma 7: Under this design for uL and VL, the no collision" claim of Lemma 7 follows

directly from Lemmas 13 and 15. In addition, the above design is guaranteed to satisfy the passenger comfort requirement as long as x 2 V2 i.e. as long as safety is not an issue. The only point that remains to be investigated is that there exists a nite time, TL , such that, if the state leaves VL , it returns at most TL seconds later. Assume, for the sake of contradiction, that x exits VL and never re-enters. Note that in XC n V2 which contains the complement of VL  u is applied i.e. vehicle 1 A decelerates as hard as possible. By Lemma 13 vehicles A and B will not collide. Moreover, if xt stays outside VL and u is applied, vehicle A will stop after at most T3 equation 6.18 seconds. But once A 1 stops without colliding, it will be safe from then on in other words, all initial conditions with x0 = 0 are 1 in VL . This contradicts our original assumption. The above discussion reveals that the state will return to VL at most T3 seconds after it leaves it. Equation 6.18 indicates that T3 depends on x0 ; x0 ; vC and the system constants. Therefore, taking: 1 2 TL = maxfT3 jx0 2 0; vH ; x0 2 aA ; aA ; vC 2 va ; 0 g 6.28 1 2 min max will give an upper bound on the time it takes the state to return to VL for any initial condition. 2 Note that the controller uL can be easily encoded by a hybrid automaton. The structure can be directly inferred from equation 6.26 Figure 6.8. Lemma 7 can then be interpreted as saying that in the interconnection of the hybrid automata of Figures 6.4 and 6.8, the states corresponding to the state CRASH are unreachable.

6.6 The Join Control Law

In the previous section a design for the leader of a platoon was pieced together. The design consists of a control law, uL , and a set of initial conditions, VL , such that if uL is switched on while the state is in VL no collision will occur, provided that the behavior of the neighboring vehicles satis es certain constraints. Can this design also be used for joining and splitting platoons? Figure 6.9 shows three slices of the boundary of VL for x0 = 1; 13 and 16m respectively starting from top to bottom and x0 = 20m=s. The gure was 3 1 obtained by allowing vehicle B to collide once, with relative velocity at most va and not allowing vehicles A and C to collide. Recall that the join law should regulate to the follower xed point, which is x4 = 0m=s and x3 = 1m. From the gure it should be immediately apparent that, even for this reduced disturbance, safe operation with uL is impossible under these conditions, as any point with x4 = 0m=s is well outside the safe set for x1 = 1m. In fact, for a meaningful design we would like the point x4 = 0m=s; x2 = 0m=s2 to be within the safe set. The gure indicates that for this to happen a spacing of at least 13m is needed. Ideally we would like the system to be safe for the entire range of x2 . For this spacings of 16m or greater are needed. What can be done to resolve this problem? The rst option is to reduce the disturbances that vehicle A can face. This can be done in four ways: 79

Comfortable

Efficient u = u*(x)
3

u* < C 3

u =C

Track y
2

u* > C
3

x 1 >VH

x 3 <yd1

u* > -C 3

u =-C

Track v
2

u* < -C
3

x V2

x V2

x V2

x V2

Safe u = u* u=j
A

min

x 3 = a min

u=0

Figure 6.8: Controller Hybrid Automaton Prevent vehicle B from colliding with the vehicle ahead of it. Prevent vehicle C from colliding with vehicle A. Reduce the range dmin ; dmax in particular increase dmin . Provide vehicle A with additional information, for example, assume the follower mode where xB is communicated to vehicle A. For the purposes of join and split alternatives 1 and 2 will be considered. Eliminating collisions of vehicles B and C will be achieved by the one maneuver at a time" assumption of Theorem 7.11 Alternative 3 is also feasible, but we will not investigate it here. In order to reduce the range of xB we need to make assumptions about vehicles downstream of vehicle B, which will signi cantly complicate the problem12. Alternative 4 will be investigated in the follower design. Technological limitations such as the range of the infrared communication device used to transmit xB information make this alternative unattractive for joining and splitting. Is this reduction in disturbance su cient to allow us to solve the problem? The discussion in Section 6.5 indicates that eliminating the possibility of collisions of vehicle B has the e ect of moving the boundary of the safe set in Figure 6.9 down by va = ,3m=s in the x4 direction. This makes a few more initial conditions safe, but is still inadequate for example the condition x3 = 1m; x4 = 0m=s; x2 = 0m=s2 is still unsafe. Clearly joining of a platoon can not take place under the requirement that no collision occurs. The next step therefore is to relax that requirement, in view of the fact that collisions with relative velocity smaller than va are considered safe see also 27 and 28 . Therefore, in the subsequent design we will consider
apply the leader control law of the previous section. 12 Interesting design alternatives can be obtained in this way. It may turn out for example that di erent join controllers can be used depending on whether a vehicle exists within the sensor range downstream of B or not.
11 This implies that if vehicle A is applying a join or split control law, vehicles B and C must be in the leader mode and

1. 2. 3. 4.

80

Safe set boundary for x1(0)=20m/s, x3(0)=1, 13 and 16m 5

2
x4(0)

2 5

1 x2(0)

Figure 6.9: Boundary of the safe set for x0 = 20ms,1 1 the cost function: where:

J10 x0 ; u; d = jx4 T j T = minft  0jx3 t = 0g

6.29

is the rst collision time. The safety requirement will be encoded by: J10 x0 ; u; d  C1 = jva j The design of the join and split controllers can also be treated as a two player, zero sum game with cost 0 function J1 . In this case one player is the control that the joining splitting vehicle applies, while the other player is the acceleration of the preceding platoon. In our standard notation13 : d 2 D0 = fxB t 2 dmin ; dmax ; ti ; 0; ti ; 0g By abuse of notation we will use dt to denote xB t. This reduction in the disturbance has the following hybrid automata theoretic interpretation: the communication protocols designed in 34 are such that their interconnection with the hybrid automaton of Figure 6.4 for the joining splitting vehicle and the vehicle being joined to split from has the e ect of disabling the transitions Coll B and Coll C. 0 Guessing a saddle solution for cost J1 is not as easy as it was for J1 . It turns out that, for certain combinations of state constraints and initial conditions not necessarily in the safe set, full deceleration by both the joining and the preceding vehicle is not the optimum. We will use u ; d  to denote the saddle 0 0 0 solution for J1 . Lemma 16 The saddle solution u ; d  is such that: 0 0  and d assume either their minimum or their maximum possible values bang-bang 1. at every time, u0 0 solution
13 There are other ways of encoding the no collision" requirement in terms of d but, they are all equivalent.

6.6.1 Design for Safety

81

2. at the time of impact, u and d assume their minimum values 0 0 3. If x4 T  0 non zero relative velocity collision, u and d involve at most one switching from their 0 0 maximum to their minimum values. In particular, three kinds of behavior are possible: a Both optimum trajectories assume only their minimum. b d assumes its maximum rst and then its minimum while u assumes only the minimum. 0 0 c Both optimum trajectories start by assuming their maximum values and u switches to the 0 minimum before d . A non zero amount of time elapses between the two switchings. 0

Proof: The proof will be based on writing J10 as an integral cost function and applying the maximum
principle 53 . Recall that at the time of impact the relative velocity of the vehicles will have to be negative. Therefore: J10 x0 ; u; d = jx4T j = ,x4T  Z 0 , T x4 tdt = ,x4 _
Z

= ,x0 + x2 t , dtdt 4 0 0 De ne Lx; u; d = x2 , d. Then, for a given initial condition, the extrema of J1 x0 ; u; d occur at the extrema of: Z T Lxt; ut; dtdt 0 The following discussion will assume the existence of optimal controls and proceed to classify them using the Maximum Principle. Existence and uniqueness questions will not be investigated further here 14 as the justi cation for the analysis will be provided by numerical examples in Section 6.6.2 We form the Hamiltonian: ~ H x; p; p0 ; u; d = p0 Lx; u; d + p0 Ax + Bu + Dd = p0 + p1 , p4 x2 + p3 x4 + p2 u + p4 , p0 d ~ As the terms in u and d conveniently decouple, we can take the maximum of H over u and the minimum  ; d  for the two player game. over d to obtain a saddle solution u0 0 ~ H x; p; p0  = u2 u max x d2 d min x H x; p; p0 ; u; d min x;umax min x;dmax ~ = min max H x; p; p0 ; u; d ~ = H x; p; p0 ; u ; d  0 0 where:
d2 dmin x;dmax x u2 umin x;umax x

u t = 0 d t = 0

 

umax x uminx dmax x dmin x

if p2 t 0 if p2 t  0 if p4 t , p0 0 if p4 t , p0  0

6.30 6.31

are linear, the cost function is a linear function of the nal state, all state constraints are convex and closed and the input constraints are compact intervals.

14 Existence or even uniqueness of optimal controls is likely to be guaranteed by the fact that the unconstrained dynamics

82

The Euler equation for p gives:

which can be integrated to: p1 t  p1T  p2 t = , p3T  T , t2 + p0 + p1 T  , p4 T T , t + p2 T  2 p3 t  p3T  p4 t = p3T T , t + p4 T  Boundary condition 2.23 gives pT  = 0 0 c 0 0 for some c 2 R. Substituting this into the expression for pt results in: p1 t  0 c p2 t = , 2 T , t2 + p0 T , t p3 t  c p4 t = cT , t Note that p4 T  = 0  p0 , hence d T  = dmin . 0 It should be noted that the solution is not completely characterized at this stage. There are three constants p0 ; c and T  to be determined from the boundary conditions: x3 T  = 0 H x; p; p0  = 0 for all times The last constant can be normalized out as c and p0 can not both be zero and they appear everywhere homogeneously. Unfortunately, the boundary conditions are too complicated to tackle analytically. Numerical calculations will be given in the next section. Here we will only try to determine whether it is possible for the optimal controls to switch. For this purpose we will make use of boundary condition 2.22, which at time T becomes: p0 x2 T  , dminxT  + cx4 T  = 0 6.32 First assume p0 0. We will distinguish a number of subcases, depending on the values of x2 T  and dmin xT .

6 p = , @H = 6 p4 , p00 , p1 _ 4 @x 6 ,p3

3 7 7 7 5

Case 1: x2T  dmin xT . As x4 T   0, we must have c 0. Then, for all t, p2  0 and p4  0 p0,
therefore d t = dmin ; u t = umin . 0 0

Case 2: x2T  dmin xT . This time c 0. Consider the following cases: p0  cT 0: No switching is possible d t = dmin ; u t = umin. 0 0  exhibits switching. In particular: 2p0  cT p0 : d0
u t = umin 0  d t = dmax if t 2 0; T , p0 =c 0 dmin otherwise
83

cT

2p0 : Both u and d exhibit switching. In particular: 0 0

u t = 0 d t = 0

 

umax umin dmax dmin

if t 2 0; T , 2p0 =c otherwise if t 2 0; T , p0 =c otherwise

Case 3: x2T  = dmin xT . If x4T 

0, c = 0, which results in no switching. If x4 T  = 0, the condition on H at time T used here can lead to no information about c. The condition at time 0 may provide more insight. In any case, collisions with zero relative velocity are not crucial from the point of view of vehicle safety. The case p0 = 0 can be treated similarly. By equation 6.32 and the fact that c 6= 0, x4 T  = 0. Therefore, as in Case 3, this situation will result in a crash with zero relative velocity collision. The comments made for Case 3 also hold here. 2

The above analysis indicates that, at least in theory, it is possible that the optimal controls may take their maximum values for certain segments of time. In terms of vehicles this implies that the best worst strategy involves not only decelerating, but also accelerating. In particular, three behaviors are possible in theory: 1. Both optimum trajectories involve only maximum deceleration. 2. The optimum for B involves maximum acceleration rst and then maximum deceleration while the optimum for A involves only maximum deceleration. 3. Both optimum trajectories start by maximum acceleration and vehicle A switches to maximum deceleration before vehicle B. A non trivial amount of time elapses between the two vehicles switching to full deceleration. We would like to determine if there exist realistic conditions under which the optimum trajectories involve acceleration. The terminal condition 6.32 provides considerable insight, when interpreted in the vehicle framework. First recall that, by the assumptions of the maximum principle Theorem 1, p0  0 and p0 = 0 implies c 6= 0. The last observation together with equation 6.32 indicate that p0 = 0 results in x4 T  = 0, i.e. collisions with zero relative velocity. In terms of safety zero relative velocity collisions are not particularly interesting. We will therefore restrict our attention to the case p0 0 and in particular situations where x4 T   va = ,3ms,1 . In order to gain some insight into situations that might lead to switching in u and d assume that 0 0 the constraints on the jerk of vehicle A are large in absolute value compared with the constraints on the acceleration15 . Recall that at the time of impact, T :

6.6.2 Numerical Investigation

u T  = umin xT  dT  = dmin xT 

Therefore, if the jerk is high enough:
15 If the jerk is unbounded, vehicle A can be described by a second order model without delay. In this case we can think of

x2 T   aA dxT   aB min min

vehicle A as controlling its acceleration directly.

84

depending on whether vehicle B has stopped or not. Moreover, x4 T  0. If:

aA  aB min min
i.e. vehicle A can decelerate at least as fast as vehicle B the terminal condition 6.32 implies that c 0. This in turn implies that the saddle inputs do not involve switching Case 1 above. Moreover, even if B aA min amin but: equation 6.32 implies that unsafe collisions can only occur if kc=p0 k is small. This makes it unlikely for switching to occur in either u or d as, by Case 2 above, switching requires at least: 0 0

aA  aB min min

k pc kT 1
0

Summarizing, if the deceleration capabilities of both vehicles are comparable and the bounds on the jerk are not too conservative, we expect u and d to involve only maximum deceleration. This observation 0 0 is also supported by the results of 28 where, assuming a second order model with delay for vehicle A and uniform deceleration capabilities, it was shown that the saddle solution involves only minimum deceleration by both vehicles. These observations limit the conditions under which saddle trajectories that involve acceleration can be expected. Are there any conditions for which such behavior can be observed? To answer this question a very crude C-program was written to estimate the value of c under the assumption that p0 0 which is a necessary condition for switching. The state and input constraints used were:

amin = ,5:0 amax = 3:0 vmin = 0:0 vmax = 50:0 aB = ,8:0 aB = 3:0 jmin = ,15:0 jmax = 15:0 max min
Vehicle B is assumed to have greater deceleration capabilities than vehicle A, within reasonable limits though. Also the bounds on the jerk are slightly conservative. Consider the initial condition: 20:0ms,1 6 ,2 x0 = 6 2:00:ms 6 4 01m 3:0ms,1
2 3 7 7 7 5

Note that, while the initial spacing is very small, the initial relative velocity is in favor of vehicle A, as it is moving away from vehicle B. The examples will indicate that this initial condition is unsafe. Figure 6.10 shows the state trajectory for ut  umin t and dt  dmin t, i.e. both vehicles braking as hard as possible. Because of the di erence in deceleration capability, a collision occurs at T = 1:075 seconds, with relative velocity x4 T  = ,1:858ms,1 . This is quite a mild collision, well below the threshold va = ,3ms,1 . Assume that the trailing vehicle decides that braking as hard as possible is the best policy. Does Figure 6.10 depict the worst that can happen? Consider for d the trajectory shown in Figure 6.11, which involves accelerating before decelerating, making the time to collision larger. The resulting state trajectory is shown in Figure 6.12. Note that the collision which takes place at T = 3:290 seconds is a lot more severe: x4 T  = ,4:822ms,1 , substantially greater than the threshold. Can vehicle A do better by accelerating? Consider the saddle solution u ; d  shown in Figure 6.13. 0 0 Note that vehicle A also accelerates slightly before putting on its brakes. The resulting state trajectory is shown in Figure 6.14. The collision at time T = 3:995 seconds is still severe x4 T  = ,3:945ms,1  but not as severe as that of Figure 6.12. 85

Velocity 21 20 19 2 18 17 16 0 0.5 Spacing 0.6 0.4 0.2 0 0 0.2 0 1 0.5 1 1.5 2 0 3 2 1 1 1.5 4 6 0 2 0

Acceleration

0.5

1.5

Relative Velocity

0.5

1.5

Figure 6.10: State trajectory under maximum braking

6.6.3 Completing the Design

The analysis presented above allows us to determine the optimal control u and the worst case disturbance 0 d for each initial condition x0 . The corresponding J0 x0  = J10 x0 ; u ; d  as well as the safe set of initial 0 0 0 conditions can then be determined numerically. It should be noted that x0 does not in uence the position 1 of the boundary between safe and unsafe initial conditions. The boundary corresponds to initial conditions where collisions do happen but with a small relative velocity, which implies that the joining vehicle will have to be moving x1 T  0. Therefore, the value of x0 plays no role in the calculation. 1 Having established the set of safe initial conditions, the design of the join controller can be completed to take into account passenger comfort and e ciency. A controller needs to be designed to track the desired J xed point, yd . Any controller that tracks this xed point for example 60, 27  will su ce. Designs that try to minimize the cost function J3 are preferable. Let uJ denote the chosen tracking controller, VJ denote 3 the safe set whose computation was discussed in Section 6.6.1 and V2J its interior in the relative topology of XC . Then the complete safe, comfortable and e cient design for the join maneuver will be: 8 uJ if x 2 V2J and uJ 2 ,C2; C2 3 3 C2 if x 2 V2J and uJ C2 uJ x = ,C if x 2 V J and u3 ,C 6.33 J 2 2 2 3 :  u0 if x 2 XC n V2J Note that, unlike VL the safe set VJ is an invariant set, under the assumed set of disturbances.

86

Control 0

10

15 0

0.5

1.5

2.5

3.5

Disturbance 4 2 0 2 4 6 8 0 0.5 1 1.5 2 2.5 3 3.5

Figure 6.11: Maximum braking by vehicle A, worst case d

6.6.4 The Split Design

Most of the previous discussion still holds for the split controller as well. For example the controller design of equation 6.33 will also work for split, if uJ is replaced by some uS designed to track the split xed 3 3 point. Letting VS = VJ and V2S = V2J as above: if x 2 V2S and uS 2 ,C2 ; C2 3 if x 2 V2S and uS C2 3 uS x = 6.34 if x 2 V2S and uS ,C2 3 : if x 2 XC n V2S With a little more e ort better designs that avoid collisions whenever possible, switch to the leader law as soon as VL is reached, etc. can be constructed.
8

uS 3 C2 ,C2 u 0

Overall, the above discussion provides guidelines for designing safe, comfortable and e cient controllers for the join and split maneuvers uJ and uS respectively and determining the sets of initial conditions VJ and VS for which these controllers can guarantee safe operation. Note that, as for the leader law, uS and uJ are easily implemented by means of a hybrid automaton. If a design is carried out under these guidelines, the proofs of Lemmas 8 and 9 follow.

6.7 The Lane Change Laws

It should be noted that collisions between vehicles A and C are not possible in the lane change scenario. Because of the one maneuver at a time" restriction, it is impossible for vehicle C to be joining or splitting 87

Velocity 25 20 15 10 5 0 2 0 2 4 6 0

Acceleration

2 Spacing

Relative Velocity 5

4 3 2 1 0 0

5 0

Figure 6.12: State trajectory under minimum u worst case d to from vehicle A while the lane change is taking place the only situation that can lead to a collision in a single lane under corollaries 1 and 2. We still need to consider the possibility of vehicle D getting rear ended though. We will start with the move part of the lane change. The decelerate part will essentially follow as a special case. Consider the following control laws for vehicles A, C and D: uMA = minfuAB ; uAE g 6.35 L L CA uMC = uL 6.36 DA uMD = uL 6.37 6.38 and de ne VMA = VMD = VMC = VL .

Proof of Lemma 11: Note that, as u = umin , uMA = u if either uAB = u or uAE = u. Moreover, 1 1 1 1 L L
uMAx =


the design of uL equation 6.26 dictates that uL = u if the state leaves the interior of VL . In other words: 1 minfuAB ; uAE g if xAB 2 VMA and xAE 2 VMA L L u otherwise 1 6.39 By assumption, both x0 2 VMA and x0 2 VMA . Therefore, the proposed controller 6.39 satis es the AB AE conditions of Lemma 7 for both xAB and xAE , hence vehicles A, B and E will not collide. Similarly, as uMC = uCA and x0 2 VMC vehicles A and C are guaranteed not to collide. Is it possible L CA for vehicle C to collide to vehicle B after vehicle A leaves the lane? The geometry" of the problem imposes at all times the condition that:

xCB3  xAB3
88

Control 15 10 5 0 5 10 15 0 0.5 1 1.5 2 Disturbance 4 2 0 2 4 6 8 0 0.5 1 1.5 2 2.5 3 3.5 4 2.5 3 3.5 4

Figure 6.13: Saddle solution u ; d  0 0 As the safe set VL is monotone in x3 and vehicle A is safe with respect to vehicle B, vehicle C will also be safe with respect to vehicle B. A similar argument for vehicle D completes the proof.2 For the decelerate law consider:

uD = minfuAB ; uAE g L S VD = VL
of Lemma 11. 2

6.40 6.41

Proof of Lemma 10: Lemma 10 is a direct corollary of Lemma 7 and the argument given in the proof
It should be noted that vehicle A is safe with respect to both the origin and target lanes throughout the move maneuver. Likewise, vehicles C and D are also safe with respect to both the moving vehicle A and vehicles in their own lanes. Therefore the move can be safely aborted at any stage. This is a very desirable property as it allows us to safely decouple the lanes. For example it seems advantageous to abort the move if the deceleration in target lane gets large to avoid the propagation of decelerations from lane to lane. The monotonicity of the safe set with respect to x3 implies that occlusion of vehicles by other vehicles during a lane change does not a ect safety. For example, vehicle C will have sensory information about vehicle B only after vehicle A has e ectively moved out of the lane. The proof of Lemma 11 indicates that this should not be a problem from the safety point of view. Finally, note that di erent control laws can be used instead of uAE in the decelerate maneuver. For example control laws producing more modest S decelerations the initial condition is likely to be extreme for the split law may be designed. The law used is not critical from the point of view of safety. 89

Velocity 25 20 15 10 5 0 0 1 2 Spacing 6 4 2 2 0 4 2 0 1 2 3 4 6 0 1 0 2 6 4 3 4 4 2 0 2 4 6 0 1

Acceleration

Relative Velocity

Figure 6.14: State trajectory under u ; d  0 0

6.8 Key Points and Discussion

A few remarks are in order, concerning the results presented here. supports the formation of platoons. It should be noted, however, that the analysis on which the design was based is considerably more general. Most of the calculations rely on little more than the vehicle model and are therefore applicable verbatim to other designs. For example, our calculations trivially lead to to safety requirements for concepts such as autonomous vehicle operation. We are currently working on the more complicated follower mode calculations which will allow us to fully analyze variations such as vehicles entering and exciting as followers, lane changes from to the middle end of a platoon, platoon lane changes, etc. For example, the above calculations also indicate the requirements that the system needs to satisfy for a collision free AHS design to be possible. Such a design could involve a simple join maneuver that brings the vehicle to the boundary of the leader safe set under the assumption that no collisions of vehicle B and vehicle C are allowed. If it turns out that this spacing is within the domain of attraction of the follower control law and is small enough to allow the communication necessary for follower operation the join can be declared complete and the follower operation can be established. A similar modi cation can be used for the split maneuver, where the follower law brings the vehicle to the edge of the follower safe set or the range of the communication devices, whatever is smaller. At that point leader operation is established, provided of course that the initial condition is within the leader safe set. 90

Design Alternatives: The results presented above constitute a safe AHS regulation layer design that

Collision Free AHS: The design presented here and can easily be trimmed to t other speci cations.

These simple changes, which are easily implemented using the calculations of Sections 6.5 and 6.6, will produce a regulation layer for which no collisions are possible. The feasibility of the design will depend heavily on subtle issues such as the communication device range, follower law domains of attraction etc. To avoid these complications we will not investigate it further here. The calculations of Section 6.5 can be used to obtain the minimum communication device range follower domain of attraction for collision free operation.

Necessary & Su cient: The conditions obtained by solving the game theoretic problems are su cient

and necessary in the sense of being tight. They are su cient from the point of view of controller design: they state that if the designer ensures that the initial condition lies in the safe set Vi before controller ui is invoked and that ui equals to the saddle input on XC n intVi , then the closed loop hybrid design is guaranteed to be safe. The need for further veri cation is eliminated in this case by the design procedure. The conditions are necessary from the point of view of veri cation. They state that, if the discrete design does not guarantee that x 2 Vi when controller ui is applied, then there exist system trajectories that violate the closed loop requirements in this case safety.

Improving performance using coordination: In the algorithm introduced in Chapter 3 the role of

inter-agent coordination is to reduce the set of allowable disturbances. This has the e ect of biasing" the game in favor of the controller and hence increasing the range of conditions under which game winning controls exist and improving the overall system performance. Here we only investigated the possibility of using coordination to limit the discrete disturbances. In particular, communication protocols were used to allow vehicle A to join split from vehicle B, without having to worry about vehicle C hitting it from behind or vehicle B hitting the vehicle ahead of it. Similarly, timing constraints on the protocols guaranteed adequate separation between collisions to allow vehicle A to recover while executing the lead controller. Communication can also be used to reduce the continuous disturbances in this case the acceleration range of vehicle B. The discussion of Section 6.6 indicates that the safety of the join split maneuvers is very sensitive to the relative deceleration capabilities of the vehicles involved. In fact, safe joining splitting may not be possible in some cases. In these cases, communication can be used to limit the set of decelerations of vehicle B, to make joining splitting safe and improve the performance of the AHS. For example, when vehicle A contacts vehicle B with the intention of joining splitting it may also ask for a promise that vehicle B will not decelerate faster than a certain rate which makes the maneuver safe for the given deceleration capabilities of vehicle A. Before giving permission to start the maneuver, vehicle B should decelerate with at most the requested rate to create adequate spacing from the platoon ahead of it. The spacing should be such that safe leader operation can be obtained under the limited deceleration expected from vehicle B. Vehicle A can then join safely knowing that the deceleration of vehicle B will not exceed the promised value. The discrete steps of this extended communication protocol request to join split, acknowledge the request, etc. will be the same as the steps of the protocol of 34 , but more continuous information deceleration promises etc. will need to be added. The calculations of Sections 6.5 and 6.6 can be used to determine the feasibility of such maneuvers for given ranges of sensors and communication devices and estimate its impact in terms of highway throughput.

Re ning the space requirements: The design presented here is rather conservative. We can actually

do a little better by getting into the details of the protocols and the structure of the plant. If vehicle C is joining to splitting from vehicle A, then the disturbance set that needs to be considered is indeed D. However, according to corollaries 1 and 2, if vehicles A and C are not involved in a join or split maneuver, the allowable disturbances reduce to: D0 = fxB t 2 dmin ; dmax ; TB ; vB 2 maxfva ; x4 TB  + x1 TB g; 0 ; 0; 0g 91

i.e. the possibility of vehicle A being rear-ended is eliminated. The reduced disturbance implies that vehicle A can follow vehicle B a little closer if it is not involved in a join or split with vehicle C. This o ers obvious advantages in terms of packing more vehicles into the same length of highway. On the other hand, if this space is utilized by an appropriate design, vehicle A has to decelerate slightly to increase its spacing from vehicle B before giving permission to vehicle C to begin a join or split maneuver. The choice to use this extra space or not will probably have to be based on more global tra c considerations, such as demand, spacing between entries and exits, etc. Similar savings can be expected for the case of a lane change. Before moving vehicle A establishes communication with either vehicle D or vehicle E in the target lane. If communication with vehicle D is established, then vehicle D can not be involved in a a join or split maneuver with the vehicle behind it. Hence the possibility of vehicle D being rear ended is again eliminated and A can change lane a little closer to D. Similarly, if communication with vehicle E is established, E can not be involved in a a join or split maneuver with the vehicle ahead of it. Hence the possibility of vehicle E being involved in a collision is eliminated and A can change lane a little closer to E. Finally, as A can not be involved in a join or split maneuver throughout the lane change, it can not be rear ended by C, therefore the pairs A E and A C can be allowed to be a little closer. The above modi cations are based on information that is available to vehicle A through the current communication protocols. E ectively the design is being re ned by reducing the disturbance set of vehicle A in particular eliminating the possibility of certain collisions by interconnecting the communication protocols with the hybrid automaton of Figure 6.4. It should be noted that the analysis of Section 6.5 can be trivially modi ed to produce safe sets for all these cases by setting vB and vC to zero appropriately.

Reducing the application of u: As discussed earlier, collisions of vehicles B and C may push the 1

state of vehicle A well outside the region VL . The system remains safe as the assumptions of the theorem guarantee that, once a collision of a certain kind has occurred, a second collision of the same kind will not occur before the system has had time to recover. Therefore, vehicle A has to deal with a reduced set of disturbances for some time after each collision. This fact can be utilized to reduce the magnitude of the inputs, as the boundary of the safe set where u needs to be applied to guarantee safety moves after a 1 collision occurs. Points in the interior of the safe set V2 are mapped to points in the interior of the safe set for the reduced disturbance, therefore the milder controller for the interior can again be safely applied. This modi cation will increase the domain of applicability of the comfortable and e cient controller. The down side is that the proofs become a lot messier, especially the part concerning the time it takes for the state to return to VL .

Re ning u3 : In the interior of the safe sets all the standard control tricks and any intuition the designer
might have can safely be used. In particular, at no extra cost, one can add safety margins around the boundary of VL , smoothing functions to avoid control discontinuities hysteresis to avoid chattering, time varying and or nonlinear controllers, or controllers based on reference trajectories, adaptive controllers to deal with plant uncertainties etc. As long as the designer guarantees that the safety boundary @VL is not crossed e.g. by applying u on @VL  all the lemmas and theorems still hold. Similarly, as long as 1 the designer guarantees that the input does not exceed C2 in the interior of the safe set e.g. by using a saturation function as in 6.26, the control will be guaranteed to be comfortable as much as possible.

Sensor ranges: The calculations of Sections 6.5 and 6.7 can also be used to provide speci cations for
the enabling sensor technology. For example, one would like vehicle A to be able to stop if it detects a stopped vehicle B at the end of it's front sensor range. The calculation of Section 6.5 can be used to 92

100 90 80 70
Sensor Range (m)

60 50 40 30 20 10 10

12

14

16

18

20 22 Velocity (m/s)

24

26

28

30

Figure 6.15: Minimum front sensor ranges for safe operation obtain speci cations for the minimum front sensor range as a function of velocity Figure 6.15. The gure was obtained by assuming zero initial acceleration for vehicle A x0 = 0ms,2  and allowing one rear end 2 collision with vehicle C. If the collision with vehicle C is eliminated the speci cation curve moves right by va . In other words, a vehicle being joined to split from has to move slower, if no vehicle exists in its sensor range. Similar calculations can be carried out for the adjacent lane sensors. Hopefully, the sensor technology can provide the minimum ranges required for safety at the desired speeds. If this is not the case we may still be able to do something making use of the sensors of neighboring vehicles and the communication capabilities16. For example, if the adjacent lane sensor range is smaller than the front sensor range, vehicle A in Figure 6.5 may be unable to detect vehicle D, but vehicle D may be able to detect vehicle A, after the middle of the maneuver. In this case safe lane changes may be carried out at higher speeds if vehicle D establishes communication the moment it detects vehicle A and forces it to abort the lane change if xDA is not in VMD .

Throughput Abstractions: The time it takes leaders and followers to recover from collisions TL and
TF respectively as well as the spacing requirements generated by the above calculations directly lead to
abstractions of the various controllers discussed here in terms of their highway space-time requirements. The impact of these results on the expected throughput is currently under investigation.

16 This approach is further explored in 15 , where the sensors of neighboring vehicles and the communication devices are used to recover information that has been lost due to sensor faults. In 15 , a vehicle may even use communication and the actuators of neighboring vehicles to recover actuation capabilities lost to faults for example brake faults of platoon followers. These alternatives are explored in Chapter 7

93

Chapter 7

Automated Highway Systems: Fault Tolerant Design

The work presented in this chapter can be thought of as an attempt at relaxing Assumption 2 of Section 6.2.5. Our goal is to extend the normal mode architecture outlined in Section 6.1 to make it capable of dealing with faults and adverse environmental conditions. Because the system performance is likely to degrade in this case, we will use the term degraded modes to describe operation under the extended control scheme. The extended control scheme will be such that graceful and gradual degradation in performance is obtained. The design process will follow along the lines of Chapter 5. We will assume that fault detection has already been carried out and proceed to specify the fault handling module. It will become apparent that the task is formidable, even in this reduced setting. Here we will only highlight the main points; the details of the design as well as the proofs of the performance claims can be found in 15 and 16 .

7.1 Extended Information Structure

We rst try to extend the information structure. We will address both the causes that can lead to discrete transitions in the controller structure i.e. initiation of degraded modes discussed in Chapter 5. For AHS these two causes can be thought of as faults and extreme environmental conditions. Faults induce discrete, qualitative changes in the system dynamics, that dictate discrete transitions in the control scheme. Their e ect on the system will be monitored by a hierarchical structure, called the capability monitor. Extreme environmental conditions on the other hand lead to gradual, quantitative changes in the system dynamics. The e ect of such changes is continuous as opposed to discrete degradation in the system performance. This can eventually lead to discrete changes in the control scheme if, at some point, the degradation is severe enough so that the performance speci cations can not be met. The e ect of environmental conditions on the system performance will be monitored by a hierarchical structure, called the performance monitor. The control scheme for normal operating conditions presented in 74 relies on a number of sensors, actuators and communication devices, both on the vehicles and on the roadside. All this additional hardware, as well as the standard mechanical parts of the vehicles, are prone to failure. Such failures will directly in uence the capabilities of the system as a whole and therefore restrict the strategies that the control scheme can implement. Because the faults will be modeled as discrete events, we propose a design based on a hierarchy 94

7.1.1 Capability Monitor

of predicates for the capability monitor. Each predicate will monitor a single functional capability and will return a 1 True if the system possesses the capability in question or a 0 False otherwise. The predicates will be arranged in a hierarchy similar to that of the normal mode control architecture. The values returned by the higher layer predicates will depend on the values of the lower layer predicates. This scheme can be used to systematically go through combinations of faults and design specialized control laws that utilize the remaining capabilities so that the impact of the faults on the system is minimized. We will start describing this hierarchy at the bottom and work our way up.

Physical Layer Predicates

The vehicles and the roadside need to have access to resources such as sensors, actuators and communication devices. We model each one of these resources as a predicate, that returns 1 if the resource is available and functioning and 0 otherwise. Assuming that the control scheme requires na actuators, ns sensors and nc communication devices, the capability of the physical layer can be though of as a vector, cP , of zeros and ones of dimension ns + na + nc :

cP 2 CP = f0; 1gns +na +nc

This vector re ects which resources are functioning and which are not. For simplicity, the actuator predicates are interpreted as re ecting the capability of the vehicle to accelerate, decelerate and turn. Therefore they incorporate information about basic vehicle functionality, like engine and tires being in proper working order. Predicates for these basic functionalities can explicitly be added at the cost of a small increase in the complexity of the monitor. Similarly a sensor" predicate re ects the ability of a vehicle to sense its environment. A more complicated structure for the sensor predicates can be constructed to re ect things like sensor redundancy.

Regulation Layer Predicates

The regulation layer contains laws for controlling both the longitudinal and lateral vehicle motion. Each one of these laws makes use of physical layer resources, primarily sensors and actuators. For a regulation layer controller to be functional, all of these resources need to be available. Therefore, the applicability of a regulation layer controller can be modeled by a predicate whose value depends on the values of the predicates for the physical layer. Consider, for example, the lead controller proposed in 6. This longitudinal control law uses sensor readings of velocity and acceleration of the vehicle and of the spacing and relative velocity with respect to the preceding vehicle to calculate inputs for the throttle and brake actuators. Without getting into the details of the control law, we can see that the lead controller predicate can be viewed as an AND of the predicates for the velocity, acceleration, spacing and relative velocity sensors and the brake and throttle actuators. Likewise, the law proposed in 72 for the followers in a platoon makes use of additional information about the state of the leader of the platoon. It is assumed that this information is transmitted to all the followers using an infrared communication link. Therefore, the predicate for the longitudinal follower law should depend on the predicate for the infrared communication link as well as the predicates for the sensors and actuators listed above. In this formalism the capability of the regulation layer can be encoded by a vector, cR , of zeros and ones, of dimension equal to the number of control laws available to the layer. If there are nlong longitudinal laws and nlat lateral laws this vector will have the form:

cR 2 CR = f0; 1gnlong +nlat

95

actuator capabilities
brakes throttle steering

1 1 1

sensor capabilities &

velocity acceleration relative distance relative velocity magnetometer magnets

1 1 1 1 1 1 & &

leader law join law split law follower law lane keep lane change Catch up

communications capabilities
infrared

Platoon break up Stop sign law

1 1 &

Accelerate to enter

radio

Figure 7.1: Connection between Physical and Regulation layer capabilities As shown in the examples, the design of the control laws implies a mapping from the vector coding the capabilities of the physical layer to the vector coding the capabilities of the regulation layer: FR : CP ,! CR This mapping is easiest to convey by means of a gure. Figure 7.1 shows the mapping for the control designs in 97, 72, 60, 98, 87 .

Coordination Layer Predicates

From the point of view of the coordination layer, the regulation layer control laws represent resources that can be used to carry out maneuvers. Each maneuver will need to make use of two control laws, one longitudinal and one lateral. In order for the coordination layer to be able to invoke a maneuver, the relevant control laws should be operational. For example, for the coordination layer to command a platoon leader to join, at least one of possibly many longitudinal join law and one lateral lane keeping law should be operational. The capabilities of the regulation layer, when seen from the point of view of the coordination layer, can thus be modeled by predicates that depend on the regulation layer capability vector. Let nman denote the number of maneuvers that may be requested by the coordination layer. Then the system capability at the regulation coordination layer interface can be modeled by a vector, cI , of zeros and ones of dimension nman . The design of the interface induces a mapping: FI : CR ,! CI = f0; 1gnman For the normal maneuvers presented in 34, 98 , the control laws of 97, 72, 60, 98, 87, 27 and the interface design of 91 , the map FI can be seen in Figure 7.2. The coordination layer of 34 requires that a normal vehicle should be able to enter the AHS, lead a platoon, join a platoon, be a follower, split from a platoon, change lane and exit the AHS. The capability 96

Radio Comm.

infrared link

&

Stop sign law Accelerate to enter leader law join law OR safe join law

& ENTRY LEAD FREE AGENT & JOIN & SPLIT & NORMAL MODE

&

split law decelerate law follower law Catch up

DECELERARE TO CHANGE FOLLOW MOVE TO ADJACENT LANE

&

& Platoon break up

EXIT

& lane keep lane change &

Figure 7.2: Coordination layer capabilities to carry out these maneuvers will be coded by the regulation coordination interface capability vector. In addition, to execute the protocols that organize the maneuvers, the coordination layer needs to be able to communicate with neighboring vehicles. Therefore, the capability of the coordination layer to operate in its normal mode can be expressed as a predicate on the values of the capability vector for the interface and the communication device predicates. This dependency is also shown in Figure 7.2. From the gures it should be clear that if a fault damages any of the vehicles basic functions toggling a physical layer predicate to 0 the normal mode of the coordination layer is likely to be rendered inoperable. For this purpose additional coordination layer strategies that are capable of operating in these reduced circumstances need to be designed. These new strategies will probably have to make use of new regulation layer control laws that try to make the best of the remaining capabilities of the vehicle. An example of such a strategy is Take Immediate Exit, described in Section 7.2.1. The objective of this strategy is to take a vehicle that has developed a fault and therefore can not function in the normal mode out of the highway as soon as possible. The design makes use of additional maneuvers, which can easily be added to the predicate structure for the regulation layer and its interface. As we shall see in Section 7.2.1, some of these maneuvers will require close cooperation with the neighboring vehicles. Therefore, the applicability of the Take Immediate Exit strategy can be expressed as a predicate on the values of the interface capability vector and the communication device predicates of the faulty vehicle as well as the interface capability vectors of the neighboring vehicles. It is assumed that knowledge about the capabilities of the neighbors will be obtained once communication has been established. Once many such strategies have been designed the coordination layer capability can be expressed as a vector of zeros and ones. The dimension of this vector will be equal to the number of these strategies, which will be denoted here by ncoord . The design of the strategies induces a mapping: FC : CI  f0; 1gnc  CIN ,! CC = f0; 1gncoord 97

Here N stands for the maximum number of neighboring platoons that may need to cooperate in an emergency maneuver. Maps similar to the one shown in Figure 7.2 can be constructed for the strategies introduced in Section 7.2.1 to deal with faulty conditions. The coordination layer predicate structure illustrates some of the features discussed in Chapter 5. For example, the explicit separation between supervisor and regulator in the control structure rst appears here. The interface predicates can be thought of as the capability of the regulator part of the coordination layer, while the strategy predicates can be hough of as the capability of the supervisor part. The transition from a non-cooperative to a cooperative game also emerges here, through the e ect of the capability vectors of neighboring vehicles on the degraded mode strategy predicates.

Link Layer Supervisor Predicates

A similar predicate structure can be constructed for the link layer. An example of such a structure for the link layer design of 75 is given in 15 . To avoid getting into the details of the various link layer designs that have so far been proposed we will not discuss the issue further here.

7.1.2 Performance Monitor

The performance monitor is designed to continuously track the e ect of disturbances on the system and draw the line between acceptable and unacceptable degradation of performance. In case of unacceptable degradation, the performance structure can either tune the parameters of the a ected controllers or initiate a degraded mode strategy.

Framework

The proposed monitor involves three elements. The rst is the causes of gradual performance degradation which the controller will have to guard against. They include adverse weather conditions such as rain, fog or snow and gradual hardware degradation such as brake ware. An extensive list of such causes, compiled by interviewing numerous PATH researchers, can be found in 75 . We will use C to denote the set of performance degradation causes. Assuming there are c such causes, C has the form:

C Rc
Each entry of C is a real number whose magnitude signi es the severity of the cause e.g. the longitudinal wind measured in meters per second 1 . The second factor is the performance parameters that can be used to monitor the system performance. These performance parameters depend on the layer of the architecture and include, for example, the maximum and minimum deceleration available to the vehicle for the physical layer and the maximum tracking error of the various continuous controllers for the regulation layer. We use P to denote the set of performance parameters. The set P is divided according to the level of the hierarchy associated with each parameter. P = PP PR PC PL PN i i where PP = fPP ; i = 1; : : : ; np g are the parameters associated with the physical layer, PR = fPR ; i = 1; : : : ; nr g the ones associated with the regulation layer, etc. The nal factor is the performance requirements. They can be thought of as thresholds on the performance parameters. More formally performance requirements are predicates on the space of performance
1 In its simplest form Ci can be thought of as a predicate that returns 1 if cause i is present and 0 if it is not. Soft" approaches, such as fuzzy logic may be used to quantify more elusive causes, such as snow or fog.

98

parameters:

The design of the performance monitor involves nding functional relationships between causes of gradual performance degradation and the performance parameters. It should be noted that the performance parameters of higher layers may depend on those of lower layers e.g. the tracking errors may depend on the acceleration bounds. However, there can be no loops in these dependencies, i.e. lower layer parameters may not depend on higher layer ones. Therefore, the overall relationship can be attened into a map:

Ri : P ,! fTrue; Falseg i = 1; : : : ; r

f : C ,! P
that determines how the causes of performance degradation a ect the performance parameters. This map will depend on the details of the control laws. Qualitatively the dependencies will be xed unless major changes are made in the design even though the map will change quantitatively with any change in the ^ controller parameters. In this framework, the range of conditions C under which the performance of the system is acceptable is given by: r ^ C = f ,1Ri,1True C Many iterations o -line may be needed in order to properly capture the system requirements in terms of ^ the above equation for C . A convenient way of thinking of performance parameters and requirements is in terms of the design speci cations, Ji ; Ci , introduced in Chapter 3. The performance parameters can be thought of as the worst case value of Ji . As we assume that the controller design has already been carried out, the control inputs are xed to u = u say and only the disturbance enters the picture. Thus: ^ The threshold Ci imposes the performance requirements. For example, if Ji denotes the time it takes to complete a join maneuver, we can introduce a performance requirement:
 L Ri = True if Ji yd   Ci False otherwise
L Recall that yd is the xed point for the leader, hence the standard initial condition for the join maneuver.


i=1

Ji x0  = max Jix0 ; u; d ^ d2D

^ Enhancing the robustness of the system involves enlarging C . This framework can be used for o -line ^ robustness enhancement, where the controllers are tuned to accommodate the largest set of conditions C . The framework can also be used to increase the system autonomy by on-line tuning of the controllers. If the performance requirements are not met by the capability parameters at any time, the control laws are tuned until the new requirements are met by the parameters.

Robustness Enhancement

Degraded Mode Initiation ^ ^ Even after the domain C has been maximized, there will probably still be some conditions in C n C which

are not covered. These conditions for which performance is unacceptably degraded will be treated in a way similar to the treatment of loss of capability due to faults. In this sense, the e ects of gradual degradation and limits of robustness can be modeled as an extra term on the predicates. The overall process is shown in Figure 7.3. 99

Lower Level Predicates

{0,1}

&

{0,1}

R i

{0,1}

Controller Tuning {0,1}

Causes of Degradation and Lower Level Performance Parameters

Qualitative Capability Parameters

Figure 7.3: Introduction of robustness predicates

7.2 Extended Control Structure

In this section we present a speci c controller design that deals with faulty conditions. As discussed in Chapter 5, controllers at each layer of the control hierarchy are divided into two levels: a supervisory level that selects an appropriate strategy and a regulator level which executes individual maneuvers to track this strategy. Pictorially this is indicated in Figure 7.4. This distinction is implicitly present in the normal mode architecture. Because there is only one xed strategy for the normal mode, the division is not explicitly stated. To produce a fault tolerant controller one needs to design new strategies at the supervisory level and new controllers for the regulator level for each layer of the control hierarchy. Here we will give a brief outline of the degraded mode controllers designed for the coordination layer. They consist of a set of new coordination layer strategies that are executed by following a sequence of maneuvers, which we call atomic maneuvers. We will only discuss the main features of the proposed design. The details can be found in 15 and 16 . These references also contain a discussion of the extensions necessary at the regulation and link layers.

7.2.1 Coordination Supervisor Strategies

The extended coordination layer strategies can be separated into two classes. For the most severe faults, strategies are required to stop the vehicle on the highway. Three strategies are de ned for this purpose, Gentle Stop , Crash Stop , and Aided Stop . Once the vehicle comes to rest it is assumed that the link layer employs special control laws to ease congestion, divert tra c away from the incident, assist emergency vehicles and get the queued vehicles out. In this case, the performance of an entire section of the highway is degraded. For less severe faults coordination layer strategies are designed to get the faulty vehicle out of the highway as soon as possible, without stopping the tra c. For this purpose three more strategies are de ned, Take Immediate Exit , Take Immediate Exit - Escorted and Take Immediate Exit - Normal . The degradation in performance is expected to be less, in fact the link layer need not be noti ed at all. In a sense faults that dictate the use of the last three strategies a ect capability predicates up to the coordination layer while faults that dictate the use of the rst three strategies a ect predicates all the way up to the link layer2 .
2 The details of the classi cation of faults induced by the predicate hierarchy can be found in 15

100

Sensory info. Capability info. Performance info.

Link Layer Supervisor

Link Layer

Density / Velocity Profiles

Sensory info. Capability info. Performance info.

COORDINATION Link Layer Regulator LAYER

Sensory info. Capability info. Performance info.

Coordination Supervisor (Strategies)

Coordination Layer

Communication with neighbors Sensory info. Capability info. Performance info. Coordination Maneuver Protocols

Sensory info. Capability info. Performance info.

Regulation Supervisor (Interface)

Regulation Layer

Sensory info. Capability info. Performance info.

Longitudinal Law

Lateral Law

Sensory info. Capability info. Performance info.

Throttle Actuator

Brake Actuator

Steering Actuator

Physical Layer

Vehicle Dynamics (Plant)

Figure 7.4: Extended Control Architecture During the execution of the degraded mode strategies, the faulty vehicle may request cooperation from neighboring vehicles. This cooperation can be encoded by means of communication protocols. In 16 the details of these protocols are speci ed in terms of interacting nite state machines. Here we only give a brief overview of the design. Gentle Stop and Crash Stop Strategies: The goal of these control strategies is to bring the faulty vehicle to a complete stop on the highway. Gentle Stop is used by a faulty vehicle that is ordered to stop and can do so by using its brakes. It is assumed that the fault is not severe enough to require maximum deceleration, hence the vehicle will use gentle braking in order to minimize the disturbance to the following vehicles. Faults that dictate the use of the Gentle Stop strategy include engine failures and communication failures. For Crash Stop, on the other hand, the severity of the fault requires the faulty vehicle to apply maximum emergency braking. Faults that dictate the use of the Crash Stop strategy include steering faults and complete longitudinal sensor failures. Both these strategies do not require any assistance from neighboring vehicles, therefore they are trivially implemented at the maneuver layer the strategies are also the maneuvers. 101

Take Immediate Exit

Two Forced Splits

Emergency Lane Change

5 6

4 7

EXIT
Index : Assisting Vehicle Faulty Vehicle

Figure 7.5: Take Immediate Exit: Highway Snapshots ahead of it in the same platoon to come to a stop. If the faulty vehicle is a leader, it uses the Front Dock maneuver to become a follower before executing the Aided Stop maneuver. The faulty vehicle uses its engine friction to slow down as much as possible, while the assisting vehicle applies gentle deceleration to let the faulty vehicle collide from behind. Then the assisting vehicle uses its brakes to bring the combined mass of both vehicles to a stop. Take Immediate Exit TIE: The Take Immediate Exit strategy Figure 7.5 is used by the faulty vehicle to get out of the AHS as soon as possible. The strategy consists of up to two Forced Split maneuvers to make the faulty vehicle a free agent. This is followed by a sequence of Emergency Lane Change maneuvers until the vehicle reaches the rightmost automated lane from where it takes the next exit. Take Immediate Exit - Escorted TIE-E: This strategy Figure 7.6 is used by a faulty vehicle that has lost the capability to be a platoon leader but can still operate as a follower. In this case, the faulty vehicle leaves the system as part of a two vehicle platoon with itself as the follower. This requires up to two Forced Split maneuvers if the faulty vehicle is already a follower or a Front Dock and possibly a Forced Split maneuver if the faulty vehicle is the leader of its platoon. The leader of this new platoon escorts the faulty vehicle out of the AHS by executing sequence of Emergency Lane Change maneuvers of the two vehicle platoon. Once out of the AHS, the escorting vehicle drops o the faulty vehicle in a special turnout and re-enters the AHS at the next entrance. Take Immediate Exit - Normal TIE-N: This strategy is similar to the TIE strategy except the faulty vehicle uses the normal lane change and split protocols of 34 instead of Emergency Lane Change and Forced Split. It is intended for mild faults that a ect predicates only up to the regulation layer. 102

Aided Stop Strategy: Here a faulty vehicle with a brakes o " failure is aided by the vehicle immediately

Take Immediate Exit

- Escorted

Two Forced Splits

Emergency Lane Change

(Platoon)

EXIT Index : Assisting Vehicle Faulty Vehicle Escorting Vehicle

Figure 7.6: Take Immediate Exit - Escorted: Highway Snapshots

Normal: This is the normal mode strategy de ned by the normal mode AHS architecture of 74 . The
strategy is implemented by means of nite state machines in 34 . The strategies described above are appropriate concatenations of atomic maneuvers. These include the the normal mode maneuvers of 34 as well as some new, emergency maneuvers. As a rule the emergency maneuvers have higher priority than the normal mode maneuvers. Therefore any normal mode maneuvers that may be in progress when an emergency maneuver is requested are aborted. The emergency maneuvers needed to implement the above strategies are: Forced Split is similar to the split maneuver. It is used by a faulty vehicle to become a free agent. If the faulty vehicle is a follower it requests the leader of the platoon to initiate a Forced Split . The leader breaks the platoon at the desired location. Any normal mode maneuvers that may be going on at the time are aborted. Emergency Lane Change is used by a free agent or a platoon. The faulty vehicle requests the leader of the platoon in the adjacent lane to create and maintain a gap so that the faulty vehicle can change lane into it. A special case of Emergency Lane Change is also de ned for a platoon of vehicles to change lane into a gap used for TIE-E. Front Dock Figure 7.7 is initiated by a platoon leader that wants to join with the vehicle in front but, because of a fault, can not apply the acceleration and deceleration required by the Join maneuver. The initiating vehicle requests the leader of the preceding platoon for a Front Dock . The leader of the preceding platoon orders the last vehicle in its platoon itself in the case of a free agent to decelerate and close the gap between itself and the initiator. In the end, the initiator becomes the rst follower of the new platoon. 103

7.2.2 Atomic Maneuvers

Front Dock

Figure 7.7: Front Dock Maneuver engine to decelerate and asks the platoon leader to assist in bringing it to a stop on the highway. The responding vehicle the vehicle immediately ahead of the faulty vehicle applies gentle braking, lets the faulty vehicle collide with it from behind and then uses its brakes to bring the combined mass of both the vehicles to a stop. Queue Buildup and Queue Management are used whenever a faulty vehicle is stopped on the highway. Vehicles in the same lane immediately behind the faulty vehicle will form a queue of stopped vehicles. The Queue Buildup maneuver is used to keep track of the number of vehicles in the queue and the identity of the last queued vehicle. The extended link layer controller is designed to stop the queue buildup by diverting tra c upstream of the stopped vehicle from the blocked lane to the other lanes. If the emergency vehicle e.g. tow truck has not appeared until the queue buildup has stopped, the Queue Management strategy is initiated to get the queued vehicles moving again. The queue is dissipated in last in - rst out" fashion, in some xed platoon sizes. For the queue buildup to stop, there must be a large gap behind the last vehicle of the queue. A platoon of appropriate size breaks away from the end of the queue and backs up into this gap. This platoon will stop its backward motion when it creates su cient spacing between its front vehicle and the last vehicle of the remaining queue. The backup distance depends on the speed of the adjacent lanes and the constraints on acceleration and jerk. The platoon is now ready to accelerate up to the speed of an adjacent lane and change lane using Emergency Lane Change  whenever an appropriate gap approaches. To facilitate the process, the link layer can order the creation of gaps in the adjacent lanes upstream of the incident. It should be noted that the initiation of the Queue Management maneuver and its e cient operation rely on cooperation from the link layer. Like all things depending on the link layer, this maneuver is not critical from the point of view of safety.

Aided Stop is initiated by a follower that has developed a brakes-o failure. The faulty vehicle uses its

7.2.3 Veri cation

Proofs of performance for the communication protocols designed to implement the above strategies can be obtained by means of automatic veri cation. A lot of useful information can be retrieved if the results are interpreted in the appropriate framework. Here we will state the major results of our veri cation e ort; the details of the proofs can be found in 16 . The rst step of the proofs is automatic veri cation using COSPAN 31 . COSPAN is a nite state machine veri cation tool and as such can not be used to answer hybrid questions like the ones tackled in Chapter 6. The theorems listed below settle purely discrete questions, such as do the protocols deadlock?" but not hybrid questions like do vehicles collide?". It should be possible to answer such questions by putting the design through the machinery developed in Chapter 3. The following theorems are based on some additional assumptions about the system operation: Assumption 9 Degraded Operation Assumptions The fault detection and emergency maneuver 104

initiation scheme operates perfectly, the normal mode protocols have been designed and veri ed and the regulation layer behavior can be adequately abstracted by a nite state machine.

The last assumption is quanti ed in 16 , where the nite state abstraction itself is speci ed. A similar assumption was made for the veri cation of the normal mode protocols in 34 . In order to substantiate this assumption a full blown hybrid analysis of the system is needed, similar to the analysis undertaken in Chapter 6 for the normal mode. To state the theorems we also need a de nition: De nition 16 Two faults are intersecting if there exists a vehicle that is involved in the degraded mode strategies for both faults at the same time. Note that this de nition is not intrinsic, as it depends on the coordination layer design. Based on veri cation using COSPAN the following can be shown: Theorem 9 For a nite collection of non-intersecting faults, all faulty vehicles either stop or exit the highway and all other vehicles return to normal operation. For intersecting faults the problem is a lot more complicated. It is easy to show that a deterministic safety de nition like De nition 15 or the one in 99  is likely to be inadequate in this case: Lemma 17 For any control design there exist a combination of two faults that can lead to high relative velocity collisions.

Proof: Consider two consecutive free agents moving in the same lane. Assume the front vehicle develops

a brakes on" failure while the trailing vehicle develops a brakes o " failure. In this situation a high relative velocity collision is likely for the initial free agent spacing. 2 The discussion in 16 indicates that, whatever de nition is given for intersecting faults, great care is needed when making safety requirements. It is proposed that a safety criterion should be obtained by assigning probabilities to fault occurrences, obtaining the resulting probability distributions for the number of collisions and their severity and setting a threshold for these distributions or their expectations. The fault probabilities can be obtained from statistical data while the resulting collision probabilities are a function of the design. Hopefully sever faults such as brakes o " will be rare, making scenarios such as the one introduced above extremely unlikely. At this stage the only thing we are able to say about the e ect of intersecting faults on the coordination layer design proposed here and detailed in 16  is: Theorem 10 For an arbitrary collection of faults the communication protocols will not deadlock.

7.3 Key Points and Discussion

Completing the Design: The degraded mode strategies outlined above represent just one possible
solution to the rst step in the top-down phase of the design process. They need to be complemented by an appropriate regulation layer design, to end the top-down and begin the bottom-up phase. The regulation layer design can be obtained by a process similar to the one described in Chapter 6. Eventually the performance of the overall fault tolerant control architecture will have to be evaluated, to complete the bottom up phase. This is likely to be a very challenging task, since the safety criteria need to be rede ned as discussed above. Safety and performance measures could be obtained essentially deterministically in the normal mode case. Because faults are rare events however, the evaluation of a fault tolerant design will probably have to be probabilistic. 105

Pseudo Sensors and Actuators: A prominent feature of the design proposed here is the use of the

sensors and actuators of neighboring vehicles together with the communication devices as pseudo sensors" and pseudo actuators" for the faulty vehicle. For example, a vehicle that is incapable of braking uses communication in particular the Aided Stop strategy and the brakes of the vehicle ahead of it to come to a stop. Similar arrangements through the TIE-E strategy for example are made for vehicles that have lost sensing capabilities. This kind of interaction is another example of how communication can be used to turn a non-cooperative game like the one used for normal mode design in Chapter 6 into a cooperative game. While in normal mode other agents are viewed as malicious opponents, in degraded modes their actions are controlled" by the faulty vehicle through communication, therefore their role is assisting. The price to pay for this convenience of course is the substantial increase in the design complexity. and design complexity. Quantifying these issues and determining the optimal tradeo is likely to be an overwhelming task however. As discussed above, even the safety requirement is di cult to quantify in this degraded mode setting. No claims of optimality are made for the design presented here. The proposed controller was derived from some intuitive understanding of what schemes are likely to be safe in a given situation. Within this framework of safe schemes the one that let the vehicle move for as long as possible was selected in an attempt to maximize throughput despite the fault. The design obtained in this way is rather complicated. Much simpler designs are possible; for example, the controller could stop the faulty vehicle whatever the fault and wait for an emergency vehicle to tow it out of the AHS. Intuitively such a scheme would lead to a more severe degradation in performance. However, the question of whether these intuitive notions really lead to a better design remains to be investigated. to play an indispensable role for the evaluation of degraded mode strategies. Even though simulation can not replace formal proof, it can still provide valuable information about the system performance. More speci cally, successful results under extensive simulation indicate that the design is likely to behave well, even though there may still be a lot of room left for situations where the system behaves poorly. On the other hand, unsatisfactory performance on the simulation testbed indicates design shortcomings and may suggest improvements. In other words, simulation results can not be taken as proof of the general system performance, but they can be taken as proof of performance in speci c cases, or, more importantly, proof of poor performance in others. Finally, Monte Carlo simulations of the overall fault tolerant system can be used to obtain estimates of the impact of the degraded mode controllers on the highway throughput and hence validate any theoretical models developed for this purpose. The AHS simulator SmartPath 92 has been successfully used in the past to carry out all these tasks for the normal mode 12, 75 . We are currently extending the capabilities of SmartPath to include operation under degraded conditions.

Design Optimality: Any controller design will represent a trade o between safety, highway throughput

The Role of Simulation: Because of the lack of formal analysis and design tools, simulation is likely

106

Chapter 8

Concluding Remarks
The control of large scale systems is one of the major problems facing control engineers today. The biggest challenge arizes from the complexity of the system, that forces designers to use controllers at di erent levels of abstraction. The modeling, design and analysis of the resulting hierarchical, hybrid control structures poses a number of interesting problems. In this dissertation we tried to address three of them: controller design, closed loop performance veri cation and extending the controller autonomy. We concentrated on physical processes that can be modeled by a collection of interacting agents. Each agent was modeled by a hybrid dynamical system with relatively few continuous and discrete states. The agent description can be modular, that is the agent can be modeled by an interconnection of even simpler hybrid automata, representing for example the plant, sensors, actuators and controllers. The interaction of the agents with one another is modeled by appropriate interconnections of the overall agent automata. This process allows us to model vastly complicated large scale systems as a hierarchy of relatively simple hybrid models. Our approach to controller design involves two phases. The overall goal is to produce a controller such that the closed loop system exhibits some desired emergent behavior. The emergent behavior is usually described linguistically increased safety, improved e ciency, reduced environmental impact, etc.. In the rst phase of the design process top-down performance speci cation are established to quantify the desired emergent behavior. The performance speci cations then get parsed to a preliminary high level discrete controller design. This process imposes requirements on the low level continuous controller. We present an algorithm to determine under what conditions these requirements can be met and produce a hybrid low level controller to meet them whenever possible. The algorithm makes use of ideas from game theory. The performance of each agent is treated as a game between two players: the actions of the agent itself control and the actions of other agents disturbance. This process imposes further restrictions on the high level controller. In the second phase of the design process bottom-up the high level controller is modi ed to take into account these restrictions. The implication is that, if the process is completed successfully, the closed loop hybrid system is guaranteed to satisfy the performance speci cations without the need for further veri cation. To conclude the bottom-up phase, the e ect of the resulting hybrid controller on the physical process needs to be abstracted so that the resultin emergent behavior can be evaluated. The bottom-up phase of the design process can also be though of as a veri cation technique in cases where a controller has already been designed. Optimal control is used to determine the worst possible evolution of the system for example each agent with respect to some desired property. If the performance speci cations are satis ed in this case, then they are guaranteed to be satis ed for all other evolutions as well. If the performance speci cations are not satis ed, the veri cation process reveals ways in which the design can be modi ed in terms of restrictions on the actions of other agents and on the switching patterns of the hybrid control scheme. If these restrictions are implemented in the discrete design e.g. by means of inter-agent coordination the modi ed controller is guaranteed to satisfy the performance speci cations. 107

The design and veri cation techniques are based on a nominal, deterministic model of the physical process. Ideally we would like the controller to be capable of dealing with physical processes that deviate from this nominal model. For deviations which are in some sense small" such extensions are usually implemented by adaptive or robust controller designs. In this dissertation we investigated extensions of the controller autonomy with respect to large", qualitative deviations from the nominal model. Such deviations can be caused by rare events, such as faults or extreme degradation in environmental conditions under which the performance speci cations can not be met. We concentrated on the hierarchical and hybrid issues that arise in this case. We discussed how these e ects can be modeled in the multiagent framework and indicated qualitative changes in the information and control structures needed to implement the autonomous design. This informal discussion and the insight gained by the application of these principles to the automated highway problem indicates that the problem is very subtle. It is unlikely that the deterministic, worst case approach used to design controllers for the nominal model can be used in this case. A stochastic formulation of the performance requirements is likely to be needed. The application of our techniques was illustrated throughout by means of small examples. In addition a more thorough investigation was carried out based on a case study on the automated highway problem. Using the work of 74, 34 as a preliminary high level design we applied our algorithm to obtain a hybrid low level controller and discrete requirements on the high level. The result was a theorem that established conditions under which the safety of the automated highway can be guaranteed. The safe controllers used in the theorem were obtained using the game theoretic principles. In most cases we were able to obtain analytical solution, but numerical calculations also need to be carried out. Because our calculations were based only on assumptions about the physical process, they can be used to investigate interesting design alternatives and provide requirements on the physical process in order to guarantee certain levels of performance. Further, we investigated extending the autonomy of the control scheme. We brie y outlined a possible fault tolerant design and investigated the conditions under which it is likely to produce the desired e ect. Clearly, there is signi cant work that needs to be done. The algorithms presented here addressed only a small part of the design problem. We still need to develop techniques for parsing the emergent behavior speci cations in terms of a discrete design, modifying the discrete design to take into account the needs of the continuous design and re-abstracting the overall system performance in terms of its emergent behavior. Moreover, our heuristic techniques for extending the system autonomy need to be made rigorous. The application of these techniques to further case studies are also worth pursuing. We are currently investigating the example of air tra c management systems. Other possible applications include power systems and the control or simulation of systems that mimic human or animal behavior. In the long run the ideas of abstracting continuous information and determining the minimum coordination needed between processes to achieve a certain goal may nd applications to image processing, image compression, distributed algorithms and parallel computation. Finally, some speculation. There seem to be two main problems in the design of large scale systems. The rst problem is how to quantify emergent behavior requirements, i.e. how to come up with appropriate quantitative performance speci cations. The second problem is how to design a hybrid controller to meet these performance speci cations. In terms of automated highway systems, the rst problem manifests itself in the modeling of the network and link layers and the second problem manifests itself in the design of the coordination and regulation layer controllers. The top-down and bottom-up phases discussed above contain a little of both problems: the rst part of the top-down phase and the last part of the bottom-up phase deal with the rst problem while the rest of the design process deals with the second. The extension of the system autonomy also seems to contain a little bit of both problems. Currently the major hurdle seems to be the rst problem, how to specify safety requirements in the presence of faults. This dissertation has little to say about the rst problem. At this stage it seems that the process is case speci c and relies heavily on intuition about the problem. It is therefore likely to be di cult to formalize 108

it in an abstract setting. The only thing our approach can be used for in this setting is abstracting the performance of the closed loop system, once a way of quantifying of quantifying the emergent behavior has been determined. Some progress is made here towards solving the second problem. It seems that game theory is a convenient framework for approaching the problem as it provides a concrete way of modeling the interaction between non-cooperating agents. Solution to the game theory problems directly provides low level, hybrid controllers and requirements for the high level controllers. It should be noted that all the high level requirements observed in the examples can be encoded in terms of languages accepted by timed automata. The work of 35 indicates that problems in timed languages are purely discrete problems. Therefore, in principle, the design of high level controllers is tractable and could be tackled using discrete tools. These observations should not lead to the mistaken impression that the work presented here solves the whole of the second problem. Unfortunately, game theory problems are very hard to solve in general. In the examples considered here we were fortunate enough to be able to obtain solutions almost analytically. This is unlikely to be the case in general. Moreover, even if computational tools for solving such problems were available, questions of existence of solutions, uniqueness and local minima limit the theoretical results that can be expected in general. Moreover, the statement that the high level design can be encoded by means of timed languages is merely an observation based on our examples. This may prove to be the case in general, however, the problem of obtaining discrete controller designs to implement the timed speci cations is still likely to be hard. A proper framework of extending the theoretical results available for nite state systems to the case of timed systems needs to be developed and implemented by means of computational tools.

109

Bibliography
1 A. Nerode and W. Kohn, Models for hybrid systems: Automata, topologies, controllability, observability," in Hybrid System R. L. Grossman, A. Nerode, A. P. Ravn, and H. Rischel, eds., no. 736 in LNCS, pp. 317 356, New York: Springer Verlag, 1993. 2 R. W. Brockett, Hybrid models for motion control systems," in Perspectives in Control H. Trentelman and J. Willems, eds., Birkhauser, 1993. 3 X. Nicollin, A. Olivero, J. Sifakis, and S. Yovine, An approach to the description and analysis of hybrid systems," in Hybrid System R. L. Grossman, A. Nerode, A. P. Ravn, and H. Rischel, eds., no. 736 in LNCS, pp. 149 178, New York: Springer Verlag, 1993. 4 P. J. Antsaklis, J. A. Stiver, and M. Lemmon, Hybrid system modeling and autonomous control systems," in Hybrid System R. L. Grossman, A. Nerode, A. P. Ravn, and H. Rischel, eds., no. 736 in LNCS, pp. 366 392, New York: Springer Verlag, 1993. 5 R. Alur, C. Courcoubetis, T. A. Henzinger, and P. H. Ho, Hybrid automaton: An algorithmic approach to the speci cation and veri cation of hybrid systems," in Hybrid System R. L. Grossman, A. Nerode, A. P. Ravn, and H. Rischel, eds., no. 736 in LNCS, pp. 209 229, New York: Springer Verlag, 1993. 6 M. S. Branicky, V. S. Borkar, and S. K. Mitter, A uni ed framework for hybrid control: Background, model and theory," Tech. Rep. LIDS-P-2239, Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, 1994. 7 M. S. Branicky, Control of Hybrid Systems. PhD thesis, Massacussets Institute of Technology, 1994. 8 A. Deshpande, Control of Hybrid Systems. PhD thesis, Department of Electrical Engineering, University of California, Berkeley, California, 1994. 9 A. Puri, Theory of Hybrid Systems and Discrete Event Systems. PhD thesis, Department of Electrical Engineering, University of California, Berkeley, California, 1995. 10 T. Henzinger, P. Kopke, A. Puri, and P. Varaiya, What's decidable about hybrid automata," in STOCS, 1995. 11 Z. Manna and A. Pnueli, Temporal Veri cation of Reactive Systems: Safety. New York: SpringerVerlag, 1995. 12 D. N. Godbole, J. Lygeros, and S. Sastry, Hierarchical hybrid control: an IVHS case study," in Hybrid Systems II P. Antsaklis, W. Kohn, A. Nerode, and S. Sastry, eds., no. 999 in LNCS, Springer Verlag, 1995. 110

13 J. Lygeros and D. Godbole, An interface between continuous and discrete event controllers for vehicle automation," Tech. Rep. PATH Memorandum 93-8, Institute of Transportation Studies, University of California, Berkeley, 1993. Also in Proceedings of the 1994 American Control Conference. 14 J. Lygeros, D. N. Godbole, and S. Sastry, Simulation as a tool for hybrid control," in AIS Conference on Distributed Interactive Simulation Environments, 1994. 15 J. Lygeros, D. N. Godbole, and M. E. Broucke, Towards a fault tolerant AHS design." SAE Paper  951894, Presented at SAE Future Transportation Technology Conference, Costa Mesa, 1995. 16 D. N. Godbole, J. Lygeros, E. Singh, A. Deshpande, and A. Lindsey, Design and veri cation of coordination layer protocols for degraded modes of operation of AHS," in IEEE Control and Decision Conference, pp. 427 432, 1995. 17 S. Sastry, G. Meyer, C. Tomlin, J. Lygeros, D. Godbole, and G. Pappas, Hybrid systems in air tra c control," in IEEE Control and Decision Conference, pp. 1478 1483, 1995. 18 J. Lygeros, D. N. Godbole, and S. Sastry, A game theoretic approach to hybrid system design," Tech. Rep. UCB ERL-M95 77, Electronic Research Laboratory, University of California Berkeley, October 1995. 19 J. Lygeros, D. N. Godbole, and S. Sastry, Optimal control approach to multiagent, hierarchical system veri cation," in IFAC World Congress, 1996. 20 J. Lygeros, To brake or not to brake? is there a question?." preprint, March 1996. 21 J. Lygeros, D. N. Godbole, and S. Sastry, A veri ed hybrid controller for automated vehicles." preprint, submitted to Special Issue on Hybrid Systems of the IEEE Transactions on Automatic Control, March 1996. 22 R. F. Stengel, Intelligent ight control systems," in IMA Conference on Aerospace Vehicle Dynamics, September 1992. 23 A. Nerode and W. Kohn, Multiple agent hybrid control architecture," in Hybrid System R. L. Grossman, A. Nerode, A. P. Ravn, and H. Rischel, eds., no. 736 in LNCS, pp. 297 316, New York: Springer Verlag, 1993. 24 P. Varaiya and S. E. Shladover, Sketch of an IVHS systems architecture," Tech. Rep. UCB-ITSPRR-91-3, Institute of Transportation Studies, University of California, Berkeley, 1991. 25 K. M. Passino and P. J. Antsaklis, Modeling and analysis of arti cially intelligent planning systems," in An Introduction to Intelligent and Autonomous Control P. J. Antsaklis and K. M. Passino, eds., pp. 191 214, Boston: Kluwer Academic Publishing, 1993. 26 C. Heitmayer and N. Lynch, The generalized railroad crossing: A case study in formal veri cation of real-time systems," in Proc. ICCC Real-Time Systems Symposium, San Juan, Puerto Rico, 1994. 27 J. Frankel, L. Alvarez, R. Horowitz, and P. Li, Safety oriented maneuvers for IVHS," in American Control Conference, pp. 668 672, 1995. 28 P. Li, L. Alvarez, and R. Horowitz, AVHS safe control laws for platoon leaders." preprint, January 1996. 111

29 T. Baar and P. Bernhard, H 1 -Optimal Control and Related Minimax Design Problems. Birkhauser, s 1991. 30 J. C. Doyle, K. Glover, P. P. Khargonekar, and B. A. Francis, State-space solutions to standard H2 and H1 control problems," IEEE Transactions on Automatic Control, vol. 34, no. 8, pp. 831 847, 1989. 31 Z. Har'El and R. Kurshan, Cospan User's Guide. AT&T Bell Laboratories, 1987. 32 Adnan Aziz, et al., HSIS: a BDD-based environment for formal veri cation," in ACM IEEE International Conference on CAD, 1994. 33 M. Heymann, Hierarchical decomposition of hybrid systems." preprint, 1994. 34 A. Hsu, F. Eska , S. Sachs, and P. Varaiya, Protocol design for an automated highway system," Discrete Event Dynamic Systems, vol. 2, no. 1, pp. 183 206, 1994. 35 R. Alur and D. Dill, A theory of timed automata," Theoretical Computer Science, vol. 126, pp. 183 235, 1994. 36 R. Alur, C. Courcoubetis, and D. Dill, Model checking for real-time systems," Logic in Computer Science, pp. 414 425, 1990. 37 R. P. Kurshan, Computer-aided veri cation of coordinating processes; the automata-theoretic approach. Princeton University Press, 1994. 38 C. Daws and S. Yovine, Two examples of veri cation of multirate timed automata with KRONOS," in Proc. 1995 IEEE Real-Time Systems Symposium, RTSS'95, Pisa, Italy, IEEE Computer Society Press, Dec. 1995. 39 K. Larsen, P. Pettersson, and W. Yi, Compositional and symbolic model checking of real-time systems," in 16th Annual Real-time Systems Symposium, pp. 76 87, IEEE Computer Society Press, 1995. 40 F. Balarin, Iterative Methods for Formal Veri cation of Digital Systems. PhD thesis, University of California, Berkeley, 1994. 41 F. Balarin, K. Petty, and A. L. Sangiovanni-Vincentelli, Formal veri cation of the PATHO real-time operating system," in IEEE Control and Decision Conference, pp. 2459 2465, 1994. 42 A. Puri and P. Varaiya, Decidability of hybrid systems with rectangular di erential inclusions," in Computer Aided Veri cation, pp. 95 104, 1994. 43 T. A. Henzinger, P. H. Ho, and H. W. Toi, A user guide to HYTECH," in TACAS 95: Tools and Algorithms for the Construction and Analysis of Systems E. Brinksma, W. Cleaveland, K. Larsen, T. Margaria, and B. Ste en, eds., no. 1019 in LNCS, pp. 41 71, Springer Verlag, 1995. 44 A. Puri and P. Varaiya, Driving safely in smart cars," in American Control Conference, pp. 3597 3599, 1995. 45 R. Alur and T. A. Henzinger, Computer-Aided Veri cation. 1996. to appear. 46 A. Deshpande, D. Godbole, A. Gollu, L. Semenzato, R. Sengupta, D. Swaroop, and P. Varaiya, Automated highway system tool interchange format." preprint California PATH Technical Report, Institute of Transportation Studies, University of California, Berkeley, 1996. 112

47 L. Semenzato, A. R. Deshpande, A. Gollu, P. Varaiya, D. Godbole, and R. Sengupta, SHIFT reference manual." preprint California PATH Technical Report, Institute of Transportation Studies, University of California, Berkeley, 1996. 48 L. Berkovitz, Optimal Control Theory. Springer-Verlag, 1974. 49 T. Baar and G. J. Olsder, Dynamic Non-cooperative Game Theory. Academic Press, second ed., s 1995. 50 J. Lewin, Di erential Games. Springer-Verlag, 1994. 51 T. Baar and A. Haurie, eds., Advances in Dynamic Games and Applications, vol. 1 of Annals of the s International Society of Dynamic Games. Birkhauser, 1994. 52 L. Pontryagin, V. Boltyanskii, R. Gamkrelidge, and E. Mishchenko, The Mathematical Theory of Optimal Processes. Wiley, 1962. 53 L. C. Young, Optimal Control Theory. Chelsea, second ed., 1980. 54 H. B. Khalil, Nonlinear Systems. MacMillan, 1992. 55 S. Wiggins, Introduction to Applied Nonlinear Dynamical Systems and Chaos. Springer-Verlag, 1990. 56 J. R. Munkres, Topology, a First Course. Prentice Hall, 1975. 57 P. J. G. Ramadge and W. M. Wonham, The control of discrete event dynamical systems," Proceedings of the IEEE, vol. Vol.77, no. 1, pp. 81 98, 1989. 58 A. L. Schwartz, Theory and Implementation of Numerical Methods Based on Runge-Kutta Integration for Solving Optimal Control Problems. PhD thesis, Department of Electrical Engineering, University of California, Berkeley, California, 1996. 59 A. Puri and P. Varaiya, Veri cation of hybrid systems using abstractions," in Hybrid Systems II, no. 999 in LNCS, Springer Verlag, 1995. 60 D. N. Godbole and J. Lygeros, Longitudinal control of the lead car of a platoon," IEEE Transactions on Vehicular Technology, vol. 43, no. 4, pp. 1125 1135, 1994. 61 S. Sastry and M. Bodson, Adaptive Control. Prentice Hall, 1989. 62 M. Krstic, I. Kanellakopoulos, and P. Kokotovic, Nonlinear and adaptive control design. Wiley, 1995. 63 L.-X. Wang, Adaptive Fuzzy Systems and Control: Design and Stability Analysis. Prenice Hall, 1994. 64 A. Morse, Control using logic based switching," in Trends in Control A. Isidori, ed., pp. 69 114, Springer Verlag, 1995. 65 R. K. Douglas, J. L. Speyer, D. L. Mingori, R. H. Chen, D. P. Malladi, and W. H. Chung, Fault detection and identi cation with application to advanced vehicle control systems." California PATH Research Report UCB-ITS-PRR-95-26, Institute of Transportation Studies, University of California, Berkeley, 1995. 66 S. Patwardhan, Fault Detection and Tolerant Control for Lateral Guidance of Vehicles in Automated Highways. PhD thesis, Department of Mechanical Engineering, University of California, Berkeley, California, 1994. 113

67 V. Garg, Fault Detection in Nonlinear Systems: An application to Automated Highway Systems. PhD thesis, Department of Mechanical Engineering, University of California, Berkeley, California, 1995. 68 A. Agogino, K. Gobel, and S. Alag, Intelligent sensor validation and sensor fusion for reliability and safety enhancement in vehicle control." California PATH Research Report UCB-ITS-PRR-95-40, Institute of Transportation Studies, University of California, Berkeley, 1995. 69 M. Sampath, R. Sengupta, S. Lafortune, and K. Sinamohideen, Diagnosability of discrete-event systems," IEEE Transactions on Automatic Control, vol. AC-40, no. 9, pp. 1555 1575, 1995. 70 S. Shaldover, Operation of automated guideway transit vehicles in dynamically recon gured trains and platoons," 1979. 71 A. Hitchcock, Casualties in accidents occuring during split and merge maneuvers," tech. rep., PATH Technical Memo 93-9, Institute of Transportation Studies, University of California, Berkeley, 1993. 72 D. Swaroop, String Stability of Interconnected systems: an application to platooning in automated highway systems. PhD thesis, Department of Mechanical Engineering, University of California, Berkeley, California, 1994. 73 B. S. Y. Rao and P. Varaiya, Flow bene ts of autonomous intelligent cruise control in mixed manual and automated tra c," Transportation Research Record, no. 1408, pp. 36 43, 1993. 74 P. Varaiya, Smart cars on smart roads: problems of control," IEEE Transactions on Automatic Control, vol. AC-38, no. 2, pp. 195 207, 1993. 75 B. S. Y. Rao and P. Varaiya, Roadside intelligence for ow control in an IVHS," Transportation Research - C, vol. 2, no. 1, pp. 49 72, 1994. 76 P. Li, R. Horowitz, L. Alvarez, J. Frankel, and A. Robertson, Tra c ow stabilization," in American Control Conference, pp. 144 149, 1995. 77 R. W. Hall, Longitudinal and lateral throughput on an idealized highway." PATH Technical Report, Institute of Transportation Studies, University of California, Berkeley, 1993. 78 M. E. Broucke and P. Varaiya, A theory of tra c ow in automated highway systems." preprint, 1995. 79 P. Ioannou, C. Chen, and J. Hauser, Autonomous intelligent cruise control," Tech. Rep. No 92-05-01, University of Southern California, Los Angeles, 1992. 80 W. Ren and D. Green, Continuous platooning: a new evolutionary and operating concept for automated highway systems," Tech. Rep. UCB ERL-M94 24, Electronic Research Laboratory, University of California Berkeley, 1994. 81 J. Forbes, T. Huang, K. Kanazawa, and S. Russel, The BATmobile: Towards a bayesian automated taxi," in International Conference on Arti cial Intelligence IJCAI-95, Morgan-Kaufmann, August 1995. 82 S. Sheikholeslam and C. A. Desoer, Longitudinal control of a platoon of vehicles," in American Control Conference, pp. 291 297, 1990. 83 S. Sheikholeslam and C. A. Desoer, Longitudinal control of a platoon of vehicles with no communication of lead vehicle information," in American Control Conference, pp. 3102 3107, 1991. 114

84 S. Sheikholeslam and C. A. Desoer, Combined longitudinal and lateral control of a platoon of vehicles," in American Control Conference, pp. 1763 1767, 1992. 85 S. Sheikholeslam and C. Desoer, Indirect adaptive control of a class of interconnected non-linear dynamical systems," International Journal of Control, vol. 57, no. 3, pp. 743 765, 1993. 86 H. Peng and M. Tomizuka, Lateral control of front-wheel-steering rubber-tire vehicles," Tech. Rep. UCB-ITS-PRR-90-5, Institute of Transportation Studies, University of California, Berkeley, 1990. 87 W. Chee and M. Tomizuka, Lane change maneuver of automobiles for the intelligent vehicle and highway systems IVHS," in American Control Conference, pp. 3586 3587, 1994. 88 D. Koller, T. Luong, and J. Malik, Binocular stereopsis and lane marker ow for vehicle navigation: Lateral and longitudinal control," Tech. Rep. UCB CSD-94-804, University of California, Berkeley, 1994. 89 J. C. Gerdes, D. B. Maciuca, P. E. Devlin, and J. K. Hedrick, Brake system modeling for IVHS longitudinal control." In proceedings of DSC-Vol. 53, Advancs in Robust and Nonlinear Control Systems, ASME Winter Annual Meeting, 1994. 90 B. Foreman, A survey of wireless communications technologies for automated vehicle control." SAE Paper  951928, Presented at SAE Future Transportation Technology Conference, Costa Mesa, 1995. 91 J. Lygeros and D. Godbole, An interface between continuous and discrete event controllers for vehicle automation," in American Control Conference, pp. 801 805, 1994. 92 F. Eska , D. Khorramabadi, and P. Varaiya, SmartPath: An automated highway system simulator." PATH Technical Report UCB-ITS-94-4. Institute of Transportation Studies, University of California, Berkeley, 1994. 93 Y. Yang and B. Tongue, Intra-platoon collision behavior during emergency operations," Vehicle System Dynamics, vol. 23, no. 4, pp. 279 292, 1994. 94 F. M. Callier and C. A. Desoer, Linear System Theory. Springer-Verlag, 1991. 95 S. Sheikholeslam, Control of a class of interconnected nonlinear dynamical systems: The platoon problem. PhD thesis, Department of Electrical Engineering, University of California, Berkeley, California, 1991. 96 J. Lygeros, D. N. Godbole, and M. E. Broucke, Extended architecture for degraded modes of operation of IVHS," in American Control Conference, pp. 3592 3596, 1995. 97 H. Peng and M. Tomizuka, Vehicle lateral control for highway automation," in American Control Conference, pp. 788 794, 1990. 98 D. N. Godbole, F. Eska , E. Singh, and P. Varaiya, Design of an entry and exit maneuvers for AHS," in American Control Conference, pp. 3576 3580, 1995. 99 A. Hitchcock, A speci cation of an automated freeway with vehicle borne intelligence," Tech. Rep. PATH Memorandum 92-8, Institute of Transportation Studies, University of California, Berkeley, 1992. 100 A. Kanaris, P. Ioannou, and F. S. Ho, Spacing and capacity evaluations for di erent AHS concepts." preprint California PATH Technical Report, Institute of Transportation Studies, University of California, Berkeley, 1996. 115

Appendix A

Sensor & Actuator Ranges

A.1 Vehicle Capabilities
The maximum acceleration and maximum jerk that a vehicle can produce depends on the engine performance, aerodynamic design of the body, tire condition and tire road interaction. The maximum braking force is a ected by the tire conditions, tire road interactions and the brake dynamics. 100 contains typical jerk and acceleration values for most of the popular passenger cars, buses and trucks on both dry and wet surfaces. We include a few representative values in the following table. The braking capability Vehicle Type Family Sedans Ford Contour Up-scale Sedans Toyota Avalon Low Priced Sedans Nissan Sentra Mid Sized Coupes Ford Thunderbird Utility Vehicles Jeep Cherokee Small Vehicles Geo Metro School Buses Transit Buses GMC 6 Truck Ford 4  2 Tractor Stuart Auto Hauler Stop Dist. Max. Decel. Stop Dist Max. Decel m DRY m s2  DRY m WET m s2  WET 42 8.43 50 7.16 38 43 39 43 45 112 87 158 87 130 9.12 8.33 9.02 8.24 7.84 3.14 4.02 3.23 4.12 2.74 44 48 46 47 52 8.04 7.45 7.75 7.45 6.86

Table A.1: Vehicle Capabilities of a vehicle decreases dramatically on icy roads where a passenger vehicles can not decelerate faster than 2m=s2 . Typical values for maximum jerk for acceleration and braking are given by, 50m=s3 for passenger vehicles, 40m=s3 for buses and 30m=s3 for trucks. 116

The above values represent the capability of the vehicles. Several studies on the driving simulator reported in the literature indicate that for a comfortable ride to the passengers, the acceleration should be bounded in magnitude by 2m=s2 and the jerk should not exceed 2:5m=s3 . For the worst case safety analysis presented in this paper, we use somewhat conservative values for vehicle capabilities: amax = 3:0m=s2 , amin = ,5m=s2 , jmax = 5m=s3 , jmin = ,5m=s3 . All vehicles were assumed to possess identical capabilities.

A.2 Relative velocity at impact

Vehicle bumpers are typically designed to withstand low relative velocity collisions. The current industry standards for passenger vehicles require that the bumpers withstand collision with a stationary wall in nite mass at a speed of 5 MPH. Thus these bumpers should withstand a collision with a vehicle of comparable mass at 10 MPH. On the other hand, in 71 , Hitchcock derived a relationship correlating the relative velocity of collision to the severity of injuries sustained by the passengers. Using real data collected by NHTSA from the actual accidents on highways he concluded that inter-vehicle collisions with relative velocity less than 3:3m=s2 did not result in signi cant injury to the passengers. In our safety calculations we have used va = ,3m=s2 .

A.3 Sensor Ranges

The above table also contains stopping distances for passenger vehicles traveling at highway speeds of 60 MPH or 27 m s. The longitudinal front distance and relative velocity sensor range should be at least as big as the stopping distance for a vehicle. Thus the passenger vehicles need sensor range of 60m whereas buses and trucks need sensor range of the order of 100-150m. Note that at the edge of their range, the sensors need not be very accurate. Radar sensors and vision sensors can provide such a range. For platooning operation, we also need highly accurate sensing at short distances. Apart from these environmental sensors, the vehicle controllers also need vehicle self-state sensors to obtain velocity and acceleration information.

117