REPORT OPTIMAL CONTROL THEORY (STOCHASTIC CONTROL)

Submitted By Shrey Bhatnagar 07255 EED

STOCHASTIC CONTROL It is a part of control theory which deals with control of systems involving probability or having random behavior. RANDOM PROCESS it is the counterpart of deterministic process. Instead of dealing with only one possible reality of how the process might evolve under time, in a stochastic or random process there is some indeterminacy in its future evolution described by probability distributions. This means that even if the initial condition (or starting point) is known, there are many possibilities the process might go to, but some paths may be more probable and others less so. RANDOM VARIABLE A variable whose value results from a measurement on some random process. A random variable is a function, which maps events or outcomes (e.g., the possible results of rolling two dice: (1, 1), (1, 2), etc.) to real numbers (e.g., their sum). A random variable's possible values might represent the possible outcomes of a yet-to-be-performed experiment, or the potential values of a quantity whose already-existing value is uncertain. PROBABILITY DENSITY FUNCTION a function that describes the relative likelihood for this random variable to occur at a given point. The probability for the random variable to fall within a particular region is given by the integral of this variables density over the region. The probability density function is nonnegative everywhere, and its integral over the entire space is equal to one. VARIANCE In probability theory and statistics, the variance is used as a measure of how far a set of numbers are spread out from each other. It is one of several descriptors of a probability distribution, describing how far the numbers lie from the mean (expected value). Continuous case

If the random variable X is continuous with probability density function f(x),

Root of variance is known as standard deviation

CO-VARIANCE In probability theory and statistics, covariance is a measure of how much two variables change together. Variance is a special case of the covariance when the two variables are identical.

As a result for random variables with finite variance the following inequality holds via the CauchySchwarz inequality:

Correlation coefficient pxy is the ratio of above equation. Its value lies between -1 to 1

STATIONARY PROCESS It is a stochastic process whose joint probability distribution does not change when shifted in time or space. As a result, parameters such as the mean and variance, if they exist, also do not change over time or position. Stationarity is used as a tool in time series analysis, where the raw data are often transformed to become stationary. Example: economic data are often seasonal and/or dependent on a non-stationary price level. NON STATIONARY PROCESS It is a stochastic process whose joint probability distribution changes when shifted in time or space. As a result, parameters such as the mean and variance, if they exist, also also change over time or position.

WHITE NOISE White noise is a random signal (or process) with a flat power spectral density. In other words, the signal contains equal power within a fixed bandwidth at any center frequency. White noise draws its name from white light in which the power spectral density of the light is distributed over the visible band in such a way that the eye's three color receptors are approximately equally stimulated. While it is usually applied in the context of frequency domain signals, the term white noise is also commonly applied to a noise signal in the spatial domain. In this case, it has an auto correlation which can be represented by a delta function over the relevant space dimensions. The signal is then "white" in the spatial frequency domain.

Figure 1: Gaussian white Noise

RESPONSE OF LINEAR CONTINOUS SYSTEMS TO WHITE NOISE If white noise is given as input to the linear continuous system, then the response of the system has been derived. In particular, mean and variance of the state is derived. Let the state equation of the system be: ( ) Where X(t)= state vector A(t)= characteristic matrix B(t) = control distribution matrix W(t) = white noise(input) The vector stochastic process w(t) is a non-zero white noise with intensity Q(t) E{w(t)} = 0 E{w(t) w(t)} = Q(t)(t-) Let the mean be and variance be P0 E{x(t0)w(t)} = 0 E{x(t0)} = 0 E{(x(t0) - )) (x(t0) - ))} = 0 The solution of the above equation is given by ( ) Here, ( ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

) is the state transition matrix and it must satisfy the following relations: ( ) ( ) ( )

From the above equation, we have: * ( )+ * ( ) ( )+ ( Now, to find the variance: P(t) = E{x(t) x(t)} On differentiating the above equation we get ( ) =E{ ( ) x(t) + Substituting ( ) from the initial equation: ( ) * ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) * ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )( ) ( )+ ( ) ( ) -+ ( )+ ( ) x(t)} ) * ( ) ( ) ( ) +

( ) ( )

* ( ) ( ) ( ) )

* ( ) ( ), ( ) ( ( ) ( ) ( )-

On placing the expectation operator inside the integrals ( ) ( ) ( ) ( ) ( ( ) * ( ) ( )+ ( )+ ( ) ( ) ( ) ( ) ( ) * ( ) ( )+ ( )+ ( )

) * ( )

* ( )

( ) ( )

The third and the fifth terms are zero ( ) ( ) ( ) ( ) ( ) ( ) * ( ) ( ) * ( ) ( )+ ( )

( )+

( ) ( ) ( ) ( ) ()

( ) ( )

( ) ( )

This equation is solved with the initial condition P( ) = E{x( ) x( )} The variance thus obtained is: ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )