2

Random Variables

Sec. 2.2. DEFINITION OF A RANDOM VARIABLE

59

:n the induced probability space of a random variable every event is a subset of R. But is fvery subset of R always an event? Are there subsets of R that could get us into trouble va violating the axioms of probability? The answer is yes, but fortunately these subsets are not of engineering or scientific importance. We say that they are nonmeasurableJ Sets of practical importance are of the form {x = a}, {x: a < x < 6}, {x: a < x <b}, {x: a < x < b}, x: a < x < b} and their unions and intersections. These five sets are usually abbreviated, respectively, as [a], [a,b], (a, b}, [a,b), and (a,b). Intervals that include the end points are od to be closed; those that leave out end points are said to be open. Intervals can be half-open, and so forth. We can define more than one random variable on the same underlying sample space Q. For example, suppose that f consist of a large, representational group of people in the United States. Let the experiment consist of choosing a person at random. Let X denote the person's lifetime and Y denote that person's daily consumption of cigarettes. We can now ask: Are X and Y related? That is, can we predict X frorn observing Yl Suppose we define a third random variable Z that denotes the person's weight. Is Z related to X atYl The main advantage of dealing with random variables is that we can define certain probability functions that make it both convenient and easy to compute the probabilities of various events. These functions must naturally be consistent with the axiomatic theory. For this reason we must be a little careful in deming events on the real line. Elaboration ;>f the ideas introduced in this section is given next.

X of all Borel subsets in R. making up the field ^ are events. What is an inverse image? Consider an arbitrary Borel set of real numbers B; the set of points EB in fl for which X() assumes vales in B is called the inverse image of the set B under the mapping X. Finally, all sets of engineering interest can be written as countable unions or intersections of events of the form (oc.o;]. The event {(:.X(() < x} &~ gets mapped under X into (oo,x] e J?. Thus if X is a random variable, the set of points (oo,x] is an event. In many if not most scientific and engineering applications, we are not interested in the actual form of X or the specification of the set fi. For example, we might conceive of an underlying experiment that consists of heating a resistor and observing the position and velocities of the electrons in the resistor. The set f is then the totality of positions and velocities of all N electrons present in the resistor. Let X be the thermal noise current produced by the resistor: clearly X: fi > R although the form of X, that is, the exceedingly complicated equations of quantum electrodynamics that map from electrn positions and velocity configurations to current. is not specified. What we are really interested in is the behavior of X. Thus although an underlying experiment with sample description space f may be implied, it is the real Une R and its subsets that will hold our interest and figure in our computations. Under the mapping X we have, in effect, generated a new probability space (f, J?, PX) where R is the real line, & is the Borel cr-algebra of all subsets of R generated by countable unions and intersections of sets of the form (00, x}, and PX is a set function assigning a number PX [A] > O to each set A .$.^ In order to assign certain desirable continuity properties to the function FX(X) at x 00, we require that the events {X = 00} and {X = 00} have probability zero. With

:he latter our specification of a random variable is complete, and we can summarize much o the above discussion in the following definition. Definition 2.2-1 Let J^be an experiment with sample description space l. Then ihe real random variable X is a function whose domain is O that satisfies the following: ' -B} is an event and (ii)

62

Chapter 2

Random Variables

Sec. 2.3.

PROBABILITY DISTRIBUTION FUNCTION

64

KL 2.3.

PROBABILITY DISTRIBUTION FUNCTION

66

68

Chapter 2

Random Variables

with parameters a, 6, that is, f x ( x ) = (2-Kb2) 1 / 2 exp{-|(^y^)2}, we would indeed find that /j, a, a = b. These calculations are postponed until Chapter 4. In the meanwhile we furnish some simple examples. Example 2.4-1 Let f x ( x ) = 1, for O < < 1 and zero elsewhere. This pdf is a special case of the uniform law discussed below. The mean is computed as
r H = I xfx(x)dx
J-oo

r1 = I dx = 0.5
JO

and the variance is computed as

oo />!

/ oo

(x - rffxdx = I (x- 0.5)2dx = 1/12.

JO

Example 2.4-2 Suppose we are given that P[X = 0] = P[X = 2] = 0.25 and P[X = 1] = 0.5. For this discrete random variable (r.v.) we use Equations 2.4-10 and 2.4-11 to obtain
\ = O x 0.25 + 1 x 0.5 + 2 x 0.25 = 1
and

o-2 = (O - I) 2 x 0.25 + (1 - I) 2 x 0.5 + (2 - I) 2 x 0.25 = 0.5

In Chapter 4 we shall discuss what these quantities actually mean. When we want to say that a random variable X obeys the Normal probability law with mean /j. and standard deviation u, we shall use the symbols X: N(/J,, u 2 ). The Normal pdf is shown in Figure 2.4-1. The Normal pdf is widely encountered in all branches of science and engineering as well as in social and demographic studies. For example. the IQ of children, the heights of men (or women), the noise voltage produced by a thermally agitated resistor, all are postulated to be approximately Normal over a large range of vales.

277-0-2

277-0-

Figure 2.4-1

The Normal pdf.

-: : 4

PROBABILITY DENSITY FUNCTION (pdf)

69
Suppose we are given

Conversin of the Gaussian pdf to the standard Normal. j. <j2) and must evalate P[a < X < 6]. We have

P[a < X < b] =

dx.

Rlth 3 = (x /u)/(T, dj3 = (I/cr)dx, b' = (b n)/cr, a' = (a fJ,)/cr, we obtain ./ P[a<X <b} =
= == /
I

fb'

e 2 X dx -- -= /

_i

fa'

dx.

V27T Jo

\/27T 70

Ibe function (2.4-12) s 5<:'metimes called the error function [erf(x)] although other definitions of erf(x) exist Tae erf(a;) is tabulated in Table 2.4-1 and is plotted in Figure 2.4-2.

Figure 2.4-2

erf(a;) versus x.

'For example, a widely used definition of erf(a;) is erf(o;) = (2/v^r) f^ e * dt. If we cali this erf(x) = i and we cali erf(a;) of Equation 2.4-12 erfi(a;), then erfi(a;) = ^er2(x/\/2).

70

Chapter 2

Random Variables

Table 2.4-1

Selected Vales of erf(o;)

erf(x)
X

1
erf(z)

/ exp(--<: Jo ^
X

)*
erf(x) 0.47981 0.48213 0.48421 0.48609 0.48777 0.48927 0.49060 0.49179 0.49285 0.49378 0.49460 0.49533 0.49596 0.49652 0.49701 0.49743 0.49780 0.49812 0.49840 0.49864 0.49884 0.49902 0.49917 0.49930 0.49941 0.49951 0.49958 0.49965 0.49971 0.49976 0490 .98 0.49983 0.49986 0.49988 0.49990 0.49992 0.49993 0.49994 0.49995 0.49996

0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 1.05 1.10 1.15 1.20 1.25 1.30 1.35 1.40 1.45 1.50 1.55 1.60 1.65 1.70 1.75 1.80 1.85 1.90 1.95 2.00

0.01994 0.03983 0.05962 0.07926 0.09871 0.11791 0.13683 0.15542 0.17364 0.19146 0.20884 0.22575 0.24215 0.25803 0.27337 0.28814 0.30233 0.31594 0.32894 0.34134 0.35314 0.36433 0.37492 0.38492 0.39434 0.40319 0.41149 0.41924 0.42646 0.43319 0.43942 0.44519 0.45052 0.45543 0.45993 0.46406 0.46783 0.47127 0.47440 0.47724

2.05 2.10 2.15 2.20 2.25 2.30 2.35 2.40 2.45 2.50 2.55 2.60 2.65 2.70 2.75 2.80 2.85 2.90 2.95 3.00 3.05 3.10 3.15 3.20 3.25 3.30 3.35 3.40 3.45 3.50 3.55 3.60 3.65 3.70 3.75 3.80 3.85 3.90 3.95 4.00

5ec. 2.4. PROBABILITY DENSITY FUNCTION (pdf)

71

Henee if X: N(fj,, cr2), then P[a<X<b}=er (2.4-13)

Example 2.4-3 _ Suppose we choose a resistor with resistance R from a batch of resistors with parameters (j. = 1000 ohms with cr = 200 ohms. What is the probability that R will have a valu between 900 and 1100 ohms? Solution Assuming that R: 7V[1000, (200)2] we compute from Equation 2.4-13 P[900 < R < 1100] = erf(O.S) - erf(-O.S). But erf( x) = erf(x) (deduced from Equation 2.4-12). Henee P[900 < R < 1100] = 0.38. Using Figure 2.4-3 as an aid in our reasoning, we readily deduce the following for X:N(0, 1). Assume x > O, then P[X < x] = \ + erf(x) P[X > -x] = i + erf(x) P[X > x} = \ - erf(x) P[-x < X < x] = 2 erf(x) P[\X\ > x] = 1 - 2 erf(rr). (2.4-14a) (2.4-14b) (2.4-14c) (2.4-14d) (2.4-14e)

Note that since erf(x) = erf(x), the first three formulas remain valid for x < O also. Four Other Common Density Functions 1. Rayleigh (a > 0):

u(x).

(2.4-15)

The function u(x) is the unit step, that is, u(x) = 1, x > O, u(x) = 0. x < 0. Thus fx (x) = O for x < 0. Examples of where the Rayleigh pdf shows up are in rocket-landing errors, random fluctuations in the envelope of certain waveforms, and radial distribution of misses around the bull's-eye at a rifle range. 2. Exponential (p, > 0): fx(x) = -e-x^u(x). (2.4-16)

The exponential law occurs, for example, in waiting-time problems, lifetime of machinery, and in describing the intensity variations of incoherent light.

72

Chapter 2

Random Variables

(a)

-x

(b)

(c)

(d)

(e)
(c)

Figure 2.4-3 The reas of the shaded regin under curves are (a) P[X < x ] ; (b) P(X > -x); P(X > x); (d) P(-x <X<x); and (e) P(\X\ > x).

3. Uniform (b > a):

fx(x) =

a < x <b
otherwise. (2.4-17)

The uniform pdf is used in communication theory, in queueing models, and in situatioiis where we have no a priori knowledge favoring the distribution of outcomes except for the

Sec. 2.4. PROBABILITY DENSITY FUNCTION (pdf)

73

Exponential , Rayleigh 0.606

Uniform

b- a

o-

Figure 2.4-4 The rayleigh, exponential, and uniform pdf's.

end points; that is, we don't know when a business cali will come but it must come, say, between 9 A.M. and 5 P.M. The three pdf's are shown in Figure 2.4-4. 4. Another pdf that has assumed importance since the introduction of computer analy;is of speech and images is the Laplacian. The pdf is defined by
f x ( x ) = -e

o 0.

(2.4-18)

The Laplacian is widely used to model speech sources and image gray levis. The Laplacian r.v. in these cases is called the adjacent-sample difference and is the difference in signal level from a sample point and its neighbor. Since the level of the sample point and its neighbor ore often the same, the Laplacian peaks at zero. The Laplacian pdf is sometime written as

(2.4-19)

diere a is the standard deviation of the Laplacian r.v. X. Precisely what this means will :-e explained in Chapter 4. The Laplacian pdf is shown in Figure 2.4-5.

Figure 2.4-5 The Laplacian pdf used n computer analysis of speech and mages.

74
The chi-square density for n = 2, 4,10
T

Chapter 2

Random Variables

10

15 20 25 30 35 Argument valu

40

45

50

Figure 2.4-6 The Chi-square probability density function for n = 2 (solid), n = 4 (dashed), and n = 10 (stars). Note that for larger vales of n, the shape approaches that of a Normal pdf with a positive mean-parameter p,. More Advanced Density Functions

5. Chi-square (n an integer) f x ( x ) = Kxx(^-1e-%u(x), (2.4-20)

where the normalizing constant Kx is computed as Kx 2 n/2r( n / 2 ) and F(-) is the Gamma function discussed in Appendix B. The Chi-square pdf is shown in Figure 2.4-6. 6. Gamma: (b > O, O 0) (2.4-21) where J7 = cb/T(b). 1. Student-: (n an integer)
x'2\ n
-(

(2.4-22)

where

The Chi-square and Student- densities are widely used in statistics.^ We shall encounter these densities later in the book. The Gamma density is mother to other densities. For
tThe Student- distribution is so named because its discoverer W. S. Gossett (1876-1937) publishcd his papers under the ame "Student." Gossett. E. S. Pearson, R. A. Fisher, and J. Neyman are regarded as the founders of modern statistics.

ec. 2.5. CONTINUOUS, DI5CRETE, AND MIXED RANDOM VARIABLES _

75

ampie with 6 = 1 , there results the exponential density; and with b = n/2 and c = 1/2, ifcere results the Chi-square density. There are other pdf 's of importance in engineering and science, and we shall encounter ajine of them as we continu our study of probability. They all, however, share the properties

fx(x)>0

(2.4-23) (2.4-24)

fx(x)dx = l.

When FX(X) is not continuous, strictly speaking. its finite derivative does not exist and, ierefore. the pdf doesn't exist. The question of what probability function is useful in ir cribing X depends on the classification of X. We consider this next.

2,5 CONTINUOUS, DISCRETE, AND MIXED RANDOM VARIABLES

If FX(X) is continuous for every x and its derivative exists everywhere except at a countable set of points, then we say that X is a continuous random variable (r.v.). At points x where FY(X) exists, the pdf is fx(%) = F'x(x). At points where FX(X) is continuous, but F'x(x) -, discontinuous, we can assign any positive number to fx(x)\ fx(%) will then be defined for every x, and we are free to use the following important formulas: Fx(x) = \
J OO

/*()<*

(2-5-1) (2.5-2)

/X2

:X<xi} = I fx()d., Ai

and
P[5] = /
^:

/x(0^>

(2-5-3)

here, in Equation 2.5-3, B e J?, that is, B is an event. Equation 2.5-3 follows from the fact that for a continuous random variable, events can be written as a unin of disjoint intervals in R. Thus, for example, let B = {: e U" =1 /,/i/j = <t> for i ^ j}. where 7 = (a,6]. Then clearly,

P(B] = '
t/dj

=
J(:(B

fx($dS.

(2.5-4)

A discrete random variable has a staircase type of distribution function (Figure 2.5-1).

76

Chapter 2

Random Variables

Figure 2.5-1

The probability distribution function for a discrete random variable.

A probability measure for discrete r.v. is the probability mass function^ (PMF). The probability mass function Px(x) of a (discrete) random variable X is defined as

Thus PX(X) = O everywhere where Fx(x) is continuous and has finite vales only where there is a discontinuity, that is, jump, in the PDF. If we denote P[X < x] by FX(X~), then at the jumps x, i = 1 , 2 , . . . , the finite vales of PX(X) can be computed from The probability mass function is used when there are at most a countable set of outcomes of the random experiment. Indeed PX(X) lends itself to the following frequency interpretation: Perform an experiment n times and let nt be the number of tries that x appears as an outcome. Then, for n large,
(2.5-6) n Because the PMF is so closely related to the frequency notion of probability, it is sometimes called the frequency function. Since for a discrete r.v. FX(X) is not continuous fx(x), strictly speaking, does not exist. Nevertheless, with the introduction of Dirac delta functions, ^ we shall be able to assign pdf's to discrete r.v.'s as well. The PDF for a discrete r.v. is given by

PX(X).
all

(2.5-7)

and, more generally, for any event B when X is discrete:

[B] = yi
all

(2.5-8)

tLike mass, probability is nonnegative and conserved. Henee the term "mass" in probability mass function. iAlso called impulses or impulse functions. Named after the English physicist Paul A. M. Dirac (1902 1984). Delta functions are discussed in Section B.2 of Appendix B.

2.5.

CONTINUOUS, DISCRETE, AND MIXED RANDOM VARIABLES

77

Lomples of Probability Mass Functions 1. Bernoulli (p > 0,q > 0,p + q = 1):

Px(x) = 0,

as ^0,1.

(2,5-9)

The Bernoulli law applies in those situations where the outcome is one of two possible states. that is. whether a particular bit in a digital sequence is "one" or "zero." A r.v. that has "_- Z-:roulli PMF is said to be a Bernoulli r.v. The Bernoulli PMF can be conveniently written as PX(X) = p1~x(l p)x for x = O, 1 and zero elsewhere. 2. Binomial (n = 1, 2 , . . . ; O < p < 1):
= 0,1,2,..., n

otherwise.

(2.5-10)

The binomial law applies in games of chance, military defense strategies, failure analysis. and many other situations. A binomial r.v. has a PMF as in Equation 2.5-10. The Bernoulli law is often called a point binomial. 3. Poisson (a > 0): Px(k)=e~a
= 0,

fe

= 0,1,2,... otherwise. (2.5-11)

The Poisson law is widely used in every branch of science and engineering (see Section 1.10). A r.v. whose PMF is given by Equation 2.5-11 is said to be a Poisson r.v. Sometimes a r.v. is neither purely discrete or purely continuous. We cali such a r.v. a mixed r.v. The PDF of a mixed r.v. is shown in Figure 2.5-2. Thus Fx(x) is discontinuous but not a staircase type-function. The distinction between continuous and discrete r.v.'s is somewhat artificial. Continuous and discrete r.v.'s are often regarded as different objects even though the only real difference

Figure 2.5-2

The PDF of a mixed r.v.

78

Chapter 2

Random Variables

between them is that for the former the PDF is continuous while for the latter it is not. By introducing delta functions we can, to a large extent, treat them in the same fashion and compute probabilities for both continuous and discrete r.v.'s by using pdf's. Returning now to Equation 2.5-7, which can be written as
00

Fx(x) =

= 00

PX(Xi)u(x

(2.5-12a)

and using the results from the section on delta functions in Appendix B enables us to write for a discrete r.v. dF (x) (2.5-12b) fx(x] = x ' >

dx

t= 00

where we recall that PX(X) = FX(X) FX(X ) and the unit step assures that the summation is over all i such that x < x.
Example 2.5-1 _

Let X be a discrete r.v. with distribution function as shown in Figure 2.5-3(a). The pdf of X is fx(x) = - = o.2J(x) + 0.66(x - 1) + 0.26(x - 3) dx and is shown in Figure 2.5-3(b). To compute probabilities from the pdf for a discrete r.v. great care must be used in choosing the interval of integration. Thus Fx(x)= i* /
J 00

which includes the delta function at x if there is one there.

1.0

0.8 0.6
0.4
-

()

1
(a)

YV)

-2(x)
()

I
I 1
(b)

0.2<S(x-3)
2 \ 3

Figure 2.5-3

(a) PDF of a discrete r.v. X; (b) pdf of X using delta functions.

.2J5.

CONTINUOUS, DISCRETE, AND MIXED RANDOM VARIABLES

79

.-ilarly P[XI < X < x%] involves the interval

i
id includes the impulse at x- (if there is one there) but excludes what happens at x\. On e ::Jier hand P[x\ < X < #2] involves the interval

X,

X2

_ : -r.rrefore
P(XI <X <x2) =

Applied to the foregoing example, these formulas give PpT < 1.5] = FX(1.5) = 0.
P[l< X < 3] = 0.2
P[l < X < 3] = 0.6
Example 2.5-2

T^e pdf associated with the Poisson law with parameter a

fc=o

Example 2.5-3 pdf associated with the binomial law b(k; n,

Example 2.5-4 _ The pdf of a mixed r.v. is shown in Figure 2.5-4. (1) What is the constant K? (2) Compute P'X < 5], P[5<X< 10]. (3) Draw the distribution function. Solution (1) Since
-oo

obtain 10AT + 0.25 + 0.25 = 1 => A" = 0.05.

80
fxM

Chapter 2

Random Variables

0.25S(x-5)

0.258U-10)

10

1.00 0.75 0.50 0.25

O
Figure 2.5-4

10

(a) pdf of a mixed r.v. for Example 2.5-4; (b) computed PDF.

(2) Since P'_X < o = P[X < 5] + P[X - 5]. the impulse at x = 5 must be included. Henee
,5+

P[X < 5] = / [0.05 + 0.255( Jo /o = 0.5. To compute P(5 < X < 10). we leave out the impulse at x = 10 but include the impulse at x = 5. Thus P[5 < A" < 10] = / h15= 0.5. [0.05 + 0.25( -

2.6 CONDITIONAL AND JOINT DISTRIBUTIONS AND DENSITIES

Consider the event C consisting of all outcomes ( e fi such that X(Q < x and C B C f2 where B is another event. The event C is then the set intersection of the two events

2.6.

CONDITIONAL AND JOINT DISTRIBUTIONS AND DENSITIES

81

ST(C) < x} and {OC B}. We define the conditional distribution function of X given event B as (26-1) Ikre P\X < x,B] is the probability of the joint event {X < x} O B and P[B] ^ 0. If - = x. the event {X < 00} is the certain event 7 and since D B = B, Fx(oo\B) = 1. Hnilarly. if x = oo, {X < 00} = (f> and since f D 4> = 4>i fx(~oo\B) 0. Continuing in tr-s fashion, it is not difficult to show that Fx(x\B) has all the properties of an ordinary L- ; ~:::uTon, that is, x\ < x2 > Fy(a;i|B) < Fx(x2\B). 7:-r example, consider the event {X < X2, B} and write (assuming 2 > ^i)

{X < a;2, B} = {X < Xl,B} \J {Xl < X < x2, B}.

-_:- ":ie two events on the right are disjoint, their probabilities add and we obtain P[X <x2,B}= P[X <xl,B} + P[Xl <X <x2, B] P[X < x2\B]P[B] = P[X < Xl\B]P[B] + P[XI < X < x2\B]P[B]. Tru5 when P[B] ^ O, we obtain after rearranging terms

P[XI <X < x2\B] = P[X < x2\B] - P[X < Xl\B]
= Fx(x2\B) - Fx(Xl\B). (2.6-2)

GriLerally the event B will be expressed on the probability space (R,J?,Px) rather than '- riginal space (f,,^"P). The conditional pdf is simply
fx(xB)

***&>.

(2.6-3)

are some examples. Example 2.6-1 3 = {X < 10}. We wish to compute Fx(x\B). __

(i) For x > 10, the event {X < 10} is a subset of the event (X < i}. Henee P[X < 10, X < x} = P[X < 10] and use of Equation 2.6-1 gives

ii) For x < 10, the event {X < x} is a subset of the event {X < 10}. Henee P[X < 10, X < x] = P[X < x] and

~ppr<ior
The result is shown in Figure 2.6-1. We leave as an exercise to the reader to compute Fx(x\B) when B = {b < X < a}.

82

Chapter 2

Random Variables

1.0

10

Figure 2.6-1

Conditional and unconditional PDF of X.

Example 2.6-2 Let X be a Poisson r.v. with parameter a. We wish to compute the conditional PMF and pdf of X given B = {X(is) even} = {X = 0.2.4....}. First observe that P[X even] is given by
Ia fc

while for X odd. we have

Ar=1.3.-..

From these relations. ire obtain

A:!

t=o
= -

2a

and

E c 2

' :n

Henee P[X = 0 , 2 , . . . ] = |(1 - e" *). Ung the definition of a conditional PMF. -w obtJ Px(k\X iseven) = P[X = k,X is even] P[X even]

If fc is even, then {X = fc} is a subset of {X even}. l k is odd. {X = fc} n {X even} = o. Henee

'
O,

fc odd.