Вы находитесь на странице: 1из 42

,

-
( )

2003

2003
V
-


1
-

2003

004.032.26 (06)
32.8185
82
2003. V - 2003:
. 1. .: , 2003. 188 .

, -
, 2931
2003 V 2003.
, ,
- .


. . ,

ISBN 5726204719

c -

( ), 2003


. . .
149
. . . . . . . . . . 150
. . . . . . . . . . . . 152
. . . . . . . . . . 152
153
. . . . . . . . . . . . . 157
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
. . . . . . . . . . . . . . . . . . . . . . . . . . . 163
. . . 166
. . . . . . . . . . . . . . . . . 167
. . . . . . . 168
. 169
. . . . . . . . . . . . . . . . . . . . . . . . 172
. 173
. . . . . . . . . . . . . . . . 175
. . . . . . . . . . . 177
. . . . . . . . . . . . . . . 180
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
. . . . . . . . . . . . . 180
. . . . 181
. . . . . . . . . . . . . . . . 181
. . . . . . . . . . . . . . . . . . . . . . 181
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

004.032.26 (06)

. .
- , . ;
, . ,
E-mail: alife@narod.ru


-
.
,
.
. .

S. A. TEREKHOFF
Snezhinsk Institute of Physics and Technology (SFTI), Snezhinsk;
NeurOK LLC, Moscow,
E-mail: alife@narod.ru
BAYESIAN NETWORKS PRIMER
Abstract
Bayesian networks represent probabilistic graph models of casual relations between variables in statistical information modelling. Bayesian networks consolidate empirical frequencies of events, subjective beliefs and theoretical representations of mathematical probabilities. It is the important practical advantage
which distinguishes Bayesian networks from other techniques of information
modelling. This survey lecture presents a brief introduction to calculation of
probabilities and statistical inference methods in Bayesian networks.

004.032.26 (06)

149

ISBN 5726204719



. . :
, [26],
.
, ( )
:
( ,
. .);
( , , ).

[5].
:
, ,
.
10 .
, ,
( ).
10 , !
, .
10 .

10 . ,
,
. . ,
, ,
150

004.032.26 (06)

. .
.
, .
,
.
,
, .
, .
, .
, - .
,

.
.
, ,
, .
, , .
. . .

, .
.

004.032.26 (06)

151

ISBN 5726204719



. , , , ,
, .
:
;
, ;

.

.

60- () [6], ,
.
. ,
, .
:
;

;
.
80- 1
[21] .
1 -,
20- . (Wright S. Correlation and
causation // Journal of Agricultural Research, 20, 557, 1921).

152

004.032.26 (06)

. .
. , ,
. . ,
?

,
. ,
, , .
[15].
, . 2 : , . , , , , .
, .
,
, .
,
.
,
. , . ,
, . , . ,
, .

, ,
: ,
,
. , , , 2


, , , ,
.

004.032.26 (06)

153

ISBN 5726204719

() ,
. , (explaining
away).
. .
. ,
, .
, ( ), (A, B) , A
B. A B, , ,
- A.
. 1.

Rain?

Sprinkler?

Watson?

Cholmes?

. 1.

. 1 .
1 0 (/). 3 . . 1
.
3
(, ).
.

154

004.032.26 (06)

. .
, , ,

.
80- , [21]. , ,
. , .
:
(. 2a), (. 2b),
(. 2c).
. 2c , -, ,
4 . , A B C.
, A ,
, (, ). ,
, B C
.
() d- (d-separation).
(d-). A B
d-, ,
, V , :
V , V
,
, V ,
.
, (. 1) ?
? d-.
?, ( ).
4 ,

, .

004.032.26 (06)

155

ISBN 5726204719

(a)

...

...

(b)

A
(c)

. 2.
(a) . A C , B . (b) .
A, . (c) .
A , ,
B, C, . . . , E, .
A .

156

004.032.26 (06)

. .
d- -, ,
d- .
, A C, B,
P (A | B) = P (A | B, C).
, , , ,

, ,
.
, : , .
!
,
. ,
( , !). ,

.
, ,
,
- . ,

.




,
()
.
.
004.032.26 (06)

157

ISBN 5726204719

. P (A | B) = x .
, B ( ,
A), A x. A B
:
P (A, B) = P (A | B) P (B) .
(), (),
.
P (A | B)
A , B.
B, A .
, A1 , . . . , An ( ) ( ). P (Aj | B) Aj ,
B, P (A j ):
P (Aj | B) =

P (Aj ) P (B | Aj )
Pn
.
P (B) = j=1 P (Aj ) P (B | Aj )

P (B | Aj ) (likelihood), P(B) (evidence) 5 .


.
.
5 evidence , . evidence
, . ,
evidence. , evidence
, . evidence -.

158

004.032.26 (06)

. .
, .
, ,
,
:
Y 

P (A1 , . . . , An ) =
P Aj | pa(Aj ) .
j

pa(Aj ) - Aj .
.
, - Aj .
, :

;

;

;
- A - B1 , . . . , Bn
P (A | B1 , . . . , Bn ).
A , () P (A).
()


.
, .

, .
, . (. 1): R , S
004.032.26 (06)

159

ISBN 5726204719

, C , W
.
0 , (f )
1 (t). P (R, S, C, W ), , 16 . ,

X
P (R, S, C, W ) = 1 .
R={t,f },...,W ={t,f }

,
. , ,
, ,
:
P (R = f | W = t) =
P

=P

S={t,f },C={t,f }

P (R = f, W = t)
=
P (W = t)

P (R = f, S, C, W = t)

R={t,f },S={t,f },C={t,f }

P (R, S, C, W = t)

[7, . 45] :
P (R, S, C, W ) = P (R) P (S | R) P (C | R, S) P (W | R, S, C) .
,
. :
P (R, S, C, W ) = P (R) P (S) P (C | R, S) P (W | R) .
,
, .
,
. , -,
.
, 6 .
6 , .

160

004.032.26 (06)

. .
1.

P (R = t)
0.7
R
t
f
R
t
t
f
f

P (R = f )
0.3

P (S = t)
0.2

P (W = t | R)
0.8
0.1

P (W = f | R)
0.2
0.9

S
t
f
t
f

P (R = t)
0.9
0.8
0.7
0.1

P (S = f )
0.8

P (R = f )
0.1
0.2
0.3
0.9

, ,

, 7 .


.
( !) 16 ,
, .
.
: , .
X
P (C = t) =
P (R) P (S) P (C = t | R, S) =
R={t,f }
S={t,f }

= 0.3 0.2 0.9 + 0.3 0.8 0.8 + 0.7 0.2 0.7 + 0.7 0.8 0.1 = 0.4 .
7
.
.
, , ,
,
.

004.032.26 (06)

161

ISBN 5726204719

,
P (W = t) =

P (R) P (W = t | R) = 0.31 .

R={t,f }

, , , () :
X
P (C = t | R = t) =
P (S) P (C = t | R = t, S) = 0.82 ,
S={t,f }

P (W = t | R = t) = 0.8 .

, , . , ,
? :
P (R = t | C = t) =

P (R = t, C = t)
0.054 + 0.192
=
= 0.615 .
P = (C = t)
0.4

P (S = t | C = t) =

0.054 + 0.098
P (S = t, C = t)
=
= 0.38 .
P = (C = t)
0.4

, ,
S W ,
- P (W |
R).
,
0.3 0.2 , . ,
, :
P (R = t, C = t, W = t)
=
P (C = t, W = t)
0.8 0.3 (0.18 + 0.64)
0.1968
=
=
= 0.9274 ,
0.8 (0.054 + 0.192) + 0.1 (0.098 + 0.056)
0.2122
P (S = t | C = t, W = t) = 0.2498 .
P (R = t | C = t, W = t) =

, .
162

004.032.26 (06)

. .
, (. . )

0.2. ,
.
, 8 .

,
,
. ,
-. 9 .


,
[19].
, 8 64 ,
. 8 , , ,
, .
8! . ,
,
, .
, , ,
.
, . , 8 NP-. (Cooper G. F. The computational complexity
of probabilistic inference using Bayesian belief networks // Artificial Intelligence. 1990.
42, (23). pp. 393405).
9 LHS Latin Hypercube Sampling.

004.032.26 (06)

163

ISBN 5726204719

, ,
, 10
.
1, 2, . . . , 8, . .

N D . (N D),
1, 2, . . . , N .
, N
, , , ,
.


- .
[4].
.

C
. 3.

,
.
(100 3), 100
. , , :
10

164

N/2 . . . + N/2

004.032.26 (06)

. .
2.

A P (A)
B P (B)
a1 0.20
a1
0.4
a2 0.35
a2
0.6
a3 0.45
a1
P (C | A, B)
c1
c2
c3

b1
0.01
0.68
0.31

a2
b2
0.04
0.93
0.03

b1
0.28
0.52
0.20

a3
b2
0.06
0.12
0.82

b1
0.18
0.50
0.32

b2
0.82
0.10
0.08

3. (100 3)
N

25

13

74

14

91

...

...

...

...

39

47

32

56

...

...

...

...

100

69

84

1 100 [0 . . . 1] 100 . i-
A LHSi A. :
( a , LHS A 6 P (a ) 100)
1
i

 1


a2 , P (a1 ) 100 6 LHSi A 6 (P (a1 ) + P (a2 )) 100
a1 ,
, LHSi B
B. 39- . 3.,
{47, 32, 56}. A a 2 ,
B, , b1 .
004.032.26 (06)

165

ISBN 5726204719

C P (C |
A, B). (a2 , b1 ) 3- ,
LHSi C = 56 C c2 .
100 .
C
.
,
, .
,
11 .
, . . ,
.
, . [24], ,
.

,
.
, ,
12 .
( )
11

.
12 .
(, 1999)
. . 1951 : , ,
, - (, . 97).

166

004.032.26 (06)

. .
, . ,
.
. , ,
,
(, ).
. ( ) ,
.

,
(, )
, , .

, , [7],
[14].



,
. ,
.
() ,
.

: 004.032.26 (06)

167

ISBN 5726204719

.
, ,
. , .

. , .
, , 13 .

,
( ), (, ).

( ).
:
;
, ,
,
;
;
- ( , ) , ;
,
- .
13 , , , .
.

168

004.032.26 (06)

. .
.
,
.
.
.

. , , ,
, ,
.

, P [x j |
pa(xj )]. xj , ,
(., , . 1).


. , , ,
. , ,
.
.
14
.
,
. N
14 , ,
. ,

.

004.032.26 (06)

169

ISBN 5726204719

h, , , t = N h. N
D. , N + 1 .

, () . , ,
N .

, . (
) p( | ).
, :
p( | D, ) =

p(D | , ) p( | )
.
R
p(D | ) , p(D | , ) p( | )d

, , ,
, :
p(D | , ) h (1 )t
:
Z
p(xN =1 = H | D, ) = p(xN +1 = H | , ) p( | D, )d =
=

p( | d, )d = hip(|D,) .


- [14]:
p( | ) = ( | h , t ) ,

170

(h + t ) h 1

(1 )t 1 .
(h ) (t )

004.032.26 (06)

. .
, -
15 .
p(xN +1 = H | D, ) =

h + h
.
h + + t + h + t


( ) h t .
, N + 1
D.
.
.
, ,
( ).

.
D = {D1 , . . . , Dk , . . . , DS }
( Dk k- ), , ,
:
L=

S
N

1 XX 
log P (xj | pa(xj ), Dk ) .
N S j=1
k=1

,
. (
) , .
15 , , (conjugate priors).

004.032.26 (06)

171

ISBN 5726204719

,

, . , -
,
- .
, ,
.
, ,

. ,
, -, 16 .

, .
,
(evidence) .

.
,
.
.


( ) ,
[10, 12].
16 , , , . [1],
(www.auai.org).

172

004.032.26 (06)

. .
D, ~ Y }. {X;
p (y | ~x).
.
17 .
, () .
, ( ).

(reasoning). ,
() .
: ,
18 .
()
.


. ymin < y1 < . . . < yk < . . . ymax
,
nk = pk N0 D. 19
, :
X
pk log pk .
S0 = N0
k

17


, .
18 y
D.
(. . prior).
19 .

004.032.26 (06)

173

ISBN 5726204719

()
. () .
,
() .
,
.
.
0
00
, S1 = S1 +S1
. , , , , ,
20 .
, .

[12].
( ) M (M
X). , ,
. , .
.
( ) .

( ).
, :
20 , , , ,
- .

174

004.032.26 (06)

. .
1. (
).
2. .
3. , .



.
. , , ,
, . , (,

, ).
- - [25]. [2].
. , N , (
) . , 0, (N 1)/N .
, , (N 1) .

. 4.

.

004.032.26 (06)

175

ISBN 5726204719

1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

10

12

14

. 4. ( )

().
-. ( )
, .
.
. 5.
,
, .
. .

.
176

004.032.26 (06)

. .

, [8].

, , ,
EM-, [2].
,
.
. (ounter-propagation, R. Hecht-Nielsen, [13]).
[16] , , , , -. , ,
. ,
, ,
. , -
.

. , [11,23],
.
[11] 506 13 ( ).
, (
)
. 439
( 200).
. 5.
004.032.26 (06)

177

ISBN 5726204719

CRIM
LSTAT

ZN

INDUS

CHAS
NOX

PTRATIO
TAX
RAD

RM
DIS
AGE

. 5. Boston Housing

3/4
: ZN , ( 25000
. ), CRM , RM .
. 6.
.
,
.
, ,
,
.
() .
178

004.032.26 (06)

. .
0.2
0.1
0
5
0.2

10

15

20

25

30

35

40

45

50

10

15

20

25

30

35

40

45

50

10

15

20

25

30

35

40

45

50

10

15

20

25

30

35

40

45

50

10

15

20

25

30

35

40

45

50

10

15

20

25

30

35

40

45

50

0.1
0
5
0.4
0.2
0
5
0.4
0.2
0
5
0.4
0.2
0
5
0.5

. 6.


.
.
.

.
, .

004.032.26 (06)

179

ISBN 5726204719

. , .


, . .
.

PathFinder (Heckerman, 1990) . PathFinder 60 130 ,


.
, PathFinder-4 .
(Child, MUNIN, Painulim, SWAN .)
[15].

Vista (Eric Horvitz )
NASA (NASA Mission Control Center) .
,
.
,
, ,
. [9]
Netica (. ) Operation Dardanelles. ,
, .
180

004.032.26 (06)

. .

.

Microsoft
- Office ( ),
wizard-.


,
(Jordan, 2002).

[22]
.
,
, , .
, . .


, .
,
004.032.26 (06)

181

ISBN 5726204719

, .
, .
,
, .

, , .
, , , .
( ), ( ),
, ,
.

. . , . . ,
. . .
. . , ,
. Alma Mater
.

1. DAgostini G. Bayesian reasoning in high energy physics principles and


applications. CERN Yellow Report 99-03, July 1999.
2. Bishop C.M. Neural networks for pattern recognition. Oxford University Press,
1995.
3. . . . . , 2001.
4. Cheng J., Druzdel M. J. Latin hypercube sampling in Bayesian networks // In: Proc.
of the Thirteenth International Florida Artificial Intelligence Research Symposium

182

004.032.26 (06)

. .
(FLAIRS-2000). Orlando, Florida, AAAI Publishers, 2000. pp. 287292.
URL: http://www2.sis.pitt.edu/jcheng/Latin.zip
5. Coles S. Bayesian inference. Lecture notes. Department of Mathematics, University
of Bristol, June 10, 1999.
URL: http://www.stats.bris.ac.uk/masgc/teaching/bayes.ps
6. Giarratano J., Riley G. Expert systems: Principles and programming. PWS
Publishing, 1998.
7. . . . 7- . .: ,
2001.
8. ., . . .: , 1988.
9. Das B. Representing uncertainties using Bayesian networks. DSTO Electronics and
Surveillance Research Laboratory, Department of Defence. Tech. Report DSTOTR-0918. Salisbury, Australia, 1999.
URL: http://dsto.defence.gov.au/corporate/reports/DSTO-TR-0918.pdf
10. Fukuda T., Morimoto Y., Morishita S., Tokuyama T. Constructing efficient decision
trees by using optimized numeric association rules // The VLDB Journal, 1996.
URL: http://citeseer.nj.nec.com/fukuda96constructing.html
11. Harrison D., Rubinfeld D. L. Hedonic prices and the demand for clean air // J.
Environ. Economics & Management. 1978. vol. 5. pp. 81102.
12. Hastie T., Tibshirani R., Friedman J. The Elements of statistical learning. Data
Mining, Inference, and Prediction. Springer, 2001.
13. Hecht-Nielsen R. Neurocomputing. Addison-Wesley, 1990
14. Heckerman D.. A tutorial on learning with Bayesian Networks. Microsoft Tech.
Rep. MSRTR956, 1995.
15. Jensen F. V. Bayesian networks basics. Tech. Rep. Aalborg University, Denmark,
1996.
URL: http://www.cs.auc.dk/research/DSS/papers/jensen:96b.ps.gz
16. Kohonen T. Self-organizing Maps. Springer, 1995.
17. Minka T. Independence diagrams. Tech. Rep. MIT, 1998.
URL: http://www-white.media.mit.edu/tpminka/papers/diagrams.html
18. Blake C. L., Merz C. J. UCI Repository of Machine Learning Databases, 1998.
URL: http://www.ics.uci.edu/mlearn/MLRepository.html
19. Montgomery D. C. Design and analysis of experiments. 5th. Ed. Wiley, 2001.
20. Neal R. M. Probabilistic inference using Markov chain Monte Carlo methods.
Technical Report CRG-TR-93-1, 25 Sep 1993, Dept. of Computer Science,
University of Toronto.
004.032.26 (06)

183

ISBN 5726204719

21. Pearl J. Probabilistic reasoning in intelligent systems: Networks of plausible


inference. Morgan Kaufmann, 1988.
22. Shenoy C., Shenoy P. Bayesian network models of portfolio risk and return /
Y. S. Abu-Mostafa, B. LeBaron, A. W. Lo (Eds.) Computational Finance, 85-104,
MIT Press, 1999.
23. StatLib datasets archive. CMU.
URL: http://lib.stat.cmu.edu/datasets/
24. .. -. .: , 1973.
25. . . . -
, , , 2527 2002 .
26. . .
. IV .: . ., . .-, . . ,
. . , . . , . . , . . , . . ,
. . . . , , 1998 . 101
136.
27. . .
. , 19941998.
URL: http://alife.narod.ru/lectures/neural/Neu_index.htm

.

,
. , , ,
, 1015 .

AUAI

URL: http://www.auai.org/
(Association for
Uncertainty in Artificial Intelligence AUAI) ,

(UAI).

184

004.032.26 (06)

. .
UAI2002 2002
(, ). UAI ,
1985 , .
, .

NETICA
URL: http://www.norsys.com/index.html)

Norsys Software Corp. , ().


Norsys
. Netica , 1992
1995 . Netica

.

Norsys.

. 7. () Netica
Netica , .
. ,

004.032.26 (06)

185

ISBN 5726204719

, ,
( ).
,
, .
.
, Netica ,
, . Netica
.

Knowledge Industries
URL: http://www.kic.com/

Knowledge Industries
.
.

Data Digest Corporation


URL: http://www.data-digest.com/home.html

Data Digest Corporation .

HUGIN Expert
URL: http://www.hugin.com/
Hugin Expert 1989 , (). Hugin ESPRIT,
, , - . Hugin.
Hugin 25 , ,
, , , , .

186

004.032.26 (06)

. .
BayesWare, Ltd
URL: http://www.bayesware.com/corporate/profile.html
BayesWare 1999 . , ,
, . ,
Bayesware Discoverer, .


URL: http://stat.rutgers.edu/madigan/bayes_people.html

.

C , - , - (),
.
, ,
, , ,
. 1 50 .