(A) Plot The Error E As A Function of The Number of Training Iterations

Assignment 4 Assigned 2/3/2012 Due 2/10/2012 (40 points) Problem 1.
Implement the delta training rule for a two-input linear unit. Train it to fit the target concept . (a) Plot the error E as a function of the number of training iterations.
Delta Rule Training 2.5
1.5
Error
0.5
100
200
300 Iteration
400
500
600
(b) Plot the decision surface after 5, 10, 50, 100, 150, , 500 iterations.
Decision Surface of 5th Iteration 0.5
-0.5
-1
-1.5
-2 -1
-0.5
0.5
1.5
2.5
Decision Surface of 10th Iteration 1
0.5
-0.5
-1
-1.5
-2 -1
-0.5
0.5
1.5
2.5
1.5
0.5
-0.5
-1 -1
-0.5
0.5
1.5
2.5
1.5
0.5
-0.5
-1 -1
-0.5
0.5
1.5
2.5
1.5
0.5
-0.5
-1 -1
-0.5
0.5
1.5
2.5
1.5
0.5
-0.5
-1 -1
-0.5
0.5
1.5
2.5
(c) Use different learning rates and also variable rates (decaying). Analyze which works better and explain why. If I use variable rates, it shown that it did not converge,
Delta Rule Training 3.5
2.5
Error
1.5
100
200
300 Iteration
400
500
600
If I use different etas, the larger the eat is, the faster it converge, so using different learning rates works better.
Delta Rule Training,eta=0.0001 2 1.8 1.6 1.4 1.2
Error
1 0.8 0.6 0.4 0.2 0
100
200
300 Iteration
400
500
600
Delta Rule Training,eta=0.0005 2.5
1.5
Error
0.5
100
200
300 Iteration
400
500
600
Delta Rule Training,eta=0.0008 2 1.8 1.6 1.4 1.2

Error
1 0.8 0.6 0.4 0.2 0
100
200
300 Iteration
400
500
600
(d) Now implement delta rule in an incremental fashion (as opposed to batch fashion when all the data are presented for training, the incremental approach updates the network after each example). For the same choice of other parameters (learning rate, etc.). Compare the two approaches in terms of total execution time and number of weight updates.
Batch fashion: when eta=0.0001; epochs=500; Elapsed time is 1.036147 seconds, about 225 weight updates.
Delta Rule Training, batch fashion 2 1.8 1.6 1.4 1.2
Error
1 0.8 0.6 0.4 0.2 0
100
200
300 Iteration
400
500
600
Incremental fashion: when eta=0.0001; epochs=500; Elapsed time is 1.095839 seconds, about 160 weight updates
Delta Rule Training, incremental fashion 2.5
1.5
Error
0.5
100
200
300 Iteration
400
500
600
Problem2. Derive a gradient descent training rule for a single unit with output o, where
Implement this gradient descent algorithm, train and test it on the sigmoid data provided on Blackboard
Gradient Descent Training Rule 1.3 1.2 1.1 1
Error
0.9 0.8 0.7 0.6 0.5
50
100
150
200
250 300 Iteration
350
400
450
500
Testing error is 0.67574 MATLAB commands for cpu time (from MATLAB help) CPUTIME CPU time in seconds. CPUTIME returns the CPU time in seconds that has been used by the MATLAB process since MATLAB started. For example: t=cputime; your_operation; cputime-t returns the cpu time used to run your_operation. The return value may overflow the internal representation and wrap around. See also etime, tic, toc, clock Reference page in Help browser doc cputime

(A) Plot The Error E As A Function of The Number of Training Iterations

Загружено:

Сведения о документе

Исходное описание:

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

(A) Plot The Error E As A Function of The Number of Training Iterations

Загружено:

Авторское право:

Доступные форматы

Assignment 4 Assigned 2/3/2012 Due 2/10/2012 (40 points) Problem 1.

Decision Surface of 5th Iteration 0.5

Decision Surface of 10th Iteration 1

Decision Surface of 50th Iteration 2

Decision Surface of 100th Iteration 2

Decision Surface of 150th Iteration 2

Decision Surface of 500th Iteration 2

Delta Rule Training 3.5

1 0.8 0.6 0.4 0.2 0

Delta Rule Training,eta=0.0005 2.5

Delta Rule Training,eta=0.0008 2 1.8 1.6 1.4 1.2

1 0.8 0.6 0.4 0.2 0

1 0.8 0.6 0.4 0.2 0

0.9 0.8 0.7 0.6 0.5

250 300 Iteration

Вам также может понравиться