5/25/2012

Comparison of Neural Networks and Kernel Regression

Report on a comparison of the relative performance l ti f

Comparison methodology
Comparisons are reported for 3 sets of data: 1. 2. 3. Air preheater Setpoint optimization Bearing vibrations

For each dataset, two charts are shown: 1.

The first chart shows the Predicted (Y_est) versus Actual (Y) values from a NN model (via Neuromodeler) and Kernel Regression (KR) via XLSTAT, with the R2 value for each. This indicates the degree of agreement between the actual and predicted values. (0<R2<1), with a value of 1 indicating perfect prediction. The second chart shows the actual time series plots of the actual and predicted (KR and NN) outputs. Three widely used error statistics are also shown: MSE, RMSE, and MAPE. These measure the deviation of the forecasts from the original data. They are defined in the next slide.

2.

5/25/2012

Forecast evaluation: measures

R2 or the Coefficient of determination measure the proportion of total variation in the data explained by the model. This is the most widely used single indicator of goodness of fit of a model. But for finer comparison, the error statistics defined below are used. Definitions of error statistics Mean Squared Error (MSE) is the Mean (average) of the Squared Errors. MSE = et 2/ n, where et = Yt- Y_estt , and n the number of observations

Root Mean Squared Error (RMSE) is the square root of MSE Median Ab l M di Absolute P Percent E Error (MAPE) i calculated as the median of the is l l d h di f h percentage difference between the fitted and the original data. MAPE = Median(|et/yt| * 100)

Note that while a higher R2 is better, for RMSE and MAPE, lower values are better!

Predicted Vs. Actuals: air_preheater

Thus in this case both NN and KR perform very well wellin each case 99% of the variation in Y is explained by the model predictions

KR predictions and R2 are in red, NN in green

5/25/2012

air_preheater_time_series

The time series plots confirm the good prediction made by both models, though the NN statistics look somewhat better

Actual values in blue, KR predictions in red, NN in green

Predicted Vs. Actuals:

setpoint optimization

In this case both NN and KR perform quite well- for KR , the R2=0.87, which is more than for NN

KR predictions and R2 are in red, NN in green

5/25/2012

setpoint_time_series

The time series plots confirm the good prediction made by both models. Here the KR error statistics are better than the NN

Actual values in blue, KR predictions in red, NN in green

Predicted Vs. Actuals:

bearing vibrations
In this case neither NN or KR perform very wellNN looks a little better

KR predictions and R2 are in red, NN in green

5/25/2012

bearing_vib_time series

The time series plots confirm the relative predictive ability of both models, with the NN statistics being a little better
Actual values in blue, KR predictions in red, NN in green