Fit a Gaussian process regression (GPR) model - MATLAB fitrgp - M... http://es.mathworks.com/help/stats/fitrgp.

html

fitrgp

Syntax
gprMdl=fitrgp(tbl,ResponseVarName) example

gprMdl=fitrgp(tbl,formula)
gprMdl=fitrgp(tbl,y)

gprMdl=fitrgp(X,y) example

gprMdl=fitrgp( ___ ,Name,Value) example

Description
gprMdl=fitrgp(tbl,ResponseVarName) returns a Gaussian process regression (GPR) model trained using the sample data in tbl, where ResponseVarName is the name of the response variable in tbl. example

gprMdl=fitrgp(tbl,formula) returns a Gaussian process regression (GPR) model, trained using the sample data in tbl, for the predictor variables and response variables identified by formula.

gprMdl=fitrgp(tbl,y) returns a GPR model for the predictors in table tbl and continuous response vector y.

gprMdl=fitrgp(X,y) returns a GPR model for predictors X and continuous response vector y. example

gprMdl=fitrgp( ___ ,Name,Value) returns a GPR model for any of the input arguments in the previous syntaxes, with additional options specified by one or more Name,Value pair arguments. example

For example, you can specify the fitting method, the prediction method, the covariance function, or the active set selection method. You can also train a cross-validated model.
gprMdl is a RegressionGP object. For methods and properties of this class, see RegressionGP class page.

If you train a cross-validated model, then gprMdl is a RegressionPartitionedModel object. For further analysis on the cross-validated object, use the methods of RegressionPartitionedModel class. For the methods of this class, see the
RegressionPartitionedModel class page.

Examples collapse all

Train GPR Model Using Data in Table

This example uses the abalone data [1], [2], from the UCI Machine Learning Repository [3] . Download the data and save it in your current folder with the name abalone.data'.

Store the data into a table. Display the first seven rows.

tbl=readtable('abalone.data','Filetype','text','ReadVariableNames',false);tbl.Properties.VariableNames={'Sex','Length','Diameter','Height','WWeight','SWeight','VWeight','ShWeight','NoShellRings'};
tbl(1:7,:)

ans=

SexLengthDiameterHeightWWeightSWeightVWeightShWeightNoShellRings
________________________________________________________________

'M'0.4550.3650.0950.5140.22450.1010.1515
'M'0.350.2650.090.22550.09950.04850.077
'F'0.530.420.1350.6770.25650.14150.219
'M'0.440.3650.1250.5160.21550.1140.15510
'I'0.330.2550.080.2050.08950.03950.0557
'I'0.4250.30.0950.35150.1410.07750.128
'F'0.530.4150.150.77750.2370.14150.3320
The dataset has 4177 observations. The goal is to predict the age of abalone from eight physical measurements. The last variable, number of shell rings shows the age of the abalone. The first predictor is a categorical variable. The last variable in the
table is the response variable.

Fit a GPR model using the subset of regressors method for parameter estimation and fully independent conditional method for prediction. Standardize the predictors.

gprMdl=fitrgp(tbl,'NoShellRings','KernelFunction','ardsquaredexponential',...
'FitMethod','sr','PredictMethod','fic','Standardize',1)

grMdl=

RegressionGP
PredictorNames:{1x8cell}
ResponseName:'Var9'
ResponseTransform:'none'
NumObservations:4177
KernelFunction:'ARDSquaredExponential'
KernelInformation:[1x1struct]
BasisFunction:'Constant'
Beta:10.9148
Sigma:2.0243
PredictorLocation:[10x1double]
PredictorScale:[10x1double]
Alpha:[1000x1double]
ActiveSetVectors:[1000x10double]
PredictMethod:'FIC'
ActiveSetSize:1000
FitMethod:'SR'
ActiveSetMethod:'Random'
IsActiveSetVector:[4177x1logical]
LogLikelihood:9.0013e+03
ActiveSetHistory:[1x1struct]
BCDInformation:[]

Predict the responses using the trained model.

ypred=resubPredict(gprMdl);

Plot the true response and the predicted responses.

figure();
plot(tbl.NoShellRings,'r.');
holdon
plot(ypred,'b');
xlabel('x');
ylabel('y');
legend({'data','predictions'},'Location','Best');
axis([04300030]);
holdoff;

1 of 10 26/12/2015 0:20
Fit a Gaussian process regression (GPR) model - MATLAB fitrgp - M... http://es.mathworks.com/help/stats/fitrgp.html

Compute the regression loss on the training data (resubstitution loss) for the trained model.

L=resubLoss(gprMdl)

L=

4.0064

Train GPR Model and Plot Predictions

Impact of Specifying Initial Kernel Parameter Values

Navigate to a folder containing sample data.

cd(matlabroot)
cd('help/toolbox/stats/examples')

Load the sample data.

loadgprdata2

The data has one predictor variable and continuous response. This is simulated data.

Fit a GPR model using the squared exponential kernel function with default kernel parameters.

gprMdl1=fitrgp(x,y,'KernelFunction','squaredexponential');

Now, fit a second model, where you specify the initial values for the kernel parameters.

sigma0=0.2;
kparams0=[3.5,6.2];
gprMdl2=fitrgp(x,y,'KernelFunction','squaredexponential',...
'KernelParameters',kparams0,'Sigma',sigma0);

Compute the resubstitution predictions from both models.

ypred1=resubPredict(gprMdl1);
ypred2=resubPredict(gprMdl2);

Plot the response predictions from both models and the responses in training data.

figure();
plot(x,y,'r.');
holdon
plot(x,ypred1,'b');
plot(x,ypred2,'g');
xlabel('x');
ylabel('y');
legend({'data','defaultkernelparameters',...
'kparams0=[3.5,6.2],sigma0=0.2'},...
'Location','Best');
title('Impactofinitialkernelparametervalues');
holdoff

The marginal log likelihood that fitrgp maximizes to estimate GPR parameters has multiple local solutions; the solution that it converges to depends on the initial point. Each local solution corresponds to a particular interpretation of the data. In this
example, the solution with the default initial kernel parameters corresponds to a low frequency signal with high noise whereas the second solution with custom initial kernel parameters corresponds to a high frequency signal with low noise.

Use Separate Length Scales for Predictors

Navigate to a folder containing sample data.

cd(matlabroot)
cd('help/toolbox/stats/examples')

Load the sample data.

2 of 10 26/12/2015 0:20
Fit a Gaussian process regression (GPR) model - MATLAB fitrgp - M... http://es.mathworks.com/help/stats/fitrgp.html

loadgprdata

There are six continuous predictor variables. There are 500 observations in the training data set and 100 observations in the test data set. This is simulated data.

Fit a GPR model using the squared exponential kernel function with a separate length scale for each predictor. This covariance function is defined as:

d 2
x x
1
k xi , xj = 2f exp im jm
.
2 2
m=1
m

where represents the length scale for predictor m, m = 1, 2, ...,d and is the signal standard deviation. The unconstrained parametrization is
m f

m = log m, form = 1, 2, ..., d

d + 1 = log f .

Initialize length scales of the kernel function at 10 and signal and noise standard deviations at the standard deviation of the response.

sigma0=std(ytrain);
sigmaF0=sigma0;
d=size(Xtrain,2);
sigmaM0=10*ones(d,1);

Fit the GPR model using the initial kernel parameter values. Standardize the predictors in the training data. Use the exact fitting and prediction methods.

gprMdl=fitrgp(Xtrain,ytrain,'Basis','constant','FitMethod','exact',...
'PredictMethod','exact','KernelFunction','ardsquaredexponential',...
'KernelParameters',[sigmaM0;sigmaF0],'Sigma',sigma0,'Standardize',1);

Compute the regression loss on the test data.

L=loss(gprMdl,Xtest,ytest)

L=

0.6919
Access the kernel information.

gprMdl.KernelInformation

ans=

Name:'ARDSquaredExponential'
KernelParameters:[7x1double]
KernelParameterNames:{7x1cell}
Display the kernel parameter names.

gprMdl.KernelInformation.KernelParameterNames

ans=

'LengthScale1'
'LengthScale2'
'LengthScale3'
'LengthScale4'
'LengthScale5'
'LengthScale6'
'SigmaF'
Display the kernel parameters.

sigmaM=gprMdl.KernelInformation.KernelParameters(1:end1,1)
sigmaF=gprMdl.KernelInformation.KernelParameters(end)
sigma=gprMdl.Sigma

sigmaM=

1.0e+04*

0.0004
0.0007
0.0004
4.1731
0.1018
0.0056

sigmaF=

28.1718

sigma=

0.8162
Plot the log of learned length scales.

figure()
plot((1:d)',log(sigmaM),'ro');
xlabel('Lengthscalenumber');
ylabel('Logoflengthscale');

The log of length scale for the 4th and 5th predictor variables are high relative to the others. These predictor variables do not seem to be as influential on the response as the other predictor variables.

Fit the GPR model without using the 4th and 5th variables as the predictor variables.

3 of 10 26/12/2015 0:20
Fit a Gaussian process regression (GPR) model - MATLAB fitrgp - M... http://es.mathworks.com/help/stats/fitrgp.html

X=[Xtrain(:,1:3)Xtrain(:,6)];
sigma0=std(ytrain);
sigmaF0=sigma0;
d=size(X,2);
sigmaM0=10*ones(d,1);

gprMdl=fitrgp(X,ytrain,'Basis','constant','FitMethod','exact',...
'PredictMethod','exact','KernelFunction','ardsquaredexponential',...
'KernelParameters',[sigmaM0;sigmaF0],'Sigma',sigma0,'Standardize',1);

Compute the regression error on the test data.

xtest=[Xtest(:,1:3)Xtest(:,6)];
L=loss(gprMdl,xtest,ytest)

L=

0.6928
The loss is similar to the one when all variables are used as predictor variables.

Compute the predicted response for the test data.

ypred=predict(gprMdl,xtest);

Plot the original response along with the fitted values.

figure;
plot(ytest,'r');
holdon;
plot(ypred,'b');
legend('Trueresponse','GPRpredictedvalues','Location','Best');
holdoff

Train GPR Model Using Cross Validation

This example uses the abalone data [1], [2], from the UCI Machine Learning Repository [3]. Download the data and save it in your current folder with the name abalone.data'.

Store the data into a table. Display the first seven rows.

tbl=readtable('abalone.data','Filetype','text','ReadVariableNames',false);tbl.Properties.VariableNames={'Sex','Length','Diameter','Height','WWeight','SWeight','VWeight','ShWeight','NoShellRings'};
tbl(1:7,:)

ans=

SexLengthDiameterHeightWWeightSWeightVWeightShWeightNoShellRings
________________________________________________________________

'M'0.4550.3650.0950.5140.22450.1010.1515
'M'0.350.2650.090.22550.09950.04850.077
'F'0.530.420.1350.6770.25650.14150.219
'M'0.440.3650.1250.5160.21550.1140.15510
'I'0.330.2550.080.2050.08950.03950.0557
'I'0.4250.30.0950.35150.1410.07750.128
'F'0.530.4150.150.77750.2370.14150.3320
The dataset has 4177 observations. The goal is to predict the age of abalone from eight physical measurements. The last variable, number of shell rings shows the age of the abalone. The first predictor is a categorical variable. The last variable in the
table is the response variable.

Train a cross-validated GPR model using the 25% of the data for validation.

rng('default')%Forreproducibility
cvgprMdl=fitrgp(tbl,'NoShellRings','Standardize',1,'Holdout',0.25);

Compute the average loss on folds using models trained on out-of-fold observations.

kfoldLoss(cvgprMdl)

ans=
4.6409

Predict the responses for out-of-fold data.

ypred=kfoldPredict(cvgprMdl);

Plot the true responses used for testing and the predictions.

figure();
plot(ypred(cvgprMdl.Partition.test));
holdon;
y=table2array(tbl(:,end));
plot(y(cvgprMdl.Partition.test),'r.');
axis([01050030]);
xlabel('x')
ylabel('y')
holdoff;

4 of 10 26/12/2015 0:20
Fit a Gaussian process regression (GPR) model - MATLAB fitrgp - M... http://es.mathworks.com/help/stats/fitrgp.html

Fit GPR Model Using Custom Kernel Function

Generate the sample data.

rng(0,'twister');%Forreproducibility
n=1000;
x=linspace(10,10,n)';
y=1+x*5e2+sin(x)./x+0.2*randn(n,1);

Define the squared exponential kernel function as a custom kernel function.

You can compute the squared exponential kernel function as

where is the signal standard deviation, is the length scale. Both and must be greater than zero. This condition can be enforced by the unconstrained parametrization, and , for some unconstrained
parametrization vector .

Hence, you can define the squared exponential kernel function as a custom kernel function as follows:

kfcn=@(XN,XM,theta)(exp(theta(2))^2)*exp((pdist2(XN,XM).^2)/(2*exp(theta(1))^2));

Here pdist2(XN,XM).^2 computes the distance matrix.

Fit a GPR model using the custom kernel function, kfcn. Specify the initial values of the kernel parameters (Because you use a custom kernel function, you must provide initial values for the unconstrained parametrization vector, theta).

theta0=[1.5,0.2];
gprMdl=fitrgp(x,y,'KernelFunction',kfcn,'KernelParameters',theta0);

fitrgp uses analytical derivatives to estimate parameters when using a built-in kernel function, whereas when using a custom kernel function it uses numerical derivatives.

Compute the resubstitution loss for this model.

L=resubLoss(gprMdl)

L=

0.0391

Fit the GPR model using the built-in squared exponential kernel function option. Specify the initial values of the kernel parameters (Because you use the built-in custom kernel function and specifying initial parameter values, you must provide the initial
values for the signal standard deviation and length scale(s) directly).

sigmaL0=exp(1.5);
sigmaF0=exp(0.2);
gprMdl2=fitrgp(x,y,'KernelFunction','squaredexponential','KernelParameters',[sigmaL0,sigmaF0]);

Compute the resubstitution loss for this model.

L2=resubLoss(gprMdl2)

L2=

0.0391

The two loss values are the same as expected.

Input Arguments collapse all

tbl Sample data

table

Sample data used to train the model, specified as a table. Each row of tbl corresponds to one observation, and each column corresponds to one variable. tbl contains the predictor variables, and optionally it can also contain one column for the
response variable. Multi-column variables and cell arrays other than cell arrays of strings are not allowed.

If tbl contains the response variable, and you want to use all the remaining variables as predictors, then specify the response variable using ResponseVarName.
If tbl contains the response variable, and you want to use only a subset of the predictors in training the model, then specify the response variable and the predictor variables using formula.
If tbl does not contain the response variable, then specify a response variable using y. The length of the response variable and the number of rows in tbl must be equal.

For more information on the table data type, please see table.

If your predictor data contains categorical variables, then the software uses full dummy coding for these variables. The software creates one dummy variable for each level of the categorical variable.

Data Types: table

ResponseVarName Response variable name

name of a variable in tbl

Response variable name, specified as the name of a variable in tbl. You must specify ResponseVarName as a string. For example, if the response variable y is stored in tbl (as tbl.y), then specify it as 'y'. Otherwise, the software treats all the
columns of tbl, including y, as predictors when training the model.

Data Types: double

formula Response and predictor variables to use in model training

String in the form of 'y~x1+x2+x3'

Response and predictor variables to use in model training, specified as a string in the form of 'y~x1+x2+x3'. In this form, y represents the response variable; x1, x2, x3 represent the predictor variables to use in training the model.

Use formula string if you want to specify a subset of variables in tbl as predictors to use when training the model. If you specify a formula string, then any variables that do not appear in formula are not used to train the model.

The formula does not indicate the form of the BasisFunction.

Example: 'PetalLength~PetalWidth+Species' identifies the variable PetalLength as the response variable, and PetalWidth and Species as the predictor variables.

X Predictor data for the GPR model

n-by-d matrix

5 of 10 26/12/2015 0:20
Fit a Gaussian process regression (GPR) model - MATLAB fitrgp - M... http://es.mathworks.com/help/stats/fitrgp.html

Predictor data for the GPR model, specified as an n-by-d matrix. n is the number of observations (rows), and d is the number of predictors (columns).

The length of y and the number of rows of X must be equal.

To specify the names of the predictors in the order of their appearance in X, use the PredictorNames name-value pair argument.

Data Types: double

y Response data for the GPR model

n-by-1 vector

Response data for the GPR model, specified as an n-by-1 vector. You can omit y if you provide the tbl training data that also includes y. In that case, use ResponseVarName to identify the response variable or use formula to identify the response and
predictor variables.

Data Types: double | char

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside single quotes (''). You can specify several name and value pair arguments in any order as
Name1,Value1,...,NameN,ValueN.

Example: 'FitMethod','sr','BasisFunction','linear','ActiveSetMethod','sgma','PredictMethod',fic' trains the GPR model using the subset of regressors approximation method for parameter estimation, uses a linear basis function, uses
sparse greedy matrix approximation for active selection, and fully independent conditional approximation method for prediction.

Fitting

'FitMethod' Method to estimate parameters of the GPR model

'none' | 'exact' | 'sd' | 'sr' | 'fic'

Method to estimate parameters of the GPR model, specified as the comma-separated pair consisting of 'FitMethod' and one of the following.

Fit Method Description

'none' No estimation, use the initial parameter values as the known parameter values.

'exact' Exact Gaussian process regression. Default if n 2000, where n is the number of observations.

'sd' Subset of data points approximation. Default if n > 2000, where n is the number of observations.

'sr' Subset of regressors approximation.

'fic' Fully independent conditional approximation.

Example: 'FitMethod','fic'

'BasisFunction' Explicit basis in the GPR model

'constant' (default) | 'none' | 'linear' | 'pureQuadratic' | function handle

Explicit basis in the GPR model, specified as the comma-separated pair consisting of 'BasisFunction' and one of the following. If n is the number of observations, the basis function adds the term H* to the model, where H is the basis matrix and is a
p-by-1 vector of basis coefficients.

Explicit Basis Basis Matrix

'none' Empty matrix.

'constant' H =1

(n-by-1 vector of 1s, where n is the number of observations)

'linear' H = 1, X
'pureQuadratic' H = 1, X , X2 ,

where

2 2 2
x11 x12 x1d
2 2 2
x21 x22 x2d
X2 = .

2 2 2
xn1 xn2 xnd

Function handle Function handle, hfcn, that fitrgp calls as:

H = hfcn X ,

where X is an n-by-d matrix of predictors and H is an n-by-p matrix of basis functions.

Example: 'BasisFunction','pureQuadratic'

Data Types: char | function_handle

'Beta' Initial value of the coefficients

p-by-1 vector

Initial value of the coefficients for the explicit basis, specified as the comma-separated pair consisting of 'Beta' and p-by-1 vector, where p is the number of columns in the basis matrix H.

The basis matrix depends on the choice of the explicit basis function as follows (also see BasisFunction).

fitrgp uses the coefficient initial values as the known coefficient values, only when FitMethod is 'none'.

Data Types: double

'Sigma' Initial value for the noise standard deviation of the Gaussian process model
std(y)/sqrt(2) (default) | positive scalar value

Initial value for the noise standard deviation of the Gaussian process model, specified as the comma-separated pair consisting of 'Sigma' and a positive scalar value.

Example: 'Sigma',2

Data Types: double

'SigmaLowerBound' Lower bound on the noise standard deviation

1e2*std(y) (default) | positive scalar value

Lower bound on the noise standard deviation, specified as the comma-separated pair consisting of 'SigmaLowerBound' and a positive scalar value.

Example: 'SigmaLowerBound',0.02

Data Types: double

'CategoricalPredictors' List of categorical predictors

numeric vector | logical vector | 'all' | cell array of strings

6 of 10 26/12/2015 0:20
Fit a Gaussian process regression (GPR) model - MATLAB fitrgp - M... http://es.mathworks.com/help/stats/fitrgp.html

List of categorical predictors, specified as the comma-separated pair consisting of 'CategoricalPredictors' and one of the following.

A numeric vector with indices from 1 through p, where p is the number of columns in X or variables in tbl.
A 1-by-p logical vector, with a true entry corresponding to the column of X or tbl that is a categorical variable.
A cell array of strings, where each element in the array is the name of a predictor variable. The names must match the entries in PredictorNames values.
'all', which means all predictors are categorical.

By default, if the predictor data is in a matrix (X), fitrgp assumes that none of the predictors are categorical unless specified as such using the CategoricalPredictors name-value pair argument. If the predictor data is in a table (tbl), the software
assumes that a variable is categorical if it contains logical values, categorical values, or a cell array of strings.

Example: 'CategoricalPredictors',[falsetruefalsetruefalse] specifies the second and the fourth predictors as categorical.

Data Types: double | logical | cell

'Standardize' Indicator to standardize data

0(false) (default) | logical value

Indicator to standardize data, specified as the comma-separated pair consisting of 'Standardize' and a logical value.

If you set 'Standardize',1, then the software centers and scales each column of the predictor data, by the column mean and standard deviation, respectively. The software does not standardize the data contained in the dummy variable columns that it
generates for categorical predictors.

Example: 'Standardize',1

Example: 'Standardize',true

Data Types: logical

'Regularization' Regularization standard deviation

1e2*std(y) (default) | positive scalar value

Regularization standard deviation for sparse methods subset of regressors ('sr') and fully independent conditional ('fic'), specified as the comma-separated pair consisting of 'Regularization' and a positive scalar value.

Example: 'Regularization',0.2

Data Types: double

'ComputationMethod' Method for computing log likelihood and gradient

'qr' (default) | 'v'

Method for computing the log likelihood and gradient for parameter estimation using subset of regressors ('sr') and fully independent conditional ('fic') approximation methods, specified as the comma-separated pair consisting of
'ComputationMethod' and one of the following.

'qr' Use QR factorization based approach, this option provides better accuracy.
'v' Use V-method-based approach. This option provides faster computation of log likelihood gradients.

Example: 'ComputationMethod','v'

Kernel (Covariance) Function

'KernelFunction' Form of the covariance function

'squaredexponential' (default) | 'matern32' | 'matern52' | 'ardsquaredexponential' | 'ardmatern32' | 'ardmatern52' | function handle

Form of the covariance function, specified as the comma-separated pair consisting of 'KernelFunction' and one of the following.

Function Description

'squaredexponential' Squared exponential kernel

'matern32' Matern kernel with parameter 3/2

'matern52' Matern kernel with parameter 5/2

'ardsquaredexponential' Squared exponential kernel with a separate length scale per predictor

'ardmatern32' Matern kernel with parameter 3/2 and a separate length scale per predictor

'ardmatern52' Matern kernel with parameter 5/2 and a separate length scale per predictor

Function handle A function handle that can be called like this:

Kmn=kfcn(Xm,Xn,theta)
where Xm is an m-by-d matrix, Xn is an n-by-d matrix and Kmn is an m-by-n matrix of kernel products such that Kmn(i,j) is the
kernel product between Xm(i,:) and Xn(j,:).
theta is the r-by-1 unconstrained parameter vector for kfcn.

For more information on the kernel functions, see Kernel (Covariance) Function Options.

Example: 'KernelFunction','Matern32'

Data Types: char | function_handle

'KernelParameters' Initial values for the kernel parameters

vector

Initial values for the kernel parameters, specified as the comma-separated pair consisting of 'KernelParameters' and a vector. The size of the vector and the values depend on the form of the covariance function, specified by the KernelFunction
name-value pair argument.

'KernelFunction' 'KernelParameters'

'squaredexponential' or 'matern32' or 'matern52' 2-by-1 vector phi, where phi(1) contains the length scale and phi(2) contains the noise standard deviation.
Default initial value of the length scale parameter is the mean of standard deviations of the predictors, and the noise
standard deviation is the standard deviation of the responses divided by square root of 2. That is,
phi=[mean(std(X)),std(y)/sqrt(2)]

'ardsquaredexponential' or 'ardmatern32' or 'ardmatern52' (d+1)-by-1 vector theta, where phi(i) contains the length scale for predictor i and phi(d+1) contains the noise standard
deviation. d is the number of predictor variables.
Default initial value of the length scale parameters are the standard deviations of the predictors and the noise standard
deviation is the standard deviation of the responses divided by square root of 2. That is,
phi=[std(X),std(y)/sqrt(2)]

Function handle r-by-1 vector as the initial value of the unconstrained parameter vector phi for the custom kernel function kfcn.
When KernelFunction is a function handle, you must supply initial values for the kernel parameters.

For more information on the kernel functions, see Kernel (Covariance) Function Options.

Example: 'KernelParameters',theta

Data Types: double

'DistanceMethod' Method for computing inter-point distances

'fast' (default) | 'accurate'

2 2 2
Method for computing inter-point distances to evaluate built-in kernel functions, specified as the comma-separated pair consisting of 'DistanceMethod' and either 'fast' or 'accurate'. fitrgp computes x y as x + y 2 x y when you
2
choose the fast option and as x y when you choose the accurate option.

Example: 'DistanceMethod','accurate'

Active Set Selection

'ActiveSet' Observations in the active set

[] (default) | m-by-1 vector of integers ranging from 1 to n (m n) | logical vector of length n

Observations in the active set, specified as the comma-separated pair consisting of 'ActiveSet' and an m-by-1 vector of integers ranging from 1 to n (m n) or a logical vector of length n with at least one true element. n is the total number of
observations in the training data.

7 of 10 26/12/2015 0:20
Fit a Gaussian process regression (GPR) model - MATLAB fitrgp - M... http://es.mathworks.com/help/stats/fitrgp.html

fitrgp uses the observations indicated by ActiveSet to train the GPR model. The active set cannot have duplicate elements.

If you supply ActiveSet, then:

fitrgp does not use ActiveSetSize and ActiveSetMethod.

You cannot perform cross-validation on this model.

Data Types: double | logical

'ActiveSetSize' Size of the active set for sparse methods

an integer m (1 m n)

Size of the active set for sparse methods ('sd', 'sr', 'fic'), specified as the comma-separated pair consisting of 'ActiveSetSize' and an integer m, 1 m n, where n is the number of observations.

Default is min(1000,n) when FitMethod is 'sr' or 'fic', and min(2000,n), otherwise.

Example: 'ActiveSetSize',100

Data Types: double

'ActiveSetMethod' Active set selection method

'random' (default) | 'sgma' | 'entropy' | 'likelihood'

Active set selection method, specified as the comma-separated pair consisting of 'ActiveSetMethod' and one of the following.

Method Description

'random' Random selection

'sgma' Sparse greedy matrix approximation

'entropy' Differential entropy-based selection

'likelihood' Subset of regressors log likelihood-based selection

All active set selection methods (except 'random') require the storage of an n-by-m matrix, where m is the size of the active set and n is the number of observations.

Example: 'ActiveSetMethod','entropy'

'RandomSearchSetSize' Random search set size

59 (default) | integer value

Random search set size per greedy inclusion for active set selection, specified as the comma-separated pair consisting of 'RandomSearchSetSize' and an integer value.

Example: 'RandomSearchSetSize',30

Data Types: double

'ToleranceActiveSet' Relative tolerance for terminating active set selection

1e-06 (default) | positive scalar

Relative tolerance for terminating active set selection, specified as the comma-separated pair consisting of 'ToleranceActiveset' and a positive scalar value.

Example: 'ToleranceActiveset',0.0002

Data Types: double

'NumActiveSetRepeats' Number of repetitions

3 (default) | integer value

Number of repetitions for interleaved active set selection and parameter estimation when ActiveSetMethod is not 'random', specified as the comma-separated pair consisting of 'NumActiveSetRepeats' and an integer value.

Example: 'NumActiveSetRepeats',5

Data Types: double

Prediction

'PredictMethod' Method used to make predictions

'exact' | 'bcd' | 'sd' | 'sr' | 'fic'

Method used to make predictions from a Gaussian process model given the parameters, specified as the comma-separated pair consisting of 'PredictMethod' and one of the following.

Method Description

'exact' Exact Gaussian process regression method. Default, if n 10000.

'bcd' Block Coordinate Descent. Default, if n > 10000.

'sd' Subset of Datapoints approximation.

'sr' Subset of Regressors approximation.

'fic' Fully Independent Conditional approximation.

Example: 'PredictMethod','bcd'

'BlockSizeBCD' Block size

minimum of 1000 or n (default) | integer in the range from 1 to n

Block size for block coordinate descent method ('bcd'), specified as the comma-separated pair consisting of 'BlockSizeBCD' and an integer in the range from 1 to n, where n is the number of observations.

Example: 'BlockSizeBCD',1500

Data Types: double

'NumGreedyBCD' Number of greedy selections

minimum of 100 and BlockSizeBCD (default) | integer value in the range from 1 to BlockSizeBCD

Number of greedy selections for block coordinate descent method ('bcd'), specified as the comma-separated pair consisting of 'NumGreedyBCD' and an integer in the range from 1 to BlockSizeBCD.

Example: 'NumGreedyBCD',150

Data Types: double

'ToleranceBCD' Relative tolerance on gradient norm

1e-3 (default) | positive scalar

Relative tolerance on gradient norm for terminating block coordinate descent method ('bcd') iterations, specified as the comma-separated pair consisting of 'ToleranceBCD' and a positive scalar.

Example: 'ToleranceBCD',0.002

Data Types: double

'StepToleranceBCD' Absolute tolerance on step size

1e-3 (default) | positive scalar

Absolute tolerance on step size for terminating block coordinate descent method ('bcd') iterations, specified as the comma-separated pair consisting of 'StepToleranceBCD' and a positive scalar.

Example: 'StepToleranceBCD',0.002

Data Types: double

'IterationLimitBCD' Maximum number of BCD iterations

8 of 10 26/12/2015 0:20
Fit a Gaussian process regression (GPR) model - MATLAB fitrgp - M... http://es.mathworks.com/help/stats/fitrgp.html

1000000 (default) | integer value

Maximum number of block coordinate descent method ('bcd') iterations, specified as the comma-separated pair consisting of 'IterationLimitBCD' and an integer value.

Example: 'IterationLimitBCD',10000

Data Types: double

Optimization

'Optimizer' Optimizer to use for parameter estimation

'quasinewton' (default) | 'fminsearch' | 'fminunc' | 'fmincon'

Optimizer to use for parameter estimation, specified as the comma-separated pair consisting of 'Optimizer' and one of quasinewton, fminsearch, fminunc, fmincon.

Use of 'fminunc' and 'fmincon' requires an Optimization Toolbox license.

Example: 'Optimizer','fmincon'

'OptimizerOptions' Options for the optimizer

structure | object

Options for the optimizer you choose using the Optimizer name-value pair argument, specified as the comma-separated pair consisting of 'Optimizer' and a structure or object created by optimset, statset('fitrgp'), or optimoptions.

Optimizer Create Optimizer Options Using

'fminsearch' optimset (structure)

'quasinewton' statset('fitrgp') (structure)

'fminunc' or 'fmincon' optimoptions (object)

The default options depend on the type of optimizer.

Example: 'OptimizerOptions',opt

Cross-Validation

'CrossVal' Indicator for cross-validation

'off' (default) | 'on'

Indicator for cross-validation, specified as the comma-separated pair consisting of 'CrossVal' and either 'off' or 'on'. If it is 'on', then fitrgp returns a GPR model cross-validated with 10 folds.

You can use one of the KFold, Holdout, Leaveout or CVPartition name-value pair arguments to change the default cross-validation settings. You can use only one of these name-value pairs at a time.

As an alternative, you can use the crossval method for your model.

Example: 'CrossVal','on'

Data Types: double

'CVPartition' Random partition for a stratified k-fold cross validation

cvpartition object

Random partition for a stratified k-fold cross validation, specified as the comma-separated pair consisting of 'CVPartition' and a cvpartition object.

Example: 'CVPartition',cvp uses the random partition defined by cvp.

If you specify CVPartition, then you cannot specify Holdout, KFold, or Leaveout.

'Holdout' Fraction of data to use for testing

Scalar value in the range from 0 to 1

Fraction of the data to use for testing in holdout validation, specified as the comma-separated pair consisting of 'Holdout' and a scalar value in the range from 0 to 1. If you specify 'Holdout',p, then the software:
1. Randomly reserves around p*100% of the data as validation data, and trains the model using the rest of the data
2. Stores the compact, trained model in cvgprMdl.Trained.

Example: 'Holdout',0.3 uses 30% of the data for testing and 70% of the data for training.

If you specify Holdout, then you cannot specify CVPartition, KFold, or Leaveout.

Data Types: double

'KFold' Number of folds

10 (default) | positive integer value

Number of folds to use in cross-validated GPR model, specified as the comma-separated pair consisting of 'KFold' and a positive integer value. KFold must be greater than 1. If you specify 'KFold',k then the software:
1. Randomly partitions the data into k sets.
2. For each set, reserves the set as test data, and trains the model using the other k 1 sets.
3. Stores the k compact, trained models in the cells of a k-by-1 cell array in cvgprMdl.Trained.

Example: 'KFold',5 uses 5 folds in cross-validation. That is, for each fold, uses that fold as test data, and trains the model on the remaining 4 folds.

If you specify KFold, then you cannot specify CVPartition, Holdout, or Leaveout.

Data Types: double

'Leaveout' Indicator for leave-one-out cross-validation

'off' (default) | 'on'

Indicator for leave-one-out cross-validation, specified as the comma-separated pair consisting of 'Leaveout' and either 'off' or 'on'.

If you specify 'Leaveout','on', then, for each of the n observations, the software:
1. Reserves the observation as test data, and trains the model using the other n 1 observations.
2. Stores the compact, trained model in a cell in the n-by-1 cell array cvgprMdl.Trained.

Example: 'Leaveout','on'

If you specify Leaveout, then you cannot specify CVPartition, Holdout, or KFold.

Other

'PredictorNames' Names for the predictor variables

{'x1','x2',...} (default) | cell array

Names for the predictor variables, specified as the comma-separated pair consisting of 'PredictorNames' and a cell array.

If you specify the predictors in a table (tbl), then PredictorNames must be a subset of the variable names in tbl. In this case, the software uses only the variables in PredictorNames to train the model.

Example: 'PredictorNames',{'PedalWidth','PedalLength'}

Data Types: cell

'ResponseName' Name of the response variable

'Y' (default) | string

Name of the response variable, specified as the comma-separated pair consisting of 'ResponseName' and a string.

This name-value pair argument is not valid when using the formula or ResponseVarName input arguments.

Example: 'ResponseName','Price'

Data Types: char

9 of 10 26/12/2015 0:20
Fit a Gaussian process regression (GPR) model - MATLAB fitrgp - M... http://es.mathworks.com/help/stats/fitrgp.html

'Verbose' Verbosity level

0 (default) | 1

Verbosity level, specified as the comma-separated pair consisting of 'Verbose' and one of the following.

0 fitrgp suppresses diagnostic messages related to active set selection and block coordinate descent but displays the messages related to parameter estimation, depending on the value of 'Display' in OptimizerOptions.
1 fitrgp displays the iterative diagnostic messages related to parameter estimation, active set selection, and block coordinate descent.

Example: 'Verbose',1

'CacheSize' Cache size in megabytes

1000 MB (default) | positive scalar

Cache size in megabytes (MB), specified as the comma-separated pair consisting of 'CacheSize' and a positive scalar. Cache size is the extra memory that is available in addition to that required for fitting and active set selection. fitrgp uses
CacheSize to:

Decide whether interpoint distances should be cached when estimating parameters.

Decide how matrix vector products should be computed for block coordinate descent method and for making predictions.

Example: 'CacheSize',2000

Data Types: double

Output Arguments collapse all

gprMdl Gaussian process regression model

RegressionGP object | RegressionPartitionedModel object

Gaussian process regression model, returned as an RegressionGP or a RegressionPartitionedModel object.

If you cross validate, that is, if you use one of the 'Crossval', 'KFold', 'Holdout', 'Leaveout', or 'CVPartition' name-value pairs, then gprMdl is a RegressionPartitionedModel object. You cannot use a RegressionPartitionedModel
object to make predictions using predict. For more information on the methods and properties of this object, see RegressionPartitionedModel.
If you do not cross validate, then gprMdl is a RegressionGP object. You can use this object for predictions using the predict method. For more information on the methods and properties of this object, see RegressionGP.

More About expand all

Active Set Selection and Parameter Estimation

For subset of data, subset of regressors, or fully independent conditional approximation fitting methods (FitMethod equal to 'sd', 'sr', or 'fic'), if you do not provide the active set, fitrgp selects the active set and computes the parameter estimates
in a series of iterations.

In the first iteration, the software uses the initial parameter values in vector 0 = [0,0 ,0] to select an active set A1 . It maximizes the GPR marginal log likelihood or its approximation using 0 as the initial values and A1 to compute the new parameter
estimates 1. Next, it computes the new log likelihood L1 using 1 and A1 .

In the second iteration, the software selects the active set A2 using the parameter values in 1. Then, using 1 as the initial values and A2 , it maximizes the GPR marginal log likelihood or its approximation and estimates the new parameter values 2. Then
using 2 and A2 , computes the new log likelihood value L2 .

The following table summarizes the iterations and what is computed at each iteration.

Iteration Number Active Set Parameter Vector Log Likelihood

1 A1 1 L1

2 A2 2 L2

3 A3 3 L3

The software iterates similarly for a specified number of repetitions. You can specify the number of replications for active set selection using the NumActiveSetRepeats name-value pair argument.

Tips

fitrgp accepts any combination of fitting, prediction, and active set selection methods. In some cases it might not be possible to compute the standard deviations of the predicted responses, hence the prediction intervals. See predict. And in some
cases, using the exact method might be expensive due to the size of the training data.
The PredictorNames property stores one element for each of the original predictor variable names. For example, if there are three predictors, one of which is a categorical variable with three levels, PredictorNames is a 1-by-3 cell array of strings.
The ExpandedPredictorNames property stores one element for each of the predictor variables, including the dummy variables. For example, if there are three predictors, one of which is a categorical variable with three levels, then
ExpandedPredictorNames is a 1-by-5 cell array of strings.
Similarly, the Beta property stores one beta coefficient for each predictor, including the dummy variables.
The X property stores the training data as originally input. It does not include the dummy variables.

Gaussian Process Regression Models

Kernel (Covariance) Function Options

References
[1] Warwick J. N., T. L. Sellers, S. R. Talbot, A. J. Cawthorn, and W. B. Ford. "The Population Biology of Abalone (_Haliotis_ species) in Tasmania. I. Blacklip Abalone (_H. rubra_) from the North Coast and Islands of Bass Strait." Sea Fisheries Division,
Technical Report No. 48 (ISSN 1034-3288), 1994.

[2] S. Waugh. "Extending and Benchmarking Cascade-Correlation", PhD Thesis. Computer Science Department, University of Tasmania, 1995.

[3] Lichman, M. UCI Machine Learning Repository, Irvine, CA: University of California, School of Information and Computer Science, 2013. http://archive.ics.uci.edu/ml.

[4] Rasmussen, C. E. and C. K. I. Williams. Gaussian Processes for Machine Learning. MIT Press. Cambridge, Massachusetts, 2006.

See Also
compact | predict | RegressionGP

Introduced in R2015b

10 of 10 26/12/2015 0:20