Application of Machine Learning to Finance

Z elia Cazalet & Tung-Lam Dao

-Application of Machine Learning to Finance-

Introduction
Figure: A subset of the database

ASSET MANAGEMENT BY

LYXOR
QUANT TOUCH

Introduction

2 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

Introduction

LYXOR
QUANT TOUCH

Figure: PCA of faces

Introduction

3 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

Introduction

LYXOR
QUANT TOUCH

Figure: ICA of faces

Introduction

4 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

Outline

LYXOR
QUANT TOUCH

Hedge fund replication: factor selection and the lasso method Nonnegative matrix factorization Learning algorithms Trend forecasting with L1 and L2 lterings Support Vector Machine and nancial applications

5

Outline

5 / 35

-Application of Machine Learning to Finance-

Hedge Fund replication

ASSET MANAGEMENT BY

LYXOR
QUANT TOUCH

It is principally done using factor-based models: rolling least squares or Kalman ltering algorithms.

HF replication
RtHF =
m X i =1

i ,t Rti + t

Dene the tracker portfolio as:

m

RtTracker = +1

i ,t Rti +1
Hedge fund replication 6 / 35

i =1
Hedge fund replication: factor selection and the lasso method

-Application of Machine Learning to Finance-

Problem of factor selection

ASSET MANAGEMENT BY

LYXOR
QUANT TOUCH

Considering the problem of factor selection is necessary: the universe of factor selection inuences the trackers performance. A solution: the lasso method. Trackers with dierent universes of factors

Hedge fund replication: factor selection and the lasso method

Problem of factor selection

7 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

LYXOR Lasso regression (Tibshirani, 1996)

QUANT TOUCH

It corresponds to a linear regression with regularization of coecient estimates: L1 norm constraint of exposures.

Lasso regression
After the standardization of returns, we have:

= arg min R HF R
m

R HF R

u.c.
i =1

i2

where is the shrinkage measure of the lasso model with respect to the OLS model.

Hedge fund replication: factor selection and the lasso method

Lasso regression

8 / 35

-Application of Machine Learning to Finance-

Ranking of factors
Ranking of the lasso exposures (Feb. 28, 2011)
1. SPX 7. GOLD 2. HY 8. EMBI 3. GSCI 9. RTY 4. UST 10. TPX 5. MSCI EM 11. JPY/USD

ASSET MANAGEMENT BY

LYXOR
QUANT TOUCH 6. EUR/USD 12. SX5E

Factors selection (Feb. 28, 2011)

Hedge fund replication: factor selection and the lasso method

Empirical results

9 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

Cross-validation procedure

LYXOR
QUANT TOUCH

We dene an out-of-sample procedure to choose the optimal value of .

Principle
1 2

We build training and test samples from the lag window p . For one sequence of dierent [0, 1], we estimate the exposures i ,t on the training sample. We compute a statistic of interest on the test sample: performance, TE or MSE. We nd the value of which permits to optimize the statistic of interest.

3 4

Hedge fund replication: factor selection and the lasso method

Empirical results

10 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

Trackers with cross-validation lasso regression

LYXOR
QUANT TOUCH

Results of replicating the HFRI index using dierent methods

Model HFRI CV #1 CV #2 CV #3 OLS 6.80 3.64 4.09 3.81 3.56 6.81 7.59 7.77 7.68 7.66 sh 0.57 0.09 0.15 0.11 0.08 MDD 21.42 22.32 21.56 20.20
24.07 AB 71.50 74.99 72.82 70.85 TE 3.52 3.29 3.43 3.51 0.89 0.91 0.89 0.89

Hedge fund replication: factor selection and the lasso method

Empirical results

11 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

LYXOR NMF principle and nancial interpretation

QUANT TOUCH

NMF is an alternative approach to decomposition methods like PCA and ICA with the special feature to consider nonnegative matrices:

NMF decomposition
Let A be a nonnegative matrix m p :

A BC
with B and C nonnegative matrices of dimensions m n and n p . Considering a variable/observation storage in A, interpret B as a matrix of

weights called loading matrix and C as
a factor matrix.

Nonnegative matrix factorization

NMF principle and nancial interpretation

12 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

LYXOR Factor extraction of an equity universe

QUANT TOUCH

Using the composition at the end of 2010, we compute NMF on the logarithm of the stock prices. Comparison between the EuroStoxx 50 and the rst NMF factor

The rst NMF factor is highly correlated with the index.

Nonnegative matrix factorization Factor extraction of an equity universe 13 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

LYXOR Factor extraction of an equity universe

QUANT TOUCH

NMF with two factors

We may interpret them as a factor of bear market and a factor of bull market.
Nonnegative matrix factorization Factor extraction of an equity universe 14 / 35

-Application of Machine Learning to Finance-

Pattern recognition of asset returns LYXOR

ASSET MANAGEMENT BY

QUANT TOUCH

Data: weekly returns of 20 stocks. Period: January 2000 - December 2010. NMF on positive and negative returns (four patterns)

Nonnegative matrix factorization

Pattern recognition of asset returns

15 / 35

-Application of Machine Learning to Finance-

Stock classication

ASSET MANAGEMENT BY

LYXOR
QUANT TOUCH

Some stocks are more sensible to the representative NMF factor than to their corresponding sectors.

Nonnegative matrix factorization

Classication of stocks

16 / 35

-Application of Machine Learning to Finance-

Classication of stocks: NMF classiers LYXOR

ASSET MANAGEMENT BY

QUANT TOUCH

Apply the K-means procedure directly on the stocks returns. Results of the cluster analysis

Nonnegative matrix factorization

Classication of stocks

17 / 35

-Application of Machine Learning to Finance-

Classication of stocks: NMF classiers LYXOR

QUANT TOUCH

ASSET MANAGEMENT BY

Can NMF classiers represent an alternative sector classication? Frequencies of sectors in each cluster

Nonnegative matrix factorization

Classication of stocks

18 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

Bagging and Boosting algorithmsLYXOR

QUANT TOUCH

Bagging and boosting algorithms are recent powerful techniques which permit to reduce the error of any learning algorithms. These two methods consist in determining several classiers before aggregating them by voting. Dierence between the two algorithms
bagging uses bootstrap samples to construct classiers, boosting adjusts the weights of the training instances considering errors of classication.

Learning algorithms

Bagging and Boosting algorithms

19 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

LYXOR Application to stock picking: scores

QUANT TOUCH

We work on the improvement of a score used in a stock picking model. We use the current score based on a discrete optimization and a score built with a probit model. Probit score

S = (X + )
with (x ) the cumulative distribution function of the standard normal distribution and (, ) two vectors estimated using the estimator of the maximum likelihood.

Learning algorithms

Application to stock picking

20 / 35

-Application of Machine Learning to Finance-

Application to stock picking: index tilting LYXOR

ASSET MANAGEMENT BY

QUANT TOUCH

The objective of index tilting is to maximize the score of the portfolio compared to the score of a benchmark. This optimization is under constraint of tracking error. Optimization problem x = arg max (x b ) s u.c. 1 x = 1 b = 1 and with: 2 = (x b ) (x b )

where x and b are respectively the portfolio and the benchmark weights, s is

the vector of score, the variance-covariance matrix of stocks and , the
constraint of tracking error.
Learning algorithms Application to stock picking 21 / 35

-Application of Machine Learning to Finance-

Application to stock picking: backtests LYXOR

ASSET MANAGEMENT BY

QUANT TOUCH

Backtests of the stock picking model (2002-2006)

Reporting of the stock picking model (2002-2006)

Models Benchmark Discret Score Probit Score Probit Score bagging Probit Score boosting 5.34 5.74 5.51 5.92 6.00 20.61 21.38 20.57 20.59 20.57 sh 0.26 0.27
0.27 0.29 0.29 MDD 48.76 50.01 49.25 48.86 49.07 IR 0.09 0.09 0.33 0.33 TE 4.67 1.91 1.73 1.98 0.98 0.99 0.99 0.99 22 / 35

Learning algorithms

Application to stock picking

-Application of Machine Learning to Finance-

Application to stock picking: backtests LYXOR

ASSET MANAGEMENT BY

QUANT TOUCH

Backtests of the stock picking model (2007-2011)

Reporting of the stock picking model (2007-2011)

Models Benchmark Discret Score Probit Score Probit Score bagging Probit Score boosting 7.71 6.06 8.36 7.46 8.09 27.30 28.50 27.09 27.12 27.10 sh 0.28 0.21
0.31 0.27 0.30 MDD 61.04 58.27 62.18 61.11 61.84 IR 0.29 0.23 0.10 0.14 TE 5.63 2.80 2.42 2.70 0.98 0.99 0.99 0.99 23 / 35

Learning algorithms

Application to stock picking

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

Trend ltering
Noisy signal yt can be decomposed into trend xt and noise zt : yt = xt + zt L2 lter (Hodrick-Prescott lter) detects xt by minimizing: 1 y x 2 with second derivative D :
2 6 6 D=6 4

LYXOR
QUANT TOUCH

2 L2

+ Dx

2 L2

2 1

1 2

3 1 .. . 1 7 7 7 5 2 1

` 1 L2 lter allows explicit solution x = I + 2D y

Trend forecasting with L1 and L2 lterings Method Principle 24 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

L1 ltering
Minimize the objective function with L1 pernalty: 1 y x 2
2 L2

LYXOR
QUANT TOUCH

+ Dx

L1

where D is discrete form of the rst or second derivative. Similar problems: Lasso regression (Tibshirani, 1996) or the L1 regularized least square problem (Daubechies, 2004) Properties of L1 ltering:
Using L1 norm 2nd derivation of xt must be zero. L1 norm allows xt change the trend without two much cost.

Trade-o between: residual noise and number of breaks.
Determine by minimizing prediction error within caliration procedure.

Trend forecasting with L1 and L2 lterings

Method Principle

25 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

Linear trend model

LYXOR
QUANT TOUCH

Stochastic linear trend

8 yt = xt + > > ` zt > > < zt N 0, 2 xt = xt 1 + vt > > 1p > > Pr { vt = vt 1 } = : Pr vt = b U[1,1] = p

Signal
150 150

Noisy signal

100

100

50

50

50 500 1000 1500 2000

50 500 1000 1500 2000

t
150

t HP lter
150

L1 -T lter ( =5285)

( =1217464)

Remarks
L1 lter gives hidden trend Direct trend prediction

100

100

50

50

50

50

500

1000

1500

2000

500

1000

1500

2000

Trend forecasting with L1 and L2 lterings

Method Principle

26 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

Ornstein-Uhlenbeck process

LYXOR
QUANT TOUCH

Signal

Noisy signal
30 20 10 0 10 20

OU with switching regim

8 yt = yt (t yt 1 ) + zt > > `1 + < zt N 0, 2 > > Pr { t = t 1 } =1 p : Pr t = b U[1,1] = p

30 20 10 0 10 20 500 1000 1500 2000

500

1000

1500

2000

t L1 -C lter
( =483)

t HP lter
( =2949)
30 20 10 0 10 20

Remarks
L1 is better than L2 simple for application

30 20 10 0 10 20

500

1000

1500

2000

500

1000

1500

2000

Trend forecasting with L1 and L2 lterings

Method Principle

27 / 35

-Application of Machine Learning to Finance-

Mixing trend and mean-revertingLYXOR

QUANT TOUCH

ASSET MANAGEMENT BY

Use two penalty conditions:

1 y x 2 D1 and D2 are respectively the
2000 1500 1000 500 0 500 1000 1500 2000

2 2

+ 1 D1 x 2nd

+ 2 D2 x

1st
Signal

and

derivatives.
Noisy signal
2000 1500 1000 500 0 500 1000 1500 2000

t L1 -TC lter ( 1 =8503, 2 =125683)

2000 1500 1000 500 0 500 1000 1500 2000 2000 1500 1000 500 0 500

t HP lter
( =43764340)

t

1000

1500

2000

Trend forecasting with L1 and L2 lterings

Method Principle

28 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

Cross validation: Algorithm

Training | | T1 Historical data
procedure CV Filter(T1 , T2 ) Compute an array of (max n ) of N training sets T1 , the average and variance of (n ) Compute + and 2 = Compute 1 = for i = 1 : Np do Compute = ( / )(i /Np )
i 2 2 1

LYXOR
QUANT TOUCH

Validation

Forecasting | T2 Today

-|
T2

Prediction

Scan data by the window T1 Compute the total error e (i ) end for Minimize the error e () to nd the optimal value Run the L1 lter with =
end procedure

Trend forecasting with L1 and L2 lterings

Method Principle

29 / 35

-Application of Machine Learning to Finance-

Comparison between L1 and L2 lters LYXOR

ASSET MANAGEMENT BY

QUANT TOUCH

Trend forecasting with L1 and L2 lterings

Comparison between L1 and L2 lters

30 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

History and Financial applicationsLYXOR

QUANT TOUCH

History
SVM rst introduced in 1992 as classication method SVM next interpreted as regression technique (Vapnik 1998) SVM applications in various elds: pattern recognition, bioinformation

Financial applications
SVM score: Score Binary classication SVM sector recognition: supervision method to classify stocks SVM ltering: trend extraction

SVM multi-regression: trend prediction based on multi-factors

Support Vector Machine and nancial applications

SVM at a glance

31 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

Principle and Score construction LYXOR

QUANT TOUCH

Example of SVM via the score construction

Universe of n stocks characterized by d economic factors x Rd Classify the stocks subjected to their performance indicator y = 1 SVM score is dened as the distance to the frontier

Hard margin principle

Hyperplane dened by h(x) = wT x + b = 0 Maximize the margin: T (x+ x ) /2 = 1/ w mD (h) = w

`
under constraints: yi wT xi + b > 1

i = 1...n

Support Vector Machine and nancial applications

SVM at a glance

32 / 35

-Application of Machine Learning to Finance-

Employ SVM score without overtting LYXOR

QUANT TOUCH

ASSET MANAGEMENT BY

Selection curve We construct:

High score: Q (s ) = Pr (S s ) Selection error: E (s ) = Pr (S s |Y = 1 )
1 0.9 0.8 1

Corss validation
Training set: Dene SVM classier Validation set: Minimize predicting error and SVM error SVM score constructed on both Training+Validation

SVM model Probit model

0.9 0.8

SVM Training SVM Validation SVM Testing

P r (S > s|Y = 0)

0.6 0.5 0.4 0.3 0.2

P r (S > s|Y = 0)

0.7

0.7 0.6 0.5 0.4 0.3 0.2

0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

P r (S > s)

0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

P r (S > s)

Support Vector Machine and nancial applications

SVM at a glance

33 / 35

-Application of Machine Learning to FinanceASSET MANAGEMENT BY

SVM as trend ltering

Principle
Filter yt by a trend of the form: f (x) = wT (x) + b Minimize the following tting error: . R=
n X i =1
yt
0.15

LYXOR
QUANT TOUCH

0.1

0.05

0.05

0.1

|f (xi ) yi |2 + n 2 w

0.15

0.2 0

Real signal Training Validation Prediction

50 100 150 200 250 300

Remarks
Equivalent to SVM classication.

Non-linear ltering solved by kernel approach K = (x) (x)

Support Vector Machine and nancial applications

SVM regression

34 / 35

-Application of Machine Learning to Finance-

Example on S&P 500 index

Cross validation procedure
Divide data into: training, validation and testing

ASSET MANAGEMENT BY

LYXOR
QUANT TOUCH

Learn on training, optimize parameters on validation, predict on testing

Support Vector Machine and nancial applications

SVM regression

35 / 35