Classifying S&P500 Equities

Andrew McConnell, David Lee, Jordan Friedman, and Sebastian Dobon

Abstract Our project group set out to develop a simple After combining the resulting datasets from Quandl and
binary financial classifier which could take commonly reported Yahoo Finance, we set out to conduct three experiments
financial data for a company and use that to predict how it on our data to see what modes of analysis would improve
would perform relative to the S&P 500. To this extent, we
generated a dataset by combining results from both Quandls the accuracy of classification. Initially, we checked a set of
and Yahoo Finances Python APIs and used them to build different machine learning algorithms on the whole dataset
classifiers in Weka. to see which would produce the best accuracy when 10 fold
As good financial predictions using machine learning prac- cross validation was applied. This led to, ultimately, unre-
tices have been historically difficult to build (if possible at all), alistic predictions as the model may have been using future
one of our primary goals was to investigate the best methods
of approaching this topic. To that extent, we devised three data to predict past data. To better judge the classification
experiments to figure out how best to build a classifier on accuracy, we sectioned off the most historically-recent 10%
economic data, covering different machine learning classifiers, of the data to be used as a test set, and used the other 90% to
different ranges of time for the data, and different statistical train the models. In the second experiment, we hypothesized
representations of the data. that some of the data might be chaotic, particularly around
Our results, though not the money-making breakthrough
we were hoping for, do show some promise. We found that the 2008 market crash, and therefore be inherently difficult to
(unsurprisingly) Random Forest tended to yield the highest predict. Based on this, we conducted the second experiment
accuracies, usually followed by J48 Decision Trees and K similar to the first, but only looked at the data from 2009
Nearest Neighbors. We also found that accuracy could generally onwards, using the same 90%/10% ratio for training/testing.
be improved by standardizing the data according to z-score, and The final experiment we conducted, at the suggestion of
that it helped to focus on more financially-stable time periods
for classification. Professor Downey, was to standardize our data by z-score
and try to classify those instances. Z-scores were calculated
I. CONTACT by the following equation:
Our group can be reached by email at:
(X u)
jordanfriedman2017@u.northwestern.edu Z=

andrewmcconnell2016@u.northwestern.edu
davidlee1.2019@u.northwestern.edu We believed this might help our machine learning models,
sebastiandobon2019@u.northwestern.edu as each of our forty instance attributes contain numerical
data. The results of each of our experiments are below.
II. PROJECT
III. RESULTS
We wanted to develop a reliable methodology to accurately
predict whether a certain stock will perform well or poorly TABLE I
relative to the S&P 500. If done well, this task is incredibly F ULL DATASET - 10 F OLD VALIDATION
intriguing. It would give us the ability to take much of the
Model Accuracy
uncertainty out of our investment portfolios and would allow ZeroR 53.3865%
us to reach higher and steadier rates of return on our equity J48 69.4887%
investments. IBk 75.7968%
Logistic 57.5003%
We decided to conquer this task by examining data from K Star 47.4104%
equities listed on the S&P500 since 2004 and through 2014. RandomF orest 79.7145%
We used the Quandl and Yahoo Finance APIs to scrape
fundamental data about each stockforming the backbone of
our attribute list. Private Equity firms and Investment Banks TABLE II
use metrics like Return on Equity, Enterprise Value/Revenue, F ULL DATA S ET - 10% AS T EST
Enterprise Value/EBITDA, and Beta to determine the real
value of a company. We hoped to use these metrics, along Model Accuracy
ZeroR 56.6879%
with an expanded attribute list, to make predictions about J48 51.9108%
a companys future performance. We used several different IBk 61.1465%
learners to try and discover the best predictor of over and Logistic 56.051%
K Star 43.3121%
under performers including: J48, IBK, Logistic Regressions, RandomF orest 59.5541%
K-Star and Random Forest over a specified training set of
500 examples.
 TABLE III TABLE V
F ULL DATA S ET - Z-S CORES 10 F OLD DATASET A FTER 2009 - 10% AS T EST

Model Accuracy Model Accuracy

ZeroR 56.6879% ZeroR 47.3684%
J48 57.6433% J48 62.6316%
IBk 60.1911% IBk 45.2632%
Logistic 53.5032% Logistic 55.7895%
K Star 53.502% K Star 52.6316%
RandomF orest 64.0127% RandomF orest 70.5263%

TABLE IV
F ULL DATA S ET Z-S CORES - 10% AS T EST Andrew: proposal, data scraping, experiments and
Model Accuracy analysis, final report, website
ZeroR 53.3865%
J48 69.4887% Jordan: domain knowledge, data scraping, status update,
IBk 75.7968%
Logistic 57.5003%
modeling, final report
K Star 47.4104%
RandomF orest 79.7145% David: model implementation, analysis, final report

VI. ACKNOWLEDGEMENTS
Note: Though the results from Table V are slightly more
We would like to thank Professor Downey and the
optimistic (particularly for Random Forest), it is worth
EECS349 staff for their assistance in guiding us during this
noting that these models were built and tested on much
project, as well as for teaching us about the core concepts
smaller datasets, and thus should not be taken as wholly
of machine learning.
representative. The results may not remain consistent when
scaled to larger datasets. On the other hand, we felt that the
market environment after 2009 may have been, as a whole,
quite similarwhere data from 2004 and onward included a
financial crisis and had erratic market trends that did not
continue after 2009.
IV. DISCUSSION
The models we created using Random Forest consistently
gave us the best results with accuracies both higher that the
ZeroR score and above 50% accuracy. Also, our experiments
did show a slight increase in predictability after the chaos of
the 2008 market crash had settled down, as we had hypoth-
esized. J48 and IBk mostly proved to be effective classifiers
(with IBk performing poorly in the 2009 data set relative to
ZeroR, but much better in the other experiments). We think
that IBk worked because companies with similar financial
profiles, either weak or strong, should have correlated price
movements (where companies that are performing well have
strong financial profiles and companies that perform poorly
have weak financial profiles).
Additionally, the top ten ranked attributes by InfoGain
included, in descending order and based on tenfold cross-
validation, were: Price, Cash Flow, Market Cap, Enterprise
Value, Earnings Growth, Net Income Avl to Common, Gross
Profit, Held by Institutions, Operating Margin, and Shares
Short. This highlights the relative importance of Cash Flow
as a strong indicator of a companys ability to perform well
on the S&P500.
V. CONTRIBUTIONS
Sebastian: proposal, experiments, status update, final
report