Implementation and evaluation of a movie recommender system using
Collaborative Filtering algorithm
Vishita Singh
College Station, Texas
vishita.singh@tamu.edu
Abstract
Recommender Systems have been an active area of research
since the mid-1990's and gained popularity primarily due to
a surge in the e-commerce industry and in the entertainment
industry in recent times. Recommender Systems use ma-
chine learning algorithms to suggest items to a user based
on the user’s preferences. This paper aims to implement a
recommender system using the supervised learning algo-
rithm, Collaborative Filtering (CF) through K-Nearest
Neighbors method. The implementation was done using
open-source libraries and the key challenges faced through
the process are presented in this paper. The evaluation of the
algorithm was done using different similarity indexes. The
quality of the similarity indexes was then evaluated using
Root Mean Square Error (RMSE), Mean Absolute Error
(MAE), and Fraction of Concordant Pairs (FCP) evaluation
metrics. The results are compared, and the observations are
presented with different values of k in the K-Nearest Neigh-
bors algorithm. This paper also identifies some practical
challenges while implementing a recommender system and
suggests some methods to overcome those. A literature re-
view has also been provided on Recommender Systems us-
ing CF and on evaluation of Recommender Systems.
Introduction
Making efficient decisions consistently to improve differ-
ent aspects of life has always been a challenge for human
beings. Human mind possesses limited analytical and
processing power. The probability of a human mind to
make a correct decision reduces exponentially with a line-
ar increase in dimensionality and volume of information
and data involved in making the decision.
In recent times, a growth in human interaction with the
World Wide Web has increased the scope for the study of
human behavior patterns for better decision making. This
has led to an increase in competition for businesses to
offer personalized recommendation to its customers by
analyzing consumer behavior and traits. As a result, con-
sumers are also presented with never ending alternatives
to resolve their wants and requirements. With the abun-
dance of data collection techniques and ever-increasing
volumes of raw data being collected by enterprises today,
it is a big challenge to persist and process the data to get
meaningful business acumens. All these factors steered
the creation of sophisticated data mining algorithms that
help in solving the problem of collecting huge volumes of
data from a large number of transactions and studying
patterns in these transactions to provide meaningful in-
sights into the similarities and dissimilarities in human
behaviors.
Recommender systems are a subset of these data min-
ing algorithms that study human behavior using past data
points and help make efficient decisions. These systems
also help businesses make better recommendations and
advertise their products and solutions to their customer
base by analyzing the consumer behavior.
The recommender systems are used on multiple fronts
in today’s world. They predict consumer actions and
requirements in a grocery store, recommend a user music
records, books and movies based on personal preferences
and many more. These systems are also used by business-
es to implement customer retention techniques as these
systems help businesses study consumer interaction with
various products and solutions and make them predict
customer behavior better. The systems could also be used
in predicting a new user’s behavior by calculating its sim-
ilarity within the existing customer base.
This paper deals with the problem of recommending
movie choices to users based on their personal prefer-
ences using a recommendation technique called collabora-
tive filtering. Collaborative filtering is a recommendation
technique that uses user’s previous movie ratings and
rating of similar other users to predict the affinity of this
user towards a movie entity. Although, this paper deals
with association of movies and users, the technique of
collaborative filtering is applicable to any other product
and item as well.
In this paper, a movie recommender system has been
implemented using collaborative filtering through a data
model called K-Nearest Neighbors (KNN) algorithm to
1
recommend movie choices to users based on their inputs
and preferences. KNN algorithm works by defining
neighborhoods for similar entities, either item based, or
user based. It is assumed that the entities in the neighbor-
hoods have similar interactions and behaviors.
The paper provides insights into effectively training
and designing a movies recommendation system for a
user base using the techniques of collaborative filtering.
The paper also dives deep into the implementation and
evaluation of these techniques using real data of Movie
Lens website made available by Group Lens Corporation
(Grouplens, 2018). The paper also highlights the pros and
cons associated with Collaborative filtering techniques
and presents problems that can and cannot be solved us-
ing this technique.
This paper can be summarized as follows -
• Provides insights into the design, development,
and training of a Movie recommender system
with collaborative filtering using Correlation and
K-Nearest Neighbor methods.
• Presents an overview of the evaluation of the da-
ta model created for the movie recommender
system using correlation and k-NN algorithm.
• Highlights key challenges and provides solutions
while implementing a recommender system us-
ing open-source libraries.
Figure 1: Components of a Recommender System
Related Work
Recommender systems truly emerged as an independent
area of study in the mid-1990’s when researchers started
explicitly focusing on the rating structure and the recom-
mender system problem was reduced to the problem of
estimating rating for items that have not been seen by a
user (Adomavicius & Tuzhilin, June 2005). Since then a
lot of work has been done in academia as well as in the
industry. Recommender systems proactively predict user
preferences based on the user’s past preferences. The term
collaborative was introduced with a commercial recom-
mender system called Tapestry, which was used to provide
users with recommendations on newsgroup documents
(Melville & Sindhwani, April 2010). One of the earliest
work done in this field was by GroupLens Research
(Resnick, Iacovou, Sushak, Bergstrom, & Reidl, 1994a), a
research lab in the Department of Computer Science and
Engineering at the University of Minnesota. Grouplens has
made available multiple datasets like MovieLens, which
provides rating data sets from the MovieLens website
(Grouplens, 2018). The initial formulations for recom-
mender systems was based on statistical correlation and
predictive modeling rather than machine learning models
and subsequently collaborative filtering was mapped to
classification which also allowed the use of dimensionality
reduction to improve the model precision (Melville &
Sindhwani, April 2010).
Collaborative filtering gained traction because of its
direct use in e-commerce and a notable mention is the Net-
flix Prize competition for the best collaborative filter-
ing algorithm to predict user ratings for films for which the
company released a large dataset containing 100 million
user ratings. Collaborative Filtering (CF) is now widely
used in recommending movies, music, e-commerce, news,
research papers among others (Haruna, Akmar Ismail,
Damiasih, Sutopo, & Herawan, Oct 2017). This method is
one of the most successful techniques used in recommend-
er systems which recommend items to a user based on
what similar users have rated (Haruna, Akmar Ismail,
Damiasih, Sutopo, & Herawan, Oct 2017).
Collaborative Filtering remains an active area of re-
search simply because of its use in a plethora of applica-
tions to provide personalized recommendations that assist
users in dealing with insurmountable information. There is
much work to be done in taking into consideration the user
behavior and its representation. Using not just explicit rat-
ing but also implicit user behavior, for example - a user
closed a movie within 15 minutes of starting it, is an active
area of research. Advanced recommendation modeling
methods incorporate various contextual information into
the recommendation process, utilize multi-criteria ratings
and develop less intrusive systems that are more effective
in measuring system performance (Adomavicius &
Tuzhilin, June 2005).
AI Approach
The fundamentals of any Recommender System is based
on a combination of the type of data available in the da-
2
taset, the filtering algorithm, and the chosen model. The
data extraction techniques used are Data Mining tech-
niques. The quality of incoming data varies by the source
and nature of data (Maheshwari, 2015). Recommender
systems are divided into three main categories, depending
on the information used to drive the recommendations:
collaborative, content-based, and hybrid filtering (Portugal,
Alencar, & Cowan, May 2018). Collaborative approach
provides personalized recommendations to a user based on
ratings from the user as well as other users who have simi-
lar preferences while content-based approach recommends
items to a user based on previously rated similar items by
the user. The hybrid approach combines attributes of more
than one type of recommendation system (Recommender
Systems, 2014). Hybrid recommender systems can be im-
plemented by making content-based and collaborative-
based predictions separately and then combining them or
by adding content-based capabilities to a collaborative-
based approach.
Collaborative filtering (CF) is a popular recommenda-
tion algorithm that bases its predictions and recommenda-
tions on the ratings or behavior of other users in the system
(Phorasim & Yu, 2017). CF methods can be further subdi-
vided into memory-based and model-based approaches.
Memory-based approach is based on literal (explicit)
memory of past user ratings (Recommender Systems,
2014). Neighborhood-based methods are also commonly
referred to as memory-based approaches (Breese,
Heckerman, & Kadie, July 1998). Amazon and YouTube
recommender systems exploit the collaborative and heuris-
tic-based approaches (Park, Park, Jung, & Lee, May 2015).
Figure 2: AI Approach for a Recommender System using Collab-
orative Filtering
The assumption used in the Collaborative filtering mod-
el is that if certain users have the same preferences regard-
ing certain items then they will most likely have the same
opinion about other items as well. The concept behind this
algorithm is that if a users’ preferences are known, a simi-
larity is calculated with other users who have similar pref-
erences.
The K-Nearest Neighbors algorithm (k-NN) is a non-
parametric method used for classification and regression.
Collaborative filtering using k-NN works by gathering user
ratings of items and calculating similarities between users
(Park, Park, Jung, & Lee, May 2015). Using this similarity,
predictions can be made for unrated items for a user. In
neighborhood-based techniques, a subset of users is chosen
who are highly similar to the user and who have rated the
item for which a prediction needs to be made for the user.
There are several methods to calculate the similarities be-
tween users like adjusted cosine-based similarity, cosine-
based similarity and correlation-based similarity (Phorasim
& Yu, 2017). Because not all users rate all items, the da-
taset contains missing values and different similarity in-
dexes treat missing values differently. It is important that
missing values can be filled in with average or modal or
default values (Maheshwari, 2015). In this implementation,
the Pearson correlation (adjusted cosine) coefficient is used
as the similarity measure for the user-based CF algorithm
because it considers the weighted average rating by the
similarity value from the neighborhood. The similarity,
sim(a,n) is calculated using:
Using the similarity, sim(a,n), the ratings are predicted
for the items not viewed by the user. According to
(Herlocker, Konstan, Borchers, & Riedl, 1999), the rating
for an active user a for an item i is defined as follows:
where N(a) denotes the set of K-Nearest Neighbors of a
among the users that have rated item i; rn,i denotes the rat-
ing of item i by user n; 𝑟̅a and 𝑟̅n are the average ratings of
user a and neighbor n, respectively; sim(a,n) is the simi-
larity between a and n.
CF algorithms produce personalized movie recommen-
dations as compared to the methods that recommend “top-
N movies”, “most popular movies” or “highly rated mov-
ies” (Park, Park, Jung, & Lee, May 2015). Collaborative
Filtering is widely popular because of its simplicity and
ability to predict accurate results.
3
 Implementation
The MovieLens dataset available at the Grouplens website
(Grouplens, 2018) was used for this recommender system.
The dataset consists of 100,000 ratings applied to 9,000
movies by 600 users. The movies dataset consists of movie
ids, titles and genres. The ratings dataset consists of ratings
given by each user and each user has a distinct user-id. Figure 3: UserId – MovieId 2D Matrix containing ratings pro-
vided by the user
Table 1: Ratings Dataset with userId and movieId
The similarity was calculated from the matrix using
userId movieId rating timestamp Pearson’s coefficient. The system takes the movie and cal-
1 5060 5 964984002 culates its similarity with other movies based on user rat-
2 318 3 1445714835 ings. The system was configured to take into considera-
2 333 4 1445715029 tions only movies that have more than 275 ratings. It was
3 31 0.5 1306463578 observed that less number of ratings did not provide accu-
4 106 4 986848784 rate results primarily due to the lack of data. This number
was determined by a trial and error method and may differ
The ratings are on a scale from 0.5 to 5. For the purposes based on the data and requirements. It was observed that a
of this implementation, the user preference scale has been number significantly lower than 275 provided more movie
related to the ratings as below: title recommendations but there was a trade-off in terms of
rating count. To ensure more accuracy, in the current im-
plementation, a higher rating count is preferred. Below is
Table 2: User preference – rating scale
an example of the movies similar to “The Shawshank Re-
User Preference Rating scale demption, (1994)”:
Strongly Like 5
Like 4
Average 3
Dislike 2
Strongly Dislike 1

The implementation is done using Python open-source

libraries pandas, numpy, and sklearn. The top-rated movies
can simply be calculated by the movies that are rated the Figure 4: Pearson coefficient for movies corresponding to “The
most and their average ratings. Shawshank Redemption, (1994)”
To recommend movies using the k-NN method, the line-
ar co-relation between movies is calculated using the Pear-
The corresponding movies titles that are most like the
son correlation. To calculate the coefficient, we need the
movie “The Shawshank Redemption, (1994)” as computed
average rating and total number of rating as calculated pre-
above are:
viously. To ensure statistical significance for the experi-
ment, the following are removed from the dataset: movies
with less than 50 ratings and users who provided less than
100 ratings. This was done because movies with a low
count of ratings and a high rating value altered the results
and provided false positives. To calculate the correlation,
the ratings and movie-ids are first pivoted into a 2D matrix.
Since every user did not rate every movie, a significant
number of ratings have a null value. Following is an exam-
ple of the converted matrix: Figure 5: Movie titles similar to “The Shawshank Redemption,
(1994)”

4
 It should be noted that movie at index 0 is the same as
the one (The Shawshank Redemption, (1994)) for which
the value of similarity index is being calculated. The Pear-
son Coefficient for the movie is 1 because it is the same
movie. Thus, the most similar movies have a higher value
of the Pearson Coefficient. Other movies that are similar to
“The Shawshank Redemption, (1994)” can be selected
based on the total rating count as well as the coefficient
value. Movies, “Forrest Gump (1994)” and “Pulp Fiction
(1994)” have the highest rating count amongst all the dis-
played movie titles and thus these are most likely to be
similar to “The Shawshank Redemption, (1994)”. These
are the movie titles based on co-relation.
Using the k-NN model, movie recommendations are
made by calculating the distance between items and deter-
mining the closest neighbors. The neighbors with the least
distance from the item are most likely to be the user’s pref-
erence. The most popular neighbors are picked based on
the total rating count. The number of neighbors k can be
specified based on the available data and system require-
ments. The value of k was determined by experimenting
with different values starting from 3, which is a reasonable
number of neighbors to start with. In this implementation,
the value of k was set as 5 because this value provided the
most accurate results. The below values were observed for
the evaluation metrics of Root Mean Square Error
(RMSE), Mean Absolute Error, and Fraction of Concord-
ant Pairs (FCP):
Figure 6: Comparison of RMSE, MAE, and FCP for values k=3
and k=5 in this implementation of the recommender system
Below is another example, the movie “Aladdin (1992)”,
based on which recommendations are were made and the
following are the recommendations generated by the mod-
el:
Figure 7: Movie Recommendations generated with k-NN model
showing distance values
Evaluation
A good recommender system is as good as its core recom-
mender algorithm and its performance and accuracy needs
to be measured before the recommendations can be made
to the user. All evaluation was completed offline i.e. there
was no user-interaction required but only the dataset to
calculate the evaluation metrics. This practice is referred to
as Offline Analytics. The advantage of using Offline Ana-
lytics is that there is no interaction required with the user
and thus is easy to implement with low computational
costs (Chena & Liu, Dec 2017). The testing method used
was k-fold cross-validation and was implemented using
Python Scikit open-source libraries.
To determine quality of predictions and the accuracy of
the results, the most common error metrics, Mean Absolute
Error (MAE) and Root Mean Square Error(RMSE) were
used. RMSE measures the standard deviation of the errors
the system makes in its predictions (Géron, 2017). MAE
measures the average magnitude of the errors in a set of
predictions, but it does not consider the direction of the
error (positive error or negative error) (Chena & Liu, Dec
2017). MAE is the average of the absolute difference be-
tween actual and predicted values. Both metrics predict
average model prediction error but RMSE is more useful
when there is a need to remove large errors from the data.
RMSE gives high weight to large errors because errors are
squares before they are averaged. RMSE and MAE are
calculated as below:
Root Mean Square Error:
Mean Absolute Error:
5
where Q is the test set, 𝑟𝑢𝑖 represents the user’s true ratings,
ȓ𝑢𝑖 represents the prediction ratings of the recommender
system (Chena & Liu, Dec 2017).
The dataset was split into training dataset and testing
dataset. The method used for calculating similarities was
Pearson method and the value of k used was 5 for the be-
low experiment. The value of RMSE and MAE was calcu-
lated with multiple values of percentage of test dataset
used. Below are the results for the error values observed
when k-NN was implemented and tested:
Figure 8: RMSE vs. percentage of data used for evaluation
Figure 9: MAE vs. percentage of data used for evaluation
It can be observed from the above evaluations that the
error values decrease with an increase in training data and
the prediction accuracy increases. When the dataset was
split into 10% training and 90% test sets, highest values of
error metrics was observed. The values of RMSE and
MAE gradually decrease when the size of training dataset
was increased. This can be explained as the phenomenon
“Cold Start Problem”. The k-NN algorithm heavily relies
on past user ratings to make predictions and as the model
receives more data, its capability to make predictions and
the quality of results improves.
Another metric that was used to compute the quality of
precisions is the Fraction of Concordant Pairs (FCP), a
measure that generalizes the known AUC metric into non-
binary ordered outcomes (Koren & Sill, 2013). FCP
measures the proportion of pairs of well classified items.
where 𝑛𝑐 = ∑𝑢 𝑛𝑐𝑢 and 𝑛𝑑 = ∑𝑢 𝑛𝑑𝑢 are the summations of
concordant pairs and discordant pairs respectively over all
users (Koren & Sill, 2013). Concordant pairs are the ones
ranked correctly by the predictor ȓu. For each user u, the
concordant pairs are calculated as:
The discordant pairs 𝑛𝑑𝑢 are the ones that are incorrectly
ranked and are calculated in a similar fashion as the con-
cordant pairs.
FCP was calculated for the model after the dataset was
split into training dataset and testing dataset, just like
RMSE and MAE were calculated above. The observed
results are as shown below:
Figure 10: FCP vs. percentage of data used for evaluation
According to Koren & Still, the higher the FCP the
better the results. In this experiment, it was observed that
FCP significantly rose when the size of training dataset
increased. This also leads to the conclusion that the k-NN
algorithm provides better quality results when it receives
more data.
As previously mentioned, there are several methods to
calculate similarities between items. Two of those were
implemented as a part of this experiment to evaluate which
one provides better quality results. The two similarity
measuring techniques used were the Cosine similarity and
6
the Pearson similarity. Below are the results obtained for Lessons Learned
RMSE and MAE over 5 iterations:
One of the most significant lessons learnt in this project
Table 3: Comparison of evaluation metrics for the Cosine simi- was the selection of the value of k to maintain a balance
larity and the Pearson similarity between over-fitting and under-fitting the data. A higher
RMSE (Cosine) RMSE (Pearson) MAE (Cosine) MAE (Pearson) value of k provided more accurate results but is computa-
Iteration 1 1.1678 1.0388 0.9241 0.8047 tionally more expensive. This did not impact the perfor-
Iteration 2 1.1747 1.035 0.9287 0.8042
mance significantly for the experiment but may not be
Iteration 3 1.1758 1.0362 0.9279 0.7983
Iteration 4 1.1707 1.0332 0.9207 0.7989 suitable to distributed systems. After selecting a lower val-
Iteration 5 1.178 1.0461 0.9306 0.8088 ue of k it was observed that the noise impacted the results
and predictions were not accurate.
It is evident from the results that the Pearson similarity To calculate the similarities, different methods were
provides results with lower error rates and thus was the used. It was observed that the cosine-coefficient did not
preferred method to compute similarity during the final result in accurate predictions because of the null (NaN)
implementation to obtain movie predictions. values in the data. This data signifies the ratings that have
Though the Pearson similarity does provide better re- not been provided by the user. The Pearson (centered co-
sults, a trade-off in computation costs was observed. The sine) similarity considers the weighted average of the rat-
time taken by the system to fit the model and to test the ings by their similarities and thus is able to reduce the
model is significantly greater while using Pearson similari- noise caused by the null values whereas, the cosine-
ty than when using the cosine similarity. Below are the coefficient considers the values as 0. Treating null values
mean and standard deviation of the time taken by the two as 0 caused the model to provide inaccurate results.
methods over all the conducted iterations: Another challenge that was faced during the implemen-
tation of this system was to determine the statistical signif-
Table 4: Comparison of time taken to fit and test the model for icance of results. The statistical distribution of the data was
the Cosine similarity and the Pearson similarity used to determine the top movies and the movies that had
Fit time Fit time Test time Test time the largest rating count. It was observed that 1% of the
(Cosine) (Pearson) (Cosine) (Pearson) movies received more than 66 ratings. This information
Mean 4.62 4.84 4.04 4.06 was used to filter data with less significance for experi-
Std 0.05 0.07 0.05 0.06 mentation purposes. The movies with less than 50 ratings
and users who provided less than 100 ratings were exclud-
It should be noted that while the difference in time taken ed from the data to determine significant co-relations.
does not impact this experiment but in a practical imple- Another challenge was to determine the appropriate
mentation, with distributed systems, any time difference recommendations from the list of movie titles based on
can cause an exponential delay in delivering recommenda- correlation as well as k-NN. The model predicts the rec-
tions to a customer. The Pearson similarity is better, but the ommendations, but it is the designer’s responsibility to
designer of a recommender system should take into con- display the top recommendations rather than a complete
sideration the requirements of the customers and the com- list. If the user is presented with many recommendations,
pute time of the system to strike a balance between the then we circle back to the same problem we set to remedy
time taken and the quality of recommendations in a real- – to assist users in choosing items based on past prefer-
time scenario. It is often recommended to compute similar- ence. From a design perspective it was observed that if the
ity between users or items offline to save the computation number of ratings for a movie remained significantly
efforts. greater then the number of predictions made by the model
Some of the challenges encountered in evaluating per- were more accurate. In contrast, if the ratting count was
formance of the recommender system were: low, the predictions made were found to be insignificant
• Determining the appropriate combination of the and rather just a “list of movies”.
similarity index and value of k to make predic- During the prototyping phase of implementation, when
tions because both factors greatly impact error the complete dataset was not used, the system ran into
rates. “Cold Start” problem. Since the model heavily relies on
• Computing the value of k to reduce the error in explicit user ratings, there is not enough data during the
prediction. Greater values of k led to over-fitting start of the system to make predictions and it was observed
of the model and the predictions were insignifi- that the recommendations were in-accurate. Several solu-
cant. tions such as the Singular Value Decomposition, the Met-
ric Factorization and rating comparison strategy (Zhou &

7
Nadaf, Apr 2017) have been proposed to solve the “Cold
Start” problem. Another method is to use a Random Walk
with Restart along with the Recommendation Algorithm to
generate similarity of items (Sakarkar & Deshpande, May
2016). Zhou & Nadaf suggested Word Embedding, pro-
posed by Google in 2013, that uses a parameterized word
function to convert a sparse representation into a dense
representation. Zhou & Nadaf used the MovieLens dataset
to conduct experiments to test their proposed method, Em-
bedded Collaborative Filtering. Through their experiment
they observed that ECF outperforms the other methods in
“Cold Start” cases (Zhou & Nadaf, Apr 2017). The method
provided great potential to solve the “Cold-Start” problem
in applications with sparse data however does not prove
effective for systems with dense data.
Conclusion
In this paper, a system was implemented to make movie
recommendations to a user based on past ratings. The ma-
chine learning algorithm K-Nearest Neighbors was used
for the implementation. To implement the algorithm the
model computed a similarity between items. The model
then selects the k most similar items based on the user’s
past ratings. All predictions were computed using the
open-source Python machine learning libraries. The results
were compared and evaluated to determine the k value.
The evaluation was performed using Root Mean Square
Error, Mean Absolute Error and Fraction of Concordant
Pairs. The lesser the RMSE and MAE and the greater the
FCP, the better the results. It was observed that calculating
the similarities takes significant computation time and
memory and thus it is recommended to perform this task
offline. If the similarities are computed offline, the user
can be provided with results much faster. It was also ob-
served that the k-NN algorithm provided results with less
error rates when the training data increased. Thus, to pro-
vide predictions to new user it is recommended to use k-
NN in conjunction with other algorithms. To provide rec-
ommendations to a new user, it is also possible to use the
user’s preference data from social networking platforms,
but this should be done with the user’s consent to maintain
ethical standards in data sharing.
It was also observed that despite the availability of many
open-source libraries and packages, it is the designer’s
responsibility to configure the system to correctly provide
predictions. The designer should be familiar with the da-
tasets. The observations were not just limited to evaluating
the current algorithm, but some key findings were also
presented as a scope for future work. It is possible to
achieve better results by combining user behavior with
explicit user ratings. This can be done by combining K-
Nearest Algorithm with other machine learning algorithms
to provide better quality predictions and this is also one of
the key area of research in the industry.
References
Adomavicius, G., & Tuzhilin, A. (June 2005). Toward the next
generation of recommender systems: a survey of the state-of-the-
art and possible extensions. IEEE Transactions on Knowledge
and Data Engineering, 734–749. Retrieved from
https://ieeexplore-ieee-
org.ezproxy.library.tamu.edu/document/1423975
Breese, J., Heckerman, D., & Kadie, C. (July 1998). Empirical
analysis of predictive algorithms for collaborative filtering. Pro-
ceedings of the fourteenth conference on uncertainty in artificial
intelligence, Madison, Wisconsin.
Chena, M., & Liu, P. (Dec 2017). Performance Evaluation of
Recommender Systems. International Journal of Performability
Engineering, 1246-1256.
Géron, A. (2017). Hands-On Machine Learning with Scikit-Learn
and TensorFlow. Sebastopol, CA: O’Reilly Media, Inc.
Grouplens. (2018). Retrieved from https://grouplens.org/
Haruna, K., Akmar Ismail, M., Damiasih, D., Sutopo, J., &
Herawan, T. (Oct 2017). A collaborative approach for research
paper recommender system. PLoS ONE. Retrieved from
https://doi.org/10.1371/journal.pone.0184516
Herlocker, J. L., Konstan, J., Borchers, A., & Riedl, J. (1999). An
algorithmic framework for performing collaborative filtering. In
Proceedings of the 22Nd Annual International ACM SIGIR Con-
ference on Research and Development in Information Retrieval
(pp. 230-237). Berkeley, California, USA: ACM. Retrieved from
http://doi.acm.org/10.1145/312624.312682
Koren, Y., & Sill, J. (2013). Collaborative Filtering on Ordinal
User Feedback. Proceedings of the Twenty-Third International
Joint Conference on Artificial Intelligence, pp. 3022-3026. Re-
trieved from http://dl.acm.org/citation.cfm?id=2540128.2540570
Maheshwari, A. (2015). Data Analytics Made Accessible. Ama-
zon Digital Services LLC.
Melville, P., & Sindhwani, V. (April 2010). Recommender Sys-
tems. Encyclopedia of Machine Learning, 1.
Park, Y., Park, S., Jung, W., & Lee, S.-g. (May 2015). Reversed
CF: A fast collaborative filtering algorithm using a k-nearest
neighbor graph. ScienceDirect - Expert Systems with Applica-
tions, Volume 42, Issue 8, 4022-4028.
Phorasim, P., & Yu, L. (2017). Movies recommendation system
using collaborative filtering and k-means. International Journal
of Advanced Computer Research, Vol 7(29).
Portugal, I., Alencar, P., & Cowan, D. (May 2018). The Use of
Machine Learning Algorithms in Recommender Systems: A Sys-
tematic Review. Expert Systems with Applications, Volume 97,
205-227.
Recommender Systems. (2014). In G. Kembellec. John Wiley &
Sons, Incorporated. Retrieved from ProQuest Ebook Central,
https://ebookcentral-proquest-
com.ezproxy.library.tamu.edu/lib/tamucs/detail.action?docID=18
84200.
8
Resnick, P., Iacovou, N., Sushak, M., Bergstrom, P., & Reidl, J.
(1994a). GroupLens: An open architecture for collaborative filter-
ing. ACM Press New York, NY, USA.
Sakarkar, G., & Deshpande, S. (May 2016). Clustering Based
Approach to Overcome Cold Start Problem in Intelligent e-
Learning system. International Journal of Latesr Trends in Engi-
neering and Technology (IJLTET).
Zhou, Y., & Nadaf, A. (Apr 2017). Embedded Collaborative Fil-
tering for “Cold Start” Prediction. Q.I. Leap Analytics Inc.