OTC-29288-MS

Application of Data Science and Machine Learning Algorithms for ROP

Prediction: Turning Data into Knowledge

Christine Ikram Noshi and Jerome Jacob Schubert, Texas A&M University

Copyright 2019, Offshore Technology Conference

This paper was prepared for presentation at the Offshore Technology Conference held in Houston, Texas, USA, 6 – 9 May 2019.

This paper was selected for presentation by an OTC program committee following review of information contained in an abstract submitted by the author(s). Contents of
the paper have not been reviewed by the Offshore Technology Conference and are subject to correction by the author(s). The material does not necessarily reflect any
position of the Offshore Technology Conference, its officers, or members. Electronic reproduction, distribution, or storage of any part of this paper without the written
consent of the Offshore Technology Conference is prohibited. Permission to reproduce in print is restricted to an abstract of not more than 300 words; illustrations may
not be copied. The abstract must contain conspicuous acknowledgment of OTC copyright.

Abstract
A high ROP is considered one of the most sought-after targets when drilling a well. While physics-based
models determine the importance of drilling parameters, they fail to capture the extent or degree of influence
of the interplay of the different dynamic drilling features. Ensuring an adequate ROP while controlling the
tool face orientation is quite challenging. Nevertheless, its helps follow the planned well trajectory and
eliminates excessive doglegs that lead to unwanted wellbore deviations. Five different Machine Learning
algorithms were tested and trained on forty wells to optimize ROP and create a less tortuous borehole.
The collected data was cleaned, preprocessed and used to structure and train Random Forest, Artificial
Neural Networks, Support Vector Regression, K-Nearest Neighbor, and Gradient Boosting Machine and
the appropriate hyperparameters were selected. Parameters such as WOB, RPM, flowrate, MSE, bit run
distance, gamma ray for each rock formation in the data set were examined. A successful model was chosen
based a minimized deviation from planned trajectory, minimized tortuosity, and maximized ROP. A MAE
of 10% was achieved using Random Forest. The algorithms have demonstrated competence in the historical
dataset; accordingly, it will be further tested on blind data to serve as a real-time system for directional
drilling prediction and optimization to enable a fully automated system.

Introduction
The future of drilling operations can be summed up in an ever-increasing implementation of data-driven
modeling and its applications in predicting and optimizing highly uncertain downhole environments. With
various wellbore complexities exaggerating the overall well cost, it is now more imperative than ever to
achieve the highest possible rate of penetration (ROP). Although there are numerous Machine Learning
methods and approaches present, each algorithm has its particular limitations and leverages. Each dataset
and problem should be considered on a case by case basis. In essence, there is no universal objective
model for the entirety of conditions as the intricate correlations between the features themselves are quite
complicated. Generally, typical approaches fail in generating accurate ROP predictions given the added
complications of bottom-hole conditions. Conventionally, ROP optimization requires discovering the sweet
spot between weight on bit (WOB) and rotary speed (RPM) for effective drilling. However, ROP has a
complex correlation with a multitude of different attributes. In any drilling phase, ROP is the chief measured/
2 OTC-29288-MS

calculated output. There are several aspects controlling the process of drilling and ROP is chiefly dependent
on the subsequent parameters (Mantha and Samuel 2016):

• Drilling parameters (RPM, WOB, etc)

• Bit size and type (PDC, diamond impregnated, rollercone diamond hybrid)

• Type of vibration (lateral, torsional, axial, whirl)

• Properties of lithology (rock type, pressure gradient, compressive strength, etc.)

• BHA (PDM, RSS, turbine, and vertical control system)

• Hydraulic efficiency

• Mechanical efficiency

In several occasions, ROP decreases with increasing RPM and WOB. As a result, it is crucial to
understand the underlying complex interaction between these parameters.
A collection of field data was grouped from a multitude of different horizontal wells in this study. A
collective effort was made to group as much data as possible to perform comprehensive modelling for
predicting optimized ROP values. Several models were implemented and developed to analyze the dataset
based on statistical Regression and Machine Learning algorithms. The main objective of this study is to
improve the understanding of the different parameter interactions without limitations through leveraging
the predictive power of data analytics for ROP optimization at any given condition or scenario.

Exploratory Analysis
Exploratory data analysis was implemented on individual wells and data clusters to discover correlations
between the different parameters. Initially, data cleaning and processing was achieved in order to account
for missing values in the dataset. The rotary drilling phase was extracted from the entire dataset while
secluding NPT activities to clean the data set. It was apparent that the abnormally high logged values
could be destructive to model accuracy as they could be taking place as a result of measurement errors
or indicative of some potential incidents of stuck pipe, bit wear out, etc. Accordingly, those outliers were
excluded from the dataset. Outlier analysis was executed owing to the sizable sum of outliers for each
attribute. The first outlier layer was excluded after consultation with subject matter experts regarding model
construction variables including GR, ROP, RPM, and WOB. This process provided a number of insights
concerning data behavior. Feature engineering was implemented to segregate the important features from
irrelevant data columns without jeopardizing the impact of significant attributes. The following coupled
with the understanding of the physical behavior of the attributes contributed to the selection of eighteen
variables from the data set. Two groups of features were plotted against time to check if there is any specific
feature governing the response as shown in (Figs. 1-2). Principal component analysis (PCA) was performed
for dimensionality reduction purposes for some models to reduce the number of features from eighteen to
twelve. The analysis continued to measure predictor importance via feature selection to deduce the relative
weights and contribution of each attribute. Comparative feature importance discovered that amongst the
twelve features, block height governed the most, as shown in (Fig. 4). ROP shows a complex relationship
with motor differential pressure, torque, standpipe pressure, hook load.
OTC-29288-MS 3

Figure 1—A plot showing ROP, WOB, and RPM against time.

Figure 2—A plot showing motor differential pressure, block height, mud volume, GR, and annular pressure loss against time.

To detect the degree of correlation between the different attributes, that would later affect the models,
correlation matrices were performed. A Heat map showing eighteen variables with ROP showing their
different correlations is shown in (Fig. 3).
Using the entire data set, the correlations computed suggested that:

• Linearly speaking, there was no obvious trend amongst those different variables (hook load, block
height, motor torque, annular pressure loss, and mud volume against ROP
• Motor RPM strongly correlated with bit/hole depth

• Annular pressure loss correlated with standpipe pressure

• Motor RPM correlated with standpipe pressure and Annular pressure loss

• A direct relationship between ROP and rotary torque and RPM, motor differential pressure, and
standpipe pressure
• A negative correlation between ROP and Mechanical Specific Energy (MSE)

• Linear relationship between motor torque vs on-bottom differential

4 OTC-29288-MS

Figure 3—A Heat map showing 18 variables and ROP showing their correlation.

In this work, the modeling phase was designed to deduce how input features are organized as well as the
essential drilling and environmental features. The main factors for this study included: drilling parameters
WOB, RPM, bit RPM/torque, offset well ROP, bit/hole depth, block height, hook load, motor differential
pressure, standpipe pressure, mud volume, GR, annular pressure loss, downhole MSE, and motor RPM/
torque.
Fig. 5 shows the order of importance in descending order starting with block height, rotary RPM,
motor differential pressure, rotary torque, GR, annular pressure loss, bit depth, motor RPM, mud volume,
downhole MSE, WOB, motor torque, SPP, hole depth, and hook load. Rotary torque and GR seem almost
intuitive. But block height, motor differential pressure, mud volume, and annular pressure loss might not
seem obvious contributors and won’t necessarily be considered in a physics-based approach. According
to Random Forest, block height seems to be the most relevant contributor. This supports the claim that
numerous features in fact need to be exhaustively appraised for successful ROP enhancement.
OTC-29288-MS 5

Figure 4—Feature relative importance plot according to Random Forest.

Model Building and Results

After executing preprocessing and cleaning operations, the ultimate data set was prepared for the modeling
of Machine Learning techniques. Several models based on supervised algorithms and statistical regression
were intended to predict ROP. Diagnostic plot analysis discovered that modeling linear approaches will fail
to give good results with these datasets. Consequently, there was a requirement to implement non-linear
approaches and advanced ensemble techniques. Mostly non-linear algorithms were implemented on the
dataset. The study involved forty different wells, but the graphical representation is shown for only one
well. Training was done on 20% of the data set while the remaining 80% was used for testing for all the
different algorithms on the lateral section of the horizontal wells. Consequently, it is of crucial importance to
split the data conserving its reliability and resemblance in all data sets to prevent impartiality in appraising
varied algorithms while selecting the top ones. Two error metrices (R2 and MAE) were utilized to evaluate
the performance of different algorithms. The designed algorithms were verified on forty wells and was
revealed to forecast results with substantial accuracy. The algorithms developed in this study depend on
multiple predictor algorithms rather than a single model. To hinder model overtraining, cross-validation is
implemented. The models included SVR (Support Vector Regression), GBM (Gradient Boosting Machine),
Neural Networks, KNN (K-Nearest Neighbor), Ensemble methods such as Random Forests and Boosting
facilitated in enhanced accuracy and error reduction. Accordingly, the models presented here can be
efficiently implemented regardless of lithology type and location. Cross validation showing an accuracy
plot displaying R2 metric for all the different models for well 1 is illustrated in (Fig. 10).
The result of testing several models in contrast to a single example dataset is an approximation of how
different models act with respect to the problem against a selected degree of performance. If the entirety of
6 OTC-29288-MS

the different models fail to accomplish good results, it may be indicative of a structural absence present for
these models to acquire. Even though the following may happen owing to the absence of a definite learnable
arrangement in the selected data, it also provides a prospect to try multiple data alterations to construe the
structure to the learning models.
Regarding the generalization error, SRM describes an upper limit, thus, the topmost boundary for the
hyperplane regarding its training error, training sample number and model difficulty. SVR performance can
be enhanced by a procedure known as hyper parameter optimization. Via this method, the top features for
the model can be chosen (epsilon and a cost feature) to prevent overfitting. Typically grid search is chosen
for this task. Twelve predictors were utilized using SVR for ROP prediction. Fig. 5 represents the actual vs.
predicted ROP for well 1 with MAE of 0.19 and R2 of 0.72. The results improved for the majority of the
wells compared to Linear Regression models. SVR catches non-linearity better. However, the disadvantage
with SVR is its high computational time requirement.

Figure 5—Actual vs. Predicted ROP for well 1 using SVR model.

One of the straightforward Machine Learning techniques is KNN. This is an instance of case-based
learning, where novel data are categorized based on warehoused labeled occurrences. KNN is oblivious to
outliers which makes it robust to errors in the process of classification. The model’s effectiveness is largely
reliant on the quantity of nearest neighbors selected, in other words, defining the k value. Consequently,
tuning was attained by reiterating various k values to measure data performance outcome generalization
using the KNN model. Fig. 6 represents the actual vs. predicted ROP for well 1 with MAE of 0.19 and R2 of
0.76. Moreover, in instances of limited time and computational power, a square root of the k observations
value could be automatically used.
OTC-29288-MS 7

Figure 6—Actual vs. Predicted ROP for well 1 using KNN model.

Neural networks (NN) technique was performed on the training dataset. NN is a diagram of computational
nodes that accept input features and produce an output that passes through hidden layers. The attributes of
an input vector are associated to the variables of an output vector by assembling the nodes into different
layers (Moran et al. 2010; Noshi and Schubert 2018; Noshi et al. 2018). Commissioning a tune grid search
or varying neuron values manually can be performed. A loop for several neuron values was implemented
and then the optimized neurons with the lowest error metric (MAE and R2) was chosen. Modeling involved
all the possible obtainable input/factors. Several iterations were computed to create an association between
the different trends and features. Furthermore, this compounded progression minimalized the variances until
a practically matching output was achieved.
In comparison to KNN, GBM, SVR, NN did not perform well in predicting ROP. It would seem that ANN
modeling would yield an improved ROP outcomes compared to the other tested models given its robust
learning procedure. Nevertheless, enlarging the input attributes for ANN through learning complicates
modeling and augments ambiguity to the final outcome. Nevertheless, it performed better compared to the
Ridge regression model. Fig. 7 illustrates the learning outcome for ROP prediction by ANN by the red line,
while, the blue line represented actual measured ROP values. An R2 value of 0.59 and a MAE value of
0.49 were attained (a value of 1 is a perfect correlation and 0 is no correlation and a negative value is an
inverse response).

Figure 7—Actual vs. Predicted ROP for the correlation between predicted ROP (red) and actual ROP (blue) using ANN.
8 OTC-29288-MS

A popular ensembling technique to Decision Trees is the Random Forest model. Random Forest produces
multiple trees as an alternative to a single tree and permeates arbitrariness into each tree to generate a
forest of such distinct trees. In regression, the projected value is a weighted average of the value forecasted
by each individual tree. Random Forest is considered amongst the most substantial bagging ensemble
learning technique. The model works by creating a number of new training sets. Every new training set
chooses observation samples with proxy from the actual dataset. Accordingly, these models are grouped
using bootstrapped instances produced then grouping them by averaging the outcome in case of regression
problems i.e. ROP prediction. This model uses roughly two-third of the total training sample for evolving
each tree. The residual data samples are then alienated from the rest of the tree building process. Discovering
the comparative importance of each specific feature is another useful application for Random Forest, as
shown in Fig. 4. Random Forest also aids in recognizing the direction of impact of each variable on the
outcome. The Random Forest technique performed the best in comparison to the remainder of the models
and especially to regression. Given the intrinsic testing and cross-validation, the dissimilar trees help in
creating a great model.
The algorithm was implemented to test the drilling features for the remainder of the wells. The aim was to
examine the data to detect any obscure trends that could assist in improving bit life to optimize ROP values
and reduce MSE. The chief features under investigation were WOB, surface RPM, MSE, motor differential
pressure, torque, standpipe pressure, and hook load. The error metrics are similarly considerably small in
this model (R2 value of 0.88 and a MAE value of 0.10) and it performs very well in predicting actual ROP
values as displayed in (Fig. 8).

Figure 8—Actual vs. Predicted ROP using the Random Forest model for well 1.

The final technique executed in the work is GBM (Gradient Boosting Machine). In GBM, several basic
decision trees are formed, where each tree is constructed for error prediction of the preceding trees. As
soon as the initial tree is constructed, weighted trees are formed ensuing the determination of weight and
following iterations. The weighted sum of the "decisions" finalized by trees as a whole, comprises the final
model prediction.
The GBM model possesses a sizable sum of hyper-parameters to modify. Most imperatively are the
interaction depth, GBM iterations, and shrinkage value. GBM does a great prediction job; the error metrics
are the second least for GBM (An R2 value of 0.88 and a MAE value of 0.13) for the entire dataset. Visual
examination of the actual vs. predicted ROP plots shown in (Fig. 9) exhibited that GBM performed the
second best of the entirety of algorithms after Random Forest.
OTC-29288-MS 9

Figure 9—Actual vs. Predicted ROP using the GBM model for well 1.

A list of all algorithms implemented on well 1 were then implemented to the remainder of the wells.
Although, SVR was performing well on some wells, KNN, Random Forest, and GBM were applied owing
to the lower computational time required. The best modeling algorithms for all the wells were found to be
Random Forest, GBM, SVR, KNN, and NN. The accuracy of eleven models were represented in a Boxplot
on the entirety of the data set as shown in (Fig. 10).

Figure 10—Boxplot showing accuracy by comparing between eleven different models for all the wells.

Conclusion
This study was conducted to find an alternate solution to traditional models regarding ROP optimization
by leveraging the power of Machine Learning and predictive analytics. Considerable modeling efforts were
implemented to test the robustness of the best models developed. Several algorithms were developed and
implemented on a single well then tested on the rest of the data set. However, the results of five different
models are presented. Random Forest successfully estimated and optimize the drilling parameters with a
MAE of 10%. The accuracy of the models was tested using statistical regression and CART models to
compare accuracy and performance against traditional methods.
The following observations were concluded from this study:
10 OTC-29288-MS

1. ROP shows a compound interrelationship which cannot be expansively described by conventional

physics-based methods solely. The application of data-motivated predictions using Machine Learning
algorithms and regression analysis coupled with a hybrid approach for feature engineering can better
predict ROP.
2. Comparative feature importance discovered that amongst the eighteen features, block height governed
the most importance according to the RF model. ROP showed a complex relationship with motor
differential pressure, torque, standpipe pressure, hook load.
3. Modelling using comprehensive drilling data, formation GR, and survey data can better model
the complex relationship and uncover innate relationships among predictors, compared to using
conventional parameters such as RPM, WOB and RPM for modeling ROP.
4. The influence of parameters such as block height, differential pressure, mud volume, rotary torque,
GR, and annular pressure loss have a direct contribution in ROP optimization and bit wear reduction.
5. An ensemble of methods: GBM and Random Forest helped achieve the best prediction with the least
error metric for the dataset. Algorithms such as KNN and SVR also performed well and can be used
if there is a constraint on computing capabilities.

References
Noshi, C. and Schubert, J.J. 2018. The Role of Machine Learning in Drilling Operations; A Review. Presented at the SPE/
AAPG Eastern Regional Meeting, 7–11 October, Pittsburgh, Pennsylvania, USA. SPE-191823-18ERM-MS. https://
doi.org/10.2118/191823-18ERM-MS.
Noshi, C. I., Assem, A. I., Schubert, J. J. 2018. The Role of Big Data Analytics in Exploration and Production: A Review of
Benefits and Applications. Presented at the SPE International Heavy Oil Conference and Exhibition, 10–12 December,
Kuwait City, Kuwait. SPE-193776-MS. https://doi.org/10.2118/193776-MS.
Noshi, C. I., S. F. Noynaert, Schubert, J.J. 2018a. Casing Failure Data Analytics: A Novel Data Mining Approach
in Predicting Casing Failures for Improved Drilling Performance and Production Optimization. Presented at the
SPE Annual Technical Conference and Exhibition, 24–26 September, Dallas, Texas, USA. SPE-191570-MS. https://
doi.org/10.2118/191570-MS.
Noshi, C. I., S. F. Noynaert, Schubert, J.J. 2018b. Failure Predictive Analytics Using Data Mining: How to Predict
Unforeseen Casing Failures? Presented at the Abu Dhabi International Petroleum Exhibition & Conference, 12–15
November, Abu Dhabi, UAE. SPE-193194-MS.https://doi.org/10.2118/193194-MS.
Noshi, C. and Schubert, J.J. 2019. Natural Language Processing and Text Mining Applications in Drilling. IPTC-19382-
MS. Presented at the International Petroleum Technology Conference. 26–28 Mar 2019. Beijing, China.
Mantha, B. and Samuel, R. 2016. ROP Optimization Using Artificial Intelligence Techniques with Statistical Regression
Coupling. SPE Annual Technical Conference and Exhibition., 26–28 September, Dubai, UAE. SPE-181382-MS.
https://doi.org/10.2118/181382-MS.
Moran, D. P., Ibrahim, H. F., Purwanto, A. et al. 2010. Presented at the IADC/SPE Asia Pacific Drilling
Technology Conference and Exhibition, 1–3 November, Ho Chi Minh City, Vietnam. SPE-132010-MS. https://
doi.org/10.2118/132010-MS.