Академический Документы
Профессиональный Документы
Культура Документы
Big data and analytics are top IT initiatives for oil and gas companies in 2017. These initiatives are
especially important as oil and gas companies strive to further reduce their operating costs through
interconnecting systems for access and to manage the vast amounts of data generated in the oil field.
Companies must exploit this data to gain deeper insight into their operations and performance.
There are key advantages to leveraging analytics that will benefit C-level executives. By giving the
CEO, COO, CFO, and CIO immediate, accurate, and relevant information about operations
performance, these executives gain the ability to make informed decisions.
There are multiple types of analytics including BI, descriptive, predictive, and more advanced analytics
such as cognitive/augmented reality-based human-assisted remote sensing and inspection. The
objectives of augmented reality analytics are to assist humans in the field remotely to perform data
collection and inspection tasks using multimodal augmented reality and gamification.
This IDC Energy Insights White Paper explores best practices, approaches, challenges, and expected
benefits of using analytics to tap big data to solve complicated problems. Analytics can be used in a
variety of ways to improve efficiencies; for the oil and gas industry today, upstream operations and
production are two high-priority use cases for analytics.
SITUATION OVERVIEW
The advent of low energy prices worldwide over the past couple of years has been a catalyst for oil
and gas companies to reinvent themselves to become operationally efficient and effective. When oil
prices started to plummet, energy companies could no longer just drill another well and count on the
profits to support the high costs of E&P. Initially, oil and gas companies responded by cutting staff,
delaying projects, and renegotiating contracts with vendors to rapidly lower costs.
Few oil companies can maintain profitability given today's low energy prices. Most companies must be
more efficient, particularly if they have already made cuts and renegotiated agreements. To
compensate for staff reductions, oil and gas companies are looking to quickly automate processes
wherever it makes good sense.
Companies are adding new devices and software in the oil field to facilitate automating some activities,
but value will not be realized until the associated data is collected and combined with other relevant
While only a small number of companies have demonstrated maturity with their use of analytics, IDC
Energy Insights believes that big data and analytics are now mainstream in oil and gas: 16% reported
enterprise installations, and others are testing high-potential pilots for impact and returns. This is fairly
consistent with other industry averages, and the number of enterprise installation is growing rapidly.
For the most part, companies are using analytics on a limited basis, but the use is growing fast as the
benefits of analytics are realized.
As oil and gas companies seriously invest in analytics solutions, many need a strategy and an
architecture to help them make the transformation to the 3rd Platform. Companies often require the
expertise of third parties, especially during times of low oil prices and staff reductions. It is important to
work with professionals that possess deep oil and gas domain knowledge and technical expertise to
help energy companies identify and develop top analytics opportunities and solutions.
Ongoing business conditions in the energy industry are pushing companies to more rapidly adopt
innovative cloud and analytics technologies. Adopting these technologies often requires help from
trusted partners to deploy innovative solutions so companies can become more efficient, agile, and
capable of responding to change.
One of the main goals of analytics is to help companies leverage the vast amounts of data that are
continuously generated in the oil field. This data is critical to gain deeper insight into optimizing
operations performance as well as improve productivity and achieve other positive outcomes such as
adhering to compliance requirements and minimizing risk.
Big data and analytics encompass a broad set of initiatives — it is important therefore to first determine
and prioritize what areas of the business will benefit the most from analytics. It is also important to
identify which initiatives can be quickly deployed to achieve immediate value.
Oil and gas companies plan to increase IT spending in 2017 — most notably in upstream operations in
both production and drilling. As companies digitally transform their businesses and cloud, big data, and
analytics all come together with mobility applications, innovative technology initiatives are made
possible. The resulting integrated IT environment will be developed incrementally over time and will
connect internal and external partners, customers, mineral rights owners, vendors, suppliers, and
others into an ecosystem to provide a holistic view of upstream operations.
It is this kind of environment in which analytics can have the most impact. Oil and gas companies
reported that they have three main purposes for enabling analytics: to help optimize processes, to
allocate resources more effectively, and to predict outcomes. When selecting business objectives for
analytics, it is helpful to prioritize each objective in terms of how they will be impacted by analytics.
For example, if a company selects production optimization as the top objective, it is important to
understand how analytics will impact the processes and resource allocations involved. It is also
necessary to predict the expected outcomes to determine the potential value. It is advisable for
companies to study their IT environments for how and what type of analytics can be applied to realize
the optimal results.
Keep in mind, however, that the decision of where to begin with a big data and analytics initiative
depends on each company's strategy; drilling or another function may be more important than
production at some companies.
It is imperative to first understand the purpose of the analytics and the expected outcomes to be able
to identify the requisite data and determine the integration requirements — integration is often the
biggest challenge. The technologies and approach will depend on the type of data to be analyzed, as
well as the volumes, variety, velocities, reporting tools, and the accuracy required.
Solutions can range from using a relational database for managing critical structured data for analytics
to using newer and more open technologies like Hadoop and other similar technologies for managing
high volumes of unstructured data. The implications of each approach can be significant as techniques
will vary — for example, using master data management for detailed accuracy or, alternatively,
sampling large amounts of data if that is more appropriate. The most effective design, software, and
road map are best determined by a DX or data architect and a competent team of IT and oil and gas
business professionals.
Some of the promising areas where analytics can benefit oil and gas companies achieve operations'
optimization include drilling and production as the highest priorities. Production is the top LOB in terms
of budget increases for 2017 followed by drilling. Analytics is also useful in finding oil and in horizontal
ways like improving processes, supply chain, and other functional ways, but most of the IT budget will
be for improving production efficiencies and outcomes.
Most companies reported they expect that analytics will continuously help improve processes and
optimize output performance somewhere around 8-30% according to some studies. Analytics will also
be applied to help solve safety issues and mitigate risk as well as improving production and reservoir
performance.
Each company has its own approach for producing hydrocarbons, and analytics is helpful for solving
problems, determining best practices, and predicting failures before they occur. Several oil and gas
companies are leveraging analytics to monitor the health of equipment and devices. This use of
analytics is valuable for predicting potential downtime and even predicting an impending event to take
remedial action before it occurs. Upstream asset management is based on this concept of using
analytics to monitor the health of the devices in the oil field; some companies have integrated AI for
deeper learning and better decision making.
Integrating the requisite data is the most difficult problem for most operators. There are different
architectures to be explored as to which one is best suited for the objectives in consideration. This is
an arduous task and why it is important to set achievable metrics, as data from multiple systems must
be cleansed, formatted, and aligned with workflows to be meaningful.
As an example, consider optimizing the production of 3,500 wells, all of which are in the same
geographic region. Some wells are larger, some are deeper, and usually there are multiple different
systems involved with various formats that must be addressed to manage and provide meaningful data
to apply descriptive and predictive analytics about production performance.
Collecting near-real-time time series data is complicated and is usually collected in the operations’
data historian software, and this same data can even be taken directly from the device, sensors, and
other instruments and equipment with meters. This data is used to take a snapshot for a current status
of well performance, which is valuable by itself for making decisions. In addition to meter data, other
systems data can also be included for different perspectives such as financial, maintenance,
accounting, land, and GIS systems — all of which provide a more holistic view of production operations.
There are a host of automated interfaces and visualization tools and applications that companies can
use to monitor, manage, and apply logic, algorithms, and techniques to convert and present data in the
desired format for interpretation and understanding. For example, analytics can be applied ad hoc or in
the form of KPI-driven dashboards. Many companies like the dashboard approach for running
upstream operations as dashboard data can be color coded to easily depict potential and urgent
problems that are violating KPI thresholds. Some operations dashboards also have drilldown
capabilities to quickly assess the underlying problems, thereby enabling companies to take some
positive action to resolve them.
Top-producing well
Least producing well
AFE budget costs by well and exception-based reporting
Number of failures of each well
Types of failures ranked by dollar impact or downtime
Mean time between failures and resolution by well
Equipment and device reliability for each well
Aggregate information about each well should also be an option as analytics results can be rolled up
by field, location, and project. Individual wells can be rolled up with other wells to be optionally
presented by day, week, month, and other time frames.
The kind of analytics tools companies use depends on the desired results, but it is always critical to
first make sure there is access to accurate data. Some other analytical tools and presentation
applications include ad hoc queries, report cards, advanced modeling, simulation, 3D virtual reality,
cognitive, and human language processing, and even AI is being added for rapid analysis of trends
and patterns.
Analytics is a powerful tool for engineers to examine on a regular basis and collaborate about any
changes in the IT environment that need to be addressed to optimize operations performance.
One major oil company deploys analytics every morning to look at activity performance and determine
if wells are performing to expectations. In this manner, multiple professionals can be included in the
discussions to decide what the problems are, what action should be taken, and the predicted outcomes.
In the future, the COO may want to expand analytics to include data from other systems to improve
operations performance. For this reason, a data management framework approach is ideal for
establishing standards and governance and developing an easy-to-use interface for adding new
applications data, including data from third parties.
Drilling Analytics
According to IDC Energy Insights research, companies are planning to increase IT spending related to
drilling significantly in 2017. Analytics can play an important role in drilling-related activities.
Companies primarily use drilling analytics to monitor, manage, and analyze critical drilling data to help
set benchmarks for gauging performance as well as test the impact of using different technologies and
methods. Benchmarks can be set that meet expectations about length of time to drill a well on one well
pad and time to move to the next site to spud.
Other metrics can also be set to help companies manage expectations and determine best practices.
Analytics for drilling will play a large part in helping companies select the ideal drilling equipment,
resources, services partners, and other conditions to improve operations performance.
Information from other applications can also add value to analytics. For example, HR data about
employees can be combined with analytics to help select those workers with the most appropriate
skills and experience for specific drilling projects. Including financial data with drilling or production
operations provides an immediate view of the AFE budget status.
FIGURE 1
Source: IDC's Oil and Gas Flash Poll Study, November 2016
There are multiple benefits for implementing analytics to help run upstream oil and gas operations. An
integrated IT production framework will help achieve a company's vision of operations optimization by
connecting all the requisite systems and managing the critical data.
Analytics will provide the flexibility to transform organizational behavior and help identify and enable
the best resources to improve activity performance across the ecosystem from drilling and production
to distribution and beyond. With a comprehensive analytics strategy, companies can help to reduce
operations costs and improve performance as well as ultimately improve productivity and profitability.
Applying analytics to real-time SCADA and process control systems to help operators optimize
resource allocation and prices by making more real-time decisions with fewer engineers
Being able to predict well performance based on historical results
Determining sub-par production zones to shift assets to more productive areas
Providing guidance for fracking and making changes in lifting methods
CHALLENGES
Advancements in technologies in the past few years have exceeded most energy companies' ability to
keep up. Companies are now faced with prioritizing budgets based on keeping the old systems running
while investing in innovation. For example, new applications, devices, and equipment continue to
grow, as do the number of sensors. The CIO must keep up with technology while transforming IT to the
3rd Platform at the same time.
Today, companies face several challenges: They strive to become tightly integrated, to implement
automated processes, and to rapidly adapt to change — all while maintaining operational excellence.
Developing an IT environment to support these goals requires a business plan with clear objectives
and metrics.
TCS works with each oil and gas client to understand and prioritize its key business objectives to
develop an analytics environment that best meets the client's vision of how it wants to apply analytics.
It is critical for organizations to leverage professionals with domain and business expertise of this
caliber and an understanding of the nuances and complexities involved. This knowledge is essential to
know what data is needed to integrate multiple disparate systems into a holistic analytics environment
that provides a 360-degree view of oil field operations.
One end goal is to help an operator drive business agility by enabling faster and smarter business
decisions and sustained business outcomes. TCS has developed ValueBPS, which is a special
approach that enables an enterprise to achieve significant and sustained business outcomes by
leveraging the team's deep domain expertise and operations redesign methodologies.
The TCS approach also encompasses robotic process automation (RPA), analytics and insights,
business process as a service (BPaaS), and business process management (BPM). TCS deploys
oil and gas-specific analytics services, many of which are proprietary models, that are aligned to meet the
client's end-to-end needs in the upstream, midstream, and downstream that span the entire analytics value
chain including data management, reporting, descriptive analytics, predictive modeling, and optimization.
In 2014, most oil and gas companies reported limited experience with big data and analytics. Today,
oil and gas companies have escalated the adoption curve. A majority of companies are working on
developing a solid data management framework environment for deploying analytics, while less than
half of companies believe they already have sufficient data management frameworks in place to
deploy analytics.
With the advent of drones, the IoT, and continued adoption of 3rd Platform technologies such as cloud
and mobile, there will be additional volumes of data available. An analytics framework will become the
hub for the adoption of such new technologies and approaches to rapidly adapt to changing
environments. Hence the need for a solid analytics platform and strategy will only grow over time.
ESSENTIAL GUIDANCE
Analytics is a top IT initiative for oil and gas companies, and IDC Energy Insights recommends gaining
active support from leadership due to the strategic nature and high impact analytics will have on the
culture and business workflows. Companies need to realize that data is a valuable asset, but only if it can
be used to solve some critical problem or enable a new capability that realizes value for the company.
This is why it is so important to first identify the business opportunity that data can help solve.
IDC Energy Insights advises companies to demonstrate value to the executives and other
stakeholders every three months by detailing progress and discussing lessons learned. One goal of
this exercise is to develop a framework for adding applications and data to be integrated over the next
one to five years. Over time, this framework approach will grow to connect with other systems and
sources of data to become the big data and analytics platform of the future.
Clear governance backed by top levels of the organization is required to ensure that analytics
becomes an integral part of the corporate culture. In the absence of corporate governance, business
units will work at cross-purposes or may not even use an analytics system. Governance percolates
throughout the organization, supporting consistent approaches at each level — a critical component to
support the alignment of IT with business objectives across all departments. It is crucial for each
department head to collaborate on system design and operations among different business units.
CONCLUSION
IDC Energy Insights recommends that oil and gas companies identify and configure their team of
internal and/or external experts before tackling analytics solutions. A competent team of professionals
is required with the business and technical knowledge and skills to identify, prioritize, and select the
key objectives for analytics and a technical architecture of the solution. The team should include
professionals with oil and gas domain and application knowledge and the requisite skills to develop an
integrated frameworks solution.
Analytics is a top IT initiative for oil and gas companies, and IDC Energy Insights recommends gaining
active support from leadership because of the strategic nature and high impact it will have on the
culture and business workflows in the company. It is wise to assess the current IT environment and
leverage what is already there before making investments in new technologies.
Professional services companies are excellent sources for domain knowledge and good digital
transformation consulting knowledge as well to help in other critical areas like determining strategy,
architecture design, process refinement, and development. Each company will need to assess what
resources are required to successfully deploy an analytics initiative and determine what roles to
outsource.
IDC Energy Insights advises analytics initiatives should be led by a data architect expert with oil and gas
knowledge and skills who can collaborate with team members to identify clear objectives and help design
and develop the underlying technical solution. One goal is to develop a framework for adding applications
and data to be integrated over the next one to five years. Over time, this framework approach will grow to
connect with other systems and sources of data to become the big data and analytics platform of the future.
Global Headquarters
5 Speen Street
Framingham, MA 01701
USA
508.872.8200
Twitter: @IDC
idc-community.com
www.idc.com
Copyright Notice
External Publication of IDC Information and Data — Any IDC information that is to be used in advertising, press
releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or
Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right
to deny approval of external usage for any reason.