G O V E R N A N C E

CONTINUOUS
OVERSIGHT IN
THE CLOUD
How to Improve Cloud Security, Privacy and
Compliance
2 CONTINUOUS OVERSIGHT IN THE CLOUD

CONTENTS
4 Today’s Technology Landscape:
Continuous Oversight for Continuous
Evolution
6 / New Processing Realities Require a
Paradigm Shift in Long-Standing Practices
7 Benefits of Continuous Oversight
8 Strategy, Challenges and Execution
9 / Strategies for Identifying and
Mitigating Risk in the Cloud
9 / Get Started With Fundamentals
10 / Define Strategy, Assign
Responsibilities and Take Action
12 / Maintain a Continuous Cloud Service
Assurance and Oversight Program
13 / Capture the Right Metrics
13 / Key Supply Chain Metrics
14 / Key Incident and Breach Metrics
14 / Example Metrics for Common
Challenges
15 Call to Action
16 Acknowledgments

© 2019 ISACA. All Rights Reserved.

3 CONTINUOUS OVERSIGHT IN THE CLOUD

ABSTRACT
Many emerging technologies and practices—including artificial intelligence (AI), big data
analytics, Internet of Things (IoT) devices and third-party services—directly or indirectly
access a wide variety of cloud services. Today’s dense hyperconnectivity not only
envelops products and consumers, but also links enterprises and business processes in
ways that often erase traditional boundaries between internal and external domains.
Cloud computing services are increasingly mediating the connections. In this
environment, information assurance professionals must not only address longstanding
information security threats and vulnerabilities, but they also face new challenges relative
to their experience in the field just a few years—or even months—before.

This white paper advocates for continuous oversight of the wide variety of cloud services
used by organizations—a set of distinct, but related, management and assurance
practices that address critical emerging risk domains, including security, privacy and
compliance. Continuous oversight includes:

• Continuous assurance for data and processes, including continuous monitoring,

continuous risk awareness and continuous compliance throughout the full data life
cycle. The life cycle begins from the time data leaves the organization, through to
when data are deleted or otherwise destroyed. These processes align with
requirements for privacy and data protection, and support meeting the associated
assurance activities’ target maturity levels.

• Continuous cloud assurance tailored appropriately for each type of service being
provided, and for the associated type (private, community, public or hybrid) of cloud.
This ensures risk is being appropriately addressed, and compliance obligations are
being met.

• Continuous supply chain management and oversight for cloud vendors, and their
subcontractors. Ensuring security, privacy and compliance activities are monitored in
addition to associated processing environments.

• Continuous improvement (CI) to ensure that cloud services’ security, privacy and
compliance activities remain relevant and effective, to provide risk and maturity level
metrics, and to indicate where improvements are necessary.

© 2019 ISACA. All Rights Reserved.

 4 CONTINUOUS OVERSIGHT IN THE CLOUD

Today’s Technology Landscape:

Continuous Oversight for
Continuous Evolution
Even before facing current regulatory requirements, • Twenty percent of organizations have experienced a cloud

enterprises of all sizes struggled to address information incident in the past year. Absence of documented policies,

security management effectively. In the late 1990s to early standards and supporting procedures for information

2000s, legislatures and other authorities worldwide security, privacy and compliance management across all

enacted new laws and regulations. At the same time, types of cloud environments can result in ad hoc or

novel technologies emerged that further complicated the uninformed assurance activities, which, in turn, may

management of information security and privacy conflict with other enterprise policies and procedures.

programs. Today, on top of accumulated regulatory and • Bring your own device (BYOD)—It is becoming the norm (and

technology challenges, enterprises face several tectonic no longer an exception) for employees to use their own devices

shifts in the digital business landscape including: when performing business activities. Eighty-seven percent of

organizations allow employees to use personal devices to

• Cloud computing—In 2017, cloud computing leaped forward as
access business applications, perform processes or work with
cloud service providers (CSPs) like Amazon Web Services
data files, and 64 percent of employees use personal devices
(AWS)1 (followed soon after by Google® and Microsoft®) offered
1

for work regardless of whether there is a policy in place or not;

next generation cloud-computing capabilities, including new
nevertheless only 59 percent of organizations have a formal
data storage options and processing capacity. Cloud services
BYOD policy in place.3 These shadow IT4 devices likely use
3 4

contracted to support business activities and made part of the

more than one type of cloud service, which may or may not
business environment—but used outside of the direct control of
have proper security controls in place. The risk created by lack
the enterprise’s IT department—must be managed to mitigate
of insights for users of BYODs were reported in a recent
information security and privacy risk, which varies across
research report5 which included the following startling findings:
5

different kinds of cloud environments. A few findings from a

• Forty-three percent of organizations do not know if BYODs
recent report2 highlight the challenges facing organizations
2

accessing corporate data have downloaded malware.

while adequately addressing the new risk that cloud computing
• Fifty-one percent of organizations believe the number of
introduces to organizations.
threats targeting mobile devices have increased in the
• Sixty-five percent of IT professionals still underestimate
past year.
the damage cyberattacks against cloud-based targets can
• Twenty percent of organizations lack visibility into basic,
cause.
native mobile apps (like email) on personal devices.
• Only 30 percent of respondents affirmed security was the
• Big data analytics—Enterprises are increasingly migrating big
responsibility primarily of the cloud provider.
data analytics to public clouds and creating more proprietary in-

1
1
IT Svit, “New AWS Tech Introduced During AWS Re:Invent 2017,” 12 May 2017, https://itsvit.com/blog/new-aws-tech-introduced-aws-reinvent-2017
2
2
Check Point, “The 2019 Security Report,” https://www.cloudcomputing-news.net/news/2019/feb/22/check-point-exposes-yet-more-shared-
responsibility-misunderstandings-cloud-security/
3
3
Syntonic Research, “BYOD Policy Requirements For a Secure BYOD Environment,” 18 March 2019, https://solutions.pcmcanada.com/byod-policy-
requirements-for-a-secure-byod-environment
4
4
Shadow IT includes applications or devices used by employees but not approved/implemented by the enterprise. For more on shadow IT, see ISACA,
“Shadow IT Primer,” http://www.isaca.org/knowledge-center/research/researchdeliverables/pages/shadow-it-primer.aspx
5
5
Bitglass, “Mission Impossible: Securing BYOD,” November 2018, https://pages.bitglass.com/MissionImpossibleSecuringBYOD_LP.html

© 2019 ISACA. All Rights Reserved.

 5 CONTINUOUS OVERSIGHT IN THE CLOUD

house applications for supporting and integrating those the US Congress its third bill on IoT security in March 2019.10 10

analytics within the business environment. Big data analytics Enterprises must consider the complete risk environment when
6
produces three exabytes of data per day. This data can be
6
addressing IoT devices.

overwhelmingly sensitive for the business, and often qualifies • Artificial intelligence (AI)—Investments in AI have drastically

as personal data as well. Organizations are also increasingly accelerated since 2016.11 New and emerging AI, including
11

12
using big data security analytics as an integral part of security machine learning (ML), 12
predictive analytics13 and deep
13

14
control decision-making. However, such use creates additional learning, 14
all pose complex security challenges and raise

security and privacy issues as seen in a recent survey that privacy concerns that have not existed to date. Because of

found the single biggest obstacle when using or planning to use these new security and privacy challenges, there are new

security analytics solutions for 68 percent of organizations, considerations information assurance professionals must

especially those bared in the European Union, was data privacy evaluate, in addition to their current responsibilities. Many
7
and security. Another consideration is that an enterprise that
7
information assurance professionals will need a plan to deal

deploys a big data infrastructure in one vendor’s public cloud with these challenges effectively and cost efficiently since their
may discover that the implementation does not meet enterprise budgets do not usually increase in light of risk associated with

risk standards or privacy requirements—or may simply find that these new technologies. Consider the following example which

the price increase is prohibitive—and thus, a new service illustrates why these new challenges must be addressed.

provider must be sought. In this context, the security of cloud Software developers increasingly use AI in applications that

services and associated applications—including data affect the lives and livelihoods of consumers and workers

confidentiality, integrity, availability and portability—are all throughout a wide range of populations. The developers need to
8
constant concerns. 8
ensure the AI algorithms used within those applications have

• Internet of Things (IoT) devices—IoT devices are becoming enough accuracy, that the integrity of the calculations will be

ubiquitous in business environments: 20.4 billion IoT devices preserved, and that resulting privacy risks will be successfully
9
will be in use by 2020. IoT devices and cloud services are
9
identified. However, without guidance from information

inherently related, because most IoT devices are designed as assurance professionals, these important actions will likely not

end points that collect data for processing elsewhere, usually in be performed.

cloud processing centers. IoT end points and cloud processing Alongside new technology challenges, information
may mutually compound security risk and the complexity of assurance professionals commonly inherit security and
assurance, because most IoT devices and apps are not compliance issues associated with legacy systems and
sufficiently secure, if secured at all. US lawmakers introduced to decades-old personal data—sometimes stored in

6
6
SecurityScorecard, “Looking Ahead to RSAC 2019: How Can We Work for BETTER Privacy and Security?,” 1 March 2019,
https://securityscorecard.com/blog/looking-ahead-to-rsac-2019
7
7
KuppingerCole, “Big Data and Information Security: How Big Data Technology Can Help in Increasing Cyber Attack Resilience by Better Detection of
Attacks, Enabling Real Time Response,” 2016, https://bi-survey.com/security-big-data-challenges
8
8
Data privacy and security in the cloud reflect the greatest obstacles for security analytics solutions, according to 68 percent of enterprises responding
to a recent big data security analytics survey. The response is especially characteristic of enterprises in the European Union (EU). See KuppingerCole,
“Big Data and Information Security: How Big Data Technology Can Help in Increasing Cyber Attack Resilience by Better Detection of Attacks, Enabling
Real Time Response,” 2016, https://bi-survey.com/security-big-data-challenges
9
9
Boufis, Eli; “Profit, Peril And The Internet Of Things,” Forbes, 2 January 2019, https://www.forbes.com/sites/eliboufis/2019/01/02/profit-peril-and-the-
internet-of-things
10
10
Lemos, Robert; “New IoT Security Bill: Third Time’s the Charm?,” DarkReading, 18 March 2019, https://www.darkreading.com/iot/new-iot-security-bill-
third-times-the-charm/d/d-id/1334190
11
11
Organisation for Economic Co-operation and Development (OECD), “Private Equity Investment in Artificial Intelligence,” OECD Going Digital Policy Note,
December 2018, http://www.oecd.org/going-digital/ai/private-equity-investment-in-artificial-intelligence.pdf
12
12
According to ISACA, “Machine learning, which includes predictive analytics, covers cognitive systems that go beyond big data analytics.” See ISACA,
“Machine Learning Drives Big Business Benefits,” 2015, http://www.isaca.org/Knowledge-
Center/Research/Documents/machine_whp_eng_0615.pdf?regnum=492717.
13
13
According to SAS Insights, “Predictive analytics is the use of data, statistical algorithms and machine learning techniques to identify the likelihood of
future outcomes based on historical data. The goal is to go beyond knowing what has happened to providing a best assessment of what will happen in
the future.” See SAS Institute, Inc., “Predictive Analytics: What it is and Why it Matters,” https://www.sas.com/en_us/insights/analytics/predictive-
analytics.html.
14
14
According to Bernard Marr, “Deep learning is a subset of machine learning where artificial neural networks, algorithms inspired by the human brain, learn
from large amounts of data.” See Marr, Bernard; “What Is Deep Learning AI? A Simple Guide With 8 Practical Examples,” Forbes, 1 October 2018,
https://www.forbes.com/sites/bernardmarr/2018/10/01/what-is-deep-learning-ai-a-simple-guide-with-8-practical-examples/#385f2948d4ba.

© 2019 ISACA. All Rights Reserved.

 6 CONTINUOUS OVERSIGHT IN THE CLOUD
outdated, disparate legacy media, across sundry
geographic locations—all of which still need to be
maintained, protected or migrated. These factors
combined magnify into much larger and complex risk than
have ever existed before.
Once the initial goal of establishing information security
and privacy controls and processes, and meeting all
applicable legal requirements for security and privacy
compliance has been met, information assurance
professionals must continue to maintain maturity levels
on an ongoing basis, and perform the actions necessary
to reach the targeted maturity levels15 for their 15
information security and privacy compliance programs.
New Processing Realities
Require a Paradigm Shift in
Long-Standing Practices
The complexity of current business processing
environments requires rethinking the longtime tradition of
having one central set of security, privacy and compliance
policies and procedures that apply generally to all
applications, systems, networks and data. Certainly,
having an overarching set of high-level corporate rules to
establish an overall security, privacy and compliance
framework is necessary. However, traditional governance
documents must be expanded to cover multiple, and often
hybrid, systems in multiple locations, that are typically
managed by multiple entities.
A growing number of enterprises collect, process, store
and access data within cloud services managed internally,
but owned by other entities. These environments are
accessed by a wide range of mobile apps, with significant
amounts of data collected, processed and stored in an
endless number of possible end points, including worker-
owned devices that are likely vulnerable in many ways. In
this context, organizations should update their
15
15
For more information about maturity levels, see CMMI V2.0 at https://cmmiinstitute.com/cmmi.
© 2019 ISACA. All Rights Reserved.
information security, privacy and compliance
management strategies to support continuous
monitoring, assurance and compliance that aligns with
associated goals to support the new processing realities.
Factors in the general business environment can place
additional strain on legacy technology and business
practices, not only for large organizations spanning
multiple countries, but also for small and medium
enterprises. These create major challenges, some of
which include:
• Cloud security and privacy risk—Lack of understanding about
the unique types of cloud security and privacy risk, along with a
lack of insight of the full scope of issues involved, result in an
absence of clarity for determining the appropriate roles and
teams to be accountable for the various necessary actions to
support effective cloud information assurance activities. As a
result, key responsibilities are often not assigned.
• Budget constraints—Tight budgets may complicate efforts to
acquire sufficient, effective cloud security, privacy and
compliance resources. This means critical supporting tools may
not be acquired and implemented appropriately.
• Acquiring and retaining IT talent—After gaining experience in
small and medium-sized enterprises, IT resources may leave for
larger organizations that provide better salaries and/or benefits.
Even within larger organizations, entry-level resources may find
opportunities with other organizations after gaining experience.
Many organizations are simply left with open positions or less
experienced workers to mitigate complex risk environments.
Taken together, all the foregoing factors contribute to
continuous evolution in technology, regulatory and
business environments—especially considering the
overarching trend toward using cloud computing services.
New levels of enterprise risk demand an equally broad,
comprehensive and holistic response: continuous
oversight for continuous evolution.
 7 CONTINUOUS OVERSIGHT IN THE CLOUD

Benefits of Continuous Oversight

Continuous oversight and monitoring can provide visibility continuously monitored, using meaningful metrics that surface

into current security, privacy and compliance levels, new risk and compliance issues and provide insights for

through the use of a risk-benefit analysis. Continuous improvement.

oversight and monitoring also lay the groundwork for real- Business leaders often get lost in debates about
time metrics and facilitate effective maintenance of information security, privacy and compliance
16
ongoing security and privacy management. 16
The requirements, cyberattacks, privacy breaches,
following oversight and monitoring activities, when management frameworks, legal requirements, acceptable
overseen by a technical-minded governing board, and are assessment and controls, risk management
applicable to all types and sizes of organizations: responsibilities, metrics—and a laundry list of other
• Continuous internal monitoring—Continuous monitoring issues.
throughout the data life cycle supports continuous awareness
Amidst the ongoing debates, business leaders sometimes
of risk levels at any point in time and allows for the most
forget why information security and privacy ultimately
expedient response to help ensure continuous compliance and
matter. Focusing on the benefits of continuous oversight
effective risk management.
can help remind leaders that information security, privacy
• Continuous external cloud assurance—Enterprises cannot
and compliance assurance supports the overall success
assume that external cloud services meet all their information
of the business—and thus may help alleviate many
security, privacy and compliance requirements. Historically, and
recurring debates.
to date, such assumptions have blindsided organizations and

have led to breaches.17 Using continuous assurance practices

17
Among their many benefits for the business, continuous

for external cloud services will help provide timely insights to oversight and monitoring programs:

critical functions and risk areas, allowing appropriate controls to
be implemented to reduce risk.
• Continuous supply chain management—Digital environments
of third-party vendors and their subcontractors constantly
change. An assessment or audit may have confirmed
acceptable practices a year ago, but enterprises cannot assume
the practices are still the same a year later, or that risk levels
remain acceptable. Continuous oversight must extend to all
supply chain vendors, contractors and other third-parties.
• Continuous improvement (CI)—Establishing security, privacy
and compliance controls is not a onetime effort. Business
environments change on a daily basis; new technologies
emerge, new vulnerabilities are introduced as software is
updated, and new threats surface as malicious actors discover
weaknesses in technology. Digital environments must be
• Promote real time information security, privacy and compliance
risk management; identify and enumerate risk early; help
anticipate incidents and prevent breaches; and avoid potential
costs, fines, and damage to business reputation
• Support valid and appropriate ongoing information system and
common controls authorization, helping to ensure that
appropriate controls are in place at any point in time for each
associated business process
• Provide senior leaders and executives with information to make
timely, cost-effective risk management decisions
• Support design and implementation of information security and
privacy controls within development life cycles, reducing the
need to return to planning stages after deployment, and
addressing security and privacy issues

16
16
For a full discussion, see National Institute of Standards and Technology (NIST), “Information Security Continuous Monitoring (ISCM) for Federal
Information Systems and Organizations,” NIST Special Publication 800-137, September 2011,
https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-137.pdf.
17
17
For example, in late 2018 it was reported that NICE Systems, a cloud service provider for Verizon, had a misconfigured file repository that exposed the
names, addresses, account details, and account personal identification numbers (PINs) for as many as 14 million US Verizon customers. See “Cloud
Leak: How A Verizon Partner Exposed Millions of Customer Accounts,” Upguard, 12 December 2018, https://www.upguard.com/breaches/verizon-cloud-
leak

© 2019 ISACA. All Rights Reserved.

 8 CONTINUOUS OVERSIGHT IN THE CLOUD

• Connect risk management processes within data, applications • Support proactive responsibility and accountability for controls

and systems to risk management processes at organizational and risk management throughout the enterprise and its third-
levels parties, including CSPs and their subcontractors

Strategy, Challenges and Execution

Many common—but nonetheless wildly diverse—issues obligations they have created for themselves. As a result, new

must be addressed as enterprises adapt to new business controls are not implemented or sufficiently addressed. These

processing realities. Current threats, vulnerabilities, legal actions create more risk for the business. Legal and regulatory

requirements and associated consequences must all be penalties and fines—often levied against noncompliant

considered. Some of the most common challenges companies—are increasing, as are civil suits21 against
21

include: organizations. For example, as of 22 March 2019, the UK

Information Commissioner’s Office (ICO) data protection

• External bad actors and insider threats—There will always be
enforcement actions have included:22 22

external bad actors seeking to access business data,

• Ninety-one monetary penalties
applications and systems to steal data, to cause business
• Thirty-two enforcement notices
interruptions, or as an act of hacktivism.18 Insider threats can
18

• Fifteen undertakings
be unintended or malicious, but nonetheless, just like the
• Twelve prosecutions
external bad actor, will always exist. Some internal personnel
• Damage to brand and bottom line—The publicity from security
simply lack awareness of appropriate security practices and
incidents and privacy breaches can significantly damage brand
engage in risky behaviors. Others innocently make mistakes
value and result in lost customers. Fines and penalties for
and some choose to act maliciously.
breaches, incidents and noncompliance can be huge,23 even23

• Poorly designed and buggy applications and systems—Lack of

large enough to put an enterprise out of business.
rigor in developing software, failing to implement security by
• Insufficiently secured CSPs—Information security incidents,
design19 and inattention to systems management can
19

privacy breaches and noncompliance with data protection

compromise response time and data availability at best. At
requirements on the part of the cloud services used by
worst, these can lead to business failure or even deadly
enterprises can directly impact the enterprises themselves.
consequences.20 20

• Vendor contracts and relationship management—Vendor

• Increasing legal requirements—Legal requirements for data
contracts are often written to complicate or even prevent easy
protection and privacy worldwide are becoming more detailed
or clean migrations to a different vendor. The time, effort and
and prescriptive. New laws are increasingly addressing personal
resources expended to incorporate a cloud vendor into business
and sensitive data that are stored outside the country where
processes can seem prohibitive when enterprises consider
data subjects reside. Enterprises often implement cross-
moving to other providers. Simply accepting vendor risk can
boundary storage without realizing the additional legal
seem relatively expedient but, nonetheless, may increase risk in
18
18
See a description of hacktivists at https://ics-cert.us-cert.gov/content/cyber-threat-source-descriptions#hack.
19
19
For a good discussion of security by design, see Casola, Valentina; Alessandra De Benedictis; Massimiliano Rak; Umberto Villano; “Security-By-Design in
Multi-Cloud Applications: An Optimization Approach,” Information Sciences 454–455, July 2018, pgs 344-362,
https://www.sciencedirect.com/science/article/pii/S0020025518303517?via%3Dihub.
20
20
For examples, see Loeffler, John; “When Bad Programming Turns Deadly: A Look at Programming Disasters and How Bad Programming Can be Deadly,”
22 November 2018, Interesting Engineering, https://interestingengineering.com/when-bad-programming-turns-deadly.
21
21
For discussion on the changing norms of civil actions related to personal data security and breaches, see DeMarco, Joseph V.; Brian A. Fox; “Data Rights
and Data Wrongs: Civil Litigation and the New Privacy Norms” Yale Law Journal, 1 April 2019, https://www.yalelawjournal.org/forum/data-rights-and-
data-wrongs.
22
22
Information Commissioner's Office (ICO), "Enforcement action," https://ico.org.uk/action-weve-taken/enforcement
23
23
For example, under GDPR Article 83, organizations in noncompliance are “subject to administrative fines up to € 20,000,000, or in the case of an
undertaking, up to 4 percent of the total worldwide annual turnover of the preceding financial year, whichever is higher.” See Official Journal of the
European Union, “Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016” on the protection of natural persons with
regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection
Regulation),” https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32016R0679&from=EN.

© 2019 ISACA. All Rights Reserved.

9 CONTINUOUS OVERSIGHT IN THE CLOUD
ways that regulators and clients deem unacceptable—or even
contractually negligent. The results could result in brand
damage and lost customers.
Strategies for Identifying and
Mitigating Risk in the Cloud
In the 1980s and 1990s, information security was largely a
matter of protecting intellectual property and the network
to help ensure data confidentiality, integrity and availability
(CIA), and mitigating associated risk. In the late 1990s, as
enterprises went online, connected to other businesses,
and started collecting, processing and sharing more
personal information, the challenges expanded, and
required insight into the types of personal information
being processed and exchanged, and third-parties with
access to the data. In the 2000s and 2010s, businesses
everywhere developed a web presence, and many created
social media sites. These new online presences expanded
risk boundaries, far beyond the corporate network that
was formerly controlled almost exclusively by internal
staff. Suddenly, enterprises needed to assess risk
associated with CSPs and then develop security and
compliance oversight capabilities accordingly. In this era
of cyberattacks and breaches, key business leaders and
decision makers have grown increasingly concerned
about the ability to deliver core business capabilities and
also reduce risk. Their concerns include:
• Working with insufficient budgets and resources—There is a
tendency within information security teams to underestimate
the budgets and resources necessary to support successful
information security, privacy and compliance programs.
Business leaders making resourcing decisions typically believe
that if cloud services are being implemented then fewer internal
security resources are needed, since it is assumed that the
cloud services will already be performing all the necessary
outsourced security, privacy and compliance activities. This
assumption often leads to a reduction in already insufficient
budgets. Then, as has been the case throughout decades, when
IT leaders need more funding for new technologies or initiatives,
the information security budget is often the first place where
funds are siphoned to compensate. This leaves information
security, privacy and compliance budgets even more scarce and
limits resources even further. Such budget reductions make
© 2019 ISACA. All Rights Reserved.
implementing continuous oversight of cloud services much
more challenging.
• Managing full supply chain security, privacy and compliance
oversight—Keeping track of all the players within the
organization’s supply chain is often more than a full-time job.
The amount of risk organizations need to track within the full
supply chain ecosystem is large and includes tracking the
specific vendors, their locations, primary contact information,
data involved, applications involved, information security,
privacy and compliance activities being performed by each of
the vendors, regular audits, review of assessments, and more.
Processes need to be established to identify the risk each
vendor throughout the entire supply chain brings to the
organization, along with determining how to monitor vendor risk
levels on an ongoing basis. Additionally, the organization must
determine the best ways to ensure vendors report security
failures, incidents and privacy breaches in a timely and
sufficient manner, and also ensure vendor security, privacy and
compliance management programs are validated.
• Full systems, applications, data life cycle improvement—This
includes ensuring proper information security controls, privacy
controls and compliance requirements are built into each
application and system from the point in time that they are
being designed, through implementation, updating, and end of
use. Historically, information security issues have not been
addressed until late in the life cycle, often just before
implementation into production. And more recently, privacy
controls are also usually considered and addressed late in the
life cycle.
Get Started With Fundamentals
A few fundamental components are critical for
information security and privacy programs, including
associated risk management. These principles enable
professionals to anticipate and respond to new threats,
identify new vulnerabilities, and ensure that all legal
requirements for data protection and privacy are
addressed. These fundamental components include:
1 Defining, identifying and categorizing systems, applications, and
data according to the following needs:
a Confidentiality
b Availability
c Integrity
 10 CONTINUOUS OVERSIGHT IN THE CLOUD

2 Identifying legal requirements for compliance with:

Define Strategy, Assign
a Laws and regulations
b Contracts, including data processing agreements (DPAs) Responsibilities and Take
c Privacy and security notices and other legally binding Action
statements
A common lapse in many organizations is failing to
3 Identifying and planning to address risk on an ongoing basis by:
formally assign responsibilities for continuous oversight
a Performing risk assessments
of information security, privacy and compliance
b Assigning mitigation responsibilities
requirements and risk. Key responsibilities need to be
c Determining how best to mitigate risk and enable
identified and documented to be effective. For continuous
mitigation effectiveness and CI, which also include:
oversight, management and improvement, these
i Establishing a corrective action plan (CAP) for the
responsibilities fall under four primary activities.
risk findings
• Determine accountability—What person or role is ultimately
ii Establishing a plan of action and associated
accountable for developing and implementing an
milestones (POA&M)24 24

organizationwide strategy for continuously monitoring control

Implementing these actions will require proper
effectiveness? Is this person or role the most appropriate for
consideration and design of a security, privacy and
such continuous monitoring of cloud services? Or, is this best
compliance governance structure for each of the
performed by a different role? Who will ultimately be
organization’s business environments. The security,
accountable for ensuring continuous oversight activities are
privacy and compliance activities must then be
performed? Many organizations assign accountability to the
documented and implemented considering sustainability,
individual that is accountable for overall information security,
role responsibilities and assignments for the associated
privacy and compliance. However, in large organizations, this
business organizational structure. This will help to ensure
accountability may be given to a management position within a
that ongoing continuous monitoring and improvement will
team led by the chief information security officer (CISO), chief
support not only the immediate organizational needs, but
privacy officer (CPO), chief risk officer (CRO) or chief
also future organizational needs. Following a feasible,
compliance officer (CCO).
applicable and proper governance structure aligned to the
• Identify key stakeholders—Who are the key stakeholders in
business environment is of critical importance.
continuous cloud services oversight, monitoring, assurance,

A POA&M should consider the following fundamental supply chain management and improvement? Key stakeholders

assessments: 25 25
typically include:

• Board and executive management

• Describe the current disposition of discovered vulnerabilities
• Business unit management
and system findings, and include the CSP’s intended corrective
• Supply chain management
actions for those findings.
• IT management
• Devise a well-organized, structured approach to track risk
• Physical security/safety management
mitigation activities.
• Internal audit
• Identify tasks that need to be accomplished to mitigate risk.
• Information security leaders (CISOs, etc.)
• Establish continuous monitoring activities to address all
• Privacy leaders (CPOs, etc.)
identified vulnerabilities and findings.
• Risk management leaders (CROs, etc.)

• Compliance leaders (CCOs, etc.)

24
24
Generally, a CAP provides specific information as to remediation of findings/weaknesses, and includes a determination of causal factors and trends. A
POA&M is a more high-level management tool for tracking the mitigation of cybersecurity program and system level findings/weaknesses.
25
25
For a useful reference explaining how to write a POA&M, see “FedRAMP Plan of Actions and Milestones (POA&M): Template Completion Guide,” Version
2.1, 21 February 2018, https://www.fedramp.gov/assets/resources/documents/CSP_POAM_Template_Completion_Guide.pdf

© 2019 ISACA. All Rights Reserved.

11 CONTINUOUS OVERSIGHT IN THE CLOUD

• Define the enterprise’s continuous assurance and oversight • Establishing responsibilities throughout the continuous

strategy—Establishing an enterprisewide strategy for monitoring processes, which includes creating and
consistent, standardized continuous monitoring methodologies reviewing the associated data, reports and metrics, and

and practices will maximize the program’s effectiveness, making changes and adjustments based upon results and

efficiencies, and value. An effective strategy results when communications with the associated cloud service

stakeholders thoughtfully consider requirements and activities representatives

throughout the organization, then extend out to the supply • Define organizational roles for continuous assurance and

chain, and apply to the cloud services being used. The strategy oversight implementation—Key stakeholders are needed to

should include monitoring security, privacy and compliance identify and support those with responsibility for continuous

metrics; performing assessments with appropriate frequency; oversight, monitoring, assurance, supply chain management

and providing reports regularly to key stakeholders. High-level and improvement. Stakeholders should ensure that responsible

components include documented policies and supporting personnel have the skills, time, resources and authority for all

procedures for: continuous assurance and oversight activities.

• Performing security, privacy and compliance impact and After key resources are assigned responsibilities, they can
risk analyses implement the strategy. Key activities specific to cloud
• Determining sources and sample sizes for data used to services include:
create metrics that are appropriate for each type of cloud
• Analyzing data including:
service used
• Determining the specific types of data to collect about the
• Implementing enterprisewide cloud services monitoring
cloud services security and privacy risk levels and
tools with the appropriate frequencies for data gathering
associated metrics, along with the specific sources of
and metrics calculations established for the business
data
environment
• Determining when it is necessary to collect supplementary
• Defining key continuous monitoring metrics for each type
data, such as through performing audits, or by
of cloud service
implementing automated vulnerability scanning tools, to
• Establishing and implementing security, privacy and
clarify security-, privacy- and compliance-related
compliance status monitoring and reporting
information that is being analyzed
• Assessing threats and vulnerabilities through appropriate
• Determining the best ways to communicate data and
information sources
associated reports to the appropriate department or staff,
• Determining effectiveness of security, privacy and
both internal and within cloud services management as
compliance controls
appropriate, such as through raw data feeds, database
• Maintaining—and, as appropriate, modifying—the
views, logs, statistics, and so on
monitoring strategy for cloud services and associated
• Analyzing data within the context of the determined risk
actions to take for metrics that fall below target minimum
tolerances, determining the possible impact of
acceptable levels
vulnerabilities within the network and systems, according
• Determining active monitoring practices within the
to the organization’s mission, the contract with each of the
business environment and extending them out through
cloud services, consideration of business processes, and
the cloud services, to determine security, privacy and
estimating the impact of mitigation activities on the
compliance controls effectiveness, status and risk
business and upon the relationship with the cloud service
impacts
• Considering new, emerging and evolving vulnerability and threat
• Ongoing authorization for using each cloud service
data during the analyses
supported with updates for requirements based on

assessment results and monitoring findings

© 2019 ISACA. All Rights Reserved.

 12 CONTINUOUS OVERSIGHT IN THE CLOUD
• Reviewing analysis reports to determine next steps, such as
whether to apply mitigation activities, or to transfer, avoid, reject,
or accept risk, or to terminate the associated CSP contract
• Recording resolution of risk along with documenting the
reasons for the associated decisions, along with the contracts
that the cloud service agreed to, and who will be accountable
for the actions necessary to implement risk resolutions
• Reporting findings of assessments and monitoring
• Communicating the effectiveness of each cloud service’s
information security, privacy and compliance controls
• Providing appropriate metrics for continuous assurance and
improvement associated with each cloud service
• Documenting each cloud service’s challenges,
recommendations and lessons learned
• Responding to findings, which may include risk mitigation
actions, risk acceptance, risk avoidance, or risk sharing and
transfer, or termination of the cloud service, all in accordance
with established business risk tolerance determinations, and as
appropriate to meet the terms for each associated cloud
service contract
• Coordinating responses with the contact responsible for each
cloud service’s appropriate information security, privacy and
compliance management activities within the security-, privacy-
and compliance-focused configuration management program,
adjusting the training provided, and updating supply chain
vendor oversight activities
• Implementing response strategies over a period of time, and
including implementation plans within the POA&M
• Going forward, documenting how each cloud service will
include new or revised controls in the overall continuous
monitoring strategy
• Keeping the cloud services continuous assurance and oversight
strategy and program updated
• Reviewing the cloud services continuous monitoring strategy to
ensure it sufficiently continues to support the organization in
continuing each cloud service relationship, by confirming they
are each operating within acceptable risk tolerance levels, that
metrics remain relevant, and that data, from all sources, is
current and complete.
• Identifying ways to improve the enterprise’s:
26
26
National Institute of Standards and Technology (NIST), “Information Security Continuous Monitoring (ISCM) for Federal information Systems and
Organizations,” NIST Special Publication 800-137, September 2011, https://nvlpubs.nist.gov/nistpubs/legacy/sp/nistspecialpublication800-137.pdf
© 2019 ISACA. All Rights Reserved.
• Understanding of each cloud service’s current security,
privacy and compliance posture
• Support of the process by which each cloud service
performs informed risk management decision-making and
ongoing authorizations
• Understanding of improvements needed within each cloud
service to better assess vendor and third-party security,
privacy and compliance management programs
• Ability to respond to known and emerging threats for each
cloud service
• Reviewing and updating, as needed, the procedures for all
aspects of the cloud services continuous monitoring strategy,
including current relevance of the overall strategy, accuracy in
reflecting organizational risk tolerance, appropriateness of
metrics being used, reporting requirements, and monitoring and
assessment frequencies
• Determining if any of the data collected to support established
cloud service metrics are no longer needed for reporting
purposes or have been determined not to be useful in
maintaining or improving the organization’s insight to the
desired security, privacy and compliance risk levels within each
cloud service
Maintain a Continuous Cloud
Service Assurance and
Oversight Program
Establishing and implementing tools and controls within
an information security, privacy and compliance program
is not a once-and-done activity. There will always be a
need for continuous assurance, continuous oversight and
CI. This is true not only for each enterprise’s own
information assurance activities, but also to support
continuous oversight of the information security, privacy
and compliance programs within the cloud services being
used. Traditionally, this process is referenced as
continuous monitoring.26 26
 13 CONTINUOUS OVERSIGHT IN THE CLOUD

The ISACA® COBIT® 2019 Design Guide,27 provides the 27

associated risk levels within each vendor’s business
following CI tasks which can be incorporated into the processing environment. There are many important
cloud services continuous assurance oversight program: metrics that can be used to provide such insights. These

1 Identify current governance context, business and IT pain points,

metrics will help monitor the risk each supply chain

events, and symptoms triggering the need to act.

vendor brings to an enterprise, and how that risk is being

2 Identify the business and governance drivers and compliance

addressed and mitigated.

requirements for improving enterprise governance of Examples of key metrics include:

information and technology (EGIT) and assess current
• Frequency (typically in number of weeks or months) by which
stakeholder needs.
the full list of vendors, suppliers, contractors and other third-
3 Identify business priorities and business strategy dependent on
parties are reviewed. With specific regard to cloud services,
IT, including any current significant projects.
questions to ask include:
4 Align with enterprise policies, strategies, guiding principles and
• Is the number of cloud services used by the organization
any ongoing governance initiatives.
greater or less than the previous review?
5 Raise executive awareness of IT’s importance to the enterprise
• Which cloud services were added or removed? For what
and the value of EGIT.
reasons?
6 Define EGIT policy, objectives, guiding principles and high-level
• How many associated access controls were appropriately
improvement targets.
modified, for those cloud services added and for those
7 Ensure that the executives and board understand and approve.
removed, in accordance with established procedures?
These tasks can be aligned with—and incorporated into—a • How many access controls were inappropriately modified
continuous monitoring, assurance and compliance for each of the cloud services?
program. • Number of CSPs determined to be critical to the business

environment

Capture the Right Metrics • Number of application programming interfaces (APIs)28 28

interfacing with CSPs. What are the total APIs for the total of all
Selecting and implementing the right metrics
cloud services and what are the total APIs for each cloud
methodology appropriate for the type of cloud services
service?
being used will help enterprises maintain continuous
• Number of workforce members using each cloud provider
monitoring for the cloud services the enterprise depends
interface
upon to support their business activities. The following
• Change in number of cloud service critical vendors, suppliers,
metrics should be considered.
contractors and other third-parties from the previous review

• Number of cloud services using strong (as established and

Key Supply Chain Metrics
Supply chain risk is ubiquitous. A large portion of security
incidents and privacy breaches are caused by contracted
vendors and business partners. To be able to effectively
manage an organization’s complete supply chain there
must be processes established to gain insights into
vendor risk management programs, within which cloud
services are included, and to get a good idea for the
formally documented by your organization) encryption of data
in transit
• Number of cloud services sharing sensitive and personal data
using various methods and channels
• Number of cloud services with any level of access to personal
data and associated sensitive data
• Number of cloud services with access of any kind to mission
critical (e.g., intellectual property, etc.) data

27
27
ISACA, COBIT® 2019 Design Guide: Designing an Information and Technology Governance Solution, USA, 2018,
https://www.isaca.org/COBIT/Pages/COBIT-2019-Design-Guide.aspx
28
28
API is the common term for application programming interface. See National Institute of Standards and Technology (NIST), Computer Resource Center
Glossary, “Application Programming Interface,” https://csrc.nist.gov/glossary/term/Application-Programming-Interface.

© 2019 ISACA. All Rights Reserved.

 14 CONTINUOUS OVERSIGHT IN THE CLOUD
• Number of cloud services that reported security incidents,
privacy breaches and non-compliance issues since the last
review.
• Time it took (in hours and/or days, as appropriate for each
cloud service) for cloud services to respond to security
incidents.
• Time it took for each cloud service to apply critical security
patches.
In 2018 the US Department of Defense issued guidance
for procurement requiring implementation of the National
Institute of Standards and Technology (NIST) Special
Publication (SP) 800-171, Protecting Controlled
Unclassified Information in Nonfederal Systems and
Organizations. 29 29
It includes a requirement to continuously
monitor system security alerts and advisories, and notify
relevant external organizations—for example, external
mission/business partners, supply chain partners,
external service providers, and peer or supporting
organizations. These requirements are good for all types
of enterprises to consider incorporating into their own
cloud services continuous assurance and oversight
programs.
Examples of key related metrics include:
• Number of cyberincidents revealed by the alerts
• Number of personal information items involved with each
individual supply chain involved with each specific incident
• Number of external entities notified
• Number of security alerts communicated to each cloud service
and associated external entity
• Number of cloud services, and their supply chain partners
impacted by each alert
Key Incident and Breach Metrics
Enterprises should establish and document key
categories of security incidents and privacy breaches, and
track the number of breaches and incidents by category.
Examples of key metrics include:
• Number of cybersecurity alerts
• Number of reports of security incidents from personnel
29
29
National Institute of Standards and Techonology (NIST), “Protecting Controlled Unclassified Information in Nonfederal Systems and Organizations,”
Special Publication 800-171, Revision 1, December 2016, https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-171r1.pdf
© 2019 ISACA. All Rights Reserved.
• Time it takes for security incident activity to be detected and
processed through the system
• Time to make a decision on what action(s) to take in response
to an alert
• Percentage of alerts not determined to be valid threats
• Time it takes to identify a security concern
• Time to identify a privacy breach
• Time spent by staff performing specialized security, privacy and
compliance operations
Example Metrics for Common Challenges
The challenges faced by information assurance
professionals continue to grow. New technology brings
new risk while legacy technology that has been
established for decades continues to be used. The use of
cloud services adds to this risk. Information assurance
practitioners face managing not only risk in system
security, but also mitigating risk in data security and
sustaining data protection compliance as new CSPs are
introduced. Common challenges are often overlooked, but
information and assurance professionals need to keep
them in mind.
Key metrics to support such insights—and help mitigate
the associated risks associated with cloud services—
include:
1 Amount (in number of bytes, files, etc.) of egress traffic leaving
the organization’s private cloud and sent to contracted cloud
services, in addition to unauthorized internet sites
2 Amount of traffic (in number of bytes, messages and/or files)
flowing to and from BYOD endpoints within the corporate
network
3 Number of corporate initiatives, and associated number of cloud
services, involving big data analytics
4 Volume of personal data (in number of bytes, records, etc.) used
within AI processes, and the associated cloud services
5 Number of legal requirements covering the use, protection,
sharing, and other activities involving personal data and the
number of cloud services where that personal data is stored,
accessed, or otherwise processed
15 CONTINUOUS OVERSIGHT IN THE CLOUD

6 Volume of traffic to/from IoT devices within the corporate 8 Number of employees using IoT devices within the business

network networking environment

7 Number of CSP user IDs that have not been used in the past

month, quarter or year

Call to Action
Enterprises of all sizes throughout the world face • Continuous improvement initiatives to ensure that cloud

significant, new types of information security, privacy and services security, privacy and compliance activities remain

compliance challenges. The use of cloud services relevant and effective, and provide metrics to gauge success

introduces many of these challenges—largely because and indicate where improvements are necessary.

they involve new and emerging technologies and The following actions will help meet these challenges:
practices, including AI, big data analytics, IoT and BYODs,
1 Obtain visible and strong support of executive leaders for
among others. Supply chain services and products also
implementing continuous monitoring activities for cloud
are increasingly provided through cloud connections—or
services used by the organization.
within cloud servers—so all associated risk must be
2 Implement continuous compliance, audit, assurance activities
mitigated.
and oversight for each cloud service.

Information assurance professionals can mitigate risk 3 Ensure that all applicable legal requirements for privacy

created by emerging technologies and practices— protections become the norm throughout all the cloud services

propagated in many ways by cloud computing and cloud used by the organization, and that they are continuously

services—through continuous monitoring and oversight monitored for compliance.

activities. Critical activities and initiatives for information 4 Maintain continuous oversight of all cloud services and

security, privacy and compliance programs include: associated parties throughout the entire supply chain to remain

aware of—and more quickly mitigate—the identified information

• Continuous assurance activities throughout the full data,
security, privacy and compliance risk.
applications and systems life cycle, including continuous
5 Establish, implement and consistently enforce policies
monitoring activities, appropriate for each type of cloud service,
governing continuous monitoring and cloud use.
continuous risk awareness and continuous compliance.
6 Provide regular training and send frequent reminders to all
• Continuous supply chain management activities for cloud
personnel involved in continuous oversight activities and cloud
services security, privacy and compliance management and
use.
oversight.
7 Understand the shared responsibility model when it comes to
• Continuous cloud information assurance actions, as applicable
managing CSPs.
for each of the various types of clouds currently in use.

© 2019 ISACA. All Rights Reserved.

16 CONTINUOUS OVERSIGHT IN THE CLOUD

Acknowledgments
ISACA would like to acknowledge:

Lead Developer Sergiu Sechel Tichaona Zororo

CISA, CRISC, CISM, CBP, CEH, CSSLP, CISA, CRISC, CISM, CGEIT, COBIT 5
Rebecca Herold
GICSP, PMP Assessor, CIA, CRMA
CISA, CISM, CIPM, CIPP/US, CIPT, CISSP,
Advisory Manager, Romania EGIT | Enterprise Governance of IT (Pty)
FIP, FLMI
Ltd, South Africa
CEO, The Privacy Professor, and Founder,
SIMBUS, LLC
ISACA Board of Directors Theresa Grafenstine
Des Moines, IA, USA ISACA Board Chair, 2017-2018
Rob Clyde, Chair
CISA, CRISC, CGEIT, CGAP, CGMA, CIA,
CISM
Contributing Editor CISSP, CPA
Clyde Consulting LLC, USA
Deloitte & Touche LLP, USA
Fouad Khalil
CISA Brennan Baybeck, Vice-Chair
Chris K. Dimitriadis, Ph.D.
VP of Compliance, SecurityScorecard, Inc. CISA, CRISC, CISM, CISSP
ISACA Board Chair, 2015-2017
New York, NY, USA Oracle Corporation, USA
CISA, CRISC, CISM
Tracey Dedrick INTRALOT, Greece
Expert Reviewers Former Chief Risk Officer with Hudson
Rufina Achieng City Bancorp, USA
CISA, CISM, PROSCI Change Practitioner
Leonard Ong
Management Consultant, Kenya
CISA, CRISC, CISM, CGEIT, COBIT 5
Mais Barouqa Implementer and Assessor, CFE, CIPM,
CISA, CRISC, COBIT 5 Foundation Level, CIPT, CISSP, CITBCM, CPP, CSSLP, GCFA,
GRCP ITIL, ISO 27001 LA GCIA, GCIH, GSNA, ISSMP-ISSAP, PMP
Manager, Jordan Merck & Co., Inc., Singapore

Graham Carter R.V. Raghu

CISA, CGEIT CISA, CRISC
UK Versatilist Consulting India Pvt. Ltd., India

Gabriela Reynaga
Mohammed Khan
CISA, CRISC, COBIT 5 Foundation, GRCP
CISA, CRISC, CIPM
Holistics GRC, Mexico
Senior Global Audit Manager, USA
Gregory Touhill
Abbas Kudrati
CISM, CISSP
CISA, CISM, CGEIT
Cyxtera Federal Group, USA
Chief Security Advisor, Australia
Ted Wolff
Tim Sattler
CISA
CISA, CRISC, CISM, CGEIT, CCSP, CISSP
Vanguard, Inc., USA
Corporate Information Security Officer,
Germany

© 2019 ISACA. All Rights Reserved.

17 CONTINUOUS OVERSIGHT IN THE CLOUD

About ISACA
Now in its 50th-anniversary year, ISACA® (isaca.org) is a global association
1700 E. Golf Road, Suite 400
helping individuals and enterprises achieve the positive potential of
Schaumburg, IL 60173, USA
technology. Today’s world is powered by information and technology, and
ISACA equips professionals with the knowledge, credentials, education and
community to advance their careers and transform their organizations. ISACA Phone: +1.847.660.5505
leverages the expertise of its 460,000 engaged professionals—including its
Fax: +1.847.253.1755
140,000 members—in information and cybersecurity, governance, assurance,
risk and innovation, as well as its enterprise performance subsidiary, CMMI® Support: support.isaca.org
Institute, to help advance innovation through technology. ISACA has a
presence in more than 188 countries, including more than 220 chapters Website: www.isaca.org
worldwide and offices in both the United States and China.

About SecurityScorecard
Provide Feedback:
Headquartered in the heart of New York City, SecurityScorecard’s vision is to
create a new language for measuring and communicating security risk. The www.isaca.org/continuous-oversight
company was founded in late 2013 by Dr. Aleksandr Yampolskiy and Sam
Kassoumeh, two former cybersecurity practitioners who had served, Participate in the ISACA Online
respectively, as chief information security officer and head of security and Forums:
https://engage.isaca.org/onlineforums
compliance. With cloud solutions becoming an increasingly integral part of
the security technology stack, Yampolskiy and Kassoumeh recognized the
Twitter:
need to address third- and fourth-party risk as well as better understand the www.twitter.com/ISACANews
security capabilities of their business partners. Since its founding, the
company has grown dramatically, and now counts hundreds of leading brands LinkedIn:
www.linkedin.com/company/isaca
as customers. SecurityScorecard is backed by leading venture capital
investors including Sequoia Capital, GV, NGP Capital, Evolution Equity Facebook:
Partners, Boldstart Ventures and AXA Venture Partners, among others. For www.facebook.com/ISACAHQ
more information, visit securityscorecard.com.
Instagram:
www.instagram.com/isacanews/
DISCLAIMER

ISACA has designed and created Continuous Oversight in the Cloud: How to
Improve Cloud Security, Privacy and Compliance (the “Work”) primarily as an
educational resource for professionals. ISACA makes no claim that use of any
of the Work will assure a successful outcome. The Work should not be
considered inclusive of all proper information, procedures and tests or
exclusive of other information, procedures and tests that are reasonably
directed to obtaining the same results. In determining the propriety of any
specific information, procedure or test, professionals should apply their own
professional judgment to the specific circumstances presented by the
particular systems or information technology environment.

RESERVATION OF RIGHTS

© 2019 ISACA. All rights reserved.

Continuous Oversight in the Cloud: How to Improve Cloud Security, Privacy and Compliance

© 2019 ISACA. All Rights Reserved.

Instantly Rate
& Understand the
Security Risk of
Any Company
Mitigate your third-party cyber risk with our
instant and continous monitoring platform.

Learn More & Get a Free Demo at: SecurityScorecard.com