Vulnerability Management Maturity Model

-Work in progress-
Pegah Nikbakht Bideh Martin Hell Martin Höst
Department of Electrical and Information Department of Electrical and Information Department of Computer Science
Technology Technology Lund University
Lund University Lund University Lund, Sweden
Lund, Sweden Lund, Sweden Martin.host@cs.lth.se
Pegah.nikbakht_bideh@eit.lth.se Martin.hell@eit.lth.se

Abstract—Finding and handling vulnerabilities is a difficult
task, especially in organizations with many third party open
source components. Thus, formal or structured methods helping
organizations managing their product’s vulnerabilities, and
planning future improvements, are needed. We design and
evaluate a vulnerability management maturity model, building
upon other security maturity models, but with more focus on
vulnerability management. In other words, the vulnerability
management maturity model is a type of software security
maturity model which aims to help organizations formulating
their strategy of finding and managing vulnerabilities in robust
and cost efficient way.
Keywords—Software Vulnerability, Maturity Model, Software
Security
I. INTRODUCTION
In general, software security management is addressed by a
number of maturity models such as BSIMM [3], OWASP
SAMM [1], Microsoft SDL [4] and SSE-CMM [2]. However
although they provide efficient functionalities for assessing the
organization maturity level, lack of vulnerability identification
and management is apparent in most of these models. At the
same time because of increased number of vulnerabilities and
increased use of third party components, software updates and
patches are important enough to consider them in isolation.
Our aim is to formulate a maturity model which can be used to
improve the management of vulnerabilities and maintenance of
software products which includes open source components
(OSS). In Section II we briefly explain the existing maturity
models and in Section III we describe our vulnerability
management maturity model.
II. BACKGROUND
A. OWASP SAMM
The purpose of SAMM [1] (Software Assurance Maturity
Model) is to help organizations evaluating their software
security practices, build a balanced software security assurance
program, demonstrate concrete improvements to a security
assurance program and define security related activities
throughout an organization. SAMM consists of four main
business functions and each of those is made up of three
security practices. These business functions and their
correspondence security practices are as follows:
- Governance, which includes strategy and metrics,
policy and compliance and also education and
guidance.
- Construction, which includes threat assessment,
security requirements and secure architecture.
- Verification, which includes design review,
implementation review and security testing.
- Operation, which includes issue management,
environment hardening and operational enablement.
Each practice in the above business functions is further a
set of specific security related activities which build assurance
for the related business function. The security practices have in
turn three defined maturity levels, namely initial understanding
and adhoc provision of security practices, increase efficiency
and effectiveness of security practice and comprehensive
mastery of security practice. For each security practice area a
set of questions have been designed for evaluating and
assessing the maturity level of the organization.
B. Microsoft SDL
Microsoft SDL [4] (Security Development Lifecycle) is
another software security maturity model, which focuses on
applying security practices at distinct points in the software
development life cycle. Microsoft’s maturity model consists of
five capability areas that correspond to the different phases in
the software development lifecycle. Each capability area in
turn consists of security activities. These capabilities and their
related activities are as follows:
- Training, policy and organizational capabilities,
which can contain core security training.
- Requirements and design, which can contain
establishment of security requirements, creating
quality gates/bug bars, security and privacy risk
assessment, establishment of design requirements,
analyzis of attack surface and threat modeling.
- Implementation, which can include the use of
approved tools, deprecation of unsafe functions and
static analysis.
 - Verification, which can include dynamic analysis,
fuzz testing and attack surface review.
- Release and response, which can contain an incident
response plan, a final security review, release archive
and execution of an incident response plan.
Each of the security activities that are executed as a part of
the software development leads to more security gains for the
organization. Based on the number of executing activities the
maturity level of the firm can be determined.
C. SSE-CMM
SSE-CMM [2] (Systems Security Engineering - Capability
Maturity Model) is a tool for an engineering organization to
evaluate their security engineering practices and define
improvements, establish confidence in capability of
organizations and providing a mechanism for customers to
evaluate a provider’s security capabilities. SSE-CMM contains
base practices and process areas. Base practices have been
organized into process areas in order to meet a broad
requirements of security engineering organizations. The
process areas show the state of the organization that is
performing the process area to the life cycle of their products.
SSE-CMM process areas are listed below:
- PA01 Administer security controls
- PA02 Assess impact
- PA03 Assess security risk
- PA04 Assess Threat
- PA05 Assess vulnerability
- PA06 Build assurance argument
- PA07 Coordinate security
- PA08 Monitor security posture
- PA09 Provide security input
- PA10 Specify security needs and
- PA11 Verify and validate security
SSE-CMM defines five capability levels which represent
the maturity of the security engineering organization. These
levels, from low to high maturity, are: performed informally,
planned and tracked, well defined, qualitatively controlled and
continuously improving.
D. BSIMM
BSIMM [3] (Building Security In Maturity Model) is
another study of software security initiatives which reflects the
current state of software security. The purpose of BSIMM is to
quantify the activities that are carried out by software security
initiatives. BSIMM’s goal in to provide the same measuring
stick for most organizations. BSIMM consists of 12 practices
which are organized into four domains:
- Governance which includes strategy and metrics,
compliance and policy and training.
- Intelligence which includes attack models, security
features and design and standards and requirements.
- SSDL Touchpoints which contains architecture
analysis, code review and security testing.
- Deployment which contains penetration testing,
software environment and configuration management
and vulnerability management.
BSIMM also defines three maturity levels, from low to
high, and based on the answers of questions in each security
practice, the maturity level of an evaluated organization is
determined. Therefore, the organizations, by utilizing BSIMM,
can determine where their approach currently stands in
comparison to other firms. Once they have determined their
current status, they can devise a plan to enhance security
practices. BSIMM makes it possible for the organizations to
make a long-term plan for software security issues, and also
track their progress against that plan.
In the next section, our proposed vulnerability management
maturity model is described in detail.
III. VULNERABILITY MANAGEMNET MATURITY MODEL
In the above mentioned maturity models there is little focus
on finding vulnerabilities and management of these after
product release. Still, there are some vulnerability management
practices some, such as SAMM, Microsoft SDL and SSE-
CMM, but their main focus is on development phase and there
is little effort on finding vulnerabilities after production phase.
On the other hand, the mentioned models try to manage
vulnerabilities in their own code but it is not mentioned how
they are going to address vulnerabilities in third party
components. Also there are some other requirements that are
related to the process of identifying and evaluating
vulnerabilities in third party components, as well as managing
software updates. These will be included as part of our
proposed security model.
Our vulnerability management maturity model consists of 7
question areas, the first area will help organizations to identify
how much knowledge they have about their active product
components (which can be their own developed or OSS
components). The second and third areas indicate which
sources they use to identify the existence vulnerabilities in each
component, and how often they monitor these sources. The
fourth and fifth areas show how they evaluate the criticality of
found or reported vulnerabilities and how they remedy their
components to fix these vulnerabilities. Finally the last two
areas indicate how they deliver the patches or updates to their
customers and how they communicate these changes internally
or externally. The areas and questions in each area are listed
below:
 Product knowledge
- How do you keep track of which products are activated
and used?
- How do you keep track of which third party OSS and
COTS are included in developed and maintained
products?
- How do you keep track of which version of third party
OSS and COTS are used in all active and used
products?
- How do you identify which third party OSS
components are maintained by the community?
- What process do you have to identify the code
dependencies of different components?
- How do you keep track of possible threats that the
products are facing?
- How do you specify the intended use and the operating
environment of the products?
- What strategy do you have to keep track of and
communicate the end-of-life of your products?
 Identification of sources
- What process do you have for identifying which
external resources you should use to identify
vulnerabilities?
- Which process do you have for receiving
vulnerabilities reported to you by third parties?
 Monitoring sources
- What process and organization do you have to monitor
sources of vulnerabilities?
- With what time interval do you monitor vulnerability
sources?
 Evaluating vulnerabilities
- What process and organization do you have to evaluate
the criticality of identified vulnerabilities?
- What process and organization do you have to take
decision on how to handle critical vulnerabilities?
 Implementing vulnerability remedies
- What process and organization do you have to handle
critical vulnerabilities that need urgent changes?
- What process and organization do you have to handle
critical vulnerabilities that are updated in a planned
release?
- What process and organization do you have to handle
critical vulnerabilities that need no changes?
 Delivering updates
- What process is used to apply upgrades and patches to
products in the field?
- What process do you have to protect the integrity of
patches when they are delivered to products?
 Communication
- How do you communicate internally when critical
vulnerabilities are identified and resolved?
- How do you communicate externally when critical
vulnerabilities are identified and resolved?
- How do you communicate the patching status of
devices?
- How is security related information delivered with
updates and new versions of your products?
We consider five possible answers to the questions above
which shows the maturity level of the organization from low
mature to high mature. These possible answers are:
- We don’t have any structured approach.
- We can explain how we do for each product.
- We have standardized way for all products.
- We collect experience and metrics for our approach.
- We have cyclic improvements based on collected
experience.
For assessing an organization based on our vulnerability
management maturity model, we should consider three phases:
pre-assessment, assessment and post assessment. The purpose
of pre-assessment is to understand the different products of
organization, understanding their needs, meeting with security
responsible persons within the organization and finally take a
decision. This decision based on the maturity level of the
organization can be: no need for assessment, limited need and
large need for assessment. The focus of our vulnerability
management maturity model is to assess organizations which
have limited need.
The assessment phase consists of the following steps:
- Planning
In this step we should define the assessment
boundaries, select assessment team, train the
assessment team and plan for fact gathering which can
be include which roles to interview or which products
to include.
- Data gathering
In this step we should select the fact gathering
approach, identify target interviewees, define a
detailed plan and finally gather the data.
- Data analysis
In this step we conduct qualitative analysis of data,
review of findings and gathered data by the
organization and finally identify strengths and
weaknesses and areas of improvement.
- Reporting
In the last step the findings and recommendations
should be documented and the activities for post
assessment phase should be planned.
IV. ONGOING WORK
We are going to give our assessment sheet to some IoT
organizations and ask them to answer the questions based on
their real procedures. Then, we are going to interview the
companies who had answered the questions and during the
interview, the company will provide us some feedbacks about
our questions (whether they understand each or not, which
terminology was confusing for them, which questions were
hard to answer and etc). Also, we ask them about our answers
that they really fit to their maturity level or not (if any level
need to be added or if any level is redundant). Then, based on
each company feedback we will make the required changes on
our assessment sheet and on our answers as well. We will
conduct the assessment on other companies with the new
version of our assessment sheet and do the iterative changes
until we reach a satisfactory version which can fit almost all
IoT organizations.
V. CONCLUSIONS
We designed a new vulnerability maturity model which
focus on and expands certain aspects of existing maturity
models. The focus is on security management of third party
software components, i.e., identifying new vulnerabilities,
evaluating these vulnerabilities, and implementing and
deploying software updates. The need for more structured and
formal methods for handling this is motivated by the ongoing
development where more vulnerabilities are discovered, more
third party software components are used, and the increase of
connected devices. We selected relevant questions from
existing models, such as OWASP SAMM, Microsoft SDL,
SSE-CMM and BSIMM. Then we added new questions in
order to make a more complete coverage and finally we
formulate assessment based on a standard procedure.
REFERENCES
[1] OWASP, "Software Assurance Maturity Model :A guide to
building security into software development," 2017.
[2] "Information technology — Security techniques — Systems
Security Engineering — Capability Maturity Model (SSE-
CMM)," 2008.
[3] G. McGraw, S. Migues and J. West, "Building Security In
Maturity Model (BSIMM) version 6," 2011.
[4] Microsoft Coporation, "Simplified Implementation of
Microsoft SDL", 2010.