2016 International Conference on Software Security and Assurance

Security Requirements Engineering: A

Systematic Mapping (2010-2015)
Naurin Farooq Khan, Naveed Ikram

Ibn-e-sina Empirical Lab

Riphah International University

Islamabad

Abstract—We perform systematic mapping study in the
field of security requirements engineering covering the
time period of 2010-2015. Our aim is to find problems
that the literature has addressed during the five year
period. Our primary studies comprised of 251 studies. We
identified 15 clusters regarding problems faced in
security requirements engineering and identified their
solutions. Our future work includes conducting a
Systematic Literature Review on security requirements
engineering. Index Terms—Systematic Mapping, Security
Requirements Engineering, Literature Survey
I.INTRODUCTION
Considering security at the requirements level has
recently gained momentum. There was a time when
it was considered as an after-thought but the
vulnerabilities associated with information systems
made considering security at requirements level an
essential issue. Currently there exists a synthesized
evidence from the literature about the security
requirements engineering and a frame work has been
developed by carrying out a systematic literature
review [1]. The SLR comprise of the evidence from the
literature till 2009. In order to see what that security
requirement’s engineering momentum has brought
to surface in the last five years there is a need to
carry out a systematic review for the period of 2010
till the end of 2015. As a first step we carry out a
systematic mapping study to identify evidence
clusters and maps. We aim to answer some generic
questions along with a specific question. Our research
questions are:
RQ1: What type of studies have been researched in
security requirements engineering?
RQ2: Which continent and country is leading in the
security requirements engineering research?
RQ3: What is the distribution of studies with respect
to journal and conferences?
RQ4. What are the problems and their solutions that
have been reported in the literature in the last five
years?
II. MAPPING STUDY PROCESS
A.Scope, Search String And Selection Criteria
We, as a first step set the scope of our search
strategy in terms of considering those studies which
consider security. Safety and security terms are
ambiguous in a way that both of them talk about
protection of the system under consideration. In
order to find out if the literature takes safety in the
similar
terms as security we searched first 100 papers of IEEE
Xplore database with the search string. “safety” AND
“requirements engineering” We found that security and
safety although are to some extent similar, have been
addressed differently in the literature. Different standards
have been made for each. Each has a different lifecycle.
Literature had come up with the view that both are
interdependent on four fronts [2]. To such extent the
safety and security are different in literature that it is
considered conflicting at some parts. Things are going in
the direction to align both the concepts but presently
safety and security are dealt differently in the literature
[3]. We found only one paper that address security but did
not have the term “security” [4] with the above
mentioned search string. Since problems can arise when it
 comes to different lifecycles and artifacts and
methodologies we left out safety from our search terms.
B. Strategy for Deriving Search Terms and Search String In
order to finalize our search string, we first finalized our
search terms. We searched the IEEE Xplore database with
the key word “threat” and “secure software
development” to find out if there were any studies that
were pertaining to security without explicitly using these
words. We found only two articles [5] [6], in the former
while found only one [7] paper in the latter case.
Therefore we decided to leave these two terms out of our
search strategy. The databases selected were IEEE Xplore
and Scopus. Our final search string is as follows:
“Security” AND “Requirements Engineering” OR
“Requirements Analysis” OR “Requirements
Elicitation” OR “Requirements Modeling”
C. Inclusion and Exclusion Criteria
We selected those studies that have been published in the
time period of 2010-2015 and which considered building
security in at the requirements level. On the other hand
we did not consider studies published prior to year 2010
and also left out those which were not written in English.
D. Primary Studies
IEEE Xplore resulted in 293 studies while Scopus gave 496
results. We discarded the duplicates and irrelevant
studies. The number of relevant studies found were 251,
out of which 20 % were from IEEE Xplore with number of
51 studies as shown in Table I. The 200 studies (80 %)
were from Scopus database.
TABLE I. TOTAL NUMBER OF RELEVANT /IRRELEVANT
STUDIES
Database Total Relevant Perecentage
Studies Studies of Relevant
Studies
IEEE Xplore 293 51 20%
Scopus 496 200 80%
Total 798 251 - Greece, Saudi Arabia, Morocco, Brazil, Finland, Hungry
III. ANALYSIS AND CLASSIFICATION
We analyzed and classified primary studies according to
the generic data that has to be extracted for mapping
study. In order to show the trends and build systematic
maps we give answers to our generic research questions in
this section.
RQ1: What type of studies have been researched in
security requirements engineering?
RQ2: Which continent and country is leading in the
security requirements engineering research?
RQ3: What is the distribution of studies with respect to
journal and conferences?
A.Study Type
In order to answer research question 1, we categorized
the type of studies into empirical and non-empirical ones.
It was found that in the last five years much of the
emphasis is being paid to the non-empirical studies
comprising of 39 % examples followed by 27 % solutions
and 2 % reporting about the tools developed for different
security requirements initiatives (Fig. 1). In empirical
studies we found that 18 % contributed as case studies, 6
% as experimental studies followed by evaluation with 4
% and surveys/ interviews as 3 %. While 2% studies were
experience reports.
B. Top Countries and continents
To answer our research question 2, we plotted a bubble
chart on the world map as shown in Fig. 2. The top
continent was Europe with 138 studies being conducted
there. Asia was second with 61 studies followed by
America with 40 studies. Australia and Africa contributed
towards 7 and 5 studies respectively. Among the
European countries, Germany was at the top with 50
studies. United States of America contributed to security
requirements engineering with 29 studies (Fig. 3). It
should be noted that, as a world map we added those
countries to America continent bubble that belonged to
South America. Italy came third with 18 studies followed
by United Kingdom with 16 studies. China, Norway and
India and Japan contributed towards 14 and 12 studies
respectively. Countries that contributed with less than 3
but more than 12 studies were; Spain, France Australia,
Canada, South Korea, United Arab Emirates, Malaysia and
Netherland. While countries such as Belgium, Estonia,
and Taiwan contributed with 3 or 2 studies. Those
countries that had an individual publications were;
Algeria, Argentina, Austria, Costa Rica, Czech Republic,
Ireland, Nigeria, Pakistan and Kuwait (not shown in the Fig
3.).
C. Conference/Journal Distribution
 In order to answer the third research question, we
differentiated between journal and conference papers
(Fig. 4). We found 48 studies had been published as
journal articles contributing towards 48 %. 198 studies
were conference papers contributing towards a
percentage of 79 %. We also found some studies that
were compiled as chapters in lecture notes of conference
proceedings. We differentiated them as book chapters FG 2 Continent wise distribution of studies
that contributed as 5 % of the total studies. Among the
journal paper studies we found the Requirements
Engineering (RE) journal on the top with 10 studies
followed by Information and Software Technology (IST)
and Journal of Systems and Software (JSS) journal with 4
studies each. International Journal of Security and its
Applications contributed towards 3 studies. Journal of
Universal Computer Science and Communications in
Computer and Information Science contributed towards 2 Journal(79%)
studies not shown in the Fig. 5 while other journal Conference(19%)
contributed towards individual study each.
Book Chapters(2%)
D. Year Wise Distribution
FG 3 Conference/Journal
The year wise distribution of studies is shown in Fig. 6.
Year 2014 was the top year when 62 studies were carried
out, followed by year 2012 with 48 studies. The year wise
distribution shows that the number of studies gradually
Column1
increased in between 2010 and 2012 with a little dip in
2013 and then peaked in 2014. However in 2015 the 10
number significantly lowered with only 24 studies being 8
reported in that year.
6
Column1
4
2
0

FIG 4 Top Journals

Example(39%)
Experiment(6%)
Experience(2%)
Tool(2%)
Evaluation(4%)
Survey/Interview(3%)
Solution(27%)

FIG 1 Study Type


years
70
60
50
40
30 years
20
10
0
2011
2010
2012
2013
2014
2015
FIG 5 year
IV. HOT SPOTS AND DISCUSSION
In order to find hot spots and to answer our research
question 4, we extracted the problems that the studies
were trying to address. We identified different problems
that have been discussed in the literature and tried to
cluster the similar problems. The reason was to
understand the nature of problems on which research is
being carried out and to find hot spots. The clusters were
based on the similarity of the problem and on how many
studies reported the problem. The number of studies
greater than 4 were considered as a cluster. The studies
which were stand alone in terms of the problem that they
were reporting or problem being reported by less than 4
studies were put into the ‘other’ category. There were 15
clusters as shown in Figure. 7 that were identified namely;
C1-Domain Security: Studies that reported security
requirements engineering problems related to a specific
domain
C2-Methodologies: Studies that reported problem related
to security requirements/elicitation/specification etc
methodologies
C3-Integration of security: Problems that addressed the
integration of security requirements with other non-
functional requirements
C4-Lack of evaluation: Studies that reported lack of
evaluations.
C5-Architecture: Problems related with the lack of
considering architecture with security requirements
C6-Documents: Problems of eliciting security requirement
from documents or development of necessary documents.
C7-Legal Requirements: Problems related with
eliciting/specifying legal or compliance requirements
C8-Experts: Insufficient knowledge or expertise of experts
and the problems it created. C9-Security not considered:
Those studies that generically talked about problems of
not considering security from the initial stages of software
development life cycle.
C10-ISO: Problems related with ISO 27001 standard.
C11-Threats: Problems related with threat centric security
requirements engineering
C12-Human/environment not considered: Inconsideration
of stakeholders and environment during security
requirements engineering.
C13-Automatic Support: Lack of automatic support for
security implementation from specification
C14-Change: Problems that arise due to changing security
requirements or topology.
C15-Ontologies: Lack of security ontologies.
We found that 74 studies (29 %) were from the domain
security followed by methodologies with 17 (7%) studies
reporting the problem. The third and fourth clusters-
integration of security and lack of evaluation had 14 (6%)
studies each. Problems related with the architecture were
10 (4 %) in number, followed by problems with natural
language documents with 9 studies. Seventh cluster,
reporting the problems of legal nature comprised of 8 (3%)
studies while the C8 and C9 clusters comprised of 7 studies
each. Threats, Human/environment not considered, ISO
and change clusters had 5 (2 %) of the studies while
ontologies cluster had 4 studies that reported the problem
with lack of ontologies. It should be noted that the clusters
consisted of studies that reported the similar but not
identical problems. Therefore they can further be
categorized into subcategories that specifically define the
types of problem of identical nature. We identified top
four clusters for such subcategories and formed a word
cloud based on the number of studies falling into each as
shown in Figure. 8. In domain security cluster, there were
10 types of sub categories as shown in the Table II. The top
problems were related to the cloud domain with 12
studies falling into this sub category followed by web
domain with 10 studies. There were 8 studies related with
service oriented application and environment while 5
studies were related with the mobile environment.
Embedded systems, e-governance, internet of things and
business processes sub-categories – each of these sub-
categories comprised of 4 studies. Software product line
and large projects, each consisted of 3 studies, while the
rest of the studies did not form any sub-category due to
individual domain being considered. The methodologies
cluster had studies which fell into three subcategories;
Conformity of methodologies with the next phases of
Legal Threats, 5
SDLC, in need of extension and problems with insufficiency
Requiremen
of one single methodology to elicit/specify security
requirements. There were 6, 5, 6 studies falling into each Architectur ts, 8
,9
sub-category respectively. The third cluster – Integration e, 10
of security was further divided into three sub-categories;
problems of integration of security requirements with that Integration
of privacy, safety and usability requirements. 7 studies of Security, Domain
reported problems of security integration with privacy, 4 14 Security, 74
with safety and 3 studies with problems related with
usability integration with security. Lack of evaluation
cluster had 7 studies reporting lack of empirical
evaluation, 5 reporting lack of simple evaluation while 2 Other, 63
studies reported lack of comparisons. There were 63
studies that came in the other category – we did not make Ontologies,
any independent clusters since the number of similar 4
Methodolgi
studies was less than 4. However we found that there es, 17
were 3 studies that reported problems with trust Change
requirements, security patterns and problem of different (2%), 4
Human/Env
terminologies being used in the literature for security that Lack of ironment, 5
were in need of a consensus among them. There were 2 Security
Evaluation, Experts, 7 Automatic
studies each reporting problems related with fault non
14 ISO, 5 Suport, 5
tolerance of systems and formal methods. There were considered,
some studies that did not fall into any of the subcategory 7
in the other category, however at a more abstract level we FIG 6 Problem Clusters
were able to identify that they were addressing problem
like lack of methods, techniques, frameworks, process,
security requirements analysis, modularity and criteria etc.
for different security issues. Due to dissimilar nature of the
security issues, they were considered in the ‘other’
category. The rest of the studies discussed unique
problems within the ‘other’ category. We extracted the
solutions that are present in the literature to solve the
above classified problems. We classified the solutions into
8 categories namely; New method, New framework, New
process, Automation, New model, Extension of some
existing techniques, Evaluation carried out of existing
techniques/method and Combination or integration of
existing techniques/methods. We mapped the problem
hotspots with the solution categories. The Table III below
shows the number of studies that present a specific
solution for a specific problem cluster. For instance the C1
cluster (domain security) had 22 studies that solved the
similar problems with proposing a new method, 14 studies
which proposed a new framework, 1 study proposed a
new process, 1 study proposed a new model, 10 studies
that presented the extensions of existing
approaches/techniques, 1 study each contributed towards
presenting automation, evaluation and combining
different methods to solve the problem. Any solution that
we were unable to classify into any of the eight categories
was put into uncategorized column. 23 studies in the C1
cluster were unclassified so there were put into
uncategorized column. Rest of the mapping between
problem clusters and solution categories are shown in
Table III.
V. CONCLUSION AND FUTURE WORK
We have performed a mapping study in the field of
security requirements engineering as a preliminary steps
towards a deeper systematic literature review. We have
covered the time period of 2010-2015 and have tried to
analyze, classify and found hot spots in the literature. Our
classification included the type of studies being carried
out, the top journals and the top contributing continents
and countries. We also found hot spots of different sort of
security requirements engineering problems and their
solution the literature is addressing. The hot spots rather
formed problem clusters having similar studies in its
circumference. We found 15 problem clusters and further
divided into sub-categories that comprised of more
specific related problems. We also carried out initial
mapping between these two; problem clusters and
solution categories. We plan to carry out a deeper
systematic literature review as a future work.
REFERENCES
[1] D. Mellado, C. Blanco, L. E. Sánchez, and E.
FernándezMedina, “A systematic review of security
requirements engineering,” Comput. Stand. Interfaces, vol.
32, no. 4, pp. 153– 165, 2010. [2] G. Sabaliauskaite and A.
P. Mathur, “Aligning Cyber-Physical System Safety and
Security,” in Complex Systems Design & Management Asia,
Springer, 2015, pp. 41–53. [3] A.-L. Carter, “Safety-critical
versus security-critical software,” Syst. Saf., pp. 1–6, 2010.
[4] I. Alexander, “Misuse cases: Use cases with hostile
intent,” Softw. IEEE, vol. 20, no. 1, pp. 58–66, 2003. [5] P.
Ongsakorn, K. Turney, M. Thornton, S. Nair, S. Szygenda,
and T. Manikas, “Cyber threat trees for large system threat
cataloging and analysis,” in Systems Conference, 2010 4th
Annual IEEE, 2010, pp. 610–615. [6] D. Ayed, M. Asim, and
D. Llewellyn-Jones, “An event processing approach for
threats monitoring of service compositions,” in Risks and
Security of Internet and Systems (CRiSIS), 2013
International Conference on, 2013, pp. 1–10. [7] G. Yee, X.
Xie, and S. Majumdar, “Automated threat identification for
UML,” in Security and Cryptography (SECRYPT),
Proceedings of the 2010 International Conference on,
2010, pp. 1–7.