A Tertiary Study: Experiences of Conducting Systematic Literature Reviews in Software Engineering

Sr# 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 Id 1 3 6 9 14 15 16 17 18 19 20 21 22 28 30 31 33 34 38 40 41 42 44 45 46 47 48 50 51 52 54 55 56 Included citation E. Mendes, A systematic review of Web engineering research, in Empirical Software Engineering, 2005. 2005 International Symposium on, 2005, p. 10 pp. C. Mair, M. Shepperd, and M. Jrgensen, An analysis of data sets used to train and validate cost prediction systems, in ACM SIGSOFT Software Engineering Notes, 2005, vol. 30, pp. 16. T. Dyba, V. B. Kampenes, and D. I. . Sjoberg, A systematic review of statistical power in software engineering experiments, Information and Software Technology, vol. 48, no. 8, pp. 745755, 2006. D. Budgen, S. Charters, M. Turner, P. Brereton, B. Kitchenham, and S. Linkman, Investigating the applicability of the evidence-based paradigm to software engineering, in Proceedings of the 2006 international workshop on Workshop on interdisciplinary software engineering research, 2006, pp. 714. P. Brereton, B. A. Kitchenham, D. Budgen, M. Turner, and M. Khalil, Lessons from applying the systematic literature review process within the software engineering domain, Journal of Systems and Software, vol. 80, no. 4, pp. 571583, 2007. Scientific research ontology to support systematic review in software engineering http://www.sciencedirect.com/science/article/pii/S147403460600070X V. Malheiros, E. Hohn, R. Pinho, and M. Mendonca, A Visual Text Mining approach for Systematic Reviews, in Empirical Software Engineering and Measurement, 2007. ESEM 2007. First International Symposium on, 2007, pp. 245254. V. B. Kampenes, T. Dyb, J. E. Hannay, and D. I. K. Sjberg, A systematic review of effect size in software engineering experiments, Information and Software Technology, vol. 49, no. 1112, pp. 10731086, 2007. J. Bailey, C. Zhang, D. Budgen, S. Charters, and M. Turner, Search Engine Overlaps: Do they agree or disagree?, in Second International Workshop on Realising Evidence-Based Software Engineering, 2007. REBSE07, 2007, p. 22. M. Staples and M. Niazi, Experiences using systematic review guidelines, Journal of Systems and Software, vol. 80, no. 9, pp. 14251437, 2007. T. Dyba, T. Dingsoyr, and G. K. Hanssen, Applying systematic reviews to diverse study types: An experience report, in Empirical Software Engineering and Measurement, 2007. ESEM 2007. First International Symposium on, 2007, pp. 225234. J. E. Hannay, D. I. K. Sjoberg, and T. Dyba, A Systematic Review of Theory Use in Software Engineering Experiments, Software Engineering, IEEE Transactions on DOI - 10.1109/TSE.2007.12, vol. 33, no. 2, pp. 87107, 2007. D. Budgen, B. Kitchenham, S. Charters, M. Turner, P. Brereton, and S. Linkman, Preliminary results of a study of the completeness and clarity of structured abstracts, in Proc. of the 11th Int. Conf. on Evaluation and Assessment in Software Engineering, 2007, pp. 6472. F. O. Bjornson and T. Dingsoyr, Knowledge management in software engineering: A systematic review of studied concepts, findings and research methods used, Information and Software Technology, vol. 50, no. 11, pp. 10551068, 2008. F. J. Pino, F. Garca, and M. Piattini, Software process improvement in small and medium software enterprises: a systematic review, Software Quality Journal, vol. 16, no. 2, pp. 237261, 2008. O. Dieste, M. Lopez, and F. Ramos, Formalizing a Systematic Review Updating Process, in Software Engineering Research, Management and Applications, 2008. SERA 08. Sixth International Conference on, 2008, pp. 143150. T. Dyb and T. Dingsyr, Empirical studies of agile software development: A systematic review, Information and Software Technology, vol. 50, no. 910, pp. 833859, 2008. M. Staples and M. Niazi, Systematic review of organizational motivations for adopting CMM-based SPI, Information and Software Technology, vol. 50, no. 78, pp. 605620, 2008. R. C. de Boer and R. Farenhorst, In search ofarchitectural knowledge, in Proceedings of the 3rd international workshop on Sharing and reusing architectural knowledge, 2008, pp. 7178. D. Budgen, M. Turner, P. Brereton, and B. Kitchenham, Using mapping studies in software engineering, in Proceedings of PPIG, 2008, vol. 2008, pp. 195204. D. Budgen, M. Turner, S. Charters, J. Bailey, B. Kitchenham, and P. Brereton, Lessons from a Cross-domain Investigation of Empirical Practices, in Proceedings of EASE, 2008, pp. 112. Reflections on 10 Years of Software Process Simulation Modeling: A Systematic Review (2008) http://www.springerlink.com/content/l523wh65hw414471/ V. B. Kampenes, T. Dyb, and J. E. Hannay, A systematic review of quasi-experiments in software engineering, Information and Software Technology, vol. 51, no. 1, pp. 7182, 2009. B. Kitchenham, O. Pearl Brereton, D. Budgen, M. Turner, J. Bailey, and S. Linkman, Systematic literature reviews in software engineering-A systematic literature review, Information and Software Technology, vol. 51, no. 1, pp. 715, 2009. J. Nicols and A. Toval, On the generation of requirements specifications from software engineering models: A systematic literature review, Information and Software Technology, vol. 51, no. 9, pp. 12911307, Sep. 2009. Dapeng Liu, Qing Wang, and Junchao Xiao, The role of software process simulation modeling in software risk management: A systematic review, presented at the Empirical Software Engineering and Measurement, 2009. ESEM 2009. 3rd International Symposium on, 2009, pp. 302311. M. A. Babar and He Zhang, Systematic literature reviews in software engineering: Preliminary results from interviews with researchers, in Empirical Software Engineering and Measurement, 2009. ESEM 2009. 3rd International Symposium on, 2009, pp. 346355. D. Budgen, J. Bailey, M. Turner, B. Kitchenham, P. Brereton, and S. Charters, Cross-domain investigation of empirical practices, Software, IET DOI - 10.1049/iet-sen.2008.0106, vol. 3, no. 5, pp. 410421, 2009. P. Brereton, M. Turner, and R. Kaur, Pair programming as a teaching tool: A student review of empirical studies, in Software Engineering Education and Training, 2009. CSEET09. 22nd Conference on, 2009, pp. 240247. B. Kitchenham, P. Brereton, M. Turner, M. Niazi, S. Linkman, R. Pretorius, and D. Budgen, The impact of limited search procedures for systematic literature reviewsA participant-observer case study, in Empirical Software Engineering and Measurement, 2009. ESEM 2009. 3rd International Symposium on, 2009, pp. 336345. W. Afzal, R. Torkar, and R. Feldt, A systematic review of search-based testing for non-functional system properties, Information and Software Technology, vol. 51, no. 6, pp. 957976, 2009. M. Ciolkowski, What do we know about perspective-based reading? An approach for quantitative aggregation in software engineering, in Empirical Software Engineering and Measurement, 2009. ESEM 2009. 3rd International Symposium on, 2009, pp. 133144 G. S. Walia and J. C. Carver, A systematic literature review to identify and classify software requirement errors, Information and Software

34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58

57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 80 82 83

Technology, vol. 51, no. 7, pp. 10871109, 2009. B. Kitchenham, P. Brereton, and D. Budgen, The educational value of mapping studies of software engineering literature, in Software Engineering, 2010 ACM/IEEE 32nd International Conference on, 2010, vol. 1, pp. 589598. B. Kitchenham, R. Pretorius, D. Budgen, O. Pearl Brereton, M. Turner, M. Niazi, and S. Linkman, Systematic literature reviews in software engineering A tertiary study, Information and Software Technology, vol. 52, no. 8, pp. 792805, Aug. 2010. Synthesizing evidence in software engineering research How Reliable Are Systematic Reviews in Empirical Software Engineering? A. Ampatzoglou and I. Stamelos, Software engineering research for computer games: A systematic review, Information and Software Technology, vol. 52, no. 9, pp. 888901, Sep. 2010. S. P. Shashank, P. Chakka, and D. V. Kumar, A systematic literature survey of integration testing in component-based software engineering, in Computer and Communication Technology (ICCCT), 2010 International Conference on, 2010, pp. 562568. Supporting evidence-based Software Engineering with collaborative information retrieval D. mite, C. Wohlin, T. Gorschek, and R. Feldt, Empirical evidence in global software engineering: a systematic review, Empirical Software Engineering, vol. 15, no. 1, pp. 91118, 2010. M. S. Ali, M. Ali Babar, L. Chen, and K.-J. Stol, A systematic review of comparative evidence of aspect-oriented programming, Information and Software Technology, vol. 52, no. 9, pp. 871887, Sep. 2010. A critical appraisal of systematic reviews in software engineering from the perspective of the research questions asked in the reviews Refining the systematic literature review process--two participant-observer case studies http://www.springerlink.com/content/1632750n8670r125/ S. Lane and I. Richardson, Process models for service-based applications: A systematic literature review, Information and Software Technology, vol. 53, no. 5, pp. 424439, May 2011 Identifying relevant studies in software engineering An Empirical Investigation of Systematic Reviews in Software Engineering Research synthesis in software engineering: A tertiary study Using mapping studies as the basis for further research - A participant-observer case study F. Q. B. da Silva, A. L. M. Santos, S. Soares, A. C. C. Frana, C. V. F. Monteiro, and F. F. Maciel, Six years of systematic literature reviews in software engineering: An updated tertiary study, Information and Software Technology, vol. 53, no. 9, pp. 899913, Sep. 2011. M. Host and A. Orucevic-Alagic, A systematic review of research on open source software in commercial software product development, Information and Software Technology, 2010. X. Bai, H. Zhang, and L. G. Huang, Empirical Research in Software Process Modeling: A Systematic Literature Review, in Empirical Software Engineering and Measurement (ESEM), 2011 International Symposium on, 2011, pp. 339342. S. Ghanavati, D. Amyot, and L. Peyton, A systematic review of goal-oriented requirements management frameworks for business process compliance, in Requirements Engineering and Law (RELAW), 2011 Fourth International Workshop on, 2011, pp. 2534. A. C. C. Franca, T. B. Gouveia, P. C. F. Santos, C. A. Santana, and F. Q. B. da Silva, Motivation in software engineering: A systematic review update, in Evaluation & Assessment in Software Engineering (EASE 2011), 15th Annual Conference on, 2011, pp. 154163. D. S. Cruzes and T. Dyba, Recommended Steps for Thematic Synthesis in Software Engineering, in Empirical Software Engineering and Measurement (ESEM), 2011 International Symposium on, 2011, pp. 275284. F. Q. B. da Silva, M. Suassuna, R. F. Lopes, T. B. Gouveia, A. C. A. Franca, J. P. N. de Oliveira, L. F. M. de Oliveira, and A. L. M. Santos, Replication of Empirical Studies in Software Engineering: Preliminary Findings from a Systematic Mapping Study, in Replication in Empirical Software Engineering Research (RESER), 2011 Second International Workshop on, 2011, pp. 6170. D. Budgen, A. J. Burn, O. P. Brereton, B. A. Kitchenham, and R. Pretorius, Empirical evidence about the UML: a systematic literature review, Software: Practice and Experience, vol. 41, no. 4, pp. 363392, 2011. E. Murugesupillai, B. Mohabbati, and D. Gaevi, A preliminary mapping study of approaches bridging software product lines and service-oriented architectures, in Proceedings of the 15th International Software Product Line Conference, 2011, p. 11.

Sr# 1 2

Id 70 71

3 4 5 6 7 8 9 10 11 12 13

4 7 9 72 73 13 14 74 18 19 21

Citation K. Petersen, R. Feldt, S. Mujtaba, and M. Mattsson, Systematic mapping studies in software engineering, in 12th International Conference on Evaluation and Assessment in Software Engineering, 2008, pp. 7180. O. Gmez, H. Oktaba, M. Piattini, and F. Garca, A systematic review measurement in software engineering: state-of-the-art in measures, Software and Data Technologies, pp. 165176, 2008. http://www.springerlink.com/content/m172m11447734h22/ R. F. Barcelos and G. H. Travassos, Evaluation approaches for software architectural documents: a systematic review, in Ibero-American Workshop on Requirements Engineering and Software Environments (IDEAS), La Plata, Argentina, 2006. E. Hossain, M. A. Babar, and H. Paik, Using scrum in global software development: a systematic literature review, in Global Software Engineering, 2009. ICGSE 2009. Fourth IEEE International Conference on, 2009, pp. 175184. S. G. MacDonell and M. J. Shepperd, Comparing Local and Global Software Effort Estimation Models--Reflections on a Systematic Review, in Empirical Software Engineering and Measurement, 2007. ESEM 2007. First International Symposium on, 2007, pp. 401409. . Hauge, C. Ayala, and R. Conradi, Adoption of open source software in software-intensive organizationsA systematic literature review, Information and Software Technology, vol. 52, no. 11, pp. 11331154, 2010. R. Prikladnicki, D. Damian, and J. L. . Audy, Patterns of evolution in the practice of distributed software development: quantitative results from a systematic review, in Proceedings of the 12th Conference on Evaluation and Assessment in Software Engineering (EASE08), 2008. F. Q. . Da Silva, C. Costa, A. C. . Frana, and R. Prikladinicki, Challenges and solutions in distributed software development project management: A systematic literature review, in Global Software Engineering (ICGSE), 2010 5th IEEE International Conference on, 2010, pp. 8796. Q. Gu and P. Lago, Exploring service-oriented system engineering challenges: a systematic literature review, Service Oriented Computing and Applications, vol. 3, no. 3, pp. 171188, 2009. M. Ivarsson and T. Gorschek, Technology transfer decision support in requirements engineering research: a systematic review of REj, Requirements engineering, vol. 14, no. 3, pp. 155175, 2009. R. Rabiser, P. Grnbacher, and D. Dhungana, Requirements for product derivation support: Results from a systematic literature review and an expert survey, Information and Software Technology, vol. 52, no. 3, pp. 324346, 2010. E. Engstrm, M. Skoglund, and P. Runeson, Empirical evaluations of regression test selection techniques: a systematic review, in Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement, 2008, pp. 2231. S. Jalali and C. Wohlin, Agile practices in global software engineering-A systematic map, in International Conference on Global Software

14 15 16 17 18

75 24 25 29 76

19 20 21 22 23 24

77 37 78 39 79 46

25 26 27 28 29 30 31 32 33 34 35 36 37

80 81 82 52 83 84 55 85 57 86 87 88 61

Engineering (ICGSE), 2010. S. U. Khan, M. Niazi, and R. Ahmad, Critical success factors for offshore software development outsourcing vendors: A systematic literature review, in Global Software Engineering, 2009. ICGSE 2009. Fourth IEEE International Conference on, 2009, pp. 207216. R. Pretorius and D. Budgen, A mapping study on empirical evidence related to the models and forms used in the uml, in Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement, 2008, pp. 342344. L. B. Lisboa, V. C. Garcia, D. Lucrdio, E. S. De Almeida, S. R. de Lemos Meira, and R. P. de Mattos Fortes, A systematic review of domain analysis tools, Information and Software Technology, vol. 52, no. 1, pp. 113, 2010. M. Khurum and T. Gorschek, A systematic review of domain analysis solutions for product lines, Journal of Systems and Software, vol. 82, no. 12, pp. 19822003, 2009. E. de Souza Filho, R. de Oliveira Cavalcanti, D. Neiva, T. Oliveira, L. Lisboa, E. de Almeida, and S. de Lemos Meira, Evaluating domain design approaches using systematic review, Software Architecture, pp. 5065, 2008. http://www.springerlink.com/content/h0l6t67k8025805r/ P. Mohagheghi, V. Dehlen, and T. Neple, Definitions and approaches to model quality in model-based software development-A review of literature, Information and Software Technology, vol. 51, no. 12, pp. 16461669, 2009. C. Guerra-Garca, A systematic literature review of how to introduce data quality requirements into a software product development, in 5th International Conference on Evaluation of Novel Approaches to Software Engineering, 2010, pp. 1219 O. Dieste and N. Juristo, Systematic Review and Aggregation of Empirical Studies on Elicitation Techniques, Software Engineering, IEEE Transactions on, no. 99, p. 11, 2011. M. Jimnez and M. Piattini, Problems and Solutions in Distributed Software Development: A Systematic Review, Software Engineering Approaches for Offshore and Outsourced Development, pp. 107125, 2009. http://www.springerlink.com/content/n445663h43636487/ L. Chen, M. A. Babar, and C. Cawley, A status report on the evaluation of variability management approaches, in 13th International Conference on Evaluation and Assessment in Software Engineering (EASE), 2009, pp. 110. C. Pardo, F. J. Pino, F. Garca, M. Piattini Velthius, and M. T. Baldassarre, Trends in Harmonization of Multiple Reference Models, Evaluation of Novel Approaches to Software Engineering, pp. 6173, 2011. http://www.springerlink.com/content/t03283g584421118/ S. Montagud and S. Abraho, Gathering current knowledge about quality evaluation in software product lines, in Proceedings of the 13th International Software Product Line Conference, 2009, pp. 91100. M. Unterkalmsteiner, T. Gorschek, A. Islam, C. Cheng, R. Permadi, and R. Feldt, Evaluation and Measurement of Software Process Improvement-A Systematic Literature Review, Software Engineering, IEEE Transactions on, no. 99, p. 11, 2011. M. T. Baldassarre, D. Caivano, B. Kitchenham, and G. Visaggio, Systematic review of statistical process control: An experience report, in 11th International Conference on Evaluation and Assessment in Software Engineering (EASE07), UK, 2007. A. Rainer and S. Beecham, A follow-up empirical evaluation of evidence based software engineering by undergraduate students, in 12th International Conference on Evaluation and Assessment in Software Engineering, University of Bari, Italy, 2008. H. P. Breivold, M. A. Chauhan, and M. A. Babar, A systematic review of studies of open source software evolution, in Software Engineering Conference (APSEC), 2010 17th Asia Pacific, 2010, pp. 356365. D. A. Callegari and R. M. Bastos, A Systematic Review of Dynamic Reconfiguration of Software Projects, XXII Simpsio Brasileiro de Engenharia de Software, 2008. F. C. Ferrari and J. C. Maldonado, Experimenting with a multi-iteration systematic review in software engineering,, in Proceedings of the 5th Experimental Software Engineering Latin America Workshop (ESELAW), 2008. J. Calvo-Manzano, G. Cuevas, L. Grossi, J. Mejia, and T. San Feliu, Outsourcing Contracts: A Systematic Review, International Book Series" INFORMATION SCIENCE & COMPUTING, vol. 11, pp. 107115, 2009. Z. Zakaria, R. Atan, A. A. . Ghani, and N. F. . Sani, Unit testing approaches for BPEL: a systematic review, in Software Engineering Conference, 2009. APSEC09. Asia-Pacific, 2009, pp. 316322. R. B. Svensson, M. Hst, and B. Regnell, Managing Quality Requirements: A Systematic Review, presented at the Software Engineering and Advanced Applications (SEAA), 2010 36th EUROMICRO Conference on, 2010, pp. 261268. H. P. Breivold, I. Crnkovic, and M. Larsson, A systematic review of software architecture evolution research, Information and Software Technology, 2011. P. A. da Mota Silveira Neto, I. Carmo Machado, J. D. McGregor, E. S. De Almeida, and S. R. de Lemos Meira, A systematic mapping study of software product lines testing, Information and Software Technology, vol. 53, no. 5, pp. 407423, 2011. R. Burrows, A. Garcia, and F. Taani, Coupling metrics for aspect-oriented programming: a systematic review of maintainability studies, Evaluation of Novel Approaches to Software Engineering, pp. 277290, 2010. http://www.springerlink.com/content/p7m675r0n013lv75/ D. S. Cruzes and T. Dyba, Recommended steps for thematic synthesis in software engineering, in Empirical Software Engineering and Measurement (ESEM), 2011 International Symposium on, 2011, pp. 275284. E. Y. Nakagawa and L. B. . Oliveira, Using systematic review to elicit requirements of reference architectures. WER, 2011. E. Y. Nakagawa, D. Feitosa, and K. R. Felizardo, Using systematic mapping to explore software architecture knowledge, in Proceedings of the 2010 ICSE Workshop on Sharing and Reusing Architectural Knowledge, 2010, pp. 2936. V. Alves, N. Niu, C. Alves, and G. Valena, Requirements engineering for software product lines: A systematic literature review, Information and Software Technology, vol. 52, no. 8, pp. 806820, 2010. K. R. Felizardo, E. Y. Nakawaga, D. Feitosa, R. Minghim, and J. C. Maldonado, An approach based on visual text mining to support categorization and classification in the systematic mapping, in Proc. of EASE, 2010, vol. 10, pp. 110. C. Guerra-Garca, I. Caballero, and M. Piattini Velthius, A Survey on How to Manage Specific Data Quality Requirements during Information System Development, Evaluation of Novel Approaches to Software Engineering, pp. 1630, 2011. http://www.springerlink.com/content/m2240lg3t4m7h317/ H. Koziolek, Sustainability evaluation of software architectures: a systematic review, in Proceedings of the joint ACM SIGSOFT conference--QoSA and ACM SIGSOFT symposium--ISARCS on Quality of software architectures--QoSA and architecting critical systems--ISARCS, 2011, pp. 312. R. Vivian, E. Huzita, G. Lapasini Leal, and A. Chaves Steinmacher, Context-awareness on software artifacts in distributed software development: a systematic review, Collaboration and Technology, pp. 3044, 2011. http://www.springerlink.com/content/q2k2448t2611736n/ E. Hasnain, An overview of published agile studies: a systematic literature review, in Proceedings of the 2010 National Software Engineering Conference, 2010, p. 3. P. Brereton, A Study of Computing Undergraduates Undertaking a Systematic Literature Review, Education, IEEE Transactions on, no. 99, p. 11, 2011.

38 62 39 40 41 42 43 72 44 45 74 46 47 76 77 73 89 90 91 71

48 49

78 79

50 51 52 53 54 55 56 57 58

80 81 82 84 86 87 88 89 90

A. M. Magdaleno, C. M. . Werner, and R. M. Araujo, Reconciling software development models: A quasi-systematic review, Journal of Systems and Software, 2011. O. Dieste, N. Juristo, and H. Saxena, Quantitative determination of the relationship between internal validity and bias in software engineering experiments: consequences for systematic literature reviews, in Empirical Software Engineering and Measurement (ESEM), 2011 International Symposium on, 2011, pp. 285294. A. Rainer, D. Jagielska, and T. Hall, Software engineering practice versus evidence-based software engineering research, in ACM SIGSOFT Software Engineering Notes, 2005, vol. 30, pp. 15. B. A. Kitchenham, O. P. Brereton, S. Owen, J. Butcher, and C. Jefferies, Length and readability of structured software engineering abstracts, Software, IET DOI - 10.1049/iet-sen:20070044, vol. 2, no. 1, pp. 3745, 2008. B. Kitchenham, Whats up with software metrics?A preliminary mapping study, Journal of Systems and Software, vol. 83, no. 1, pp. 3751, 2010. K. Petersen and N. B. Ali, Identifying Strategies for Study Selection in Systematic Reviews and Maps, in Empirical Software Engineering and Measurement (ESEM), 2011 International Symposium on, 2011, pp. 351354. E. Barreiros, A. Almeida, J. Saraiva, and S. Soares, A Systematic Mapping Study on Software Engineering Testbeds, in Empirical Software Engineering and Measurement (ESEM), 2011 International Symposium on, 2011, pp. 107116. K. R. Felizardo, M. Riaz, M. Sulayman, E. Mendes, S. G. MacDonell, and J. C. Maldonado, Analysing the use of graphs to represent the results of Systematic Reviews in Software Engineering, in Software Engineering (SBES), 2011 25th Brazilian Symposium on, 2011, pp. 174183. B. Kitchenham, P. Brereton, Z. Li, D. Budgen, and A. Burn, Repeatability of systematic literature reviews, in Evaluation & Assessment in Software Engineering (EASE 2011), 15th Annual Conference on, 2011, pp. 4655. H. Zhang, M. A. Babar, X. Bai, J. Li, and L. Huang, An empirical assessment of a systematic search process for systematic reviews, in Evaluation & Assessment in Software Engineering (EASE 2011), 15th Annual Conference on, 2011, pp. 5665. B. A. Kitchenham, O. P. Brereton, D. Budgen, and Z. Li, An Evaluation of Quality Checklist Proposals-A participant-observer case study, in 13th International Conference on Evaluation and Assessment in Software Engineering.