A Map of Threats to Validity of Systematic Literature Reviews in Software Engineering

Resource type
Conference Paper
Authors/contributors
Title
A Map of Threats to Validity of Systematic Literature Reviews in Software Engineering
Abstract
Context: The assessment of Threats to Validity (TTVs) is critical to secure the quality of empirical studies in Software Engineering (SE). In the recent decade, Systematic Literature Review (SLR) was becoming an increasingly important empirical research method in SE. One of the mechanisms of insuring the level of scientific value in the findings of an SLR is to rigorously assess its validity. Hence, it is necessary to realize the status quo and issues of TTVs of SLRs in SE. Objective: This study aims to investigate the-state-of-the-practice of TTVs of the SLRs published in SE, and further support SE researchers to improve the assessment and strategies against TTVs in order to increase the quality of SLRs in SE. Method: We conducted a tertiary study by reviewing the SLRs in SE that report the assessment of TTVs. Results: We identified 316 SLRs published from 2004 to the first half of 2015, in which TTVs are discussed. The issues associated to TTVs were also summarized and categorized. Conclusion: The common TTVs related to SLR research, such as internal validity and reliability, were thoroughly discussed in most SLRs. The threats to construct validity and external validity drew less attention. Moreover, there are few strategies and tactics being reported to cope with the various TTVs.
Date
December 2016
Proceedings Title
2016 23rd Asia-Pacific Software Engineering Conference (APSEC)
Conference Name
2016 23rd Asia-Pacific Software Engineering Conference (APSEC)
Pages
153-160
Library Catalogue
IEEE Xplore
Extra
ISSN: 1530-1362
Citation
Zhou, X., Jin, Y., Zhang, H., Li, S., & Huang, X. (2016). A Map of Threats to Validity of Systematic Literature Reviews in Software Engineering. 2016 23rd Asia-Pacific Software Engineering Conference (APSEC), 153–160. https://doi.org/10.1109/APSEC.2016.031