Checklists for external validity: a systematic review

Rationale, aims and objectives The quality of the current literature on external validity varies considerably. An improved checklist with validated items on external validity would aid decision-makers in judging similarities among circumstances when transferring evidence from a study setting to an implementation setting. In this paper, currently available checklists on external validity are identified, assessed and used as a basis for proposing a new improved instrument. Method A systematic literature review was carried out in Pubmed, Embase and Cinahl on English-language papers without time restrictions. The retrieved checklist items were assessed for (i) the methodology used in primary literature, justifying inclusion of each item; and (ii) the number of times each item appeared in checklists. Results Fifteen papers were identified, presenting a total of 21 checklists for external validity, yielding a total of 38 checklist items. Empirical support was considered the most valid methodology for item inclusion. Assessment of methodological justification showed that none of the items were supported empirically. Other kinds of literature justified the inclusion of 22 of the items, and 17 items were included on the basis of consensus. On 36 occasions, the items were presented without methodological justification for inclusion. Assessment of frequency/occurrence showed that items were mentioned in one to at most 17 checklists. Conclusion This paper provides building blocks for the development of a new checklist for external validity. The next step is provision of empirical evidence for the checklist items to be selected, and finally, development and validation of a checklist on external validity.

_eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/jep.12166