Developing CAMELOT for assessing methodological limitations of
qualitative research for inclusion in qualitative evidence syntheses
Abstract
Introduction Qualitative evidence is increasingly incorporated into
decision-making processes. Assessing the methodological limitations of
primary studies is critical to making an overall assessment of
confidence in findings from qualitative evidence syntheses (QES) using
GRADE-CERQual. Current critical appraisal tools were not developed
specifically for use in Cochrane reviews or GRADE-CERQual, and few are
evidence-based. The aim of CochrAne qualitative Methodological
LimitatiOns Tool (CAMELOT) was to address this gap. Methods We undertook
this project in four stages: (1) systematic literature search to
identify existing tools, (2) identification of evidence to support
inclusion of potential CAMELOT domains (3) consensus survey to agree on
the inclusion and definition of CAMELOT domains, and (4) human-centred
design approach to develop and refine CAMELOT by exploring user
experience. Results CAMELOT is a new evidence-based tool for assessing
the methodological strengths and limitations of primary qualitative
research studies in a QES. CAMELOT is comprised of twelve domains: four
Meta domains that encourage review authors to consider those
characteristics of the primary study that are beyond how the study was
carried out, but which inform the conduct and design of the study, and
eight Method domains which encourage review authors to consider how the
study was designed, planned and/or conducted, and how study conduct and
design fits with the information provided in the four Meta domains.
Review authors make an assessment by identifying any concerns regarding
the methods used in the study and considering the appropriateness of fit
between the Meta and Method domains. Conclusion CAMELOT provides review
authors with a transparent and systematic method to assess
methodological limitations of primary qualitative studies. CAMELOT
incorporates qualitative principles and focuses on appropriateness of
fit between Meta and Method domains. In line with iterative tool
development approach, CAMELOT will continue to be revised over time
following extensive user testing and piloting.