DISCUSSION
MAIN FINDINGS
Our international multi-stakeholder consensus provides the first specific integrity statement for promoting and protecting RCT integrity. It was developed in a robust and comprehensive manner, covering the entire RCT lifecycle. The general statements on RCT integrity emphasize the need for global harmonization and action. The statements relating to RCT design, approval, conduct and monitoring make clear that integrity needs embedding throughout the research lifecycle. The responsibilities of the publishing community are covered in statements concerning manuscript submission, peer-review, reporting and complaints. Further statements highlight the need for continuing research and development to advance responsible research conduct in RCTs. Drafted in a simple and clear language, the set of statements needs implementation by the clinical trialist community and related institutions to take forward the health research integrity agenda.
LIMITATIONS AND STRENGTHS
There are several issues to consider in the weaknesses and strengths of this consensus development study. Defining research integrity to determine the statement scope was not straightforward. Although there is no agreed definition,3,32 it is important to recognise that there is no controversy. To confidently use research results, society expects that the highest ethics standards and professionalism are deployed to conduct and report research.1 Defining integrity narrowly, focusing on post-submission or post-publication dishonesty assessments, fails to recognize that the whole research journey needs addressing.33 Our work is subject to other limitations including the possibility that the consensus group, which may be seen as having been derived from convenience sampling with a bit of snowballing, may not have included all perspectives despite an extensive effort to capture the widest possible range (Figure 1); our stakeholder group sample size was larger than the median of 22 experts included in previous reporting guideline development groups.34 The surveys and voting were, by nature of the consensus, opinion-based. Not every stakeholder endorsed every statement (see percentages of agreement in Table 3). For example, despite the high level of overall support (92.3% approval with good level of agreement among stakeholders in the first round), there was a strong individual objection to the role of data monitoring committee in providing oversight for data integrity (Table 3, statement 26). In another example, where two statistics experts disagreed over the interpretation of the underlying evidence35,36 used to formulate the statement concerning statistical significance (Table 3, statement 33), the overall level of support just crossed the threshold for consensus (69.2% approval in the second round). For implementing this statement, examples of valid analytic strategies in the presence of multiple outcomes reported in the published literature can be helpful.37–39 The use of the umbrella review20 added breadth and objectivity.40 For example, the statement concerning the input of professional medical writers arose from a systematic review (Table 3, statement 40).20 It did not emerge from the input of any stakeholder. If a reader suspects a conflict of interest, we provide all the disclosures of stakeholders´ interests (Appendix 1). Another criticism may be that the stakeholders may have been too lenient, inclined to promote integrity softly, instead of creating challenges for researchers, committees, publishers, etc. through hard-to-implement recommendations. By explicitly reporting the agreement levels and openly sharing the consensus data we intended to maximize transparency for readers. The consensus statement would, no doubt, need updating and revisions in the future.
Our strength is that we captured integrity issues across the RCT lifecycle, advancing on previous general statements.2,3 Using established, scientifically-based consensus techniques21–25 we developed a specific statement that is comprehensive, methodologically replicable and transparently reported (see appendices concerning author contributions, disclosure statements, and data sharing). The umbrella review20 contributed a high proportion of statements to those provided by stakeholders, who had a wide and appropriate range of expertise and experience including consumer representation.41 It is important to note that stakeholders themselves were not authors of RCTs with active expressions of concerns or retractions related to integrity. The lay member of the stakeholder group had experience of representing patients and public in research,42 assisting trialists in design and conduct, serving as member of oversight committees, and scoring RCT grant applications for funding.
Surveys were anonymised with objectively determined statement approval thresholds and subjected to sensitivity analysis. Several statistics are available in the literature to determine the degree of consensus among respondents within a panel, including stipulated number of rounds, subjective analysis, APMO, mode, mean/median rating and others.24 Our chosen statistics, APMO and the predefined arbitrary threshold, are among the most commonly used.24 Additionally, we used IQR to quantify the strength of agreement among the stakeholders as a measure of how closely they agreed or disagreed with each other. The approval threshold was determined arbitrarily during the final voting round, something that should be improved in future consensuses. Through various consensus and feedback cycles, each statement was worded for maximum clarity of meaning and avoiding ambiguities. With focus on practicality, the statement set provides recommendation for embedding and enhancing RCT integrity standards. All the statements in the final set had high level of consensus across our stakeholder group.
INTERPRETATION OF THE FINDINGS
Our statement provides the basis for creating implementation plans and policies at stakeholder institutions and organisations to help inculcate integrity in RCTs. It is necessary to invest in the clinical research infrastructure required to support trustworthy RCTs. Protecting and promoting RCT integrity requires a multifaceted approach, e.g. a combination of continuing education in best research practice in clinical trials targeting a range of audiences, improved governance and audit, and editor and peer-reviewer training in methodology. (Un)intentional errors can be reduced but cannot completely be eliminated. Admission of mistakes without the risk of persecution is a key aspect of continuous improvement.43 To improve RCT credibility in health research, strategies to reduce the probability of errors are urgently required,44 something that our statement emphasises. As far as trial oversight is concerned, the statement suggests that ethics committees, in addition to their traditional protocol appraisal and approval function before a trial can begin, should be given a role in monitoring the conduct of the trial. Deliberations of the trial oversight committees should be formally documented and, in the future, may need to be made publicly available during the course of the trial to match the growing transparency demands. On completion of the trial, chairs of ethics and oversight committees may provide certificates of authenticity to the authors for submission with their trials’ manuscripts.
The statement recognises biomedical journals as key stakeholders in RCT integrity, as is obvious from the proportion of editors and peer-reviewers represented on our consensus group. It was recognised that majority of the journals’ instructions to authors lacked sufficient detail to guide trialists to report their trial findings with integrity.45 This was specifically highlighted to be the case for the information related to reporting of ethics approval, sources of finding, potential conflict of interests, trial registration and statistical analysis plans.46–50 When an allegation of possible scientific misconduct is made, journals have an obligation to investigate in an unbiased manner with an explicit policy about managing conflicts of interests of their editors, peer-reviewers and advisors. Our statement advises authors to actively engage with journal investigation process and submit their de-identifiable raw data to be examined if required. As a matter of good practice with respect to promoting transparency, authors can voluntarily electronically submit their data in a repository at the same time as submission of the trial manuscript. There is no logical reason to not be proactive, waiting for this to be made a mandatory requirement, which no doubt is the natural next step in the development of the ICMJE data sharing statement.51 Hopefully, it will help in reducing the risk of complaints.
The reported prevalence of scientific misconduct is 2-14%.52 During an investigation misconduct may appear obvious, for example when repeated duplications of observations (coping and pasting of rows and columns) or a formula to generate false data in a spreadsheet raise suspicion. However, in every case before arriving at a decision about flagging an RCT as being fraudulent a careful investigation of the raw data is required. If tools for detecting misconduct perform poorly, this would lead to false positive findings.53 Wrongful accusations will damage science and healthcare.43,54 Accurately detecting misconduct should therefore be a focus of future research to support peer-review and evaluation of post-publication concerns. Education in good research ethics, governance and monitoring may be currently more effective in generating trustworthy randomised evidence.55,56