DISCUSSION
MAIN FINDINGS
Our international multi-stakeholder consensus provides the first
specific integrity statement for promoting and protecting RCT integrity.
It was developed in a robust and comprehensive manner, covering the
entire RCT lifecycle. The general statements on RCT integrity emphasize
the need for global harmonization and action. The statements relating to
RCT design, approval, conduct and monitoring make clear that integrity
needs embedding throughout the research lifecycle. The responsibilities
of the publishing community are covered in statements concerning
manuscript submission, peer-review, reporting and complaints. Further
statements highlight the need for continuing research and development to
advance responsible research conduct in RCTs. Drafted in a simple and
clear language, the set of statements needs implementation by the
clinical trialist community and related institutions to take forward the
health research integrity agenda.
LIMITATIONS AND STRENGTHS
There are several issues to consider in the weaknesses and strengths of
this consensus development study. Defining research integrity to
determine the statement scope was not straightforward. Although there is
no agreed definition,3,32 it is important to recognise
that there is no controversy. To confidently use research results,
society expects that the highest ethics standards and professionalism
are deployed to conduct and report research.1 Defining
integrity narrowly, focusing on post-submission or post-publication
dishonesty assessments, fails to recognize that the whole research
journey needs addressing.33 Our work is subject to
other limitations including the possibility that the consensus group,
which may be seen as having been derived from convenience sampling with
a bit of snowballing, may not have included all perspectives despite an
extensive effort to capture the widest possible range (Figure 1); our
stakeholder group sample size was larger than the median of 22 experts
included in previous reporting guideline development
groups.34 The surveys and voting were, by nature of
the consensus, opinion-based. Not every stakeholder endorsed every
statement (see percentages of agreement in Table 3). For example,
despite the high level of overall support (92.3% approval with good
level of agreement among stakeholders in the first round), there was a
strong individual objection to the role of data monitoring committee in
providing oversight for data integrity (Table 3, statement 26). In
another example, where two statistics experts disagreed over the
interpretation of the underlying evidence35,36 used to
formulate the statement concerning statistical significance (Table 3,
statement 33), the overall level of support just crossed the threshold
for consensus (69.2% approval in the second round). For implementing
this statement, examples of valid analytic strategies in the presence of
multiple outcomes reported in the published literature can be
helpful.37–39 The use of the umbrella
review20 added breadth and
objectivity.40 For example, the statement concerning
the input of professional medical writers arose from a systematic review
(Table 3, statement 40).20 It did not emerge from the
input of any stakeholder. If a reader suspects a conflict of interest,
we provide all the disclosures of stakeholders´ interests (Appendix 1).
Another criticism may be that the stakeholders may have been too
lenient, inclined to promote integrity softly, instead of creating
challenges for researchers, committees, publishers, etc. through
hard-to-implement recommendations. By explicitly reporting the agreement
levels and openly sharing the consensus data we intended to maximize
transparency for readers. The consensus statement would, no doubt, need
updating and revisions in the future.
Our strength is that we captured integrity issues across the RCT
lifecycle, advancing on previous general
statements.2,3 Using established, scientifically-based
consensus techniques21–25 we developed a specific
statement that is comprehensive, methodologically replicable and
transparently reported (see appendices concerning author contributions,
disclosure statements, and data sharing). The umbrella
review20 contributed a high proportion of statements
to those provided by stakeholders, who had a wide and appropriate range
of expertise and experience including consumer
representation.41 It is important to note that
stakeholders themselves were not authors of RCTs with active expressions
of concerns or retractions related to integrity. The lay member of the
stakeholder group had experience of representing patients and public in
research,42 assisting trialists in design and conduct,
serving as member of oversight committees, and scoring RCT grant
applications for funding.
Surveys were anonymised with objectively determined statement approval
thresholds and subjected to sensitivity analysis. Several statistics are
available in the literature to determine the degree of consensus among
respondents within a panel, including stipulated number of rounds,
subjective analysis, APMO, mode, mean/median rating and
others.24 Our chosen statistics, APMO and the
predefined arbitrary threshold, are among the most commonly
used.24 Additionally, we used IQR to quantify the
strength of agreement among the stakeholders as a measure of how closely
they agreed or disagreed with each other. The approval threshold was
determined arbitrarily during the final voting round, something that
should be improved in future consensuses. Through various consensus and
feedback cycles, each statement was worded for maximum clarity of
meaning and avoiding ambiguities. With focus on practicality, the
statement set provides recommendation for embedding and enhancing RCT
integrity standards. All the statements in the final set had high level
of consensus across our stakeholder group.
INTERPRETATION OF THE FINDINGS
Our statement provides the basis for creating implementation plans and
policies at stakeholder institutions and organisations to help inculcate
integrity in RCTs. It is necessary to invest in the clinical research
infrastructure required to support trustworthy RCTs. Protecting and
promoting RCT integrity requires a multifaceted approach, e.g. a
combination of continuing education in best research practice in
clinical trials targeting a range of audiences, improved governance and
audit, and editor and peer-reviewer training in methodology.
(Un)intentional errors can be reduced but cannot completely be
eliminated. Admission of mistakes without the risk of persecution is a
key aspect of continuous improvement.43 To improve RCT
credibility in health research, strategies to reduce the probability of
errors are urgently required,44 something that our
statement emphasises. As far as trial oversight is concerned, the
statement suggests that ethics committees, in addition to their
traditional protocol appraisal and approval function before a trial can
begin, should be given a role in monitoring the conduct of the trial.
Deliberations of the trial oversight committees should be formally
documented and, in the future, may need to be made publicly available
during the course of the trial to match the growing transparency
demands. On completion of the trial, chairs of ethics and oversight
committees may provide certificates of authenticity to the authors for
submission with their trials’ manuscripts.
The statement recognises biomedical journals as key stakeholders in RCT
integrity, as is obvious from the proportion of editors and
peer-reviewers represented on our consensus group. It was recognised
that majority of the journals’ instructions to authors lacked sufficient
detail to guide trialists to report their trial findings with
integrity.45 This was specifically highlighted to be
the case for the information related to reporting of ethics approval,
sources of finding, potential conflict of interests, trial registration
and statistical analysis plans.46–50 When an
allegation of possible scientific misconduct is made, journals have an
obligation to investigate in an unbiased manner with an explicit policy
about managing conflicts of interests of their editors, peer-reviewers
and advisors. Our statement advises authors to actively engage with
journal investigation process and submit their de-identifiable raw data
to be examined if required. As a matter of good practice with respect to
promoting transparency, authors can voluntarily electronically submit
their data in a repository at the same time as submission of the trial
manuscript. There is no logical reason to not be proactive, waiting for
this to be made a mandatory requirement, which no doubt is the natural
next step in the development of the ICMJE data sharing
statement.51 Hopefully, it will help in reducing the
risk of complaints.
The reported prevalence of scientific misconduct is
2-14%.52 During an investigation misconduct may
appear obvious, for example when repeated duplications of observations
(coping and pasting of rows and columns) or a formula to generate false
data in a spreadsheet raise suspicion. However, in every case before
arriving at a decision about flagging an RCT as being fraudulent a
careful investigation of the raw data is required. If tools for
detecting misconduct perform poorly, this would lead to false positive
findings.53 Wrongful accusations will damage science
and healthcare.43,54 Accurately detecting misconduct
should therefore be a focus of future research to support peer-review
and evaluation of post-publication concerns. Education in good research
ethics, governance and monitoring may be currently more effective in
generating trustworthy randomised evidence.55,56