Objective: To assess the acceptability, safety and efficacy of the PPH Butterfly, a new uterine compression device, in women with postpartum haemorrhage (PPH). Design: A phase two clinical device trial using matched historical controls, with accompanying grounded theory study. Setting: UK university consultant obstetric unit. Population: women with PPH after vaginal birth unresponsive to initial oxytocin therapy. Outcomes were compared to historical controls matched on blood loss, parity and type of birth. Methods: after oral consent, trained staff used the device in additional to normal care. Main Outcome Measures: The primary outcome was additional blood loss >1000mls. Qualitative interviews assessed device feasibility and acceptability. Results: Of the 57 recruits, two-thirds were primiparous and almost half had undergone operative birth. Two percent of recruited women had additional blood loss of over 1000mls compared to 8% of 113 controls (adjusted odds ratio 0.13, 95% CI (0.02 to 1.09)). Women treated with the device received significantly more additional treatments and had higher rates of exclusive breast-feeding at discharge. There were no serious adverse events related to the device. In 47 interviews, participants, birth partners, clinicians and attending midwives viewed the device positively. Clinicians found it useful to stop blood loss and diagnose the source of bleeding. Conclusions: the PPH Butterfly is acceptable and may have clinical benefits: it is a promising device for PPH management. Funding: National Institute for Health Research invention for innovation (i4i) program (II-LA-0715-200008) Keywords: postpartum haemorrhage, childbirth, oxytocin, third stage of labour, uterine compression. Registration: prospective ISRCTN (15452399); www.isrctn.com/ISRCTN15452399

BJOG-20-2353.R2 What should we believe when systematic reviews disagree?For many years, uterine balloon tamponade (UBT) has been used to treat severe postpartum haemorrhage (PPH), despite a lack of randomised trials to demonstrate its effectiveness. With commercial devices being expensive, clinicians in low resource settings have made their own using 2 low-cost, widely available items (Foley catheters and condoms). Public health experts have been so confident of their benefit that large programmes have been set up to disseminate the necessary skills worldwide.Recently, however, the global maternal health community has been thrown into disarray when not one, but two randomised controlled efficacy trials suggested that outcomes with condom catheter UBTs were actuallyworse than normal care in low resource settings. The Cochrane meta-analysis concluded that “in [low resource] settings, balloon tamponade [should be] only introduced alongside multi‐system improvements in PPH care” (Kellie et al. Cochrane Database of Systematic Reviews 2020(7): CD013663).Systematic reviews may be the pinnacle of evidence-based medicine, but even they can differ in how to interpret evidence. And so, proponents of the condom catheter conducted their own systematic review with far wider inclusion criteria – they not only included the randomised trials but examined success rates from 15 non-randomised trials and 69 case series (Suarez et al. Am J Obstet Gynecol 2020;222(4):293.e1–e52). Furthermore, the primary outcome was the success rate of the technique (overall 86%) rather than the risk of morbidity and mortality compared to controls, as used in the Cochrane review. This is problematic, as reported ‘success rates’ without controls can be very difficult to interpret: in initial case series misoprostol showed similar success rates against life-threatening haemorrhage before RCTs eventually showed it to be less effective than oxytocin.A third version of the same review is published today by a WHO team (Pingray et al. BJOG 2021;XXX,XXX). This time they include only 4 high quality studies in which UBT was compared to standard care. With a composite maternal morbidity / mortality outcome, they found no evidence of benefit and concluded that “the effect … is unclear, as is the role of the type of device and the setting”. WHO studies are now underway to address this uncertainty.But why is this all so important? The difficulty comes because the World Health Organisation has been updating its guidance on PPH management, and had to declare a position on UBT. The recently-published guideline, drawn up by independent experts, accepts the validity of the RCTs but recognises that there is wide acceptability of the technique and that the evidence of harm is only for condom catheters and onlyin resource-poor settings. They put a high emphasis on minimising harm and conclude that UBTs should only be used in settings where there is already a good standard of care, including recourse to blood transfusion and surgery if needed (WHO. Geneva: World Health Organization; 2021).Until further studies are published, the debate will continue. But this episode shows how the choice of inclusion criteria and outcomes in systematic reviews are critical, both for their conclusions, and for global policy.

Re: Training in the use of intrapartum electronic fetal monitoring with cardiotocography: systematic review and meta‐analysis. Cardiotocography training is a complex intervention and requires complex evaluations: a letter to editor.Lightly K, Weeks AD, Scott HCorrespondence to Dr Kate Lightly – [email protected] Research Fellow, Sanyu Research Unit, University of Liverpool and Liverpool Women’s NHS Foundation Trust, members of Liverpool Health Partners, Crown Street, Liverpool, L8 7SS.Professor Andrew D WeeksProfessor of International Maternal Health, Sanyu Research Unit, University of Liverpool and Liverpool Women’s NHS Foundation Trust, members of Liverpool Health Partners, Crown Street, Liverpool, L8 7SS.Professor Hazel ScottDean of School of Medicine, School of Medicine, University of Liverpool, Cedar House, Liverpool, L69 3GE.Running title - CTG training requires complex evaluationsWe congratulate Kelly et al on their review on the effects of training in cardiotocography (CTG).i It is a critical step towards understanding how to correctly implement CTG training. However, we query the relevance of some of the included studies and whether using Kirkpatrick’s model adequately captures all of the relevant complexities. We believe that further work is needed to understand how such training will impact on practice.Some of the CTG research projects reviewed were poorly representative of the needs of clinicians tasked with improving fetal monitoring in their hospitals. For clinicians, their population of interest is practising clinicians who work on labour wards (sometimes infrequently); studies involving undergraduates alone may not be generalisable. Training in intermittent auscultation is also relevant. Considerable detail on the training intervention is required, including not only the format of teaching, but the duration, curriculum and proportion of relevant staff trained. Detail is also required of supporting interventions and context, as training alone is unlikely to impact change. Many would consider ‘no training’ unethical and therefore not a relevant comparator . In the UK, CTG training and competency is now required for all maternity staff.ii The ultimate aim of CTG (and therefore CTG training) is to detect the hypoxic fetus, so that timely intervention can be undertaken to avoid perinatal harm, without unnecessary intervention. Therefore, the outcomes of interest have to include intrapartum stillbirths, hypoxic ischaemic encephalopathy and mode of delivery.Kirkpatrick’s model was used in this review to evaluate training at four levels (reaction, knowledge, behaviour change and organisational performance). However, it does not attempt to understand why interventions work, or the context, or the causal pathways between training and change in practice.iii Whether participants like training (‘reaction’) is of little relevance, and knowledge acquisition (‘knowledge’) is a proxy which does not equate with improved on the job performance and outcomes. Maternal and perinatal outcomes (‘performance’) were only collected in a small number of studies.CTG training is a complex intervention which aims to create change, not simply knowledge acquisition. We therefore believe that a formal Realist Evaluation is needed. This emerging methodology has been used successfully to understand healthcare change processes and supplement traditional Cochrane style reviews. It aims to understand why complex interventions work, how, for whom, in what context and to what extentiv. Collaboration with the relevant authors to gain detailed intervention descriptions, with a realist approach, may add some much needed explanatory power to this critical subject.Training alone is unlikely to impact change. Even the best educational package will fail without the necessary support - it needs an educational and working culture which supports learning and change, aligned and clear policies, and motivated, well supported drivers and leaders.How doctors learn and what supports them to put new knowledge into practice are key research questions. High quality, methodologically appropriate, properly funded studies are needed to address these questions. Not answering them means many research findings are redundant, as they simply will not be implemented.ReferencesI Kelly S, Redmond P, King S, Oliver‐Williams C, Lamé G, Liberati E et al. Training in the use of intrapartum electronic fetal monitoring with cardiotocography: systematic review and meta‐analysis. BJOG. 2021; 00: 1–12. https://doi.org/10.1111/1471-0528.16619ii NHS England. Saving Babies’ Lives Version Two. A care bundle for reducing perinatal mortality. London: NHS England 2019. [cited 2021 Feb 3]. Available from: www.england.nhs.ukiii Moreau KA. Has the new Kirkpatrick generation built a better hammer for our evaluation toolbox? Med Teach. 2017 Sep; 39(9): 999-1001. https://doi.org/10.1080/0142159X.2017.1337874iv Wong G, Westhorp G, Greenhalgh J, Manzano A, Jagosh J, Greenhalgh T. Quality and reporting standards, resources, training materials and information for realist evaluation: the RAMESES II project. Health Services and Delivery Research. 2017 5 (28): 1–108. https://doi.org/10.3310/hsdr05280.Acknowledgements – NilDisclosure of interests - NilContribution to authorshipKL wrote the first draft of this letter and then it was revised by AW and HS.Details of ethics approval – N/AFunding – Dr Lightly’s PhD entitled “Improving intrapartum fetal monitoring in India: A mixed methods approach” is funded by MRC/DfID/Wellcome Trust Joint Global Health Trials Fund. MR/R006/1801

Mini-commentary on BJOG-20-1239.R3: Intramuscular oxytocin versus Syntometrine versus carbetocin for prevention of primary postpartum haemorrhage after vaginal birth: a randomised double blinded clinical trial of effectiveness, side effects and quality of life