Parent-led tool opens up NHS children's heart surgery data to families

Researchers are calling for the end to an era of confusion and alarm about children's heart surgery statistics by launching an innovative communication tool that will help people make sense of published survival data about children’s heart surgery in the UK and Ireland.

The website, Understanding Children’s Heart Surgery Outcomes, which launches today, shows decision makers and parents that hospitals should not be ranked by their survival rates, because hospitals treat different patients — high performing hospitals can have lower survival rates simply because they are taking on the most complex cases. An individual hospital’s actual survival rate should only be compared to its own predicted range, which is determined by the complexity of the procedures it undertakes, among other factors. The website also sets out why if a hospital's survival rate is below its predicted range, it need not indicate alarm, but rather serves as a trigger for further investigation.

The website was developed by Christina Pagel from University College London and Sir David Spiegelhalter from the University of Cambridge, in collaboration with the charity Sense about Science and experimental psychologist Tim Rakow from King’s College London. It explains a risk adjustment method known as PRAiS (Partial Risk Adjustment in Surgery).

“This website draws a line under an era of poor risk communication of hospital surgery statistics,” said Tracey Brown, director of Sense about Science. “In 2013 over-interpretation of faulty data resulted in temporary hospital closure of Leeds General Infirmary's paediatric heart unit. Parents and children were faced with all the additional stress, risks and costs of travelling further for operations, and for others the horrendous unnecessary guilt as they wondered if their child’s outcome would have been better at another unit. There could not be a stronger case for professionals and decision makers using the risk adjustment model and communicating it well.”

Each hospital that performs children’s heart surgery in the UK and Ireland has had its overall survival rates published by the National Congenital Heart Disease Audit (NCHDA) since 2013. The researchers used PRAiS to calculate a predicted range of survival for each specific hospital, taking into account the complexity of each individual child’s medical condition and surgery. No hospital will have exactly the same predicted range of survival as another hospital, because each hospital treated different children.

However, the report the NCHDA publishes is lengthy, hard to find and hard to understand without expert knowledge. Sense about Science ran user-testing workshops to involve the public, patients’ families and medical charities in co-designing the website with Pagel and Spiegelhalter, a first in this area.

“There is an understandable urge to put hospitals in a league table when comparing survival rates,” said Spiegelhalter, Winton Professor for the Public Understanding of Risk at Cambridge. “But rather than try and directly compare hospitals with each other, we need to compare a hospital’s survival rate with what we would predict it to be, taking into account how severe their cases are. This is a tricky idea but, with the help of many families, I think we have made it clear.”

“Because different hospitals treat different children and some children can have more complex medical problems than others, it is not valid to directly compare survival rates between hospitals,” said Pagel, Reader in Operational Research at UCL, who helped develop the formula the NHS uses to evaluate hospital survival rates. “We involved families from the beginning of the project and throughout to help us researchers communicate these complicated concepts clearly. I definitely learned that incorporating their feedback was absolutely crucial to building something useful. An accountable NHS is one where we can all understand how it is doing- and for this you need to listen to patients and families.”

Spiegelhalter and Pagel are calling for other researchers, companies and government to make health statistics accessible to patients and families by making them understandable. Transparency without accessibility is not enough; improved understanding by decision makers, health care professionals, patients and families can prevent misuse, confusion and unfounded anxiety.