The need to assess eMM cost effectiveness

Prof Johanna Westbrook’s Study on the Cost-Effectiveness of eMM

Professor Johanna Westbrook is Director of the Centre for Health Systems and Safety Research (CHSSR): the largest health informatics evaluation research team in Australia. Her extensive research expertise has focused on the design and implementation of complex multi-method evaluations across the health sector. One of her most recent studies examined the evidence for the cost-effectiveness of eMM systems by comparing the costs and benefits experienced against a paper-based system.

Here is a summary of the key findings of this study which were presented as part of Prof Westbrook’s keynote address at the 2015 eMedication Management Conference in Sydney.

The need to assess eMM cost effectiveness

Prof Westbrook and her team decided to undertake this study following observations that there was limited data and resources available on the cost-effectiveness of eMM systems.

In 2012, an American review on the economic evaluations of medical-related health information technology identified 5 full economic evaluations and 26 partial evaluations where people had made references to total costs incurred during the implementations of clinical decision-support systems (CDSS) in hospitals and primary care environments. However, the data of this review contained considerable uncertainties about whether the additional costs of CDSS actually represented value for money.

In addition, Australian’s health system is entirely different to the American health care system.

In 2013,  the Victorian Auditor-General highlighted the need for an Australian study by remarking that,

“until this work is done, it will be difficult to convince taxpayers that public funds have been well spent on these systems and that any further investment on clinical ICT systems is justified or will improve clinical and patient outcomes.”

The research conducted by Prof Westbrook’s team was designed to help get a full understanding of how better to make decisions in redesigning an eMM system’s features and applications to adapt to the changing needs of the industry on which it would be implemented.

A thorough preliminary evaluation would likewise help break down the investment capital and costs needed in order for an eMM system to reap the benefits down the track.

A look into the study method

To resolve the challenge on coming up with credible end results, the research team adapted the decision tree analysis method and made use of the incremental cost effectiveness ratio (ICER), which included the following variables:

• the initial operating cost of implementing the system in the hospital
• the cost to maintain the eMM system over a 15-year period
• the offsets that might be accrued downstream

Another part of the equation looked at the effectiveness of the system in terms of actually reducing adverse drug events after implementation.

eMM Effectiveness: The study proper

The cost-effectiveness study for an eMM implementation carried out by Prof Westbrook and her team was facilitated in the 30-bed cardiology ward of Sydney’s St. Vincent’s Hospital.

For the effectiveness data for this study, the team used the diagnostic tree analysis, where the process officially starts with the patient being admitted in the hospital. The diagnostic tree branches out to the pre- and post-eMM possibilities with the basic inclusions of patients getting medication errors, patients with no potential adverse drug events (ADEs), and patients with potential adverse drug events (ADEs).

Potential ADEs have the possibility of either being intercepted by somebody before they get to the patient or they won’t be intercepted at all—which was the unique part of this decision tree study as it had not been covered in any data before this.

When an ADE is not intercepted, the diagnostic tree shows the possibilities of the patient either experiencing harm or not experiencing any harm at all. In any event that non-interception of the ADE occurs and harm results from such, the study focused on the three most significantly identified errors based on their severity, which are significant ADE, serious ADE, and severe ADE.

How robust were the results?

To ensure that the results of the study were robust and compelling, the team conducted a range of sensibility analysis where they changed all the different proportions according to other evidence they had (or if they didn’t have the evidence, they just doubled or halved it).

Carrying out these analyses across 12 different proportions revealed different results. For the vast majority however, it made no difference in terms of whether the system was actually making some savings.

Eventually, the research team wanted to come up with the figures on the threshold by which a breakeven point was reached and two important points were discovered

1. Increasing the cost of the eMM system by 2.65 times would be the breakeven point and,
2. Reducing the effectiveness of the system by 57 percent could still reach breakeven point

One area of the study was particularly sensitive and had the least amount of data where when the proportion related to the probability of the harm resulting from a non-intercepted ADE was changed, only 3 studies showed data about this. The second study used all medication errors, whereas the one used by Prof Westbrook’s team only looked at ADEs relating to prescribing errors. This other value was put into the study as well and it found that if the savings reduced from $66 per admission to $29 per admission, which was a modest difference.

On a different study however, they found that with all medication errors only 9.6% of non-intercepted ADEs cause harm, which was considered staggeringly low but if that figure is placed into the original model used it would cost $2,466 to prevent an ADE —which is generally reasonable and good for the health care system.

Research summary

Prof Johanna Westbrook summed up her presentation with three notable figures associated with the implementation of the eMM system:

1. AUD$66 to AUD$63: the estimated savings per admission
2. AUS$97,740 to AUD$102,000: the estimated annual savings per admission in the cardiology ward alone (with less 80 ADEs)
3. AUD$2.5 million:  the estimated annual savings for the entire hospital with 39,000 annual admission

Comparative results with other studies

A Canadian research across three hospitals discovered that it costs AUD$12,700 to prevent one ADE. Although, one big challenge with comparing results is that the in-house system that they used from the U.S. and did not relate with local circumstances, which served as a limitation.

On the other hand, a Dutch team which conducted a similar study discovered that it costs AUD$449 to prevent 1 ADE. They likewise used the same effectiveness data and found 15% post-eMM results. It should be noted that they took a slightly different analysis approach and have more transcription than what is locally done.

Conclusions

Australian policy makers, like their international counterparts, should be confident that investing in eMM systems delivers value for money.

The research clearly highlights that advancement in technology enables eMM systems to become more effective and continue growing into a more efficient support system for the health care industry.

Moving forward, all suggested plans, additional features, and new designs come with a price so further execution must take into account the decision to make additional expenses in maintaining and developing the system if we want to reap the benefits down the track.

Of course, the need to establish new and fresh data to examine issues should also be taken into account.

We of course, look forward to hearing about the latest data and research findings in the field!

View the agenda for the 2016 eMM conference in Sydney this March :

Click to view 2016 eMM conference key speakers and agenda

To view more of this presentation, click below:

Professor Johanna Westbrook – AIHI, Macquarie University – KEYNOTE ADDRESS | The Cost-Effectiveness of EMM from Informa Australia