Counting the benefits – evaluation of healthcare programs

Over 35% of all health care costs in Australia are attributable to four chronic diseases. In a recent paper, actuaries Bronwyn Hardy and Barry Leung canvassed disease management program issues for Australian private health insurers to consider.

Around the world, illness and death from chronic disease is increasingly widespread and represents a significant cost to both individuals and society. Over 35% of all health care costs in Australia are attributable to the top four chronic conditions – cardiovascular diseases, oral healthi, mental disorders, and musculoskeletal conditions (AIHW 2008-09).

Chronic diseases can have very broad impacts, on the individual, their friends and family, their immediate community and society generally. Some impacts include: pain and suffering, increased medical costs (to the individual or society), productive time lost and reduction in quality of life.

Disease management programs provide patients with coordinated community support and to empower them to self-manage their diseases, both over the short and longer term. These programs may not cure the disease, but many believe they may help alleviate the symptoms, slow disease progression or reduce secondary impacts.

For a long time, Australian private health insurers in have had an interest in disease management. Assisting policyholders to effectively manage their condition can reduce the long-term claims costs, as well as improving member retention and satisfaction levels. It also aligns with the objectives of many insurers to foster policyholder wellbeing.

An important part of the disease management framework is the evaluation and continuous improvement of such programs. Given the relatively short history of disease management in Australia, existing evaluation approaches remain highly variable and are under constant review and development. In our paper we provide a high level introduction to disease management program evaluation for Australian private health insurers. We do not provide an assessment of different methodologies, but focus instead on the key principles and issues to consider. 

Evaluation framework

We develop the following checklist to ensure that critical elements of program design are captured in a timely manner, in order to assist with effective evaluation.

Program objectives

o   Ensure these are clearly articulated, and that there is a common understanding of the language used

o   Test whether there are secondary, or tertiary, objectives

  • Marketing
  • Retention
  • Customer satisfaction

o   Consider the most effective ways to measure success

Agree high level evaluation techniques

o   Clinical (if appropriate)

o   Financial

  • Return on investment (need to agree how savings and costs will be determined)
  • Loss ratio (accepting this is also impacted by pricing changes)
  • Admission Rates
  • Bed Days per 1,000 lives
  • Absolute cost of program (pre or post risk equalisation)
  • Cost of capital

o   Other

Specify timeframes for evaluation

o   Pre-program evaluation (the period selected can significantly influence the reported results)

o   Intervention period

o   Post-intervention period

Determine whether participants will be classified into cohorts (the initial members often have a different experience to those who join later)

o   How will cohorts be determined

Agree the comparative group – noting again that the reported results will vary depending upon the group selected

o   The control group if a RCT is performed

o   This may be the same cohort pre-intervention (with or without allowance for trends)

o   Alternatively it may be a different group (e.g. those who declined to participate)

o   Consider also comparing to a broader baseline of product or insurer membership

Ensure required data will be collected and available in a usable format

o   How will the data be collected?

o   Is there anything that needs to be recorded differently

Consider adjustments

o   Will allowance be made for reversion to the mean effects?

o   Will the comparative group be adjusted to reflect differences in characteristics with intervention cohort?

Ensure the evaluation will assist the business in making strategic decisions

o   Would the results of this evaluation enable you to choose appropriately between competing options? Would the results help the business to improve existing programs and other practices? 



Disease management programs can improve self-management practices amongst chronic disease patients, and slow the rate of disease progression while maximising health and wellbeing. Although disease management in Australia is still underdeveloped compared to the US and Europe, it is gaining momentum. As the number of programs increases, the need for program evaluation is going to grow.

We believe that actuaries have a role to play as the actuarial skill set is closely aligned with the requirements of a health program evaluator. We hope the paper will provide some useful insights for those who undertake program evaluation. Every program is different in nature and evaluators should adapt their methodologies to suit the circumstances. There are many considerations that evaluators need to take into account when performing an evaluation.

While there is no recognised “gold standard” in program evaluation, the exercise is still worthwhile for both the funder as well as the wider community. As the knowledge of disease management increases through quality evaluations, better programs can be designed, and most importantly, we can help improve future patient outcomes.

View the Presentation and Audio around this paper, presented at the Actuaries Summit 2015.

Oral health may not typically be considered a chronic disease or condition; however the AIHW ranks it with other leading diseases due to the costs attributed to this area. Refer for example to  

CPD: Actuaries Institute Members can claim two CPD points for every hour of reading articles on Actuaries Digital.