As you would expect we support any move that switches the focus of providing services away from the activity that takes place to the achievements that result. Effectiveness will always be more important than how busy you look. Having said that we are always concerned when the most important outcomes lose sight of the people using services.
In the realms of outcome measurement there two types of outcome that need to be separated out.
Put simply, they are individual outcomes and system outcomes. Individual outcomes focus on the change that has happened to the person, as a result of accessing a service (or hopefully, integrated group of services). System outcomes focus on the impact of the wider system, as a result of the service being operational.
In designing services sometimes it is easy to forget who the service is designed to benefit. That can be a particular problem when system outcomes dominate. It’s not surprising to see system outcomes become the driving force behind assessing whether or not a service is effective. In the age of the business case the cost savings, in other parts of the system, make the most compelling case for diverting scarce resources. Changes in patterns of use of other services, interpreted as a consequence of a service existing, are easy to measure.
There are numerous examples of system outcomes that have become regularly embedded in contracts and service level agreements but for purposes of illustration I will stick to two of the most topical; reduced A&E attendance and reduced GP appointments. These are all too common outcomes in contracts that often have a tenuous impact on them.
It’s understandable that both have become priorities; these are both significant costs to the system and reductions will free resources to, hopefully, meet need elsewhere. Unfortunately both assume that in addition to helping the system, they also help the individual.
System outcomes present three major challenges that are often overlooked in service design.
Causality
There is often no provable link between a service being provided and system outcomes. In the two examples I’ve provided what you are actually measuring is not outcome but activity. In a system where there is an almost limitless amount of need, people will flow through it like water. Much like water, people will try and find the point of least resistance to find the help and support they need.
How people use the system is more indicative of the structure of the system than the changing need. For example both the example system outcomes have a strong relationship with the perception of how accessible A&E and GP appointments are. A perception that GP appointments are hard to get drives traffic to A&E. The perception that there are long waits in A&E drive people away.
This is further confused when system outcomes are as interlinked as my two examples. Ostensibly, people with long term conditions being managed regularly by a GP could prevent A&E attendances.
It is more productive to judge commissioned services on outcomes that demonstrate a line of causality between their actions and health and wellbeing improvement than patterns of service use.
Shift of Emphasis
As the emphasis of contracts shifts towards the metrics of how the system functions then the way that services are delivered also shift to achieve those aims. For example we begin to provide convoluted services that divert people from A&E at the door rather than focussing on those preventative services that can help manage future demand. We focus on measuring the somewhat nebulous concept of whether someone has attended A&E in the last six months, ignoring that the measurement tells us nothing on an individual basis and only hypothesises benefit.
We develop an ever more complicated patchwork of services ignoring that it is that complexity that is driving people to use services “inappropriately” rather than changing the underlying need.
Benefit
Why do we design services? We hope that the main driver is to improve the health and wellbeing of the people that use them. Once we reach the point that the efficiency of the system itself becomes the focus then potential for benefit has been lost.
With my example system outcomes, both are fairly easy to achieve. Aggressive triage of GP appointments and providing inconvenient time windows for making appointments by phone reduce appointments. Stigmatising people who use A&E, as being a burden, reduce attendances. These can be cost saving measures, in the short term, but they can also deter people from seeking timely help and create greater future cost when need becomes critical.
These are extreme illustrations of how activity that isn’t beneficial to the individual can be seen as successful outcomes elsewhere in the system.
It is important for parts of the system to be aware of their own activity in order to manage their own resources. It’s not helpful for that activity to be the driver for other parts of the system that have little influence over them.
Maintaining a focus on the outcomes for individuals means you concentrate on those things that there is evidence for health and wellbeing improvement. In many cases you can use these as proxies for how they might impact the wider system.
For example we know that a reduction in social isolation will decrease the level of contact individuals have with public services. We know this because it is evidence based; reducing the impact on wider services does not need to the main driver behind a contract so reduce social isolation.
The system exists to benefit the individual, not to feed itself.