The world is constantly changing and new diseases as well as new preventions or interventions are constantly emerging. So we need to be able to keep track of those when we're looking at population health. So we're going to create new population health indicators along the way. So the big question is, what makes a really great population health indicator? In this module, we're going to cover some of the characteristics of really good population health indicators and help you to discern between good and not so good health indicators. There are 10 basic characteristics that I'd like to go over in this module. There are many more, but these tend sort of run the whole gamut of what makes a really good Population Health Indicator. They deal with things like relevance, validity, whether or not the statistic is well behaved and clearly specifies whether or not the population health indicator is timely, can be deconstructed so other people can see what's going on, whether or not it's feasible to measure it, whether it's free from bias, does it put forth a balanced account of what's going on and is it repeatable? Can we see what's going on over time? The first major characteristic that I'd like to focus on is relevance. So is this indicator relevant to the thing you actually want to measure? If there's a clear rationale and it fits the purpose, it's a relevant population health indicator. Suppose we want to estimate the prevalence of people with undiagnosed type two diabetes. What kind of indicator do we build? A good indicator is going to have who we're looking for in the numerator and everyone eligible in the denominator. For example, if we go into a community and measure fasting plasma glucose on 1,000 individuals and then take the number of people with fasting glucose above 126 milligrams per deciliter and divide by the number of people without a previous diagnosis of type two diabetes, that's probably our best estimate of the prevalence of undiagnosed type two diabetes in our community. A not so good indicator might be the proportion of new patients with fasting glucose above 126 milligrams per deciliter. Why isn't this good? Well, you're only looking at new patients, that's the denominator. But what about existing patients that just developed diabetes? Second, you're assuming that everyone that goes to see a doctor and has their fasting plasma glucose measured. Lots of people don't go to see their doctor and even if they do, they don't get their fasting glucose measured. The second characteristic is validity and there's two different types of validity. One is the face validity. This is really whether or not the indicator measures what it's meant to measure and you can tell it. I mean, it's right in the name of the indicator that, oh, I understand what this is measuring. The second kind is construct validity. This is where all the components in a composite measure makes sense. You're not mixing apples and oranges, you're really looking at things that all make sense to go together to create a composite score that actually validly reflects what's going on in the population. So let's go over an example of good face validity. Suppose that we want to estimate the fraction of type two diabetics with their disease under control. Well, a good indicator would be something like the proportion of type two diabetics with blood glucose control that's less than 126 milligrams per deciliter. A not so good estimate of face validity would be something like looking at the proportion of type two diabetics without a co-morbidity. Co-morbidity isn't defined and it doesn't directly assess the disease status that we're looking for. The third characteristic is how well these statistics behave. So what does that mean? It means that when you have the change in the value of the indicator, it's interpretable. So for instance, things that have random score and no units are sometimes difficult to interpret. Especially in composite indicators, if the indicators value is not changing and flowing with its components, it can be pretty difficult to apply to a community or a population setting. So we want all of these statistics to actually flow in the direction that's expected as the data is changing. That's what makes it well-behaved. Fourth, is it clearly specified? Can basically anybody with some kind of background or interest in the area look in and say, "Oh, I think I know how this indicator was constructed". So that usually means it has enough detail that you can figure out what's in the numerator, what's in the denominator, and then how to calculate that indicator. I'll give you an example of what we mean by clearly specified. So in the field of health analytics, that is now sort of a part and parcel of many health care systems, they often are looking at the patient population, but they're not really defining it as anything more than at a point in time or for instance, a change over two time points. So ultimately, you don't have an understanding of what's really in the denominator, or you don't really know when the same person is counted twice or not counted twice. So if you're not actually understanding what is in both the numerator and the denominator of a health indicator, because you're really just looking at statistics, so those are the summaries of the individual data, then you're not going to be able to really interpret and then apply that information. So let's say that we're looking at the percent increase in emergency department visits for people with type two diabetes? Well, from one to one, how do we know that there's just one observation per person? Or maybe we don't care about that, but how many people are contributing to this? Is it really just a few people that are constantly going to the emergency department or is it really that there's now a big increase, there's a lot of new people. So this particular health indicator wouldn't be able to discern between the two. The fifth characteristic is whether or not the indicator is timely. So good indicators reflect current trends and current needs. So if a new disease is popping up on the horizon. So for instance, when we first started to understand AIDS, then the mapping of HIV as the culprit really became an important population health indicator. Well, we have that with other diseases like Lyme disease, West Nile virus, the Zika virus. Things like that actually become an important new indicator to develop so people can see the emerging threats or the current trends and important population health aspects. Another great characteristic of a good health indicator is whether or not it can be deconstructed. Can we see inside and know what are its component measures? For complex measures, it's really valuable to be able to deconstruct it and to look inside because sometimes we can't act on the big composite or complex measure. We have to break it down. So looking at life expectancy as an example, it has some good qualities but really we want to be able to look inside. So looking at cause-specific mortality inside of that or mortality within specific age groups. Because when we actually formulate plans around population health interventions or preventions, we want to be able to tailor it to specific causes and to prioritize those that have the biggest impact. A lot of people these days are trying to develop summary scores that will help people in better understanding their risk. So for instance, cognitive function scores are used to predict risk for Dementia or Alzheimer's. The Framingham heart score here is used to create sort of 10-year risk scores for having a heart attack or a cardiovascular event and there are many good attributes about them. One of them that is difficult is, we don't really know what's inside of it and it's hard to act on different components in it. It really is representing kind of a composite that is not able to be deconstructed and so it limits its usability. Another important characteristic is feasibility, and this has a lot to do with the data. Is it feasible to measure this indicator? Can you measure it routinely? Can you compare it across jurisdiction? So we're thinking about the quality of the data in the numerator and the denominator. So if you've got good quality data sources from electronic medical records, the census, annual surveys, public health surveillance data sets, those things that you can count on really make good population health indicators feasible. If we're using convenience samples and one of indicators that nobody else can use or have access to and they are not repeatable, that really doesn't make a very feasible population health indicator for other people to use or for your community to sort of standby. We also want our indicators to be free from bias. So we want indicators that have hardwired into them, a neutrality that keeps people from selectively presenting data that is either overly optimistic or is too pessimistic. We don't want to lead people one way or the other. You want to play it straight in terms of here, what the facts of what we know is going on in this population. So this is an important characteristic to keep in mind. What's related to this freedom from bias is balance. That balance is really associated with how you're going to use the information and are you really excluding things that could be important to that, I'll call it temperature gauge in your population? If possible, you want to be able to consider how improvements in one area might have a negative impact on other areas when you're constructing these population health indicators. Last but not least is repeatability. Most of the time we are using indicators to be able to track not only where we are, but how we're doing and even predicting into the future. So repeatability has to do with how sort of stable the estimate is and most indicators are going to be, I'll call it insensitive to fluctuations if they have a longtime view and very sensitive to time fluctuations in the data if we're looking at very short time periods. So for example, looking at monthly mortality rates is not nearly going to give you the kind of clear view of what's going on than an annual mortality rate. If we are looking for good repeatability and stability in our inferences following these Indicators, you just want to be sensitive to how repeatable is that actual indicator when you measure it. Really good population health indicators use good epidemiological and statistical reasoning as their basic foundation. Ultimately, any indicator is got to be used by people and professionals hopefully to better understand what's going on in a community and to track the impact of interventions or programs, see if it's really making a big difference in people's lives. I hope that you get a chance to try your hand at developing your own population health indicators. It's a lot of fun. It's a creative sort of scientific process. It can be really rewarding. But if nothing else, what you have from this module is 10 good characteristics from which you can actually give constructive feedback to other people who may be trying to create these metrics but haven't quite got it right yet.
