Second, I'm going to look at some studies that compare results of
randomized clinical trials to non-randomized studies
of the same type of interventions.
And finally, I'm going to review some high profile cases where
the results were in conflict between clinical trials and observational studies.
And at the bottom of the slide I've got a little
glossary box for you so you can go back and refer
to that if some of the abbreviations don't make sense later on.
So to start out with a framework
for evaluating evidence for a particular healthcare intervention.
And we have to remember that what we're trying to do is to evaluate
a body of evidence and put together
evidence from clinical trials and observational studies.
And the focus really isn't on comparing whether one particular trial is better
than a particular observational study. It's really synthesizing the evidence.
But we need some guidelines for how to weight evidence.
And here's one proposed framework for how we should weight evidence.
And the idea of this pyramid is that
at the bottom of the pyramid is unsystematic clinical
observations, sort of case series, and those
would have the least weight in developing guidelines.
That we would be most suspect of those types of studies
that might have bias, that they don't have an appropriate control group.
And then we might go up to physiological studies that use surrogate outcomes, which
may provide some evidence, but don't provide
us evidence of the clinical effect of treatments.
And then you can see in this particular framework that
we have observational studies and single studies versus a systematic review
of observational studies where you put a lot of observational
studies together in what is commonly referred to as meta analysis.
And then, at the top, we have single randomized trials or
systematic review at the very top of more than one clinical trial.
So that framework has been elaborated on in this GRADE program, which is a system
for grading recommendations, assessment, and development of clinical evidence
that is a framework for combining evidence from randomized trials and observational
studies, and coming to an overall decision about a particular medical intervention.
So we
start here with study designs.
And you note that randomized clinical trials start at being considered
high quality evidence and observational studies not as high quality of evidence.
But there are factors that we might look at in a particular trial
or a particular group of trials that might lower the quality of that evidence.
And they have defined five areas to look at.
So whether there was a risk of bias in a particular trial, did they
have good allocation concealment, if they had
a subjective outcome was it masked assessment?
And those kinds of things to see if
there might have been bias in the clinical trial.
And they would lower the weight they put on that evidence if they found that bias.
And the same might be true for an observational
study if they felt like the control group and
the experimental group were very different and there was a lot of confounding.
They might lower that evidence from the observational study.
And other areas are inconsistency that there is lot of heterogeneity.
In the results from different studies, if the
evidence isn't really directly relevant to the question your
answering, perhaps the question your looking at is
whether a drug works in a pediatric population, and
all your studies are adult population. That would be one form of indirectness,
that it's not directly applicable to your population and the clinical question.
Imprecision would mean that there were wide confidence intervals.
And publication bias would be looking at, if the complete body of
information is out there to be reviewed. And then there are some features of
a particular set of evidence that may strengthen it.
That if they saw a large treatment effect, if there was a dose response.
And if they really thought that all of the confounding was either taken care of
or would actually operate against the results, so
that it was making the result more conservative.
So that's kind of more elaborate framework to grade individual
studies and also to know how to weight
them in final decision about what kind of recommendation.
I want to remind you of the key strengths of
a randomized clinical trials, and why it's considered a gold standard.
And the key strength is randomization, that we
can break the link between prognosis and prescription.
That there isn't confounding by indication
that patients that come in with a particular
set of risk factors aren't given a particular drug.
So that we can ensure that we have comparable groups.
And that is really the biggest strength of randomized clinical trials.
And it allows us to apply the statistical methodology based on random sampling.
We also have, in randomized clinical trials, generally more standardization
of the treatments, so that we can be more precise about
exactly what we're comparing, the
experimental treatment and the control treatment.
That can vary across different types of trials.
But it is generally more constrained than in an observational study.
And we also usually have a standardization of the outcomes assessment.
So we select primary outcomes,
we have rigorous protocols about how they're measured.
And try to ensure that it's unbiased, whether that's
by masked assessment or masking of the treatment groups.
And overall, then we assume everything else between the groups is equal.
Those are some of the key factors that allow clinical
trials to provide us with unbiased estimates of treatment effectiveness.
So that ends
our brief discussion of the framework of how we evaluate evidence.
