Features
Reader Beware: Interpreting Clinical Trial Data
Numbers don’t lie – except when people cause them to.
W
ith an abundance of scientific
research being presented and
published, many health-care
practitioners rely on the researchers’
interpretation of their data to glean clini-
cally relevant information. But, as some
recent examples have shown, they might
need to pay closer attention to how data
are being reported.
“It is incumbent on clinicians to be
able to read the literature with a critical
eye, rather than relying on others to vet
the findings,” said Ian F. Tannock, MD,
PhD, DSc, emeritus professor of medical
oncology at Princess Margaret Cancer
Centre in Toronto. “There are [statements]
that get into even the highest-impact
journals that are questionable.”
What appears in scientific journals –
even reputable, high-impact ones – may
represent an accumulation of mistakes,
errors, and miscalculations that occur
throughout the research process – from
trial design, conduct, analyses, and even-
tually interpretation. Statistics can be
misleading, and most medical profession-
als are armed only with a basic under-
standing of this branch of mathematics
and may not be able to determine when
the wrong test is applied to the right data.
30
ASH Clinical News
Visual representations can be useful for
digesting large amounts of data but they,
too, can be hard to interpret and easy to
manipulate.
ASH Clinical News spoke with Dr.
Tannock, statisticians, and other experts in
trial design about the strengths and limita-
tions of research designs, the ways in which
statistics can point to an incorrect conclu-
sion, and advice for how clinicians can arm
themselves against misinterpretation.
Statistics and Humans:
A Poor Match
According to Nobel Prize–winning behav-
ioral economist Daniel Kahneman, PhD,
humans are not intuitive statisticians. We
learn grammar intuitively, along with a
wealth of other skills important for surviv-
al; when it comes to statistics, though, our
brains prefer simple answers and cognitive
ease. We tend – without even realizing
we’re doing it – to accept more straight-
forward, less brain-taxing explanations. In
effect, we jump to conclusions based on
limited information.
Most readers appreciate a simple
graph, infographic, or other visual tool
that allows them to “get the point” without
having to slog through tedious text. But
visualization is another method by which
researchers can deceive with data – either
intentionally or unintentionally. (Can you
really trust what a graph is saying? See the
SIDEBAR on page 32.)
“Much of how we perceive data is
through confirmation bias,” said Sara R.
Machado, PhD, a fellow at the Depart-
ment of Health Policy at the London
School of Economics who studies blood
donation systems and donor retention.
“We’re more likely to believe results that
align with our prior hypothesis, so if the
statistics show what we are expecting to
see, we don’t look with a critical eye.
“Part of the problem is that we tend
not to recognize when our understand-
ing is limited,” she added. “For example, I
know basic Italian. If you give me a book
to read in Italian, my imperfect knowl-
edge of Italian will lead me to misinterpret
the meaning.”
Take the statistical phenomenon
“regression to the mean”: An unusually
large or small measurement typically will
be followed by a value that is closer to the
mean because of random measurement
error. In other words, things tend to even
out over time.
Even researchers who know not to
April 2019