Data Stream
Missing the Mark
Paging Dr. Watson
A new cancer analytic tool that combines the artificial intelligence of IBM’s Watson super-
computer with whole-genome and tumor RNA sequencing analysis identified potential
drug targets in a fraction of the time it typically takes a team of genomic experts, ac-
cording to results from a proof-of-concept study published in Neurology: Genetics.
Researchers tested how quickly the Watson Genomics Analytics system could identify
potentially actionable somatic variants in samples from one patient with glioblastoma,
finding that:
• The Watson analytic tool
returned a list of potential drug
targets within 10 minutes.
• Human analysis of the same
patients’ information took an
estimated 160 hours.
Just how good are scientists at predicting which preclinical cancer
studies would have reproducible results? They may need a little help,
according to a survey published in PLoS Biology.
To test whether scientists can accurately assess which initially-suc-
cessful experiments will work a second time, study authors asked 190
basic and preclinical cancer researchers to predict the reproducibility of
six replication studies conducted by the Reproducibility Project’s Cancer
Biology section.
The participants forecast a
75% probability
of replicating the statistical
significance
and a
50% probability
of replicating the
effect size.
But the participants went
0 for 2: Of these studies,
“This [study] highlights one of the challenges of the clinical application of precision
medicine technology,” the authors concluded, adding that this system could “address a
key bottleneck in cancer genomics.” 0% successfully replicated
on either criterion.
However, the supercomputer isn’t quite the revolution in cancer care that its creators
are promising, according to a recent investigation by STAT News. Experts who weighed
in on the “Watson for Oncology” program said the program is still in its infancy, six
years after launching, citing difficulties “training” the system, inputting massive
amounts of data, and a bias toward American research. “Experts were overconfident about the Reproducibility Project repli-
cating individual experiments within published reports,” the authors
concluded. “Researcher optimism likely reflects a combination of
overestimating the validity of original studies and underestimating the
difficulties of repeating their methodologies.”
Sources: Wrzeszczynski KO, Frank MO, Koyama T, et al. Comparing sequencing assays and human-machine analyses in
actionable genomics for glioblastoma. Neurol Genet. 2017;3:e164; STAT News, September 5, 2017. Source: Benjamin D, Mandel DR, Kimmelman J. Can cancer researchers accurately judge whether
preclinical reports will reproduce? PLoS Biol. 2017 June 29. [Epub ahead of print]
18
ASH Clinical News
October 2017