Common Statistical Issues
There are several statistical issues that routinely
arise during a clinical trial that, if not handled
appropriately, can cause serious problems at the
time of eventual regulatory submission.
Protocol deviations are events that occur during
the study that do not follow the protocol and
can cumulatively cause regulatory authorities to
become concerned about the underlying quality of
the trial conduct. They are graded during or at the
end of the study to determine if they are minor or
major deviations, which are handled differently.
Major deviations cannot be included in the per
protocol analysis, and multiple major deviations can
considerably reduce the number of subjects that can
be included in the per protocol analysis and negatively
influence efficacy results owing to the loss of power
with the reduction in sample size. Examples include:
• Misrandomizations
• Change in treatment during the trial
• Missed or out-of-window assessment visits:
A one-week post-treatment assessment window
might be 7 ± 1 days, and a minor deviation
would be missing that window by 1–2 days,
while a major deviation would greatly exceed
that time
• Inadequate assessments resulting in
incomplete data (e.g., scan/venography that
is not readable)
• Violation of eligibility criteria, such as
taking a concomitant medication that
was prohibited in the study protocol
Misrandomizations occur when the study
product is not provided to the subject per the
randomization process as documented in the
protocol and are common major protocol deviations.
Sites and CROs can underestimate the possibility
of misrandomizations and the seriousness of
their impact on study results. They often occur
due to insufficient training of, and lack of diligence
by, site personnel. Nevertheless, this is generally
viewed as a serious protocol violation by regulatory
authorities and has a detrimental influence
on the resulting data analysis; thus, too many
misrandomizations can undermine the integrity
of the trial.
Missing data occurs when the assessments are not
completed as specified in the protocol and has become
an increasing concern for regulatory authorities,
especially with the recent National Academy of
Sciences (NAS) report on missing data3 and European
Medicines Agency (EMA) guideline on missing data5
and particularly in phase III pivotal trials. Missing data
reflects outcome values that are meaningful for analysis
that were not collected for a portion of participants.3
Regulatory authorities typically view missing data
as a potential source of bias and are increasingly
requesting several imputation methods to address
the issue. Previous approaches that were considered
standard (e.g., baseline observation carried forward
[BOCF] and last observation carried forward [LOCF])
are increasingly being questioned; instead, regulators
are requesting multiple imputation methods and/or
sensitivity analyses where several different imputation
methods are implemented and compared, such as
inverse probability weighting, missing not at random
models, and tipping point analyses. If a plan to
address missing data is not considered during protocol
development, post hoc analysis can be performed,
but regulatory agencies, during the initial review, are
increasingly requesting that protocols address plans
for handling missing data in the planned analysis.
Secondary and sensitivity analyses are considered more
valuable than post hoc analyses and are recommended
to be included in the protocol and SAP to help address
missing data.3
Analysis populations need to be defined because
of the presence of noncompliant subjects or missing
data. They require appropriate handling, although
the issue of which participants are included in the
various analysis populations for efficacy (e.g., intentto-treat [ITT], modified ITT, and per protocol) can
often be confusing. Regulatory authorities nearly
always prefer ITT to be designated as the primary
analysis population, which typically requires that all
randomized participants be included and analyzed
as they were randomized, even if they were noncompliant with the protocol, ultimately not treated
with the assigned treatment, or not tr