Clinical Trial Protocol Evaluation Part 3 Study

Methodology and Statistics

NAFDAC Clinical Trial Regulation & Ethics Digital Training Program

Bra d ley Hu pf, P h D

S e n ior Ma n ager S tat ist ics, Ta ke da P h a rmaceut icals
Fe b 2 3 rd , 2 0 2 4
Table of Contents
Background: Framing clinical trial in a statistical context
◦ Hypothesis Testing
◦ Type I error and Power
◦ Bias

Components of a clinical trial protocol

◦ Objectives and endpoints
◦ Study Design
◦ Study Population
◦ Study Intervention
◦ Statistical analyses to be performed
 Motivating Example
Want to assess the efficacy of a new treatment in patients with non-small cell lung cancer

Ideally: 𝑝𝑇 = 0.5

Treatment Effect
Δ = 𝑝𝑇 − 𝑝𝐶

𝑝𝐶 = 0.25
All patients with non-small
cell lung cancer
 In Reality
We can’t give the control and treatment to everyone in the population let alone reach the whole
population.

𝑝Ƹ 𝑇 = 0.5 Estimated What does this say

Sample Treatment Effect about the true
෡ = 𝑝Ƹ 𝑇 − 𝑝Ƹ 𝐶
Δ treatment effect Δ

All patients with

non-small cell lung
cancer 𝑝Ƹ 𝐶 = 0.5
Hypothesis Testing
A statistical framework which uses the estimated treatment effect to make a statement about
the true treatment effect.
Null hypothesis: statement of “no difference” between treatment and control
◦ H0 : Δ = 0
◦ Treatment has the same (or worse) efficacy as the control

Alternative hypothesis: statement of difference between treatment and control

◦ HA : Δ > 0
◦ Treatment has better efficacy than the control

A p-value measures the likelihood of observing the estimated treatment effect due to random
chance
If the p-value is below a pre-specified cutoff 𝛼 (typically 0.05 or 5%) will reject H0
Power and Type I Error
The Truth
No treatment benefit Treatment Benefit
Δ=0 Δ>0
Conclusion from Fail to reject H0 Type II Error
Hypothesis Test Conclude Δ = 0 False negative
Reject H0 Type I Error
Conclude Δ > 0 False positive
Probability of a Type I Error is called the 𝛼-level:
◦ Is selected ahead of time, typically at 5%
Power is the probability of detecting a treatment effect when one is present
◦ Power = 1 – Prob(Type II error)
◦ Driven by the true treatment effect Δ and the sample size
◦ As Δ increases, power increases
◦ As n increases, power increases
 Statistical Bias
The expected difference between the estimated parameter and the true parameter. In other
෡ to incorrectly estimate Δ
words, anything which systematically causes Δ
Let’s assume there is a way to determine if a patient is going to respond to treatment ( ) or not ( )

෡ systematically
Δ ෡ systematically
Δ
over-estimates Δ under-estimates Δ

All patients with Sample only Sample only non-

non-small cell lung responsive patients responsive patients
cancer
Table of Contents
Background: Framing clinical trial in a statistical context
◦ Hypothesis Testing
◦ Type I error and Power
◦ Bias

Components of a clinical trial protocol

◦ Objectives and endpoints
◦ Study Design
◦ Study Population
◦ Study Intervention
◦ Statistical analyses to be performed
Objectives and Endpoints
An objective is the purpose for performing the study in terms of the scientific question to be
answered. Should be expressed as a state of purpose (e.g. to assess, to determine, to compare)
and include the general purpose (e.g. efficacy, effectiveness, safety)

A study endpoint is a specific measurement or observation to assess the effect of the study
intervention.
◦ Succinct but precise definition
◦ Should be prioritized and should correspond to the study objectives and hypothesis being tested

Objectives: The specific research question

Endpoint: The data collected to answer the specific research question
Primary and Secondary Objective/Endpoints
The primary objective: The secondary objective(s)
◦ The main research question ◦ Goals that provide additional information
◦ Generally, drives the statistical planning on the intervention
for the trial.
Typically, the primary objective is:
The secondary endpoints are those that
◦ Phase 1 is to assess safety and determine
the recommended phase 2 dose may provide supportive information
◦ Phase 2/3 are to assess short-term/long- about the study intervention’s effect on
term efficacy the primary endpoint or demonstrate
additional effects on the disease or
The primary endpoint is the basis for condition
concluding the study met its objective
and generally there is just one primary
endpoint
Why Does it Matter?
Each hypothesis test has an 𝛼% chance of being a false positive (Type I error).
If one perform 2 independent hypothesis tests, each at the 5% 𝛼-level, then the chance of at
least one false positive occurring in the 2 tests is more than 5%
This is the problem of multiplicity also called the multiple testing problem
Two main solutions:
◦ Adjusting the 𝛼-level for each test
◦ Bonferroni adjustment, Hochberg adjustment
◦ Hierarchical testing procedures

In general, the more hypothesis tests you perform the more difficult it is to claim significance for
an individual test. So, the number of primary/secondary objectives should be kept small
Common Endpoints in Cancer Trials
The choice of endpoint should be based on the specific disease and intervention
Time-to-event Outcomes:
Randomization CR Death
◦ Overall Survival (OS)
◦ Time from randomization to death
◦ Progression-Free survival (PFS)
◦ Time from randomization to death or disease progression OS and PFS
◦ Time to Progression (TTP) TTP is censored
◦ Time from randomization to disease progression
Randomization Progressive Death
Binary Outcomes: Disease
◦ Complete Response (CR)
◦ No detectable evidence of tumor
OS
◦ Objective Response Rate (ORR)
◦ Either a complete or a partial response PFS and TTP
Study Design
Let’s look at an example:

The study design specifies:

◦ Randomization?
◦ Blinding?
◦ Number of study arms?
◦ Type of experimental design?
Randomization
Refers to using chance to allocate patients to a study intervention
Consider two schemes to allocate patients:
◦ First 2 patients get treatment, last 2 receive control

A time-varying effect will introduce bias

◦ Flip a coin, if heads allocate to treatment

“On average” we get an unbiased sample

Types of Randomization
Randomization is one of the strongest tools to eliminate selection and allocation bias, as it will
balance both known and unknown prognostic factors
Some example of randomization:
◦ Simple Randomization
◦ Randomly assign each patient to treatment or control
◦ Does not guarantee arms will have same sample size
◦ Blocked Randomization
◦ Allocate blocks of patients at a time, within the block patients are equally allocated to treatment and control
◦ Guarantees arms will have the same sample size
◦ Stratified Randomization
◦ Split population into known groups then randomize within each group
◦ Guarantees arms will be balanced with respect to the stratifying variable
◦ I.e. Split patients into smokers and non-smokers then allocate to treatment or control with each group
Blinding
Refers to keeping the information about which treatment a patient has been assigned to hidden
until the end of the trial
Single Blinded:
◦ The patient does not know which treatment they received but the study operator does

Double Blinded:
◦ Patient and study operator do not know which treatment the patient received

Keeping treatment information hidden helps to reduce the amount of bias introduced into the
trial. For example:
◦ If a patient knows they received the placebo they may drop out or not follow all procedures
◦ If a study operator knows a patient will receive the treatment, they may consciously or unconsciously
give the patient better care introducing additional bias
Number of Treatment Arms
Refers to how many different study interventions will be given
Singe-arms Trials:
◦ A single study intervention is given and the results are compared against a benchmark
◦ Generally, used in early phase (1/2) trials and/or when it would be unethical/impossible to give a
placebo
◦ Provides less information about the true treatment effect Δ since the “control” did not come from the
same trial
◦ Survival data from single-arm trials can be unreliable

Multi-arm Trials:
◦ Allows for randomization and blinding
◦ One arm is the control, while other(s) are interventions
◦ Allows for a better estimate of the treatment effect as the control is given in similar conditions as the
treatment, reducing bias
Common Types of Experimental Designs
Refers to the statistical design used to obtain an estimate of the treatment effect
Parallel Group Design:
◦ Patients are assigned to a single
intervention group and followed until the
end of the study

Crossover Design
◦ Patients are assigned to either treatment
or control, follow by a “wash-out” period
and then the patient is assigned the other
intervention
◦ Allows each patient to be their own
control Treatment
◦ Not all interventions can “wash-out”
Wash-out time
Back to Our Example:
Generally, the study design is summarized in an illustration

Is a two-arm parallel-group design with 2:1 randomization, double-blinding and placebo-

controlled
Study Population
Refers to the specific group of patients targeted by the clinical trial.
Is specified through two sets of criteria
◦ Inclusion criteria:
◦ Patients are eligible for the study if they meet all of the inclusion criteria
◦ Exclusion criteria
◦ Patients will be excluded if they meet any of the exclusion criteria

Want the study population to be as large as possible while maintaining a clinically meaningful
treatment effect.
◦ Too general a population, the treatment effect may no longer be present
◦ Too specific a population, recruitment could be an issue
Study Intervention
This sections describes in-detail the study intervention(s)/control that are being tested for safety
and efficacy.

Should describe how the study intervention/control:

◦ Will be administered
◦ Will be prepared/handled/stored
◦ Will be given to minimize bias
◦ How the randomization and/or blinding will be done
◦ Will be discontinued
◦ When should a patient be discontinued from the trial?
◦ How will be the patient be followed off-study?

Detailing the interventions helps to reduce bias introduce by different study sites and
investigators
Statistical Considerations
This section should specify:
◦ The formal null and alternative hypothesis for primary and key secondary endpoints
◦ How the sample size was determined for the trial
◦ A general description of the statistical analyses to be performed

Null hypothesis: statement of “no difference” between treatment and control

◦ H0 : Δ = 0
◦ Treatment has the same (or worse) efficacy as the control

Alternative hypothesis: statement of difference between treatment and control

◦ HA : Δ > 𝛿
◦ Treatment has better efficacy than the control, where 𝛿 is the clinically meaningful difference
Common Statistical Analyses
The type of statistical analyses is generally determined by the endpoint and the type of
experimental design

Binary Data (ORR, CR):

◦ Exact Binomial Test
◦ Test of proportions
◦ Test of odds ratios

Time-to-event Data (OS, PFS, TTP) :

◦ Log-rank test
◦ Kaplan-Meier curves
◦ Cox-proportional hazards
Example: Kaplan-Meier Curve
This curve shows the percentage of
patients who have had an “event”
for each of the arms.
For PFS an “event” is defined to be
disease progression or death

The treatment effect Δ is measured

through the hazard ratio (HR)
◦ HR < 1, suggests treatment is
protective and prolongs survival
relative to the control
◦ HR > 1, treatment is harmful and
shortens survival relative to control
Sample Size Determination
Remember: the power of a statistical test is the ability for the test to detect a significant effect
when one is present. Further, the power is mainly affected by the treatment effect Δ and the
sample size.
Given a value of power and a predicted treatment effect can back-calculate the sample size
Generally,
◦ Power is set to be between 80%-90%
◦ The predicted treatment effect is determined using all available information and expert opinion

For time-to-event endpoints, the sample size tells you the number of events needed to achieve
the desired power
◦ Using this number of events can then predict the number of patients needed
Example: Sample Size Determination
From the study of Abemaciclib protocol:
◦ “The primary PFS analysis will be performed after 385 PFS events have occurred. Assuming a hazard
ratio (HR) of 0.703, this sample size yields at least 90% statistical power to detect superiority of
Abemaciclib plus fulvestrant arm over the placebo plus fulvestrant arm with the use of a 1-sided log-
rank test and type I error of 0.025.”

Study targeted at least 90% power, with a predicted treatment effect of HR = 0.703. To achieve this the
study would need 385 PFS events

Using this 385 events, study designers can then estimate how many patients will need to be recruited
based on predicted drop-out rates and screening failure rates
Thank you!
References
Protocol Templates for Clinical Trials: https://grants.nih.gov/policy/clinical-trials/protocol-template.htm
Clinical Trial Endpoints for the Approval of Cancer Drugs and Biologics:
https://www.fda.gov/media/71195/download
A Study of Abemaciclib (LY2835219) Combined With Fulvestrant in Women With Hormone Receptor Positive
HER2 Negative Breast Cancer (MONARCH 2)
◦ https://clinicaltrials.gov/ct2/show/results/NCT02107703
◦ https://clinicaltrials.gov/ProvidedDocs/03/NCT02107703/Prot_000.pdf
◦ Sledge GW, Toi M, Neven P, et al. The Effect of Abemaciclib Plus Fulvestrant on Overall Survival in Hormone Receptor–
Positive, ERBB2-Negative Breast Cancer That Progressed on Endocrine Therapy—MONARCH 2: A Randomized Clinical
Trial. JAMA Oncol. 2020;6(1):116–124. doi:10.1001/jamaoncol.2019.4782

Sample size Calculators:

◦ Binary: https://biostatistics.mdanderson.org/shinyapps/Nbinary/
◦ Continuous: https://biostatistics.mdanderson.org/shinyapps/Nnormal/
◦ Survival: https://biostatistics.mdanderson.org/shinyapps/Nsurvival/

Other useful free statistical software: https://biostatistics.mdanderson.org/SoftwareDownload/SoftwareOnline