GCR Statistics

Design your study

Follow the outlined steps and start writing down your methodology. When you are finished, you have the basis for your study protocol. Furthermore, you will be able to claim that you have followed CONsolidated Standards Of Reporting Trials (CONSORT) statement. Make sure to check out the CONSORT extensions for alternative trial designs. If your intervention study uses an observational study design (not recommended), follow the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) guideline.

Why perform an intervention study?

The aim of intervention research is to compare the effects of different treatments. We want to know which treatments work best for which patients, in terms of efficacy (benefit in symptom reduction, quality of life, specific functions, survival) and adverse effects (toxicity, complications). randomization ensures comparability of treatment groups, which is the key to causal inference between treatment and outcome. In order to establish the standard-of-care, new therapies need to be validated. This validation process differs between drugs and (surgical) interventions. See the next section for further elaboration.

What types of diagnostic studies are there?

We can roughly divide intervention research trials into:
- Drug trials
- Surgical/intervention/psychotherapy trials

What about drug trials?

Drug trials typically follow these phases:

- Phase 0: Pilot/exploration trial. These studies are typically performed in a handful of patients (often healthy volunteers who will not benefit from the treatment). The aim of these studies is to evaluate whether the drug does what it is supposed to do.
- Phase 1: Safety study. This is typically a prospective cohort study in which dose-escalation is performed in patients to determine the maximum tolerable dose and describe the safety profile ([serious] adverse events are noted).
- Phase 2: Safety study. This is typically a prospective cohort study in which patients with the disease (often less then 100) are treated with the drug in the maximum tolerable dose to demonstrate treatment efficacy. The researchers need to define efficacy ahead of time (for example a response rate of 40% or median overall survival of 1 year).
- Phase 3: Comparative trial. This phase is the hallmark of intervention research. This is the time to compare the treatment efficacy and/or safety of the new drug with that of the standard-of-care (pragmatic trial) or a placebo (explanatory trial) in a randomized controlled trial. Most studies are designed to demonstrate superior efficacy of the new drug over the standard-of-care (superiority trial). Some trials however, aim to demonstrate equal efficacy (non-inferiority trial). In the latter case, the novel drug should have another benefit (less adverse events, cheaper etc.).
- Phase 4: Post-marketing testing/registry study After a succesful phase 3 trial, most drugs can enter the market (with regulatory approval). Some things still need to be evaluated, requiring a large number of patients and a longer period of follow-up. Typically, a registry study is performed to record rare adverse events and determine added/reduced benefit in patient subgroups.

What about surgical and interventional trials?

The validation of (surgical) interventions is more complex. This is attributable to the operator dependency (difference in strategies, learning effect) and high costs (both financially and time). The IDEAL (Idea, Development, Exploration, Assessment, Long-term follow-up – improving the quality of research in surgery) collaboration has described the following phases for these types of studies:

- Phase 1: Idea In this phase, the innovator of a novel procedure tries to establish proof-of-principle (often times focused on technical succes) in a handful of patients (comparable to phase 0 drug trial).
- Phase 2a: Development The procedure is developed and refined in a few selected patients in a prospective development study. It is essential to describe all sequentially treated patients, adaptations to the procedure, complications and potential benefits.
- Phase 2b: Exploration In this phase, the procedure should become standardised so that it can be performed by a growing number of operators in an increasing number of patients. A prospective research database can be used to evaluate learning curves, further refine the procedure, identify optimal candidates for the treatment and assess benefits and adverse effects of treatment. In this stage, a small RCT can be performed to evaluate the feasibility of a large RCT.
- Phase 3: Assessment Comparable to phase 3 comparative drug trial.
- Phase 4: Long-term study Comparable to phase 4 post-marketing drug study.
Click here to learn more about IDEAL.

What are the differences in study design

- For phase 0-2 trials, a prospective cohort study is recommended. A so-called exact group sequential design with predefined stopping boundaries may be required.
- A ‘standard RCT’ is called a parallel group randomized trial.
- If the goal is to demonstrate that the novel therapy works just as well as another therapy, it is called a non-inferiority trial. Key to this study design is defining a range of difference in outcome ahead of time that can be regarded as negligible. - If the comparator treatment is a placebo (or sham procedure), the trial is called an explanatory trial. It is not always necessary or desirable to use a placebo. First of all, there’s a placebo effect in every treatment given in clinical practice (usually beneficial). Second of all, it is usually more informative to compare the efficacy and toxicity with the standard-of-care (a pragmatic trial).
- Within-patient-randomized controlled trials use the study subject as his/her own control. A well-known type is the before-and-after study, where certain functions or symptoms are measured before and after 1 therapy or multiple therapies. In locoregional interventions, an organ may be split into multiple study parts (for example two eyes/ears/lungs/kidneys/liver lobes or multiple teeth). This can be a very efficient study design, but there are many considerations to take into account.
- A trial that compares multiple treatments or treatment regimens (for example different doses or combinations of therapies) is called a factorial randomized controlled trial.
- A propensity score matched study is a non-randomized intervention study design. A randomized trial is preferred, but not always feasible. In a propensity score study, two different treatment cohorts can be compared after assigning a propensity score – based on his/her baseline characteristics – to each individual. Consequently individuals with equal propensity scores can be compared.

NB: Learn more about these and many other alternative trial designs as well as their statistical implications at GCR academy.

What are the biggest challenges in diagnostic research?

Feasibility. Large randomized controlled trials are very expensive, labour- and time-intensive. Many drug or medical device companies initiate or sponsor RCT’s, thereby helping out with the feasibility aspect. It is, however, important that potential conflicts of interest (science vs. profit) do not interfere with the ethics of proper scientific evaluation.

Adjusting for confounding variables. randomization should take care of confounding variables. However, especially in somewhat smaller trials, differences in baseline characteristics may persist after randomization. Stratified randomization and adjusted multivariable analysis can help solve this problem.

Selective dropout and treatment cross-over. Patients may wish to leave the study early due to adverse events or other factors (selective dropout). Furthermore, some patients may receive the standard-of-care treatment after refusal or the experimental treatment failed (cross-over). This leads to a problem in the analysis. If we analyse the patients just as they were randomized (intention-to-treat analysis), we are not really comparing the effects of the treatment that they actually received. Often times this leads to a dilution of the estimated experimental treatment effect. However, we do maintain the between-group balance created by the randomization. If on the other hand, we were to analyse patients according to the treatment that they actually received (per protocol analysis) we break the between-group balance. Don’t selectively report the most beneficial scenario at the end of the study. You should make the decision ahead of time and stick to the plan, or report both analyses.

Many different outcome types are used. Overall survival is generally regarded as the most robust and meaningful outcome type. Sometimes, for example in terminal cancer patients, quality of life is more important. Often times trials use surrogate endpoints (for example tumour response rate, serum biomarker level, time-to-progression/progression-free survival) because this requires less patients. If you use a surrogate endpoint for this reason, make sure that the surrogate endpoint has a very strong association with the actual endpoint of interest. Other trials, especially those in the cerebro/cardio-vascular domain, use composite endpoints (for example: stroke, cardiac event or death). The availability of multiple trials with different outcomes in one clinical setting makes it difficult to compare results.

Limited window of clinical equipoise. Equipoise is the situation in which there is uncertainty to which treatment is better. If a novel treatment is promising and receives a lot of media attention, people may not want to receive the standard-of-care anymore. Furthermore, due to the rapid innovation cycle, the standard-of-care may be updated during the time it takes to conduct a large RCT.

How to get started?

Hopefully, you have already defined your research question. You know your domain, determinant(s) and outcome of interest. Now, write down the background of the clinical problem, findings of previous studies and rationale of your study. The next step is to meticulously define your study methodology. Your methodology must be so clear ahead of time, that other researchers could easily replicate the study.

We can divide study design into two parts:

Data collection
Data analysis

Designing the data collection

Establish your study design

Use one of the study designs outlined above. Try to refrain from comparing multiple treatment modalities from a retrospective cohort. Due to the many sources of bias, this is rarely worth the effort.

Patient accrual

Describe:
- The study setting (primary care, secondary/tertiary hospital, ICU, ED etc.)
- The dates and period of recruitment, exposure time, and time of follow-up
- Eligibility criteria (inclusion and exclusion criteria).

Randomisation

Describe:
- The allocation ratio (1:1 balanced or other)
- By whom and when is the randomisation performed
- How the randomisation is performed (f.e. computer generated or other method
- Method of randomisation (f.e. simple/blocked randomisation or minimisation)
- Use of stratification

Blinding/Masking

Describe:
- Who does and doesn’t (patient, treating physician, outcome assessor) know treatment allocation
- Which preventative measures are taken to conceal allocation

Variables of interest

Describe:
- Which variables of interest, potential confounding factors (previously described in the literature and based on clinical reasoning) and effect-modifiers are studied
- Treatment and investigational procedure details
- The sources of data
- Detailed methods of assessment/measurement.

Outcome(s)

Describe:
- Primary and secondary outcomes, along with their exact definition
- The sources of data
- Detailed methods of assessment/measurement

NB: Try to avoid subjective, surrogate and composite endpoints

Bias prevention:

Describe how you prevent potential sources of bias (for example selection bias, information bias and confounding, detection-, performance- and attrition bias))

Designing the data analysis

Describe:
- How summary data are analysed (mean – sd; median – range). Addition of 95% confidence intervals.
- Sample size determination based on primary endpoint (use the Sample Size Calculator)
- Which analyses will be performed (Use our Test Wizard). For normally distributed data use a parametric test, non-normally distributed data require a non-parametric test. For paired/clustered data use an appropriate test
- In the case of model building: method of model assumption evaluation (graphical, numerical), method for variable selection (forward/backward/enter), way of testing (p-value, Likelihood Ratio Test, AIC value), calibration (plot), discrimination (C-index/AUC) and validation (bootstrapping)
- Transformation of data (if applicable)
- Handling of extreme outliers
- Corrections of statistical significance (Bonferroni etc.)
- Statistical ways of handling missing data: Complete case analysis (not recommended) | Multiple imputation | Reclassification => best or worst-case scenario
- Statistical package used for analyses
- Assumption of statistical significance

NB: if you use relative measures as your endpoint such as an Odds Ratio (from logistic regression analyses) or Hazard Rate (Survival analyses) try to calculate absolute measures as well. A large relative risk increase on a small a priori risk is still a small risk!

Intervention Research

Characteristics of Intervention Research