How to Understand and Create a Statistical Analysis Plan (SAP)

Clinical research serves as the foundation for advancing medical knowledge, improving patient care, and ensuring the safety and efficacy of new investigational products. As a result, clinical trials must be performed to the highest degree of scientific and ethical standards, for results to be recognised as credible and the trial deemed as successful. One document which plays an essential role in the safety, traceability and overall validity of a clinical trial is the Statistical Analysis Plan (SAP).

What is a Statistical Analysis Plan (SAP)?

The SAP is a document outlining the planned statistical methods and procedures undertaken to analyse and interpret the data collected during the lifespan of a clinical trial. The SAP should be specific and highly detailed, offering a clear description of any data manipulation or analyses required to meet trial objectives and endpoints. To ensure high reliability and validity of the clinical trial, the SAP should be written and finalised as close to the beginning of the clinical trial as possible but can be amended at any point before First Patient First Visit (FPFV) for unblinded open trials and database Lock (DBL) for blinded trials, to document any updates occurring throughout the trial. It is essential that the SAP is transparent and reproducible as it is a key contributor to the writing of Tables, Listings and Figures (TLFs) for inclusion in the Clinical Study Report (CSR), which ultimately concludes the success of the investigational product.

What Are the Key Documents Required During a Clinical Trial?

During a clinical trial, several crucial documents guide the process and ensure the integrity of the study:

1. Protocol
   The protocol is a comprehensive document which outlines the plan for the study, and provides detailed information on the study objectives, design, methodology, and statistical considerations.
   
   
2. Statistical Analysis Plan
   A detailed plan of the statistical methods and analyses planned for the clinical trial.
   
   
3. TLF Shells
   A template document including mock-ups for the Tables, Listings, and Figures, which will be used in the final presentation and reporting of study data.
   
   
4. Clinical Study Report
   A clinical study report is a comprehensive document summarising the study findings, which provides a detailed account of the methodology, conduct, and results of a clinical trial and is a key component in the process of submitting data to regulatory authorities for the approval of new drugs or therapies.
   
   

Following Database Lock, the planned datasets and TLFs are produced based on the SAP and TLF Shells.

When Should the Statistical Analysis Plan be Written?

The optimal time to create the SAP is before data collection begins, ideally during the trial design phase and before First Patient First Visit (FPFV). Benefits to developing the SAP early in the trial include:

• Bias Prevention: By developing the SAP prior to data collection, it can be proven that the analysis methods are predefined and not influenced by the data collected, reducing the risk of bias in the results.
• Clear Objectives: Having the SAP for reference helps to clarify the trial objectives, hypotheses, and primary outcomes, ensuring that the analyses aligns with the trial goals.
• Regulatory Compliance: For clinical trials and other regulated research, having a SAP before data collection can be a regulatory requirement. It ensures transparency and adherence to agreed-upon protocols.
• Resource Planning: The SAP outlines the necessary statistical methods and resources required for analysis during the clinical trial, aiding in better planning and allocation of resources.
• Error Mitigating: It allows for a thorough review of the proposed statistical methods, reducing the likelihood of errors or omissions in the analysis phase.

Creating the SAP as early as possible in the clinical trial enhances SAP robustness, but the deadline for finalising the SAP will depend on the nature of the trial. For blinded trials, the SAP must be finalised prior to DBL to ensure that planned analyses are not changed after the final data are collected. However, for unblinded or open-label trials, the primary endpoints must be well established in the SAP prior to the recruitment of the first patient, to minimise bias and enhance result credibility.

When to Submit the Statistical Analysis Plan to Regulatory Bodies

Understanding the timing of when to submit the SAP to regulatory authorities like the FDA, CHMP or MHRA (for the US, Europe and UK respectively) is highly important for achieving success in a clinical trial. While the SAP should be written and finalised as early in the trial process as possible, it is important for SAP submission to occur prior to database lock (DBL) for most blinded studies. For studies which are unblinded or have complex/adaptive designs, earlier submission may be required, sometimes during the initial protocol submission or at key milestones, such as the end-of-Phase II meeting. Ensuring timely submission of the SAP to the FDA, CHMP or MHRA helps to align the trial's statistical approach with regulatory expectations and facilitates a smoother review process.

Which Regulatory Guidelines Relate to the Writing of the Statistical Analysis Plan?

The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) guidelines are a set of internationally recognised standards and recommendations which guide the development, testing, approval, and monitoring of new pharmaceutical products. The ICH was established to harmonise regulatory requirements across different countries, particularly in the United States, Europe, and Japan, and to ensure that pharmaceutical products are safe, effective, and of high-quality.

The following three guidelines are of key interest and should be closely adhered to when writing the SAP:

• ICH Efficacy Guideline E6(R2) (Good Clinical Practice): Provides a unified standard for designing, conducting, recording, and reporting clinical trials involving human participants. The upcoming ICH E6(R3) revision, expected in 2025, will place an increased emphasis on patient centricity.
• ICH Efficacy Guideline E8(R1) (General Considerations for Clinical Studies): Emphasises quality by design, patient-centricity, and a risk-based approach, promoting pre-specification of analyses, alignment with the protocol, and ensuring the robustness and reliability of results.
• ICH Efficacy Guideline E9 (Statistical Principles for Clinical Trials): Provides recommendations on the statistical methods used in clinical trials, including the design of trials, hypothesis testing, analysis methods, and interpretation of results.

How to Write a Statistical Analysis Plan

The SAP should serve as a blueprint for all analyses, ensuring that any details relating to data manipulation and analyses methods are planned out before data collection begins. As a result, the following details are recommended to be included when writing a SAP:

• Title and Identification Information (Study title; Protocol number, Version number/date).
• Introduction and Study Overview (Background information; Study design).
• Objectives and Hypotheses (Primary, secondary, and exploratory objectives; Hypotheses).
• Endpoints/Outcomes (Primary, secondary, and exploratory endpoints and outcomes).
• Sample Size Determination (Calculation details; Interim analyses; Criteria for early termination).
• Statistical Methods (Descriptive or inferential statistics; Adjustment for covariates; Handling of missing data; Sensitivity analyses; Estimands).
• Blinding/Unblinding (Details of any blinding and unblinding processes).
• Data Presentation (TLF specifications; Statistical software and versioning).
• Assumptions and Limitations (Statistical assumptions; Potential limitations).
• Ethical and Regulatory Considerations (Compliance with guidelines; Approval and signatures).
• Appendices (Definitions; Data shells; Additional information to support the SAP).

Who Can Write a Statistical Analysis Plan?

The SAP must be written by a statistician or a biostatistician who has expertise in the relevant field of trial. However, the process often involves collaboration with other contributing key stakeholders, who are listed below:

Statisticians/Biostatisticians:

• • They are responsible for the technical aspects of the SAP, including the selection of appropriate statistical methods, sample size calculations, and the plan for data analysis.
  • They have specialised knowledge in statistics and are familiar with the regulatory requirements and guidelines pertinent to the trial's field.

Principal Investigator (PI):

• • The PI is usually involved in the development of the SAP, providing input on the trial objectives, hypotheses, and key outcomes to ensure the analysis plan aligns with the overall research goals.
  • They work closely with the statistician to ensure that the SAP is scientifically sound and practically feasible.

Clinical Researchers/Subject Matter Experts:

• • In clinical trials or specialised research fields, subject matter experts contribute to defining the clinical or experimental endpoints, and they ensure that the SAP accurately reflects the trial's clinical or scientific goals.
  • They provide insights into the practical aspects of data collection, which helps in tailoring the SAP to the trial's needs.

Regulatory Affairs Specialists:

• • For trials that require regulatory approval, such as clinical trials, regulatory affairs specialists may review the SAP to ensure it meets the necessary guidelines and compliance requirements.
  • They ensure that the SAP is aligned with regulatory expectations and standards.

Data Managers and Programmers:

• • Clinical data managers and programmers may provide input on the data handling procedures, data quality, and management processes to ensure that the SAP is realistic and executable with the available data infrastructure.
  • Both data managers and programmers work with statisticians to ensure that the data collection, storage, and retrieval processes are compatible with the analysis plan.

While the primary author of a SAP is the statistician or biostatistician, the creation of the plan is often a collaborative effort with any or all of the above disciplines to ensure that the SAP is comprehensive, scientifically valid, and aligned with the trial goals.

Should the Estimands Framework be Incorporated into the Writing of a Statistical Analysis Plan?

An estimand is a precise description of the treatment effect within a clinical trial. Contributing attributes to estimands include:

• The treatment effect of interest (e.g.: the different in mean outcomes between treatment and placebo).
• The population to which the estimand applies (e.g.: patients who adhered to the treatment).
• The outcome of interest (e.g.: reduction in blood pressure).
• The handling of potential issues/intercurrent events (e.g.: rescue medication; missing data; treatment non-compliance).

Estimands play a fundamental role during the writing of the SAP as they directly influence the design, conduct, and interpretation of statistical analyses. Specific components of the SAP which may be influenced by estimands include:

• Formulating objectives and endpoints.
• Determining statistical methods.
• Selecting analysis sets.
• Interpretation of results.

Estimands provide additional clarity and precision relating to what is being measured in clinical research, helping to reduce ambiguity, and ensuring that the selected statistical analyses align with trial objectives. Estimands also improve consistency between analysis methods and the goals of the trial, as well as improving adherence to regulatory and scientific standards by ensuring the transparency and reliability of results.

Should Interim Analyses be Considered When Writing the Statistical Analysis Plan?

Interim analyses are any planned analyses conducted at predefined points before the trial is completed. Motivations for planning interim analyses during a clinical trial include early detection of treatment efficacy or futility, which could lead to early termination of a trial if the investigational product is assessed as either beneficial or futile. Interim analyses also allow the sample size to be adjusted if the effect size and power differs to what was originally planned, as well as assessing if the trial is likely to meet its planned objectives.

However, there can be drawbacks to planning interim analyses which should be considered at the time of writing the SAP. Conducting multiple analyses throughout a clinical trial without adjustment can raise the alpha spend, which increases the chance of Type I error (false positives). Conversely, numerous interim analyses can also increase beta spend if not adjusted for correctly, which can result in a loss of power for the trial. Hence, consideration of statistical methods and spending functions (e.g.: the O’Brien-Fleming method or the beta spending function) must be considered at the time of writing the SAP to balance the benefits of early decision-making with maintaining trial integrity.

Conclusion

Clinical research is vital for advancing medical science and improving health-related quality of life. It is essential that clinical trial research in humans is performed in a safe environment which protects patient safety while accurately measuring treatment efficacy. The SAP is a key document in this process, playing a crucial role in determining the success of the investigational product, while maintaining high reliability and robustness of results.

The SAP requires input from qualified collaborators but ultimately must be written by a qualified statistician or biostatistician and meet relevant regulatory guidelines. The SAP must be detailed and comprehensive with clear instructions for suitable statistical techniques and data manipulation, which will contribute to the credibility and overall success of a clinical trial. Transparency and reproducibility are essential for honest and reliable results, for which a well-written SAP is a key contributor.

Enhance the Reliability of Your Clinical Trial with a Robust SAP

At Quanticate, our expert statisticians and biostatisticians are committed to delivering comprehensive and meticulously crafted Statistical Analysis Plans (SAPs) that ensure the credibility and success of your clinical trials. With extensive experience in adhering to regulatory guidelines and employing advanced statistical methodologies, we are here to support you in every stage of your trial. 

For more information on how we can assist with your SAP or our other biostatistical services, please contact Quanticate or submit an RFI.