A fixed one-sided significance level of 5% is commonly used to interpret the statistical significance of randomized clinical trial (RCT) outcomes. While it is necessary to reduce the false positive rate, the threshold used could be chosen quantitatively and transparently to specifically reflect patient preferences regarding benefit–risk tradeoffs as well as other considerations. How can patient preferences be explicitly incorporated into RCTs in Parkinson’s disease (PD), and what is the impact on statistical thresholds for device approval? In this analysis, we apply Bayesian decision analysis (BDA) to PD patient preference scores elicited from survey data. BDA allows us to choose a sample size (n) and significance level (α) that maximizes the overall expected value to patients of a balanced two-arm fixed-sample RCT, where the expected value is computed under both null and alternative hypotheses. For PD patients who had previously received deep brain stimulation (DBS) treatment, the BDA-optimal significance levels fell between 4.0% and 10.0%, similar to or greater than the traditional value of 5%. Conversely, for patients who had never received DBS, the optimal significance level ranged from 0.2% to 4.4%. In both of these populations, the optimal significance level increased with the severity of the patients’ cognitive and motor function symptoms. By explicitly incorporating patient preferences into clinical trial designs and the regulatory decision-making process, BDA provides a quantitative and transparent approach to combine clinical and statistical significance. For PD patients who have never received DBS treatment, a 5% significance threshold may not be conservative enough to reflect their risk-aversion level. However, this study shows that patients who previously received DBS treatment present a higher tolerance to accept therapeutic risks in exchange for improved efficacy which is reflected in a higher statistical threshold.

BACKGROUND: The statistical significance of clinical trial outcomes is generally interpreted quantitatively according to the same threshold of 2.5% (in one-sided tests) to control the false-positive rate or type I error, regardless of the burden of disease or patient preferences. The clinical significance of trial outcomes—including patient preferences—are also considered, but through qualitative means that may be challenging to reconcile with the statistical evidence.

OBJECTIVE: We aimed to apply Bayesian decision analysis to heart failure device studies to choose an optimal significance threshold that maximizes the expected utility to patients across both the null and alternative hypotheses, thereby allowing clinical significance to be incorporated into statistical decisions either in the trial design stage or in the post-trial interpretation stage. In this context, utility is a measure of how much well-being the approval decision for the treatment provides to the patient.

METHODS: We use the results from a discrete-choice experiment study focusing on heart failure patients’ preferences, questioning respondents about their willingness to accept therapeutic risks in exchange for quantifiable benefits with alternative hypothetical medical device performance characteristics. These benefit–risk trade-off data allow us to estimate the loss in utility—from the patient perspective—of a false-positive or false-negative pivotal trial result. We compute the Bayesian decision analysis-optimal statistical significance threshold that maximizes the expected utility to heart failure patients for a hypothetical two-arm, fixed-sample, randomized controlled trial. An interactive Excel-based tool is provided that illustrates how the optimal statistical significance threshold changes as a function of patients’ preferences for varying rates of false positives and false negatives, and as a function of assumed key parameters.

RESULTS: In our baseline analysis, the Bayesian decision analysis-optimal significance threshold for a hypothetical two-arm randomized controlled trial with a fixed sample size of 600 patients per arm was 3.2%, with a statistical power of 83.2%. This result reflects the willingness of heart failure patients to bear additional risks of the investigational device in exchange for its probable benefits. However, for increased device-associated risks and for risk-averse subclasses of heart failure patients, Bayesian decision analysis-optimal significance thresholds may be smaller than 2.5%.

CONCLUSIONS: A Bayesian decision analysis is a systematic, transparent, and repeatable process for combining clinical and statistical significance, explicitly incorporating burden of disease and patient preferences into the regulatory decision-making process.

The COVID-19 pandemic has raised awareness about the global imperative to develop and stockpile vaccines against future outbreaks of emerging infectious diseases (EIDs). Prior to the pandemic, vaccine development for EIDs was stagnant, largely due to the lack of financial incentives for pharmaceutical firms to invest in vaccine research and development (R&D). This R&D requires significant capital investment, most notably in conducting clinical trials, but vaccines generate much less profit for pharmaceutical firms compared with other therapeutics in disease areas such as oncology. The portfolio approach of financing drug development has been proposed as a financial innovation to improve the risk/return trade-off of investment in drug development projects through the use of diversification and securitization. By investing in a sizable and well-diversified portfolio of novel drug candidates, and issuing equity and securitized debt based on this portfolio, the financial performance of such a biomedical “megafund” can attract a wider group of private-sector investors. To analyze the viability of the portfolio approach in expediting vaccine development against EIDs, we simulate the financial performance of a hypothetical vaccine megafund consisting of 120 messenger RNA (mRNA) vaccine candidates in the preclinical stage, which target 11 EIDs, including a hypothetical “disease X” that may be responsible for the next pandemic. We calibrate the simulation parameters with input from domain experts in mRNA technology and an extensive literature review and find that this vaccine portfolio will generate an average annualized return on investment of −6.0% per annum and a negative net present value of −$9.5 billion, despite the scientific advantages of mRNA technology and the financial benefits of diversification. We also show that clinical trial costs account for 94% of the total investment; vaccine manufacturing costs account for only 6%. The most important factor of the megafund’s financial performance is the price per vaccine dose. Other factors, such as the increased probability of success due to mRNA technology, the size of the megafund portfolio, and the possibility of conducting human challenge trials, do not significantly improve its financial performance. Our analysis indicates that continued collaboration between government agencies and the private sector will be necessary if the goal is to create a sustainable business model and robust vaccine ecosystem for addressing future pandemics.

Gene therapy is a new class of medical treatment that alters part of a patient’s genome through the replacement, deletion, or insertion of genetic material. While still in its infancy, gene therapy has demonstrated immense potential to treat and even cure previously intractable diseases. Nevertheless, existing gene therapy prices are high, raising concerns about its affordability for U.S. payers and its availability to patients. We assess the potential financial impact of novel gene therapies by developing and implementing an original simulation model which entails the following steps: identifying the 109 late-stage gene therapy clinical trials underway before January 2020, estimating the prevalence and incidence of their corresponding diseases, applying a model of the increase in quality-adjusted life years for each therapy, and simulating the launch prices and expected spending of all available gene therapies annually. The results of our simulation suggest that annual spending on gene therapies will be approximately $20.4 billion, under conservative assumptions. We decompose the estimated spending by treated age group as a proxy for insurance type, finding that approximately one-half of annual spending will on the use of gene therapies to treat non-Medicare-insured adults and children. We conduct multiple sensitivity analyses regarding our assumptions and model parameters. We conclude by considering the tradeoffs of different payment methods and policies that intend to ensure patient access to the expected benefits of gene therapy.

Use of robust, quantitative tools to measure patient perspectives within product development and regulatory review processes offers the opportunity for medical device researchers, regulators, and other stakeholders to evaluate what matters most to patients and support the development of products that can best meet patient needs. The medical device innovation consortium (MDIC) undertook a series of projects, including multiple case studies and expert consultations, to identify approaches for utilizing patient preference information (PPI) to inform clinical trial design in the US regulatory context. Based on these activities, this paper offers a cogent review of considerations and opportunities for researchers seeking to leverage PPI within their clinical trial development programs and highlights future directions to enhance this field. This paper also discusses various approaches for maximizing stakeholder engagement in the process of incorporating PPI into the study design, including identifying novel endpoints and statistical considerations, crosswalking between attributes and endpoints, and applying findings to the population under study. These strategies can help researchers ensure that clinical trials are designed to generate evidence that is useful to decision makers and captures what matters most to patients.

OBJECTIVE: Provide US FDA and amyotrophic lateral sclerosis (ALS) society with a systematic, transparent, and quantitative framework to evaluate the efficacy of the ALS therapeutic candidate AMX0035 in its phase 2 trial, which showed statistically significant effects (p-value 3%) in slowing the rate of ALS progression on a relatively small sample size of 137 patients. 

METHODS: We apply Bayesian decision analysis (BDA) to determine the optimal type I error rate (p-value) under which the clinical evidence of AMX0035 supports FDA approval. Using rigorous estimates of ALS disease burden, our BDA framework strikes the optimal balance between FDA’s need to limit adverse effects (type I error) and patients’ need for expedited access to a potentially effective therapy (type II error). We apply BDA to evaluate long-term patient survival based on clinical evidence from AMX0035 and Riluzole. 

RESULTS: The BDA-optimal type I error for approving AMX0035 is higher than the 3% p-value reported in the phase 2 trial if the probability of the therapy being effective is at least 30%. Assuming a 50% probability of efficacy and a signal-to-noise ratio of treatment effect between 25% and 50% (benchmark: 33%), the optimal type I error rate ranges from 2.6% to 26.3% (benchmark: 15.4%). The BDA-optimal type I error rate is robust to perturbations in most assumptions except for a probability of efficacy below 5%. 

CONCLUSION: BDA provides a useful framework to incorporate subjective perspectives of ALS patients and objective burden-of-disease metrics to evaluate the therapeutic effects of AMX0035 in its phase 2 trial.

We review the literature on financial intermediation in the process by which new medical therapeutics are financed, developed, and delivered. We discuss the contributing factors that lead to a key finding in the literature—underinvestment in biomedical R&D—and focus on the role that banks and other intermediaries can play in financing biomedical R&D and potentially closing this funding gap. We conclude with a discussion of the role of financial intermediation in the delivery of healthcare to patients.

We develop a theory of optimal financing for R&D-intensive firms. With only market financing, the firm relies exclusively on equity financing and carries excess cash, but underinvests in R&D. We use mechanism design to examine how intermediated financing can attentuate this underinvestment. The mechanism combines equity with put options such that investors insure firms against R&D failure and firms insure investors against high R&D payoffs not being realized.

Most people will experience a severe pandemic within their lifetime, and the world remains dangerously unprepared. In fact, scientists predict a nearly 50% chance—the same probability as flipping heads or tails on a coin—that we will endure another COVID-19-level pandemic within the next 25 years. Shifting America’s pandemic response capability from reactive to proactive is, therefore, urgent. Failure to do so risks the country’s welfare.

Getting ahead of the next pandemic is impossible without government financing. Vaccine production is costly, and these expenses will hinder industries from preemptively developing the tools needed to halt disease transmission. For example, the total expected revenues over a 20-year vaccine patent lifecycle would cover just half of the upfront research and development (R&D) costs.

However, research suggests that a portfolio-based approach to vaccine development—especially now with new, broadly applicable mRNA technology—dramatically increases the returns on investment while also guarding against an estimated 31 of the next 45 epidemic outbreaks. With lessons learned from Operation Warp Speed, Congress can deploy this approach by (i) authorizing and appropriating $10 billion to the Biomedical Advanced Research and Development Authority (BARDA) (ii) developing a vaccine portfolio for 10 emerging infectious diseases (EIDs), and (iii) a White House Office of Science and Technology Policy (OSTP)-led interagency effort focused on scaling up production of priority vaccines.

INTRODUCTION: Machine learning models are increasingly applied to predict the drug development outcomes based on intermediary clinical trial results. A key challenge to this task is to address various forms of bias in the historical drug approval data.

OBJECTIVE:We aimed to identify and mitigate the bias in drug approval predictions and quantify the impacts of debiasing in terms of financial value and drug safety.

METHODS: We instantiated the Debiasing Variational Autoencoder, the state-of-the-art model for automated debiasing. We trained and evaluated the model on the Citeline dataset provided by Informa Pharma Intelligence to predict the final drug development outcome from phase II trial results.

RESULTS: The debiased Debiasing Variational Autoencoder model achieved better performance (measured by the F₁ score 0.48) in predicting the drug development outcomes than its un-debiased baseline (measured by the F₁ score 0.25). It had a much higher true-positive rate than baseline (60% vs 15%), while its true-negative rate was slightly lower (88% vs 99%). The Debiasing Variational Autoencoder distinguished between drugs developed by large pharmaceutical firms and those by small biotech companies. The model prediction is strongly influenced by multiple factors such as prior approval of the drug for another indication, whether the trial meets the positive/negative endpoints, and the year when the trial is completed. We estimate that the debiased model generates financial value for the drug developer in six major therapeutic areas, with a range of US$763–1,365 million.

CONCLUSIONS: Our analysis shows that debiasing improves the financial efficiency of late-stage drug development. From the pharmacovigilance perspective, the debiased model is more likely to identify drugs that are both safe and effective. Meanwhile, it may predict a higher probability of success for drugs with potential adverse effects (because of its lower true-negative rate), thus it must be used with caution to predict the development outcomes of drug candidates currently in the pipeline.

BACKGROUND: The authors estimate the probability of successful development and duration of clinical trials for medications to treat neuropathic and nociceptive pain. The authors also consider the effect of the perceived abuse potential of the medication on these variables.

METHODS: This study uses the Citeline database to compute the probabilities of success, duration, and survivorship of pain medication development programs between January 1, 2000, and June 30, 2020, conditioned on the phase, type of pain (nociceptive vs. neuropathic), and the abuse potential of the medication.

RESULTS: The overall probability of successful development of all pain medications from phase 1 to approval is 10.4% (standard error, 1.5%).Medications to treat nociceptive and neuropathic pain have a probability of successful development of 13.3% (standard error, 2.3%) and 7.1% (standard error, 1.9%), respectively. The probability of successful development of medications with high abuse potential and low abuse potential are 27.8% (standard error, 4.6%) and 4.7% (standard error, 1.2%), respectively. The most common period for attrition is between phase 3 and approval.

CONCLUSIONS: The authors’ data suggest that the unique attributes of pain medications, such as their abuse potential and intended pathology, can influence the probability of successful development and duration of development.

The opioid epidemic in the U.S. has resulted in significant costs in human lives as well as to the health care system, employers, and insurers. While there is great motivation and urgency to address the opioid crisis, there are currently few non-opioid pain management medications in the development pipeline. The growing regulatory pressures and stigma surrounding opioids have discouraged investments and research in the pain industry. Using estimates from the literature, our simulations show that a portfolio of pharmaceuticals and medical devices for pain treatment and opioid use disorder, diversified and optimized across different development pathways, yields single digit annualized returns. This suggests that active collaboration between the public and private sectors is needed to incentivize investments in pain research.

Using survey data gathered from grantees of the nonprofit Breast Cancer Research Foundation (BCRF), we investigated the commercial and non-commercial impacts of their research funding. We found significant impact in both domains. Commercially, 19.5% of BCRF grantees filed patents, 35.9% had a project that has reached clinical development, and 12 companies have or will be spun off from existing projects, thus creating 127 new jobs. Non-commercially, 441 graduate students have been trained by 116 grantees, 767 postdoctoral fellows have been trained by 137 grantees, 66% of grantees have used funding for faculty salaries, 93% have achieved collaboration with other researchers, and 42.7% have enacted process improvements in research methodology.  Econometric analysis identifies BCRF funding and associated process improvements as key factors associated with the likelihood to file patents. However, we also found that the involvement of more than one institution in a collaborative project had a negative impact on subsequent development. This may point to frictions introduced by multi-university interactions.

We review the recent literature on financing biomedical innovation, with a specific focus on the drug development process and how it may be enhanced to improve outcomes. We begin by laying out stylized facts about the structure of the drug development process and its associated costs and risks, and we present evidence that the rate of discovery for life-saving treatments has declined over time while costs have increased. We make the argument that these structural features require drug development (i.e., biopharmaceutical) firms to rely on external financing and at the same time amplify market frictions that may hinder the ability of these firms to obtain financing, especially for treatments that may have large societal value relative to the benefits going to the firms and their investors. We then provide an overview of the evidence for various types of market frictions to which these drug development firms are exposed and discuss how these frictions affect their incentive to invest in the development of new drugs, leading to underinvestment in valuable treatments. In light of this evidence, numerous studies have proposed ways to overcome this funding gap, including the use of financial innovation. We discuss the potential of these approaches to improve outcomes.

Following the approval by the FDA of two COVID-19 vaccines, which are administered in two doses three to four weeks apart, we simulate the effects of various vaccine distribution policies on the cumulative number of infections and deaths in the United States in the presence of shocks to the supply of vaccines. Our forecasts suggest that allocating more than 50% of available doses to individuals who have not received their first dose can significantly increase the number of lives saved and significantly reduce the number of COVID-19 infections. We find that a 50% allocation saves on average 33% more lives, and prevents on average 32% more infections relative to a policy that guarantees a second dose within the recommended time frame to all individuals who have already received their first dose. In fact, in the presence of supply shocks, we find that the former policy would save on average 8,793 lives and prevents on average 607,100 infections while the latter policy would save on average 6,609 lives and prevents on average 460,743 infections.