Research
Patient-Centered Clinical Trial Design for Heart Failure Devices via Bayesian Decision Analysis
Chaudhuri, Shomesh E., Phillip Adamson, Dean Bruhn-Ding, Zied Ben Chaouch, David Gebben, Lilian Rincon-Gonzalez, Barry Liden, Shelby D. Reed, Anindita Saha, Daniel Schaber, Kenneth Stein, Michelle E. Tarver, and Andrew W. Lo (2023), Patient-Centered Clinical Trial Design for Heart Failure Devices via Bayesian Decision Analysis, The Patient: Patient-Centered Outcomes Research 16 (4), 359–369.
View abstract
Hide abstract
BACKGROUND: The statistical significance of clinical trial outcomes is generally interpreted quantitatively according to the same threshold of 2.5% (in one-sided tests) to control the false-positive rate or type I error, regardless of the burden of disease or patient preferences. The clinical significance of trial outcomes—including patient preferences—are also considered, but through qualitative means that may be challenging to reconcile with the statistical evidence.
OBJECTIVE: We aimed to apply Bayesian decision analysis to heart failure device studies to choose an optimal significance threshold that maximizes the expected utility to patients across both the null and alternative hypotheses, thereby allowing clinical significance to be incorporated into statistical decisions either in the trial design stage or in the post-trial interpretation stage. In this context, utility is a measure of how much well-being the approval decision for the treatment provides to the patient.
METHODS: We use the results from a discrete-choice experiment study focusing on heart failure patients’ preferences, questioning respondents about their willingness to accept therapeutic risks in exchange for quantifiable benefits with alternative hypothetical medical device performance characteristics. These benefit–risk trade-off data allow us to estimate the loss in utility—from the patient perspective—of a false-positive or false-negative pivotal trial result. We compute the Bayesian decision analysis-optimal statistical significance threshold that maximizes the expected utility to heart failure patients for a hypothetical two-arm, fixed-sample, randomized controlled trial. An interactive Excel-based tool is provided that illustrates how the optimal statistical significance threshold changes as a function of patients’ preferences for varying rates of false positives and false negatives, and as a function of assumed key parameters.
RESULTS: In our baseline analysis, the Bayesian decision analysis-optimal significance threshold for a hypothetical two-arm randomized controlled trial with a fixed sample size of 600 patients per arm was 3.2%, with a statistical power of 83.2%. This result reflects the willingness of heart failure patients to bear additional risks of the investigational device in exchange for its probable benefits. However, for increased device-associated risks and for risk-averse subclasses of heart failure patients, Bayesian decision analysis-optimal significance thresholds may be smaller than 2.5%.
CONCLUSIONS: A Bayesian decision analysis is a systematic, transparent, and repeatable process for combining clinical and statistical significance, explicitly incorporating burden of disease and patient preferences into the regulatory decision-making process.
Accelerating Vaccine Innovation for Emerging Infectious Diseases via Parallel Discovery
Barberio, Joseph, Jacob Becraft, Zied Ben Chaouch, Dimitris Bertsimas, Tasuku Kitada, Michael L. Li, Andrew W. Lo, Kevin Shi, and Qingyang Xu (2023), Accelerating Vaccine Innovation for Emerging Infectious Diseases via Parallel Discovery, Entrepreneurship and Innovation Policy and the Economy 2, 9–39.
View abstract
Hide abstract
The COVID-19 pandemic has raised awareness about the global imperative to develop and stockpile vaccines against future outbreaks of emerging infectious diseases (EIDs). Prior to the pandemic, vaccine development for EIDs was stagnant, largely due to the lack of financial incentives for pharmaceutical firms to invest in vaccine research and development (R&D). This R&D requires significant capital investment, most notably in conducting clinical trials, but vaccines generate much less profit for pharmaceutical firms compared with other therapeutics in disease areas such as oncology. The portfolio approach of financing drug development has been proposed as a financial innovation to improve the risk/return trade-off of investment in drug development projects through the use of diversification and securitization. By investing in a sizable and well-diversified portfolio of novel drug candidates, and issuing equity and securitized debt based on this portfolio, the financial performance of such a biomedical “megafund” can attract a wider group of private-sector investors. To analyze the viability of the portfolio approach in expediting vaccine development against EIDs, we simulate the financial performance of a hypothetical vaccine megafund consisting of 120 messenger RNA (mRNA) vaccine candidates in the preclinical stage, which target 11 EIDs, including a hypothetical “disease X” that may be responsible for the next pandemic. We calibrate the simulation parameters with input from domain experts in mRNA technology and an extensive literature review and find that this vaccine portfolio will generate an average annualized return on investment of −6.0% per annum and a negative net present value of −$9.5 billion, despite the scientific advantages of mRNA technology and the financial benefits of diversification. We also show that clinical trial costs account for 94% of the total investment; vaccine manufacturing costs account for only 6%. The most important factor of the megafund’s financial performance is the price per vaccine dose. Other factors, such as the increased probability of success due to mRNA technology, the size of the megafund portfolio, and the possibility of conducting human challenge trials, do not significantly improve its financial performance. Our analysis indicates that continued collaboration between government agencies and the private sector will be necessary if the goal is to create a sustainable business model and robust vaccine ecosystem for addressing future pandemics.
From ELIZA to ChatGPT: The Evolution of NLP and Financial Applications
Lo, Andrew W., Manish Singh, and ChatGPT (2023), From ELIZA to ChatGPT: The Evolution of NLP and Financial Applications, Journal of Portfolio Management 49 (7), 201–235.
View abstract
Hide abstract
Natural language processing (NLP) has revolutionized the financial industry, providing advanced techniques for the processing, analyzing, and understanding of unstructured financial text. The authors provide a comprehensive overview of the historical development of NLP, starting from early rules-based approaches to recent advances in deep learning–based NLP models. They also discuss applications of NLP in finance along with its challenges, including data scarcity and adversarial examples, and speculate about the future of NLP in the financial industry. To illustrate the capability of current NLP models, a state-of-the-art chatbot is employed as a co-author of this article.
Generative AI from Theory to Practice: A Case Study of Financial Advice
Lo, Andrew W., and Jillian Ross (2024), Generative AI from Theory to Practice: A Case Study of Financial Advice, in An MIT Exploration of Generative AI, March, https://doi.org/10.21428/e4baedd9.a1f6a281.
View abstract
Hide abstract
We identify some of the most pressing issues facing the adoption of large language models (LLMs) in practical settings and propose a research agenda to reach the next technological inflection point in generative AI. We focus on three challenges facing most LLM applications: domain-specific expertise and the ability to tailor that expertise to a user’s unique situation, trustworthiness and adherence to the user’s moral and ethical standards, and conformity to regulatory guidelines and oversight. These challenges apply to virtually all industries and endeavors in which LLMs can be applied, such as medicine, law, accounting, education, psychotherapy, marketing, and corporate strategy. For concreteness, we focus on the narrow context of financial advice, which serves as an ideal test bed both for determining the possible shortcomings of current LLMs and for exploring ways to overcome them. Our goal is not to provide solutions to these challenges—which will likely take years to develop—but rather to propose a framework and road map for solving them as part of a larger research agenda for improving generative AI in any application.
Deep-Learning Models for Forecasting Financial Risk Premia and Their Interpretations
Lo, Andrew W., and Manish Singh (2023), Deep-Learning Models for Forecasting Financial Risk Premia and Their Interpretations, Quantitative Finance 23 (6), 917–929.
View abstract
Hide abstract
The measurement of financial risk premia, the amount that a risky asset will outperform a risk-free one, is an important problem in asset pricing. The noisiness and non-stationarity of asset returns makes the estimation of risk premia using machine learning (ML) techniques challenging. In this work, we develop ML models that solve the problems associated with risk premia forecasting by separating risk premia prediction into two independent tasks, a time series model and a cross-sectional model, and using neural networks with skip connections to enable their deep neural network training.These models are tested robustly with different metrics, and we observe that our models outperform several existing standard ML models. A known issue with ML models is their ‘black box’ nature, i.e. their opaqueness to interpretability. We interpret these deep neural networks using local approximation-based techniques that provide explanations for our model’s predictions.
Lo, Andrew W., and Alexander Remorov (2022), Estimation and Prediction for Algorithmic Models of Investor Behavior, Journal of Systematic Investing 2 (1).
View abstract
Hide abstract
We propose a Markov chain Monte Carlo (MCMC) algorithm for estimating the parameters of algorithmic models of investor behavior. We show that this method can successfully infer the relative importance of each heuristic among a large cross-section of investors, even when the number of observations per investor is quite small. We also compare the accuracy of the MCMC approach to regression analysis in predicting the relative importance of heuristics at the individual and aggregate levels and conclude that MCMC predicts aggregate weights more accurately while regression outperforms in predicting individual weights.
Cho, Joonhyuk, Manish Singh, and Andrew W. Lo (2024), How Does News Affect Biopharma Stock Prices?: An Event Study, PLoS One 19 (1), e0296927.
View abstract
Hide abstract
We investigate the impact of information on biopharmaceutical stock prices via an event study encompassing 503,107 news releases from 1,012 companies. We distinguish between pharmaceutical and biotechnology companies, and apply three asset pricing models to estimate their abnormal returns. Acquisition-related news yields the highest positive return, while drug-development setbacks trigger significant negative returns. We also find that biotechnology companies have larger means and standard deviations of abnormal returns, while the abnormal returns of pharmaceutical companies are influenced by more general financial news. To better understand the empirical properties of price movement dynamics, we regress abnormal returns on market capitalization and a sub-industry indicator variable to distinguish biotechnology and pharmaceutical companies, and find that biopharma companies with larger capitalization generally experience lower magnitude of abnormal returns in response to events. Using longer event windows, we show that news related to acquisitions and clinical trials are the sources of potential news leakage. We expect this study to provide valuable insights into how diverse news types affect market perceptions and stock valuations, particularly in the volatile and information-sensitive biopharmaceutical sector, thus aiding stakeholders in making informed investment and strategic decisions.
Alhamdan, Abdullah, Zachery Halem, Irene Hernandez, Andrew W. Lo, Manish Singh, and Dennis Whyte (2023), Financing Fusion Energy, Journal of Investment Management 21 (1), 5–51.
View abstract
Hide abstract
The case for investing in fusion energy has never been greater, given increasing global energy demand, high annual carbon dioxide output, and technological limitations for wind and solar power. Nevertheless, financing for fusion companies through traditional means has proven challenging. While fusion startups have an unparalleled upside, their high upfront costs, lengthy delay in payoff, and high risk of commercial failure have historically restricted funding interest to a niche set of investors. Drawing on insights from investor interviews and case studies of public–private partnerships, we propose a megafund structure in which a large number of projects are securitized into a single holding company funded through various debt and equity tranches, with first loss capital guarantees from governments and philanthropic partners. The megafund exploits many of the core properties of the fusion industry: the diversity of approaches to engender fusion reactions, the ability to create revenue-generating divestitures in related fields, and the breadth of auxiliary technologies needed to support a functioning power plant. The model expands the pool of available capital by creating tranches with different risk–return tradeoffs and providing a diversified “fusion index” that can be viewed as a long hedge against fossil fuels. Simulations of a fusion megafund demonstrate positive returns on equity (ROE) and low default rates for the capital raised using debt.
Hull, John, Andrew W. Lo, and Roger M. Stein (2019), Funding Long Shots, Journal of Investment Management 17 (4), 9–41.
View abstract
Hide abstract
We define long shots as investment projects with four features: (1) low probabilities of success; (2) long gestation lags before any cash flows are realized; (3) large required up-front investments; and (4) very large payoffs (relative to initial investment) in the unlikely event of success. Funding long shots is becoming increasingly difficult—even for high-risk investment vehicles like hedge funds and venture funds—despite the fact that some of society’s biggest challenges such as cancer, Alzheimer’s disease, global warming, and fossil-fuel depletion depend critically on the ability to undertake such investments. We investigate the possibility of improving financing for long shots by pooling them into a single portfolio that can be financed via securitized debt, and examine the conditions under which such funding mechanisms are likely to be effective.
Financial Intermediation and the Funding of Biomedical Innovation: A Review
Lo, Andrew W., and Richard T. Thakor (2023), Financial Intermediation and the Funding of Biomedical Innovation: A Review, Journal of Financial Intermediation 54, 101028, https://doi.org/10.1016/j.jfi.2023.101028.
View abstract
Hide abstract
We review the literature on financial intermediation in the process by which new medical therapeutics are financed, developed, and delivered. We discuss the contributing factors that lead to a key finding in the literature—underinvestment in biomedical R&D—and focus on the role that banks and other intermediaries can play in financing biomedical R&D and potentially closing this funding gap. We conclude with a discussion of the role of financial intermediation in the delivery of healthcare to patients.