We identify some of the most pressing issues facing the adoption of large language models (LLMs) in practical settings, and propose a research agenda to reach the next technological inflection point in generative AI. We focus on three challenges facing most LLM applications: domain-specific expertise and the ability to tailor that expertise to a user’s unique situation, trustworthiness and adherence to the user’s moral and ethical standards, and conformity to regulatory guidelines and oversight. These challenges apply to virtually all industries and endeavors in which LLMs can be applied, such as medicine, law, accounting, education, psychotherapy, marketing, and corporate strategy. For concreteness, we focus on the narrow context of financial advice, which serves as an ideal test bed both for determining the possible shortcomings of current LLMs and for exploring ways to overcome them. Our goal is not to provide solutions to these challenges—which will likely take years to develop—but to propose a framework and road map for solving them as part of a larger research agenda for improving generative AI in any application.

We identify some of the most pressing issues facing the adoption of large language models (LLMs) in practical settings and propose a research agenda to reach the next technological inflection point in generative AI. We focus on three challenges facing most LLM applications: domain-specific expertise and the ability to tailor that expertise to a user’s unique situation, trustworthiness and adherence to the user’s moral and ethical standards, and conformity to regulatory guidelines and oversight. These challenges apply to virtually all industries and endeavors in which LLMs can be applied, such as medicine, law, accounting, education, psychotherapy, marketing, and corporate strategy. For concreteness, we focus on the narrow context of financial advice, which serves as an ideal test bed both for determining the possible shortcomings of current LLMs and for exploring ways to overcome them. Our goal is not to provide solutions to these challenges—which will likely take years to develop—but rather to propose a framework and road map for solving them as part of a larger research agenda for improving generative AI in any application.

This paper explores the relationship between a firm's legal contracting environment and its innovation incentives. Using granular data from the pharmaceutical industry, we examine a contracting mechanism through which incumbents maintain market power: "pay-for-delay'' agreements to delay the market entry of competitors. Exploiting a shock where such contracts become legally tenuous, we find that affected incumbents subsequently increase their innovation activity across a variety of project-level measures. Exploring the nature of this innovation, we also find that it is more "impactful’’ from a scientific and commercial standpoint. The results provide novel evidence that restricting the contracting space can boost innovation at the firm level. However, at the extensive margin we find a reduction in innovation by new entrants in response to increased competition, suggesting a nuanced effect on aggregate innovation.

For precision genetic medicines to fulfill their potential as treatments for ultra-rare diseases, fresh approaches to academic– industry partnerships and data sharing are needed, together with regulatory change and adaptation of reimbursement models. Advances in gene therapy and gene editing technologies could revolutionize the ability to treat individuals with genetic disease, allowing treatments to be devised that target specific genetic mutations in people with even the rarest of disease indications. In 2018, a seven-year-old child with Batten disease received attention for becoming the first recipient of a customized antisense oligonucleotide (ASO) therapy specifically designed for her unique mutation1. Since then, multiple patients with ultra-rare genetic conditions have been treated with precision ASOs through academic-investigator-initiated programs. Development of these ASOs has been rapid, justified by the severity of the conditions being treated (for example, rapidly progressive neurologic degeneration), following streamlined regulatory processes. Here we discuss possible models for drug development, regulation and reimbursement that could allow these tailored genetic interventions to be scaled.

Harry Markowitz, co-recipient with Merton Miller and William Sharpe of the 1990 Nobel Prize for Economic Sciences ‘for their pioneering work in the theory of financial economics’, passed away in June 2023. As this column explains, his monumental contributions to modern finance have deeply influenced both academia and practice. His analysis of portfolio selection and risk management paved the way for a more sophisticated understanding of financial markets. And his theories continue to be integral to financial modelling and decision-making processes.

We investigate the impact of information on biopharmaceutical stock prices via an event study encompassing 503,107 news releases from 1,012 companies. We distinguish between pharmaceutical and biotechnology companies, and apply three asset pricing models to estimate their abnormal returns. Acquisition-related news yields the highest positive return, while drug-development setbacks trigger significant negative returns. We also find that biotechnology companies have larger means and standard deviations of abnormal returns, while the abnormal returns of pharmaceutical companies are influenced by more general financial news. To better understand the empirical properties of price movement dynamics, we regress abnormal returns on market capitalization and a sub-industry indicator variable to distinguish biotechnology and pharmaceutical companies, and find that biopharma companies with larger capitalization generally experience lower magnitude of abnormal returns in response to events. Using longer event windows, we show that news related to acquisitions and clinical trials are the sources of potential news leakage. We expect this study to provide valuable insights into how diverse news types affect market perceptions and stock valuations, particularly in the volatile and information-sensitive biopharmaceutical sector, thus aiding stakeholders in making informed investment and strategic decisions.

We develop a mathematical framework for constructing optimal impact portfolios and quantifying their financial performance by characterizing the returns of impact-ranked assets using induced order statistics and copulas. The distribution of induced order statistics can be represented by a mixture of order statistics and uniformly distributed random variables, where the mixture function is determined by the dependence structure between residual returns and impact factors—characterized by copulas—and the marginal distribution of residual returns. This representation theorem allows us to explicitly and efficiently compute optimal portfolio weights under any copula. This framework provides a systematic approach for constructing and quantifying the performance of optimal impact portfolios with arbitrary dependence structures and return distributions.

The regulatory approval process for new therapies involves costly clinical trials that can span multiple years. When valuing a candidate therapy from a financial perspective, industry sponsors may terminate a program early if clinical evidence suggests market prospects are not as favorable as originally forecasted. Intuition suggests that clinical trials that can be modified as new data are observed, i.e., adaptive trials, are more valuable than trials without this flexibility. To quantify this value, we propose modeling the accrual of information in a clinical trial as a sequence of real options, allowing us to systematically design early-stopping decision boundaries that maximize the economic value to the sponsor. In an empirical analysis of selected disease areas, we find that when a therapy is ineffective, our adaptive financing method can decrease the expected cost incurred by the sponsor in terms of total expenditures, number of patients, and trial length by up to 46%. Moreover, by amortizing the large fixed costs associated with a clinical trial over time, financing these projects becomes less risky, resulting in lower costs of capital and larger valuations when the therapy is effective.

We examine the leading role of the United States in the global equity markets by building daily snapshots of lead-lag price discovery networks using high-frequency country ETF returns. We find that the centrality of the U.S. equity market has been waning over time. Consistent with an explanation of gradual information diffusion, we empirically show that the shift to a multipolar system in the global equity markets can be explained by changes in information supply and demand. Using the COVID-19 pandemic as an exogenous shock, we document a causal relationship between news and country-specific price discovery network centralities.

The National Center for Advancing Translational Sciences' virtual 2021 conference on gene-targeted therapies (GTTs) encouraged multidisciplinary dialogue on a wide range of GTT topic areas. Each of three parallel working groups included social scientists and clinical scientists, and the three major sessions included a presentation on economic issues related to their focus area. These experts also coordinated their efforts across the three groups. The economics-related presentations covered three areas with some overlap: (1) value assessment, uncertainty, and dynamic efficiency; (2) affordability, pricing, and financing; and (3) evidence generation, coverage, and access. This article provides a synopsis of three presentations, some of their key recommendations, and an update on related developments in the past year. The key high-level findings are that GTTs present unique data and policy challenges, and that existing regulatory, health technology assessment, as well as payment and financing systems will need to adapt. But these adjustments can build on our existing foundation of regulatory and incentive systems for innovation, and much can be done to accelerate progress in GTTs. Given the substantial unmet medical need that exists for these oft-neglected patients suffering from rare diseases, it would be a tragedy to not leverage these exciting scientific advances in GTTs.

BACKGROUND: We consider two key challenges that early-stage biotechnology firms face in developing a sustainable financing strategy and a sustainable business model: developing a valuation model for drug compounds, and choosing an appropriate operating model and corporate structure. We use the specific example of Unravel Biosciences—a therapeutics platform company that identifies novel drug targets through off-target mechanisms of existing drugs and then develops optimized new molecules—throughout the paper and explore a specific scenario of drug repurposing for rare genetic diseases.

RESULTS: The first challenge consists of producing a realistic financial valuation of a potential rare disease repurposed drug compound, in this case targeting Rett syndrome. More generally, we develop a framework to value a portfolio of pairwise correlated rare disease compounds in early-stage development and quantify its risk profile. We estimate the probability of a negative return to be for a single compound and for a portfolio of 8 drugs. The probability of selling the project at a loss decreases from (phase 3) for a single compound to (phase 3) for the 8-drug portfolio. For the second challenge, we find that the choice of operating model and corporate structure is crucial for early-stage biotech startups and illustrate this point with three concrete examples.

CONCLUSIONS: Repurposing existing compounds offers important advantages that could help early-stage biotech startups better align their business and financing issues with their scientific and medical objectives, enter a space that is not occupied by large pharmaceutical companies, and accelerate the validation of their drug development platform.

A fixed one-sided significance level of 5% is commonly used to interpret the statistical significance of randomized clinical trial (RCT) outcomes. While it is necessary to reduce the false positive rate, the threshold used could be chosen quantitatively and transparently to specifically reflect patient preferences regarding benefit–risk tradeoffs as well as other considerations. How can patient preferences be explicitly incorporated into RCTs in Parkinson’s disease (PD), and what is the impact on statistical thresholds for device approval? In this analysis, we apply Bayesian decision analysis (BDA) to PD patient preference scores elicited from survey data. BDA allows us to choose a sample size (n) and significance level (α) that maximizes the overall expected value to patients of a balanced two-arm fixed-sample RCT, where the expected value is computed under both null and alternative hypotheses. For PD patients who had previously received deep brain stimulation (DBS) treatment, the BDA-optimal significance levels fell between 4.0% and 10.0%, similar to or greater than the traditional value of 5%. Conversely, for patients who had never received DBS, the optimal significance level ranged from 0.2% to 4.4%. In both of these populations, the optimal significance level increased with the severity of the patients’ cognitive and motor function symptoms. By explicitly incorporating patient preferences into clinical trial designs and the regulatory decision-making process, BDA provides a quantitative and transparent approach to combine clinical and statistical significance. For PD patients who have never received DBS treatment, a 5% significance threshold may not be conservative enough to reflect their risk-aversion level. However, this study shows that patients who previously received DBS treatment present a higher tolerance to accept therapeutic risks in exchange for improved efficacy which is reflected in a higher statistical threshold.

BACKGROUND: The statistical significance of clinical trial outcomes is generally interpreted quantitatively according to the same threshold of 2.5% (in one-sided tests) to control the false-positive rate or type I error, regardless of the burden of disease or patient preferences. The clinical significance of trial outcomes—including patient preferences—are also considered, but through qualitative means that may be challenging to reconcile with the statistical evidence.

OBJECTIVE: We aimed to apply Bayesian decision analysis to heart failure device studies to choose an optimal significance threshold that maximizes the expected utility to patients across both the null and alternative hypotheses, thereby allowing clinical significance to be incorporated into statistical decisions either in the trial design stage or in the post-trial interpretation stage. In this context, utility is a measure of how much well-being the approval decision for the treatment provides to the patient.

METHODS: We use the results from a discrete-choice experiment study focusing on heart failure patients’ preferences, questioning respondents about their willingness to accept therapeutic risks in exchange for quantifiable benefits with alternative hypothetical medical device performance characteristics. These benefit–risk trade-off data allow us to estimate the loss in utility—from the patient perspective—of a false-positive or false-negative pivotal trial result. We compute the Bayesian decision analysis-optimal statistical significance threshold that maximizes the expected utility to heart failure patients for a hypothetical two-arm, fixed-sample, randomized controlled trial. An interactive Excel-based tool is provided that illustrates how the optimal statistical significance threshold changes as a function of patients’ preferences for varying rates of false positives and false negatives, and as a function of assumed key parameters.

RESULTS: In our baseline analysis, the Bayesian decision analysis-optimal significance threshold for a hypothetical two-arm randomized controlled trial with a fixed sample size of 600 patients per arm was 3.2%, with a statistical power of 83.2%. This result reflects the willingness of heart failure patients to bear additional risks of the investigational device in exchange for its probable benefits. However, for increased device-associated risks and for risk-averse subclasses of heart failure patients, Bayesian decision analysis-optimal significance thresholds may be smaller than 2.5%.

CONCLUSIONS: A Bayesian decision analysis is a systematic, transparent, and repeatable process for combining clinical and statistical significance, explicitly incorporating burden of disease and patient preferences into the regulatory decision-making process.

The COVID-19 pandemic has raised awareness about the global imperative to develop and stockpile vaccines against future outbreaks of emerging infectious diseases (EIDs). Prior to the pandemic, vaccine development for EIDs was stagnant, largely due to the lack of financial incentives for pharmaceutical firms to invest in vaccine research and development (R&D). This R&D requires significant capital investment, most notably in conducting clinical trials, but vaccines generate much less profit for pharmaceutical firms compared with other therapeutics in disease areas such as oncology. The portfolio approach of financing drug development has been proposed as a financial innovation to improve the risk/return trade-off of investment in drug development projects through the use of diversification and securitization. By investing in a sizable and well-diversified portfolio of novel drug candidates, and issuing equity and securitized debt based on this portfolio, the financial performance of such a biomedical “megafund” can attract a wider group of private-sector investors. To analyze the viability of the portfolio approach in expediting vaccine development against EIDs, we simulate the financial performance of a hypothetical vaccine megafund consisting of 120 messenger RNA (mRNA) vaccine candidates in the preclinical stage, which target 11 EIDs, including a hypothetical “disease X” that may be responsible for the next pandemic. We calibrate the simulation parameters with input from domain experts in mRNA technology and an extensive literature review and find that this vaccine portfolio will generate an average annualized return on investment of −6.0% per annum and a negative net present value of −$9.5 billion, despite the scientific advantages of mRNA technology and the financial benefits of diversification. We also show that clinical trial costs account for 94% of the total investment; vaccine manufacturing costs account for only 6%. The most important factor of the megafund’s financial performance is the price per vaccine dose. Other factors, such as the increased probability of success due to mRNA technology, the size of the megafund portfolio, and the possibility of conducting human challenge trials, do not significantly improve its financial performance. Our analysis indicates that continued collaboration between government agencies and the private sector will be necessary if the goal is to create a sustainable business model and robust vaccine ecosystem for addressing future pandemics.

Gene therapy is a new class of medical treatment that alters part of a patient’s genome through the replacement, deletion, or insertion of genetic material. While still in its infancy, gene therapy has demonstrated immense potential to treat and even cure previously intractable diseases. Nevertheless, existing gene therapy prices are high, raising concerns about its affordability for U.S. payers and its availability to patients. We assess the potential financial impact of novel gene therapies by developing and implementing an original simulation model which entails the following steps: identifying the 109 late-stage gene therapy clinical trials underway before January 2020, estimating the prevalence and incidence of their corresponding diseases, applying a model of the increase in quality-adjusted life years for each therapy, and simulating the launch prices and expected spending of all available gene therapies annually. The results of our simulation suggest that annual spending on gene therapies will be approximately $20.4 billion, under conservative assumptions. We decompose the estimated spending by treated age group as a proxy for insurance type, finding that approximately one-half of annual spending will on the use of gene therapies to treat non-Medicare-insured adults and children. We conduct multiple sensitivity analyses regarding our assumptions and model parameters. We conclude by considering the tradeoffs of different payment methods and policies that intend to ensure patient access to the expected benefits of gene therapy.