The COVID-19 pandemic brought mathematical modelling into the spotlight, as scientists rushed to use data to understand transmission patterns and disease severity, and to anticipate future epidemic outcomes. However, the use of COVID-19 modelling has been criticised, in part because of a few particularly erroneous projections at the start of the pandemic. 1 More than 2 years into the pandemic, models continue to face serious obstacles as tools for informing outbreak response. 1 Population-level health outcomes are difficult to predict accurately, especially cases and hospitalisations, 2 as discussed in the International Institute of Forecaster s blog. This Comment, drawn from our experiences with real-time prospective COVID-19 modelling, details these obstacles. We aim to highlight areas where further research and investment can improve the use of models for informing outbreak responses in the USA, with a summary of recommendations in the Panel. Data quality is one of the most important drivers of model performance. If data are inconsistent or do not reflect reality, models have no reliable ground truth from which to learn or be evaluated. Unfortunately, the public health infrastructure in the USA was not equipped to provide timely, high-quality data on COVID-19 health outcomes, and required several disparate efforts to fill this need. 3 However, inherent flaws remain in the COVID-19 data reporting system. For example, decision making on how to collect and share COVID-19 data fell to individual US states. Each US state has its own reporting idiosyncrasies (eg, defining what counts as a COVID-19 case or death, whether this definition includes probable cases or deaths, and how to define a probable case or death), limiting comparative analyses across locations. Additionally, artificial spikes or drops in the reported numbers of COVID-19 cases and deaths, which can result from backlogged testing results released from resource-constrained laboratories or batch death certificate reviews conducted by states, occur frequently and with irregular pattern, and affect both the training and evaluation of models that rely on the data. Other COVID-19 data, such as vaccinations, testing, hospitalisations, and genomic surveillance, have their own quality issues, largely because of an inadequate data reporting infrastructure, absence of universal data standards, and sampling bias. 3 In addition to data on health outcomes, many modellers have relied on human behavioural data for COVID-19 forecasting and scenario analysis with the aim to predict transmission patterns more accurately, in particular at points when dynamics are rapidly changing. However, it is difficult to collect real-time behavioural data because human behaviour is inherently hard to track. Some COVID-19 risk-reduction behaviours were captured through surveys administered on Facebook, which represents a substantial step forward in collecting open and timely behavioural data; however, these data still have sampling and self-reporting bias, and data collection ended in June 25, 2022. 4 New variants have also played a considerable role in surges in the number of COVID-19 ases and deaths worldwide. To this end, increased genomic surveillance has the potential to inform and improve predictions. As of Dec 31, 2021, only 5% of cases in the USA are sequenced, compared with more than 50% in other countries, including the UK, Iceland, and Australia. 5 To give modellers the best chance of success, we need to invest in a data system that provides open, timely, and standardised data at a high spatial and temporal resolution.