RiskLabs separates earnings-call analysis from financial prediction. Explore its original workflow, reported results and limits before building a pilot.
Select a figure to open it at full size.
A language model can turn an earnings call into an organized account of management’s statements. That does not make its prediction of next month’s market risk reliable. The useful architectural question is where interpretation should end and quantitative forecasting should begin.
RiskLabs explores one answer: use language models to extract and analyze information, then feed the resulting features into a model trained to predict financial risk. Its experiments favor that division of labor over directly asking a language model for a number. They also leave important questions about reproducibility and deployment unanswered.
Source update: The paper used in the earlier version of this article now carries an author caveat that its data, methods and conclusions are being reevaluated. This revision examines the separate RiskLabs study and removes the earlier unsupported company deployments and predictive claims.
RiskLabs combines information from earnings-call text and audio with market time-series features. The language model is used for a specific intermediate job: making the transcript’s content easier for the downstream predictor to use.
The transcript analysis procedure has two paths. One divides the call into chunks, summarizes them and combines those summaries. The other asks targeted questions, retrieves relevant answers and interprets them using scenario descriptions. Those outputs are combined into an earnings-call analysis.
The original diagram gives a dividend example. A question asks about recent dividend yields; retrieved text describes an increase in the common-stock dividend; the analysis connects that statement to a scenario about increasing dividends. Dividend growth and dividend yield are different measures, so the retrieved passage does not directly answer the question as written. This illustrates why retrieval and interpretation need separate checks; it is not evidence that every dividend increase forecasts a positive return.
The resulting analysis is encoded numerically and combined with representations of the audio, transcript and market data. A trained prediction layer estimates future volatility and a downside-risk threshold. That final learning step distinguishes the framework from a chatbot issuing a confident risk forecast.
For a reader building such a system, the intermediate evidence is valuable in its own right. It can be inspected, corrected and compared with the source before it is allowed to influence a forecast.
RiskLabs evaluates volatility over horizons of 3, 7, 15 and 30 trading days. Here volatility concerns the variation in returns, not the chance of a company failing or the severity of a regulatory problem. The reported mean squared error measures how far model forecasts depart from the target values; lower is better under the same setup.
The most informative comparison adds inputs in stages. Audio and text alone produce an average error of 0.373. Adding the language-model analysis reduces it to 0.357. Adding VIX time-series information reduces it further to 0.324.
This staged comparison is more useful than saying that “generative AI predicts risk.” It suggests that the analysis contains additional information in this setup, while also showing that a conventional market signal contributes further improvement. The effect cannot all be credited to the language model.
In the baseline comparison, RiskLabs reports average error of 0.324 against 0.401 for the HTML multimodal model and 2.198 for directly prompted GPT-3.5-Turbo. But RiskLabs does not win every horizon: at 30 days, HTML reports 0.158 against RiskLabs’ 0.171. The result supports examining the feature-extraction architecture, not treating it as uniformly superior.
This is a research position paper with experiments, not an audited production forecasting service. Its appendix describes an 80/20 chronological train–test split, but does not fully specify the earnings-call sample count and date coverage. That makes it harder to reproduce the central comparisons or judge how representative they are.
Chronological splitting also does not establish that a pretrained language model lacks knowledge of later events when interpreting older calls. A forward-looking evaluation needs to address that separately. Otherwise an apparently historical prediction can benefit from information that would not have been available at the time.
The framework discusses news integration and further forecasting extensions, but the main staged comparison above contains audio, text, analysis and VIX. The paper describes parts of the news work as small-scale or ongoing. It would be misleading to attribute the table’s gain to a fully validated news-and-market intelligence platform.
These limits change the decision: the architecture is worth testing, while its published scores are insufficient to justify relying on its risk forecasts.
For a corporate research team, begin with evidence extraction rather than a trading signal. Choose a question such as how management describes cost pressures across successive calls. Preserve the exact passage, company, speaker, release date and reporting period alongside the system’s interpretation.
Review three outputs separately: whether the relevant passage was found, whether the interpretation is supported, and whether the derived feature adds predictive value. A failure in the first two should remain visible even when the final forecast happens to be close. This is the same traceability requirement that matters for annual-report summaries.
If forecasting is justified, compare market-data-only, transcript-feature and analysis-augmented models on the same future periods. Include stressed periods and a simple baseline. Track performance by horizon instead of letting a favorable average hide the period that matters to the user.
A document store with source timestamps and a structured extraction schema can support the first stage. Keep factual excerpts separate from model interpretations; missing evidence should remain missing rather than becoming a plausible answer.
For the quantitative evaluation, scikit-learn’s TimeSeriesSplit supports ordered train–test splits and a gap between them. It is a useful component, but dated filings require date-aware grouping: several companies on the same day must not leak across a split, and overlapping future-return windows need appropriate separation. The splitter does not enforce those domain rules automatically.
The smallest useful trial is a reviewed extraction task followed by a frozen historical forecast comparison. Accept the extraction stage if it improves coverage and analyst verification time without unsupported statements. Accept the prediction stage only if its incremental benefit survives later periods and the information-availability audit. Keeping extraction without adopting forecasting is a legitimate outcome.
The strongest idea in RiskLabs is its division of responsibility. Language models organize qualitative evidence; quantitative models must earn their forecasting role through disciplined evaluation. Build the evidence trail first. The paper’s incomplete experimental specification is a reason to reproduce and challenge the result, not to turn a promising architecture into a boardroom certainty.
Yupeng Cao and colleagues, RiskLabs: Predicting Financial Risk Using Large Language Model Based on Multimodal and Multi-Sources Data, arXiv version 2, May 3, 2025; presented as a position paper at the MFFM workshop at ICAIF 2024. Figures and results above refer to this version.
Check which facts, qualifications and reporting periods survive an AI-generated summary.
Match an explanation method to the decision or challenge a reviewer needs to make.
Examine a financial model that combines relationships across stocks with changes over time.
