Are Large Language Models Truly Democratizing Financial Knowledge? Identifying Knowledge Gaps

Your review of our work is very constructive and helpful. We appreciate your insights into our work and your review, noting our generalizable framework for evaluating LLM biases. We address the weaknesses mentioned and your questions below.

Regarding more background on LLMs in the financial domain: In our introduction, we mention several papers that highlight the efficacy of LLM usage across various financial use-cases. Based on your comments, we have further expanded our Section 6 “Related Work” to include further discussion of LLM usage and LLM performance in the financial domain.

Regarding Revenue Question: We agree with the reviewer. Further, we would like to draw your attention to Appendix K, where we show the impact of the bias for trading with sophisticated prompt.

Question about Gemini: We appreciate your feedback; we have incorporated Gemini results in Appendix Figure 8, Figure 10, and Table 6. The results are consistent with our findings.

Question regarding statistical significance: We appreciate the careful reading of our paper. *, $\dagger$, and $\ddagger$ indicate p-value at the 10%, 5%, and 1% levels, respectively for regression coefficients.

Thank you for your review of our paper, we appreciate your insight and suggestions, and we address the weaknesses you have brought up below.

Response to Lack of Novelty: In our paper, we aim not to evaluate the performance of LLMs in the financial domain, but to identify biases in performance. The mentioned papers do not perform an analysis of how the models perform across various stratifications of the financial domain, which was our goal in this paper. We believe that our temporal and stratified approach to evaluating performance biases is novel and has not yet been done in the financial domain or any other domain, as mentioned in lines #100-102. We also show the broad applicability of our bias evaluation framework on soccer (Spanish club) data in Appendix L.

Response to Limited LLMs Evaluated: We evaluate FinMA (which ranks last in performance among the models tested) in Appendix G of our paper and evaluate GPT-4 on a smaller sample (due to cost constraints) in Appendix F. We find that the general-domain models outperform FinMA significantly. We have added Gemini results in Appendix Figure 8, Figure 10, and Table 6. The results are consistent with our findings.

Response to Insufficient Case Studies and Error Analysis: We present an example of the biases we study in Figure 1. 

Response to Potential Copyright Concerns: The reviewer probably didn’t notice our discussion of the sources of our data in Section 2.1 (for revenue data), Section 4.1 (for cross-sectional variables data) and further in Appendix B. Our RevenuePromptDataset is novel because it contains both prompts and LLM Responses for our revenue questions, which is data that was indeed produced because of our work; it does not simply include revenue data sourced from our mentioned data sources. We use LLM-generated data from proprietary models (e.g. GPT) in accordance with applicable “terms of use”, as highlighted in Appendix M. We also specifically note the public availability of our data sources in Table 1.

Response to Related Work: The goal of our study was not to evaluate the overall performance of LLMs, but rather their biases along temporal and finance-specific stratifications. As such, our evaluation targets differ from those prioritized in previous financial LLM evaluation benchmarks. However, we have further expanded our Section 6 “Related Work” to include further discussion of LLM usage and LLM performance in the financial domain.

Thank you for your feedback on our work. We appreciate your insight into our work, but we respectfully disagree with some of the comments. We respond to your comments below.

Response to misunderstanding of Use Case of LLMs in the financial industry: The reviewer seems to misunderstand the motivation for the paper and how language models are used by retail investors in practice. We used “democratizing” in our title as our focus is not large investors. Of course, analysts at Jane Street are not going to query LLMs for revenue information. We don’t believe nor expect large brokerages and institutional investors to use LLMs for retrieval of revenue related information that they can get from their paid data sources. Our study is designed keeping in mind 10s of millions of vulnerable retail investors. As we explained in our original submission (line #162-171), our experiments are not representative of LLM usage by industry experts; however, our study is motivated by how retail investors or other financial non-professionals may use LLMs. As also highlighted in our introduction, we are motivated by demonstrated retail investors’ reliance on LLMs, as we believe this population to continue to be vulnerable to the biases described in our paper. We concern ourselves with this population specifically due to recent claims that LLMs will “democratize financial knowledge” (Yue et al. 2023) and as such we wanted to highlight biases that may be harmful to unsophisticated investors participating in this “democratization.” These financial non-professionals are unlikely to use LLMs optimally, yet their investing activities are still consequential. We further highlight our concerns regarding non-professional investors in Section 5.

Furthermore, our choice of prompt question is justified in Section 2.2. We highlight that while such knowledge-retrieval questions are not likely used in institutional financial settings, novice investors may employ such approaches and be susceptible to the biases presented in our paper. In addition, we chose our prompt to find a representative example of the type of knowledge question novice investors may ask LLMs; we did not aim to create a prompt that would capture all LLM use cases for investing.

We also like to bring your attention to the Appendix J “WHY "REVENUE"?” which explains our rationale for why we use revenue as question.

We would also like to bring your attention to Appendix K which shows the impact of the biases for trading and Appendix L which shows how the framework is also generalized to Spanish soccer league data.

Response to Temporal Exploration of LLM Performance: Previous studies have examined the performance of LLMs on information produced after the cut-off date, yet to our knowledge there is no previous work on evaluating LLM biases across a large historical range. While we agree that larger media coverage and mentions in training data is likely responsible for both temporal and cross-sectional knowledge gap, we are the first, to our knowledge, to empirically confirm this hypothesis. Our introduction clearly talks about this.

Also, contrary to the reviewer's point LLMs don't do better for larger firms as their hallucination rate is also higher for larger firms.

Response regarding Sophisticated Prompting: Again, the reviewer seems to misunderstand the motivation for the paper and how language models are used by retail investors in practice. As highlighted before, we don’t expect retail investors and unsophisticated investors to use multi-agentic approach, and few shot prompts or other sophisticated investors to reduce hallucinations. As mentioned in Section 2.2, we focus our bias evaluation on replicating LLM usage associated with retail investors and/or non-financial professionals. This population is unlikely to use the LLM prompting techniques suggested, like few-shot prompting and multi-agent approaches, and as such, we omit such techniques from our bias evaluation. We also believe that “very simple research design” is a strength not a weakness of our work.

Response regarding Fiscal Year and Calendar Year: We design our prompt to be representative of the type of prompt non-professional investors may use when using LLMs. As such, we omit nuances such as the differentiation between fiscal and calendar year in our prompt as we do not believe such details would be included in the prompt used by an retail investor. As we do not specify a quarter but specify a year, it is annual revenue. Also, we believe that most retail investors are unaware of the difference between the calendar and fiscal year.

We would like to also bring your attention to Appendix K which shows the impact of the biases for trading, and Appendix L which shows how the framework is also generalized to Spanish soccer league data.

4. While the authors claim novelty in demonstrating GPT's temporal and cross-sectional knowledge gaps in finance, similar findings are well-documented in broader LLM literature (e.g., Bender et al. 2021). Many papers have already shown that less-represented events in the training data can cause hallucinations and inaccurate answers. Surprisingly, the authors do not cite any of these papers. Of course, judging a paper’s contribution is a subjective issue. But extending this to finance, while potentially interesting, lacks sufficient novelty for a general-interest venue like ICLR.

Additionally, the conclusion that GPT hallucinations occur more for larger firms but are also more likely to provide answers is unclear. What is the takeaway? If the authors suggest LLMs are unsuitable for financial knowledge retrieval, the contribution seems limited to documentation. You might want to take your study to develop a knowledge-retrieval tool for retail investors. But again, using LLMs for financial metric retrieval does not seem like an ideal use case.

5. The authors argue again that retail investors cannot distinguish between fiscal and calendar years. However, then it becomes even more confusing how the authors evaluated their model performance. The model will surely know the difference between fiscal and calendar years. If you ask for the 2006 revenue, are you referring to the revenue released in 2006 or the revenue of fiscal year 2006? How does the model interpret this question? Either interpretation can be correct. How did you evaluate the “correctness” of the model’s answer?

Again, the authors argue that I misunderstood their motivation. I perfectly understand their motivation. However, the authors' execution and institutional understanding are far from ideal and cannot clear the bar at ICLR. Overall, the authors’ response did not resolve any of the issues raised in my previous report. I also reviewed other reviewers’ comments and agreed with the reviewer zyuC. The paper lacks novelty and does not provide convincing empirical evidence.

References

Bender, Emily M., et al. "On the dangers of stochastic parrots: Can language models be too big?🦜." Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 2021.

Lopez-Lira, Alejandro, and Yuehua Tang. "Can chatgpt forecast stock price movements? return predictability and large language models." arXiv preprint arXiv:2304.07619 (2023).

Kim, Alex, Maximilian Muhn, and Valeri Nikolaev. "Financial statement analysis with large language models." arXiv preprint arXiv:2407.17866 (2024).

Fieberg, Christian, Lars Hornuf, and David J. Streich. Using gpt-4 for financial advice. No. 10529. CESifo Working Paper, 2023.

Niszczota, Paweł, and Sami Abbas. "GPT has become financially literate: Insights from financial literacy tests of GPT and a preliminary test of how people use it as a source of advice." Finance Research Letters 58 (2023): 104333.

Niszczota, Paweł, and Sami Abbas. "GPT as a Financial Advisor." Available at SSRN 4384861 (2023).

I am not satisfied with the authors' response. Please see my answers to your response below, point by point.

1. The authors claim retail investors primarily use GPT for financial metric retrievals, such as revenues and net income. This claim is unsubstantiated and contradicts existing surveys, which consistently show retail investors use GPT for investment advice rather than metrics easily accessible via search engines. See:

https://www.investopedia.com/investors-increasingly-relying-on-chatgpt-to-manage-investments-7559052

https://www.etoro.com/news-and-analysis/press-releases/one-in-ten-retail-investors-using-chat-gpt-style-ai-to-help-pick-and-manage-investments/

I do not misunderstand the motivation of the paper. What I am not satisfied with is the authors’ execution. Retail investors, unlike the authors’ claim, do not generally use GPT to ask for revenues, net income, etc. It is the opposite in reality, and they ask for investment advice. If the authors want to claim that the primary purpose of GPT for retail investors is knowledge retrieval, provide me with any authoritative source or survey evidence. What I know from other studies and from large-scale surveys is different from what the authors claim.

If the authors want to examine whether the technology is democratizing financial knowledge (and investment), they should assess the models’ ability to provide financial advice. This has already been tested in a large sample from many other papers. Sophisticated investors will use higher-quality prompts along with their professional judgment (i.e., human + AI), while less sophisticated investors are more likely to be susceptible to GPT’s recommendations. This is akin to what the authors do in Appendix K. However, Lopez-Lira and Tang (2023) have already done this analysis.

2. I am also not very convinced that end-users do not know anything about prompting. Even under restrictive assumptions of zero-shot prompting, GPT models integrated with web search significantly enhance performance. For example, using the authors' Figure 1 prompt, GPT-4 retrieved NextEra Energy's 2006 revenue ($15.71 billion) via web search. Did the authors' API include search functionality? If not, the evaluation omits a crucial capability of real-world implementations.

3. Appendix J's focus on revenues rather than net income undermines the paper's relevance to financial performance analysis. Net income is the standard performance metric in finance. Testing irrelevant contexts (e.g., "Spanish soccer league") detracts from the paper's focus. A tighter domain-specific approach is needed to strengthen the analysis.