A curated list of resources of LLMs in Finance (FinLLMs) including their history, techniques, evaluation, and opportunities and challenges. It's based on our survey paper: A Survey of Large Language Models in Finance (FinLLMs). This survey will be actively updated including further evaluation of advanced Financial NLP tasks, a collection of financial datasets, and sharing FinLLM use-cases. Please stay tuned!🔥

• Evolution : from General LMs to Financial LMs
  • General-domain LMs
  • Financial-domain LMs
• Techniques : from FinPLMs to FinLLMs
  • Continual Pre-training : FinBert-19
  • Domain-Specific Pre-training from Scratch : FinBERT-20
  • Mixed-Domain Pre-training : FinBERT-21, FLANG
  • Mixed-Domain LLM with Prompt Engineering : BloombergGPT
  • Instruction Fine-tuned LLM with Prompt Engineering : FinMA, InvestLM, FinGPT
• Benchmark Tasks and datasets
  • Sentiment Analysis
  • Text Classification
  • Named Entity Recognition
  • Question Answering
  • Stock Movement Prediction
  • Text Summarization
• Advanced Tasks and datasets
  • Relation Extraction
  • Event Detection
  • Causality Detection
  • Numerical Reasoning
  • Structure Recognition
  • Multimodal Understanding
  • Machine Translation
  • Market Forecasting
• Other Useful Resources
• Citation

Selected papers associated with the above figure.

• GPT-1 : Improving Language Understanding by Generative Pre-Training. 2018. Paper
• GPT-2 : Language Models are Unsupervised Multitask Learners. 2019. Paper
• GPT-3 : Language Models are Few-Shot Learners. NeurIPS 2020. Paper
• GPT-4 : GPT-4 Technical Report. 2023. Paper
• BERT : BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL 2018. Paper
• T5 : Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. JMLR 2019. Paper
• ELECTRA : ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. 2020. Paper
• BLOOM : BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. 2022. Paper
• LLaMA : LLaMA: Open and Efficient Foundation Language Models. 2023. Paper
• LLaMA2 : Llama 2: Open foundation and fine-tuned chat models. 2023. Paper

• FinBert-19 : FinBERT: Financial Sentiment Analysis with Pre-trained Language Models. 2019. Paper | Github | HuggingFace
• FinBert-20 : FinBERT: A Pretrained Language Model for Financial Communications. 2020. Paper | Github | HuggingFace
• FinBert-21 : FinBERT: A Pre-trained Financial Language Representation Model for Financial Text Mining. IJCAI special track 2021. Paper
• FLANG : When FLUE Meets FLANG: Benchmarks and Large Pretrained Language Model for Financial Domain. EMNLP main 2022. Paper | Github | HuggingFace
• BloombergGPT : BloombergGPT: A Large Language Model for Finance, 2023, Paper
• FinMA : PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance. NeurIPS datasets and benchmarks track 2023. Paper | Github | Leaderboard
• InvestLM : InvestLM: A Large Language Model for Investment using Financial Domain Instruction Tuning. 2023. Paper | Github
• FinGPT : FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets. NeurIPS Workshop 2023. Paper | Github| HuggingFace

• [FinPLMs] Continual Pre-training : FinBert-19

• [FinPLMs] Domain-Specific Pre-training from Scratch : FinBERT-20
• [FinPLMs] Mixed-Domain Pre-training : FinBERT-21, FLANG
• [FinLLMs] Mixed-Domain LLM with Prompt Engineering : BloombergGPT
• [FinLLMs] Instruction Fine-tuned LLM with Prompt Engineering : FinMA, InvestLM, FinGPT

Evaluation summary of representative models across 6 financial NLP tasks

• models : FinPLMs (FLANG), FinLLMs (BloombergGPT, FinMA), LLMs (ChatGPT, GPT-4), and task-specific state-of-the-art models (SOTA) models across 6 financial NLP tasks.
• FinPLMs did not perform experiments on the more complex tasks such as Hybrid QA, SMP or Summ; hence, these data points are not included.
• Results are referenced from original or analysis research, and SOTA results from task-specific models.

• Sentiment Analysis (SA)

We include several workshops and programs for financial NLP. The link directs to the most recent workshop website.

• [FNP] Financial Narrative Processing Workshop 5th
• [FinNLP] Financial Technology and Natural Language Processing Workshop 5th
• [ECONLP] Economics and Natural Language Processing Workshop 4th
• [AiFinBridge] AAAI bridge - AI for Financial Services 2th
• [MUFFIN] Multimodal AI for Financial Forecasting Workshop 2th
• [KDF] Knowledge Discovery from Unstructured Data in Financial Services 4th

    @article{lee2024survey,
        title={A Survey of Large Language Models in Finance (FinLLMs)}, 
        author={Lee, Jean and Stevens, Nicholas and Han, Soyeon Caren and Song, Minseok},
	journal={arXiv preprint arXiv:2402.02315},
        year={2024}
    }