Sentiment Analysis has many impactful real world applications particularly in economics and finance. Financial News can give very meaningful insights about the price of a financial instrument, and we often don't have the time read every news article to make our own decisions. So, how can we leverage Natural Language Processing to aid our classification of financial news? There are three approaches to financial news sentiment analysis that I cover in this repository. The first is to build a Bag of Words model. That is, using word counts to calculate the probability that an article is negative, neutral, or positive. The second approach is to utilize word embeddings, which asssign each word a 300 dimensional word vector that represents its overall meaning. Finally, we can classify News Articles using sentence transformers that convert sentences into a 768 dimensional vectors that represent the meaning of the sentence and fit a classifier using these vectors. With these approaches we can use article headlines to classify financial news as either negative, netural or positive.

The data for for acccomplishing this task was taken from kaggle (link in sources section), and it consists of 4846 news headlines with labeled sentiments. There are not an equal amount of examples for each sentiment. This is called class imbalance. To overcome this imbalance, the metrics we are most concerned about are precision and recall scores for positive and negative sentiment headlines. Using this metric as our metric for performance has two advantages, it combats against the class imbalance that we have, and it makes our model accuracy more relavent to the real world. If an investment bank is looking to use NLP to classify financial news, we're going to want to know when we predict a positive sentiment, what is the likelihood that the article is actually positive. This metric can be described by recall. Additionally, we want a metric for the percentage of labeled positive news that was correctly classified by our model. This metric is known as precision, and for this project our goal is to have over 75 % precision and recall for our model.

• Precision
• Recall
• AUC
• Accuracy

The first approach is a BOW (Bag Of Words) model. BOW model is simply taking the entire vocabulary of all of our headlines and using each word in our voaculary as a feature. Then, the features for any given headline is the word count for every word in the vocabulary. For example, consider the headline:

“We went to the market with yield guidance of the 7.25 % area , which gave us the flexibility to go up or down by 1-8th.”

This can be represented by the following dataframe:

I did not include every column, but you can see how our model will use the information from the headline to make a prediction. Using word counts can give us some useful plots for understanding which words contribute to news sentiment. The plot below shows the probability of each sentiment given that the word said appears in our headline.

In red are the probabilities given that said appeared in the headline, and in blue are the headlines where said does not appear in the headline. We can calculate the probability of a positive article given the occurence of "said" in the headline using bayes theorem:

The calculation above confirms what we see in the plot. You can view the code in the EDA notebook to see the calculations. The following two plots illustrate the same conditional probabilities with different key words.

Using these word counts we can fit a logistic regression to the data and get decent results. The results are summarized below:

Model Metrics

• Accuracy:
• Precision:
• Recall:
• AUC score:

The TFIDF vectorizer also accomplished similar metrics, but there is another way we can improve apon bag of words.

I also compared these results to what state of the art sentence transformers from hugging face can do for sentiment analysis. Huggingface sentence transformers convert your sentence to a 768 dimensional vector representation of the general meaning of the sentence. I used this representation in a logistic regression and got the following results: The results were better than our custom made RNN's. Ideally our modeling techniques are optimized for our problem and should outperform even state of the art models that are not optimized for our problem. I have a few hypothesis on how to improve our RNNs to be able to outperform Huggingface. I didn't utilize syntatical analysis with my RNN approach. I only utilized semantics by feeding the network glove embeddings of each word. If I combine both syntatical and semantic features to my model. I could be able to outperform Huggingface transformers regardess of their pervasive use.