A grey statue of Socrates looks to his left at the title which is surrounded by smiley faces reflecting sentiment analysis.

Unless you are a psychopath, humans are by nature emotional beings. Anger, sadness, happiness, and love are emotions that almost every human has felt. Nowadays, we have a plethora of emotional AI detection methods at hand. Sentiment analysis is one of the most widely used methods, with 59% of companies using sentiment analysis to improve their customers’ experience, according to Bain & Company. But what exactly is sentiment analysis? Today we will look behind the fancy marketing gimmicks and get straight to the core of sentiment analysis and why Caplena’s stands out from the rest.

Sentiment Analysis 101

Sentiment analysis is a natural language processing (NLP) technique used to determine whether data has a positivenegative or neutral sentiment. In short: Sentiment analysis looks at the emotion expressed in text. For example, businesses can use this to monitor brand and product sentiment in customer feedback. Feedback of this type usually takes the form of textual data, such as reviews or responses to surveys.

‘Sentiment’ = ‘Emotions’?

A green and blue polar graph of the different emotions one experiences in life. Both positive and negative sentiment.

When we examine emotion in academia, we often see circles like the one above that show various emotional options. However, it is challenging – even for people – to make precise statements as to what kind of emotions are prevalent in a given text. As a result, there is a significant disparity between what different people attribute to the same text. To illustrate this point, we had eight people decide which emotion to assign to a given text out of eight options.

💡 The results -> contestants chose four emotions overall, not just one.

This shows a 75% perceptive discrepancy from choosing just one answer. Crazy, isn’t it?

As we can now see, even humans are incapable of determining complex emotional subtleties without images or audio. It is not recommended to classify into detailed emotions such as hate, sadness, or joy – this would be too inaccurate.

➡️ Through sentiment analysis we can see the polarity of texts and whether they are positivenegative, or neutral. There are two main types of sentiment analysis:

1 – Verbatim / Text Level Sentiment Analysis

Positive sentiment for "fascinating", negative sentiment for "awful" and neutral sentiment for "okay".
From Caplena’s in-app sentiment analysis feature.

Here we assign a positive, negative, or neutral sentiment towards an entire text, meaning a whole sentence – or paragraph. How can we use this? On the verbatim level, using an aggregated approach is recommended, so looking at 100s or 1000s of texts and saying that 30% had a negative sentiment, for example. A verbatim level sentiment analysis may be appropriate for practices such as social listening. Typical tweets, for example, are short and have only one sentiment per sentence. For types of text that show only one emotion, verbatim level sentiment analysis is sufficient.

➡️ In reality, however, most sentences contain more than one sentiment. Customer feedback and customer reviews are good examples of this. As a result, verbatim-level sentiment analysis can quickly become inaccurate. Assigning a single sentiment to a given sentence becomes problematic if the sentence has more than one sentiment; which leads us to the second type:

2 – Topic Level Sentiment Analysis

Review states "too costly but the service is great" reflecting a negative price topic and positive service topic.
From Caplena’s in-app sentiment analysis feature.

A more advanced version of verbatim-level sentiment analysis is topic-level sentiment analysis (also used by Caplena). Here we classify the sentiment towards a specific topic of the sentence (e.g price and service.) As shown in the example above, statements with mixed sentiments are very common in customer feedback.

Most humans would agree that it is hard to decide on just one sentiment for the above sentence. But if this was a verbatim level sentiment analysis – you would have to choose. Would negative and positive cancel each other out and make a neutral? Would it appear to be more positive than negative?

This is our humble opinion → why force a conclusion based on just one sentiment, when a sentence clearly contains multiple? This is why Caplena offers ️‍🔥sentiment analysis on the topic level️‍🔥 rather than verbatim level – because it is simply more accurate.

Topic-level sentiment analysis more accurately presents aggregated analysis findings and provides topic-level action taking potential. If you know that 20% of your customers were unhappy with the price of your service, you could reach out to each of them specifically to offer a discount, for instance.

How Reliable is Sentiment Analysis? A Short History

Starting in the 2000’s

Determining sentiment 100% reliably and automatically is still a holy grail of text analysis. But we’re getting closer. In the past, lexicons have been used, such as SentimentWortschatz, which assigns a value between -1 and 1 to each word. The word “crash” gets a value of -0.47, while wonderful gets +0.72. By adding up the values of each word in a sentence, we get a total value which announces which sentiment prevails. This works for simple statements but fails for negations because it ignores the interplay between words. The 2000s saw several attempts to address this issue, but none could account for context – and as Bill Gates said: “context is king/queen.” The real *boom* of sentiment analysis came only within the last decade, though.

2010 – Present | How Sesame Street Saved Machine Learning

Artificial intelligence, deep learning, and transformers have fundamentally changed this context issue in the last decade. It began with BERT, the character of Sesame Street – short for Bidirectional Encoder Representations from Transformers. For the first time, BERT took context into account by giving the AI many simple tasks on millions of Internet text documents. One of these tasks was to reconstruct randomly deleted words from texts. With practice, the AI gets better at guessing which word is missing from the gap, which is how machines learn context.

Popular algorithms in NLP overview: BERT; ELMo; Word2Vec; OpenAI GPT. Bert and Elmo from Sesame Street portray 2 algorithms.

Since BERT, many improvements have been made based on this, such as RoBERTa and ALBERT. One thing is clear: AI’s are becoming more complex and trained on more data, making it more difficult and expensive to build an AI from scratch. Most companies use commercial AI providers or open-source libraries. Very few perform automatic sentiment analysis on user-defined aspects (with Caplena being one of them). Some solutions additionally offer fine-tuning of the AI significantly improving the accuracy for the specific industry and question. The fine-tuning process involves manually reviewing a few sentiment assignments or injecting additional rules.

An example of fine-tuning (categorizing feedback) on the Caplena coding software platform.
Caplena provides fine-tuning in the ‘Coding View.’ Check if the topics assigned automatically are correct, incorrect, or need any additions. Within a few clicks, the AI reaches human-level accuracy.

“But What About Sarcasm?” – You Say.

Sarcasm is difficult to interpret even with the best systems. The primary difficulty is that sarcasm is very context-dependent. Let’s look at some examples of how Caplena’s AI categorizes the following statements:

"Great to see sentiment analysis being used everywhere" marked as a positive sentiment with no emoji and negative sentiment with a sarcastic emoji.
From Caplena’s in-app sentiment analysis feature.

As it turns out, our system has correctly inferred 3/4 of the cases (all but the last). What’s the takeaway?

  • AI systems can already detect irony and sarcasm in quite a few cases, although they still lack human-level performance. Without additional indicators like emojis, pictures, or the extensive use of punctuation, it is impossible even for humans to detect irony or sarcasm in text reliably.

Where does this lead us in terms of the overall analysis?

  • Irony and sarcasm should not affect the overall sentiment analysis results. Of course, mistakes happen, but in most cases, AI can detect sarcasm or irony or humans leave enough evidence of them that it is obvious.
  • Having more than a few people give sarcastic feedback is unlikely – most people want to make sure their comments are understood and thus won’t be vague in how they provide it.

How’d you like this article?  Our team will be happy to include your suggestions in future blog posts! Just email our Head of Marketing sheila [at ] caplena.com

To try Caplena for free, click here, or keep browsing through our most popular blog posts:

Related Posts:

Headspace vs Calm: A Comparative Analysis of Customer Reviews

Other Useful Links:

Sentiment and Emotion Analysis for Beginners: Types and Challenges

Leave a Reply

Your email address will not be published.

Rethinking open-ended surveys

Previous ArticleNext Article

Leave a Reply

Your email address will not be published.