How Most Sentiment Technologies Work Today
Sentiment analysis, sometimes called opinion mining, is one of the easiest and quickest ways to find out what consumers are thinking about a brand, product or event. It’s a natural language processing technique often used in social listening scenarios, that aims to systematically identify opinions in a document and give it a score of positive, negative or neutral.
That’s the theory, at least.
There are few things as mind-numbingly tedious as manually tagging documents with the right sentiment because the technology doesn’t get it.
Sentiment analysis (ironically) has a bad reputation in the social listening industry, because truth be told, it needs a lot of manual work to deliver great results.
Our data science guys (the brains behind our award winning image recognition technology) have been working on fixing this behind the scenes, and I’m excited to finally share their fantastic results.
They’ve developed a social sentiment technology based on deep learning that lets brands capture customer sentiment with 90% accuracy. This AI technology for the first time truly understands the meaning of full sentences and is able to accurately determine customer attitudes and contextual reactions in tweets, posts and articles.
But before we dive into details of why this is such a big deal, let’s take a look at why sentiment analysis is essential for brands and how the technology works.
Why is Sentiment Analysis Useful for Brands?
Sentiment analysis is one of the most versatile technologies in social listening. Brands can use and cross sentiment indicators with a variety of data to get a better understanding of what their customers are thinking. Let’s look at a few examples where sentiment analysis can be invaluable:
1. Managing Critical Posts
Many of our clients’ marketing and comms teams operate in real-time. With sentiment analysis, critical posts are detected and flagged immediately. Teams can cut down on reaction time as relevant people get an alert for high-risk posts.
2. Improving Customer Experience
Looking beyond the post level, sentiment analysis is great for improving customer experience. For example, a large client from the hospitality industry regularly evaluates different aspects of their customer service. After noticing a large number of negative posts on social media, their budget hotel chain was able to improve customer experience measurably by installing hair dryers in all available room categories.
3. Analyzing Product Launches
One of Talkwalker’s tech industry clients uses sentiment analysis technology to keep a close eye on how phone features perform after a product launch. Comparing different aspects like contrast, brightness and phone size, the company provides its R&D departments with real-time data to assess how product launches performed and which product aspects to focus on.
4. Evaluating Sponsorship or CSR Impact
Sentiment can be used to show how the public responds to sponsorship initiatives.
To analyze the public opinion around corporate social responsibility initiatives, the brands can track if and how public opinion shifts, or monitor message pull-through for key issues like sustainability, diversity or innovation.
5. Market Trends Research
Monitoring trends on a broader scale, one household brand evaluated attitudes towards kitchen cleaners on the market, and was able to see that smell was one of the most commonly disliked issues. Based on this research, they asked their R&D team to come up with a product they could market with “great smell” as a differentiator.
For one of their other product lines, the brand regularly tracks attitudes towards breakfast food trends to determine whether to launch new flavors or products.
6. Maintaining Service Quality Across Locations
Clients that operate in multiple markets or have a lot of franchise locations to manage, use sentiment technology to track opinions about suppliers, local issues and branches. This can be especially helpful to assess service quality internally.
7. Monitoring Executive Popularity
People have become one of the biggest drivers of public opinion around a brand, which is why many clients now track how the C-Suite of their company is perceived. In one case, it transpired that the CEO wasn’t present on the market. Using competitive insights, the brand identified the ideal issues to position him on, and was able to raise his public profile.
Always fun to visit @TMobile stores in NYC and 77th and Broadway was on fire when I walked by so I said hello and GoGoGo! pic.twitter.com/nPA7NuNRk3
— John Legere (@JohnLegere) 16 septembre 2017
Sentiment analysis is an incredibly helpful tool that brands can use in a variety of ways. Many brands still sideline sentiment indicators in their reports, however, because trust in the technology is low. Below, we’ll look at why that is, and what can be done to fix it.
How Does Sentiment Analysis Typically Work Today?
There are two main approaches most vendors use today: sentiment analysis based on keyword scoring, or a calculation based on predefined categories.
Keyword scoring means you give the word “good” a positive score, the word “bad” a negative. This approach has some inherent flaws that become apparent when you look at some of these examples:
Go Go and Blood, Sweat and Tears have a sound very similar to Romanian music circa 2005 and it's amazing
— Ennun (@ennunanaiurov) 18 September 2017
If the technology is only basing itself off of keywords, both tweets above are very hard to classify.
Some vendors try to fix the technology’s flaws by applying rules on top of the keyword scoring, but the underlying problem remains: keyword scoring only captures fragments of a message, and never its true meaning.
Keyword scoring can deliver some worthwhile results, if focused on broader trends or applied at scale. But at a granular level, a sarcastic tweet or a 13-year-old calling a new video game “sick” brings the system to its knees.
Because keyword-based sentiment technology doesn’t understand context, only individual words or small phrases, proven accuracy levels typically range between 50 to 80%. That’s why many brands are sidelining sentiment indicators in their reports.
The second approach is based on the idea that you let the user categorize a few dozen results (your training set), and then let an algorithm make decisions for future results.
Accuracy for predefined categories is typically higher than for keyword scoring, but that doesn’t mean it’s perfect. Categorizing results manually requires a huge time investment and an understanding of all topics that could potentially arise around your brand. Because this method works with such narrow parameters for qualifying results, it typically yields fewer results overall.
Of course, there’s a third approach I haven’t mentioned: some brands and agencies drop technology altogether and have humans code sentiment.
The upside is that accuracy isn’t something you’ll have to worry about a lot. People will disagree here and there (language is messy), but overall you’ll get an accurate picture.
The downside: on average, a person can only classify about 100 documents per hour. Brands can get thousands of mentions a day. Coding consistency aside, by the time your coders reach the critical mention that matters, it might be too late to act.
Worse yet, if brands want to determine consumer sentiment around certain topics for market research or get a snapshot of opinions around a trending topic, the significant costs involved to get timely results are seldom a valid option for brands.
How Does AI-Powered Sentiment Analysis Work?
When the introduction of our image recognition technology in early 2016 was such a success, we knew right away that we wanted to apply that knowledge to sentiment technology. The Talkwalker data science team has worked hard to deliver results, so let’s see what they came up with!
AI-Powered Sentiment works with an entirely new approach. For the first time, the algorithm understands the meaning of full sentences and is able to accurately determine customer attitudes and contextual reactions in tweets, posts and articles.
Using deep learning models that simulate the cognitive functions of the human brain, the technology can understand complex language patterns and entire sentences, and even deal with basic forms of sarcasm and irony.
While developing the algorithms that are being used now, the team did some research into determining accuracy for the posts. Accuracy of AI-Powered Sentiment scales with the size of the training set from which the algorithm “learns”, so we had to think big.
The research below shows how the size of the training correlates to the percentage of correctly identified results. That meant that the team had to classify tens of millions of results to ensure the 90% accuracy rate the algorithm now delivers.
Where’s the Technology Headed?
Sentiment technology is never going to be an exact science. But with the power of millions of results that have been accurately classified, brands will be able to take a huge step into the right direction. Our data science team is already working on ways to make the technology even smarter for brands.
We think being able to accurately classify sentiment is just the start. In a single click, brands can benchmark brand health indicators, supplement the data with demographic information or combine it with individual product features to get the best idea of how to serve their customers well.
We’re excited to see the benefits of the new technology in action, and if you want to give it a try, don’t hesitate to get in touch today. We LOVE showing this feature.
