Sentiment Analysis and Sentiment Intelligence

What is sentiment? For business intelligence and analytical purposes sentiment is defined as a collection of subjective impressions held by a party or a group of parties about a product, another party, an activity or course of action, a location (geographic and virtual), or a motivator, at some point in time or over a period of time. In analyzing sentiment the party expressing the sentiment is referred to as the “sentiment subject”, and is said to have a sentiment about a particular “sentiment object.” Sentiment objects are one or more of the parties, products, activities, locations or motivators listed in the previous sentence.

What is sentiment analysis? Sentiment analysis refers to the application of natural language processing, computational linguistics, and text analytics to identify and extract subjective information in source materials. A basic task of sentiment analysis is to classify the given text as to whether it expresses a positive, negative, or neutral sentiment on the part of the subject toward the object.

What is sentiment intelligence? Sentient intelligence is the use of sentiment analysis in a systematic way, either alone, or much more commonly, in combination with other decision support tools to answer business questions, provide feedback to new product launches and marketing campaigns, and to protect the reputation of an enterprise, especially in mass media and social networking environments.

The result of sentiment analysis can be scored along several dimensions called classifications. The first classification is Subjectivity and it has two measures, Neutrality and Polarity. In the example in Figure 1 we entered the most neutral sentence we could think of in reference to a product, and the text classification engine scored the text (“The product is neutral.”) 0.5 on Neutrality (“neutral” on the display) and 0.5 on Polarity (“polar” on the display). With the sentiment analysis engine we are using2 the Subjectivity score must always equal 1.0. The second classification is Polarity, a measure of how positive or negative a non-neutral sentiment is.

If both Neutrality and Polarity have a score of 0.5 the text is assessed as neutral and Polarity does not need to be calculated.

In the next example, Figure 2, we entered a positive statement about the product (“The product is the very best I’ve ever seen.”) The score changed, Neutrality dropped to 0.2 and Polarity rose to 0.8. Because the sentence is no longer neutral, the process also breaks Polarity down into a Positive score (“pos” on the display) of 0.8, and a Negative score (“neg“ on the display) of 0.2. Clearly the text is no longer neutral and it expresses positive polarity.

In example 3 we changed the example 2 text just slightly to express a negative statement about our hypothetical product. Notice that the Subjectivity changed as one would expect, but now the Neutrality is 0.0, and the Polarity is 1.0. Apparently the word “worst” is somewhat more negative that the word “best” is positive. This is a result of the vocabulary that was used to train the model used in these examples. The point to be made here is that the model needs to be trained on a vocabulary that is both comprehensive and relevant to the domain of the subject area being analyzed.

Polarity is a critical factor in the use of sentiment analysis in business intelligence (BI). It is a differentiator which can be associated with other dimensions in a BI query to answer questions such as “Who likes our products?”, “What do consumers as a whole think of our services?”, “Where are our customers most favored locations?”

Qualitative and Quantitative Measurements Sentiment is a qualitative measurement, like quantitative measurements such as sale price, units sold, purchase cost, and units purchased, sentiment can be used to measure the performance of many aspects on an enterprise. These aspects can include products, customers, internal divisions, exchanges, means of communication, campaigns and many others. These sentiment objects are assessed in relationship to the sentiment subject, to other sentiment objects, and to other dimensions of a sentiment model that may not be strictly subjects or objects in the same sense.

It is important to understand what is being measured. Quantitative measures are usually derived, for example, from business transactions and typically expressed in amounts of money and numbers of items associated with a sale activity, purchase activity or some other movement of goods or services. Sentiment measures are usually derived from text processing and sentiment analysis and are divided into two classifications for further analysis, Subjectivity and Polarity, as we saw in the examples above.

Therefore. sentiment can be analyzed using the same 6BI Design Framework3 that is used in assessing performance when the performance is expressed as quantitative measures. However, instead of measures that are defined as “objective”, that is, they are expressed in universal terms such as currency or units, sentiment measures consist of “subjective” impressions.4 Subjective impressions are defined by key-value pairs where the keys are the granularity of the scale (in our previous examples 0.0 to 1.0) and the values are the words or phrases that are associated with each grain of the scale (in our example the grain is one tenth, e.g., 0.1, 0.2, 0.3, etc.).

The domain of the values is enumerated by words and phrases in a particular language that are assumed to have relative polarity (i.e. distance from neutral) along the scale in either direction. The granularity of the scale can be as fine or as coarse grained as the designers of the analytics decide to make it. The finer the granularity the longer it takes to train the model and, almost counter-intuitively, the less accurate the system will be.5 For example the granularity in our examples could have been hundredths (e.g., 0.11, 0.12, 0.13, etc.) but among other concerns it is doubtful whether the English language can express consistent and minimally ambiguous sentiment at any finer grain than what we’ve used.6

A Business Intelligence Model for Sentiment AnalysisFigure 4 contains a very simple model for analyzing and reporting on the results of sentiment analysis. This model is a view of the results of sentiment analysis in a dimensional data model format. It consisting of the following.

• Sentiment Measures – This is the fact table where the Neutrality and Polarity Scores are stored for each combination of sentiment batch, sentiment source, sentiment subject and sentiment object. We capture the date of capture and the date of analysis to establish a timeline for our sentiment analysis. This is important because the “shelf life” of sentiment can be very short. A product that is loved today, can be hated tomorrow, and then felt neutral about next week.

The dimensions of the model include the following.

• Sentiment Batch – This entity represents each batch of sentiment analysis scores that are loaded into the business intelligence system.
• Sentiment Source – This entity represents the sources of sentiment score batches. For example, sources can be internal customer facing websites, social networking sites such as Facebook, Google+ and Twitter, or specialized providers of consumer sentiment.
• Sentiment Subject – This entity represents the subjects or parties that expressed the sentiment. For example, they can be specific individuals or organizations, population groups based on demographic or other criteria, or a population sample acquired from a third party.
• Sentiment Object – This entity represents the objects of the sentiment expressions. These are the things about which sentiment is expressed on the part of the sentiment subjects. Products, parties (people or organizations), activities, placements and motivators are examples of the types of sentiment objects.

Note that the assessment of Polarity into positive and negative is not captured in the model. This classification is designated by the BI front-end tool used for visual analytics.

Consumer Sentiment TypesThere can be many types of sentiment measurement, but three in particular have been studied and used for long enough to have commonly accepted definitions and a history of application in consumer sentiment scenarios.7 These common consumer sentiment types include attitude, opinion and emotion.

• Attitude. Attitude is a measure of whether a party (i.e., sentiment subject) likes or dislikes the sentiment object (e.g., product, another party, activity, etc.).
• Opinion. Opinion is a measure of whether a party is for or against the sentiment object (e.g., a course of action). 
• Emotion. Emotion is a measure of whether a party feels an outcome or result is good or bad. Emotion can also be an indirect measure, it can influence the holder, that is the sentiment subject, to amplify or change their attitude or opinion. Emotion is also the most difficult of the three to measure, and most unreliable to use for predictive analytics.

These are only three examples of the measurements that can be used to assess consumer sentiment. The more complex the language analysis that has to be performed to determine the values along a sentiment scale the more expensive the process. Coarser grained scales are preferred for early stage consumer sentiment analytics as they require less resources and produce more consistent results. Finer granularity requires more accuracy and this in turn requires more resources. At some point the real cost of consumer sentiment analytics will outweigh its potential benefits.

Application of Sentiment IntelligenceIt is believed or assumed that attitudes, opinions and emotions have an influence on behavior, that is, the actions taken by the holder of a sentiment are done so, at least in part, due to the influence of the sentiment. Without this assumption sentiment analysis would have little practical value. It is also believed that sentiments can be influenced but generally cannot be created. That is, though a sentiment is originally generated by a person it can be changed by influencers before any responding action is taken. Certain sentiments as measured by a sentiment scale have been historically correlated with certain actions. For example consumers (i.e., sentiment subjects) who express that they “like” a product, another person, a location, or an action, at a point in time, are more likely to take a favorable action toward that sentiment object.8 In the case of a product that would, of course, be to buy it, or at least give it a favorable review and perhaps influence others to buy it. This would be an anticipated, or hoped for outcome of the positive expression of any of the three sentiment types.

There is a critical period of time that an enterprise seeking to realize value from sentiment analysis has to attempt to influence the actions associated with consumer sentiment. Some sentiments can dissipate before they can motivate an action, so it is essential that consumer sentiment be analyzed in as timely a manner as possible. Action planning based on aggregated sentiment analysis should be conducted with a minimum of latency between the analysis and the planned activity. In this way an enterprise can realize the benefits of sentiment intelligence.

End Notes:

1. In these examples I used the demo sentiment analyzer at http://text-processing.com/demo/sentiment/. It is written in the Python language. It is not trained on consumer products but I believe it is a very clean and clear demo and gives a good picture of how sentiment analysis works and how it can be scored in an easy-to-understand manner.
2. Natural Language Toolkit 2.0. NLTK is a leading platform for building Python programs to work with human language data. 
3. 6BI website. See the article “Six Basic Interrogatives – Part 3, 6BI and Business Performance Management” at http://waynekurtz.wordpress.com/ for an introduction to the concept.
4. Mullen, Introduction to Sentiment Analysis.
5. Jenkins, MC, How Sentiment Analysis Works in Machines
6. Jenkins, there are more coarse-grained sentiment analysis engines that provide only two scores, plus (+) or minus (-).
7. Mullen.
8. This is most likely the theory behind the “Like” action one can take on Facebook toward some object of one’s sentiment. This sentiment can then be exploited to influence the behavior of the person expressing the “Like” who is, of course, the sentiment subject in our analysis.