Recently one of our clients had an interesting scenario, where they wanted to understand customer characteristics based on their comments and feedback given to the company. The client was a leading service provider to the logistics, manufacturing, and transportation industry.
Its proprietary data guide every Data Science project in Aryng to decision – BADIR – framework. You can read more about BADIR here.
Once we identified the problem using the “business question” framework, we formed the project’s analysis plan. Based on this plan, we collected the client’s required data – comments and feedback of their customers.
Some Example Comments:
– “Sending drivers to us without the experience to do the job.”
– “We do not get qualified members promptly.”
– “I HAVE HAD VERY LITTLE TO NO LUCK GETTING PEOPLE IN TO FILL MY OPEN POSITIONS.”
Since the task at hand involved text data – the insights were based on text analytics. Now, with the advent of deep learning algorithms, complex language models, and advanced natural language processing techniques, there is a multitude of ways to carry out text analytics. You would have already heard the name of popular models such as word2vectors, Glove, etc. as well as recent breakthroughs such as BERT (developed by Google) and GPT-3 (developed by OpenAI – whose founders include Elon Musk and Sam Altman)
Keeping in mind the core business problem, we decided to perform three critical analysis:
– Analyze the text comments through word clouds
– Build predictive models with sequence algorithms.
– Do Aspect-based sentiment analysis.
The above three analyses were chosen to get different kinds of insights from the data. Based on the above analysis, the results and recommendations helped our client increase the NPS (Net Promoter Score) by 10% point.
Let’s take a dive into each of these analyses and see what they involve.
Word Cloud of Active Customers
Word-cloud is a method to analyze the most commonly occurring words given a particular condition or context. It allows us to understand which words are associated most commonly with a particular topic, field, context, etc
The text comments were from customers who were either active or inactive. So word clouds helped us understand the most common words used by active and inactive users. It allowed us to understand the themes that ran across the two sub-set of customers and the themes that differentiated them.
Word Cloud of Inactive Customers
The larger the words in the image – the greater is its frequency in the subset. This, of course, comes with the disclaimer that the most common words like is, the, a, of, etc. have been removed before the analysis. In the world of text analytics, we often call these words stop-words. They are often removed because they would dominate in terms of frequency while lending very little to zero value to any meaningful analysis.
Predictive Analysis – LSTM
We also built a sequence model to predict whether the customer was a promoter or detractor. The sequence model took as input the customer’s comment. (Since deep neural networks only accept numerical values, the comment in English was converted to a series of vectors – each word in the statement being a numerical vector) The sequence model was a bidirectional LSTM (Long-short term memory) network – a special kind of deep neural network, designed to predict an outcome based on a sequence.
These models can become very complicated, very fast, and are more challenging to interpret than conventional Machine Learning models. But they can accurately capture patterns from a sequence of words (in our case, stories converted to numerical vectors) and understand them.
This model can be used to flag comments which indicate low NPS and help the client reach out to unhappy customers much more efficiently.
Aspect based Sentiment Analysis
An example of word association is the SpaCy model.
This analysis was probably the most complex of the three. In aspect-based sentiment analysis, models usually select an aspect and try to figure out the sentiment associated with it.
There are various ways to do this analysis. For our case, we used the language model created by the SpaCy library. This model gives us word associations, through which we can figure out if the words associated with our target words give positive or negative sentiments.
It can also help with possible problems associated with our target aspects. For example, we were able to extract possible issues that were related to candidates as:
– ‘Not received.’
– ‘Qualified not getting.’
As we can see, there are so many ways texts can be analyzed, depending on the business needs. From predicting outputs from a sequence of words to analyzing sentiments – the opportunities are endless. There are more complex models, which I mentioned before (BERT, GPT-3), which can be used to make complex predictions such as answers to questions, chatbots, and conversational AIs. In the past decade itself, the advancement in NLP and Deep Learning has opened up a whole new world.