Corporate/ PM Role/ New Logic
Customer Reviews as part of Listing Logic
Using Customer reviews sentiments to give priority in Listing based on User’s choice
In the digital age of today, all the companies believe on marketing strategy of pulling the customers for trying to use their product. To do so, they work on building a perceived value of their product in the mind of each customer. They acquire the customer needs and wants, and satisfy it through their products.
The optimized mapping of customer need with the product requires many analysis, one of which is QFD. In this blog, we present a process, which will help to optimize the said mapping when user makes a search on any digital platform. For the example, we are going to optimize the search results on Google Play store, when user search for a term. The term which is used in this example is “Hotel Booking Apps”. The same process can even be used for search listing of suppliers in e-commerce platform.
When the user searches for a keyword, different application appear in the form of listing on Play store. These applications appear in the sequence of “Keyword-Text” match or/and supplier ratings, traffic, hyperlocal etc. However, still we find out in the reviews that the product suggested by search did not meet the expectation of users. One of the major reasons is the lack of listing logic on search which considers the customers expectation on real time basis, with the help of reviews present for each listing.
To summarize the process of Customer Expectation, the reviews of all the application/listing to be collected. Based on the expert opinion/ topic extraction analysis find different topic suitable for the search category. From these lists of topics, select four to five topics which you feel are most important to customer when they make the search.
For all the extracted reviews, and selected topics, you need to perform the sentiment analysis on each review. The additional step of finding the cosine similarity for each of these sentiment analyses could also be extracted or in case if only the sentiment analysis of topics is available, you need to assign positive sentiment as 1 and negative sentiment as -1. Based on the aggregated value (mean or median) the sentiment value for each topic need to calculate for each listing.
For each of these search categories, you also need to get the conjoint analysis done for the selected topics. Based on the conjoint analysis you would get what topic is of more important to the customer. Thus, the priority needs to be given in search listing to those who have higher sentiment value for the important topics.
So, when a user searches for a keyword, the listing should appear on the hierarchy of:
- List all the result that satisfy the Keyword-Text match
- From the list of filtered result, give priority to Customer Expectation result
- Sub priority of the listing on basis of supplier ratings, traffic, hyperlocal etc.
Analysis:
The listing we selected was of Google Play store due to high availability of reviews for each of the listing and availability of APIs for easy extraction. We searched for “Hotel Booking Apps”, and found the following application in the result:
- Expedia
- Tripadvisor
- Makemytrip
- Booking
- Agoda
- Cleartrip
- Yatra
Thus, we wrote the code to extract all the reviews from these listings on google play store.
From the list of all the reviews, we performed the topic sentiment analysis and extracted the topic for the keyword “Hotel Booking Apps”.
From the list of all the topics extracted, we Questioned, a list of 20 frequent travelers, and found list of important topics, which they consider while making any booking online. The Final Selected topics are:
- Application/App
- Experience
- Customer Service
- Search
- Notification
- Destination
- Hotel
- Payment
- Book
- Navigate
- Information
- Offer
- Travel
- Deals
- Quality
- Cancellation
Out of these topics to perform the conjoint analysis and Customer Expectation Process, we pick the topics:
- Hotel
- Information
- Booking
- Application
To do the sentiment analysis for each of these topics, we make two tagged datasets: one containing all the positive keywords such as good, clean, easy etc. and the other containing all the negative bagged words such as poor, uneasy, slow, pathetic, dirty etc.
For each of these bagged keywords, for the reviews we did the sentiment analysis for each of these reviews for all the listings.
To proceed further, for each of the topic, the aggregated value of each topic needs to be calculated distinct for each of the listings. To do so, we need to consider the value negative as -1 and positive as + 1. To calculate the aggregate value, we proceed with mean in our case.
For Eg:
For Tripadvisor, the aggregate value for topic “Booking” came out to be 0.347.
Similarly, we need to calculate for Information, Hotel and Application Sentiment aggregate value for all the listings.
Parallelly we asked 6 users to rank the combination of different level of Information, Hotel, Application and Booking. For these four attributes we used the level as:
- Information
o Reliable
o Unreliable
- Booking
o Hassle Free
o Rigid
- Application
o User Friendly
o Non-User Friendly
- Hotel
o Good Suggestion
o Poor Suggestion
Based on the Rankings collected by different users, we performed the conjoint analysis to understand the factors/ topics that are important for the users. To perform the conjoint analysis on SPSS, we had to create the dummy variables, and using the regression analysis we were able to find the part worth utility for each of the options.
The lowest the score, the important it is for the customer, based on the number, we can create the weightage for each of the topic. For e.g., User Friendly application to be given 20, Followed by Good Hotel Suggestion as 15, Hassle free Booking to be 7, Unreliable info to be 5, Reliable info as 4, Rigid Booking as 2, Poor Hotel suggestion as 1 and Non-User-Friendly app as 0.
With the help of these weightage, we would calculate the final score for each of the listing and the sequence of the listing to appear on the decreasing value of the final score.
For Eg, For Trip Advisor,
Booking Sentiments came out to be: 0.347.
Application Sentiment: 0.451 (Hypothetical — Not Calculated)
Information Sentiment: -0.212 (Hypothetical — Not Calculated)
Hotel Sentiment: -0.012 (Hypothetical — Not Calculated)
So the final score for Trip advisor would be: 0.347*7+0.451*20+0.212*5+0.012*1 = 12.521
Let us assume the final score of other listings came out to be:
- Expedia: 10.524
- Makemytrip: 8.721
- Booking: 13.566
- Agoda: 6.234
- Cleartrip: 4.433
- Yatra: 5.764
So the Google Play store when searched for “Hotel Booking App” should display the listing in the Sequence of:
1. Booking
2. Trip Advisor
3. Expedia
4. Makemytrip
5. Agoda
6. Yatra
7. Cleartrip
Also, every night with new reviews coming in the value for the final score would change and so would the listing order.