This article is one of a two-part piece documenting my learnings from my Machine Learning Thesis at Spotify. Be sure to also check out the second article on how I implemented Feature Importance in this research.
Feature Importance Analysis with SHAP I Learned at Spotify (with the Help of the Avengers)
Identifying top features and understanding how they affect prediction outcomes of machine learning models with SHAP
In 2021, I spent 8 months building a predictive model to measure user satisfaction as part of my Thesis at Spotify.
Image by Author
My goal was to understand what made users satisfied with their music experience. To do so, I built a LightGBM classifier whose output was a binary response:
y = 1 → the user is seemingly satisfied
y = 0 → not so much
Predicting human satisfaction is a challenge because humans are by definition unsatisfied. Even a machine isn’t so fit to decipher the mysteries of the human psyche. So naturally my model was as confused as one can be.
From Human Predictor to Fortune Teller
My accuracy score was around 0.5, which is the worst possible outcome you can get on a classifier. It means the algorithm has a 50% chance of predicting yes or no, and that’s as random as a human guess.