A Critical Look at Spotify's Algorithms, Or: How I Learned to Stop Worrying and Love the Algorhythm

In Stanley Kubrick's deeply dark, deeply insightful, and sadly timeless classic film "Dr. Strangelove: Or How I Learned to Stop Worrying and Love the Bomb", we see people at every level of power and influence trying to figure out just how to deal with a potential catastrophe and reality of their own making. In the end (spoiler alert), well... things don't go well.

While the algorithms we have allowed to be designed have not yet grown to such a potentially calamitous end, they are growing and are deserving of our constant review and critique.

One such algorithm is Spotify's Discovery Weekly. Massively popular and also massively influential. And what do we know about it? Very little. How much do we talk about it? Very little. But we should.

So let's do that.

Can we talk about the Spotify Discover Weekly algorithm?

If you use Spotify with any regularity, chances are you are probably a fan of the Discovery Weekly playlist, Spotify's algorithmically-determined list of songs tuned to you. Since its launch in 2015, it has been met with near-universal acclaim and listenership. A quick Twitter sentiment analysis for "discovery weekly" shows overwhelming positivity - even when it misses, it still hits.

But, like all modern conveniences - from tailored ads to "customers also viewed these similar products", there's a lot going on behind its face. And with the millions of Spotify users, it has a huge impact.

With our personal data fueling this (and other) massive engines of influence, these things are worthy of understanding - and critique. I've found shockingly little critical discussion around the impact of Spotify's algorithm when the likes of Facebook, YouTube, and Google are being called before Congress to discuss their algorithms, so I've decided to take a deeper dive and consider it all.

First, How The Spotify Algorithm Works

Straightaway, I'll tell you that the Spotify is close-lipped about their algorithm because it is a private company protecting their intellectual property. Secondly, it is very complicated, but not impossible to understand. Software engineer Sofia Ciocca published an in-depth Medium piece on how the Spotify algorithm works back in 2017 that exposed from primary sources how it worked.

At the time, she wrote it was these three components:

• Collaborative Filtering: What People with Tastes Similar to You Also Like
• Natural Language Processing: What The Internet Likes
• Raw Audio Processing: Sounds Like This Other Song You Like

You could argue that the above components map to "old school" discovery - asking your friends who like the music you do what they are listening to (#1), browsing music blogs or print reviews (#2), or ... just hearing a beat you like from a radio 15 feet away (#3). There's wisdom in modeling it as such - why reinvent the wheel?

But it's important to understand that 1) these algorithms are fueled by your data and 2) we have to consider the impact (negative and positive) of a massive audience being influenced by a homogenous algorithm.

Your Data

Don't be ignorant to the fact that your personally identifiable data is being used by Spotify to provide you this service, and to provide it to other paying users. But also: that data may be used or sold to third parties. But that's the bargain, right? Thanks to laws like the European GDPR and California's CCPA - most companies have been forced to provide a way for you to request or delete your data in a timely manner.

Read: Quora: How Can I Download My Spotify Data?​

The Impact On End-Users

There have been relatively few studies on the effects of Spotify on user's consumption of and discovery of new music. But one study, ‘How Consumers’ Adoption of Online Streaming Affects Music Consumption and Discovery’, determined that discovery of "highly valued music" increased dramatically with Spotify vs iTunes, as well as overall consumption. The general sentiment on social media also implies a deep, earned affinity with Spotify for these same reasons.

Empirically speaking, this maps to my own experience and that of my friends who I've discussed how Spotify has impacted them. I've encountered a number of new artists and albums that I now "highly value" through Spotify, and specifically Discover Weekly. And of those, I've tended to purchase physical copies of the album as well.

One case in point is Pastor T. L. Burnett's Like a Ship Without a Sail - a 1971 gospel album. I am not a religious person, nor do I have an affinity for gospel music but Discover Weekly recommended a few tracks off this album to me sometime in 2016/17. Likely due to the algorithmic components described above, specifically my "favoriting" of tracks that feature "choral vocals" and "tight bass lines". This album has both, and it quickly became one of my all-time favorites. I bought the LP on vinyl.

What's more, is that later I saw this relatively unknown album in the front window of my local record store that same fall of 2017. How did that happen? Was Spotify suggesting it to me because they conspired with a tiny label to push sales of a 2010 re-released album that had little fanfare before or after? Or because, rather, it resonated with others that shared my similar tastes?

So, both objectively and empirically, I tend to think that the Impact on End-Users with Spotify is a net positive. It's been shown to increasingly expose listeners to music they find "highly valued".

But: Keep in mind these recommendations are coming from an opaque algorithm from an organization with a profit motive. While useful, we have little way of knowing what goes into it. More on this later.

The Impact On Overall Sales: Initial Disruption Followed By Some Positive Trend

The record store as we knew it pre-iPod is long gone, replaced by digital downloads or more recently, streaming. And, since the nadir of music sales in 2015, it's been climbing in large part to streaming. But there has been a recent and very interesting development. A few weeks ago, the RIAA announced vinyl sales have eclipsed CDs for the first time in three decades - and as the BBC reported in 2016, vinyl sales were being driven by streaming.

This stream-then-buy has been the case for me. I can't tell you the number of artists I've been introduced to - or re-introduced to and I've bought the vinyl. Apparently it's the same for other listeners? Emotional attachment may very well be turning into sales (again).

So, by all accounts while streaming (or perhaps the ability to stream) killed the traditional music-buying marketplace, in its place a new ecosystem has risen. The barriers to discovery and purchase are lessened, but so has the need to purchase discrete albums and tracks lessened.

One thing that an algorithmic approach to music discovery does provide is a wider view of cross-genre interests. I could not find any definitive information on the growth of genre diversity among listeners, but empirically - I've seen this in myself, and in my friend network.

The Impact On Artists: Likely Positive, But Is a Single Point of Failure

From an artist-revenue perspective, the general attitude is that Spotify isn't pulling its weight with the "middle class" of artist. But, from a music discovery and overall awareness perspective, there's never been an opportunity like Spotify.

When compared to the industrial "Top 40" machine, the "fairer" marketplace that is Spotify appears to be a win for most artists as they have a more open playing field. However, the payout (as mentioned in those prior articles) is a pittance compared to the sale of a physical disc or MP3 download.

In direct context with the algorithm - as long as the algorithm is solely based on user-generated patterns and NOT through sponsored nudges, etc - the algorithm looks to be working.

So, in sum: likely positive, but as Spotify is the 1000-pound gorilla and gatekeeper, it demands constant critique and attention. If it fails, a huge avenue for musicians fails.

The Problem With Algorithms: Transparency

Overall, my research says to me there is a positive outcome from the Spotify algorithm. However, the algorithm is opaque and proprietary, like most algorithms - including Google Search. And considering its massive impact on consumers, it demands criticism and investigation, on two fronts: 1) is Spotify allowing external influence and 2) how is the personal data being used?

Regarding external influence - it's hard to determine this except by outside observation. But, the driving force behind the recommendations appears to be data alone. Obviously we can't know that for certain - but A) Spotify has been fairly transparent in the methods it uses and B) music is so intensely personal that a song out of place in a Discover Weekly feels like your shirt is on backward. Listeners will smell (hear?) a rat.

In the Google world, there is much research, reporting, sleuthing, and experimentation done against that algorithm. From my research, this has happened, albeit on a very limited scale, with Spotify as well. And that's completely necessary for businesses (artists) who use the platform, but also for consumers to have confidence that they are not being unduly influenced. Ever hear of payola?

The homogeny of the algorithm and our intensely growing comfort and need for it needs checking on occasion. Find other sources of recommendation, otherwise we will all collapse into a Spotify singularity.

The Result

Overall, through objective research and empirical observation, I think it is fair say that the net result of Spotify and its Discover Weekly algorithm for listeners has been very positive. For musicians (ignoring for a moment the overall music marketplace) it has also been positive, though less conclusively so. But ALL of this comes with one massive asterisk.

Specifically, the Discovery Weekly suggestion algorithm appears to do a good job of recommending music users like and tends to promote music discovery and genre expansion in ways that we've never seen before on a massive scale. Not everyone has that "one friend" who is a tastemaker, after all.

However: The overall market share and audience influence that Spotify has coupled with the lack of transparency is deeply concerning and demands constant critique and investigation.

While not necessarily as important as, say, governance or legislation, music, and the environment needed to create it is a critical part of culture and the human condition. Completing ceding that it into the hands of an opaque, monolithic entity is a recipe for disaster.

And never forget that your data is running it. You are a stakeholder, and you have influence.

Coda: User Expectations Are Being Set

Shortly after Amazon Prime launched, I noticed a lot of pressure on ecommerce vendors (and really, anyone who was shipping anything) to ship quickly and ship cheaply. If it didn't say "Free Shipping", it was dead. It became a major trend because users were factoring it into their purchasing decisions. And now everyone, from Etsy to eBay offer expedited, free shipping.

I've referred to this before as , but you could also call it "The YouTube Effect" because shortly after YouTube hit the mainstream in the early 2010s users came to expect quick, cheap, high-quality, lag-free video streaming. If you or the vendor you chose to stream video didn't live up to that it was a letdown. But few for-pay video streamers could do that - though thanks to tech and bandwidth improvements, they have - because they were forced to.

Now, recommendation engines like Spotify's have changed the game for any sort of recommendations - from music to shopping to the "related posts" at the bottom of a blog post.