Posts Tagged code

The Fresh 40

Every week, thousands of artists release albums on Spotify.  Sifting through all this new music to find good stuff to listen to can be hard. Luckily, there are lots of tools from New Music Tuesday playlists to the Spotify Viral 50 to help us find the needles in the proverbial haystack of new music. However, most of these tools tend to surface up new music by artists that have been around for a while. For instance, the top artist on Spotify Viral 50 as I write this is Jeremih who has been on the charts for five years. The top of New Music Tuesday right now is Mumford & Sons who’ve been recording for at least eight years.

I’m interested in finding music by the freshest artists – artists that are at the very beginning of their recording careers. To that end, I’ve built a new chart called ‘The Fresh 40’ that shows the top albums by the freshest artists. To build The Fresh 40 I scour through all of the albums that have been released in the last two weeks on Spotify (on average that’s about 30 thousand albums), and find the albums that are the very first album release for its artist. I then rank each album by a weighted combination of the number of followers the artist has on Spotify and the popularity of the artist and album (which is related to Spotify track plays). The result is a chart of the top 40 most popular fresh artists.

2015-04-27 at 7.50 AM

The Fresh 40 updates every day and shows all the salient info including the rank, yesterday’s rank, the overall score, artist followers, artist popularity, album popularity and the number of days that the album has been on the chart. Since an album can only be on the chart for 15 days, there’s quite a bit of change from day to day.

If you are interested in finding music by the very newest artists on Spotify, you might be interested in The Fresh 40. Give the chart a look.

The Fresh 40 was built on top of the increasingly marvelous Spotify Web API.  Code is on github.

, , , , , , ,

5 Comments

Fresh Faces on Spotify

My weekend programming project this week was to explore a new feature of the Spotify Web API that allows you to find albums that have been released in the last two weeks. The result is a web app called Fresh Faces.  This app goes through all of the recent releases and finds those that are the very first release for the artist. If you are looking for new music, there’s no fresher place to start than this app – it finds the newest music by the freshest artists – artists that are barely two weeks into their recording career.

2015-04-26 at 11.40 AM Fresh Faces lets you sort the results based on artist popularity, album popularity, artist followers or release date. You can click on an album to hear a sample, find more info about the album or open it on Spotify.

How many new releases are there?

I was curious about how many releases there are in a two week period, and when releases tend to happen, so I added a chart at the bottom of the Fresh Faces app that shows the distribution of fresh and recurring releases and the dates when releases happen.  You can see that the shift of releasing music from Tuesday to Friday is ongoing.

2015-04-26 at 11.45 AM

In the past two weeks about 32,000 albums have been released – about 5,200 of these are the first release for the artist. That’s a whole lot of fresh music.

Give Fresh Faces a try and let me know what you think.

, , , , , ,

3 Comments

The Unfollower

One of the problems with working at a company like Spotify is that my Spotify account gets filled up with all sorts of work-related playlists. Over the last few years I’ve built lots of apps that create playlists. When I test these apps I end up generating lots of playlists that I will never ever listen to. If I were a tidy soul, I’d clean up my playlists after ever project, but, alas, that is something I never do. The result is that after working at Spotify for a year (and using Spotify for 8 years), I’ve accumulated many hundreds of garbage playlists. Now I could go into the Spotify desktop client and clean these up, but in the current client there’s no good way to bulk delete playlists.  Each playlist delete takes at least 3 clicks.  The prospect of doing this hundreds of times to clean up the playlist garbage is  a bit overwhelming.

I had  a few hours to kill in a coffeeshop yesterday so I decided to deal with my playlist mess.  I wrote a little Spotify web app called The Unfollower that lets you unfollow any of your playlists with a single click.  If you change your mind, you can re-follow any playlist that you unfollow.

The Unfollower screenshot

The Unfollower uses the Spotify Web API to make it all happen. In particular it relies on the  Follow/Unfollow API that was recently added by the API team.

If you are like me and have lots of dead playlists clogging up your Spotify, and you are looking for a streamlined way of cleaning them up, give The Unfollower a try.

, , , ,

3 Comments

50 Years Ago in Music

There’s a strong connection between music and memory. Whenever I here the song Lovin You by Minnie Riperton, I’m instantly transported back to 1975 when I spent the summer apprenticed to Tom, my future brother-in-law, fixing electronic organs. I was 15, Tom was 22 and super cool. He had a business (New Hampshire Organ Service) and he had a van with an 8-track player and an FM radio (a rarity in 1975). As we drove between repairs across rural New Hampshire we’d pass the time by listening to the radio.  Now, when I hear those radio songs from 1975 it is like I’m sitting in that van again.

Music can be like a time machine.  Transporting us to different times in our lives. I was interested in exploring this a bit more. Inspired by @realtimewwii which gives a day-by-day account of World War II, I created a set of dynamically updating Spotify playlists that follow the charts week-by-week.

For example there’s the 50 Years Ago in Music playlist that contains the top 100 or songs that were on the chart 50 years ago.  As I write this on April 12, 2015, this playlist is showing the top songs for the week of April 12, 1965.

The music on this playlist sends me back to when I was 5 years old listening to music on our AM radio in the kitchen in the morning while eating breakfast.

If you follow this playlist you’ll be able to re-create what it was like to listen to music 50 years ago.  If the mid-sixties doesn’t speak to you musically, there are some other playlists that you can try.

There’s 40 Years Ago in Music that brings me back to 1975 on the road with Tom.

There’s 30 years Ago in Music which is currently playing music from the mid-80s like Madonna and Phil Collins.

There’s 20 Years Ago in Music currently playing music from the mid-90s:

10 Years Ago in Music plays the music that was on the radio when Spotify was just a gleam in Daniel’s eye.

5 Years Ago in Music – the playlist of @echonest in its heyday.

All of the playlists update weekly on Monday. If you’d like a reminder about when they are updated you can follow @50yearmusic. And of course, the code is on github.

, , , ,

1 Comment

Tracking play coverage in the Infinite Jukebox

Yesterday, I upgraded the Infinite Jukebox to make it less likely that it would get stuck in a section of the song. As part of this work, I needed an easy way to see the play coverage in the song. To do so, I updated the Infinite Jukebox visualization so that it directly shows play coverage. With this update, the height of any beat in the visualization is proportional to how often that beat has been played relative to the other beats in the song. Beats that have been played more have taller bars in the visualization.

This makes it easy to see if we’ve improved play coverage. For example, here’s the visualization of Radiohead’s Karma Police with the old play algorithm after about an hour of play:
Infinite_Jukebox_for_Karma_Police_by_Radiohead

As you can see, there’s quite a bit of bunching up of plays in the third quarter of the song (from about 7 o’clock to 10 o’clock). Now compare that to the visualization of the new algorithm:

Infinite_Jukebox_for_Karma_Police_by_Radiohead

With the new algorithm, there’s much less bunching of play. Play is much more evenly distributed across the whole song.

Here’s another example.  The song First of the Year (Equinox) by Skrillex played for about seven hours with the old algorithm:

Infinite_Jukebox_for_Equinox_by_Skrillex

As you can see, it has quite uneven coverage. Note the intro and outro of the song are almost always the least played of any song, since those parts of the song typically have very little similarity with the rest of the song.

Here’s the same song with the new algorithm:

Infinite_Jukebox_for_Equinox_by_Skrillex

Again, play coverage is much more even across all of the song outside of the intro and the outro.

I like this play coverage visualization so much that I’ve now made it part of the standard Infinite Jukebox. Now as you play a song in the Jukebox, you’ll get to see the song coverage map as well. Give it a try and let me know what you think.

, , ,

3 Comments

More on “Where’s the Drama?”

My Music Hack Day Berlin hack was “Where’s the Drama?” – a web app that automatically identifies the most dramatic moment in any song and plays it for you. I’ve been having lots of fun playing with it … and even though (or perhaps because) I know how it works, I’m often surprised at how well it does at finding the most dramatic moments.  Here are some examples:

How does it work? The app grabs the detailed audio analysis for the song from The Echo Nest.  This includes a detailed loudness map of the song. This is the data I use to find the drama.  To do so, I look for the part of the song with the largest rise in volume over the course of a 30 second window (longer songs can have a bit of a longer dramatic window). I give extra weight to crescendos that culminate in louder peaks (so if there are two crescendos that are 20dB in range but one ends at 5dB louder, it will win). Once I identify the most dynamic part of a song, I pad it a bit (so we get to hear a bit of the drop after the build).

The rest is just UI – the song gets plotted as a heavily filtered loudness curve with the dramatic passage highlighted. I plot things with Highcharts, which is a pretty nifty javascript plotting and charting library. I recommend.

 Where_is_the_Drama_

Playing the music – I wanted to use Spotify to play the music, which was a bit problematic since there currently isn’t a way to play full streams with the Spotify Web API, so I did a couple of hacky hacks that got me pretty far. First of all, I discovered that you can add a time offset to a Spotify URI like so:

        

When this URI is opened in Spotify (even when opened via a browser), Spotify will start to play the song a the 1:05 time offset.  

I still needed to be able to stop playing the track – and there’s no way to do that directly – so instead, I just open the URI:

      

which happens to be the URI for John Cage’s 4’33.  In other words, to stop playing one track, I just start playing another (that happens to be silent).  The awesome side effect of this is that I’ll be slowly turning anyone who uses “Where’s the Drama?” into experimental music listeners as the Spotify recommendation system responds to all of those John Cage ‘plays’. This should win some sort of ‘hackiest hack of the year’ award.

It was a fun hack to make, and great fun to demo. And now that I have the app, I am no longer wasting time listening to song intros and outros, I can just get to the bit of the song that matters the most. 

, , , , , ,

3 Comments

Controlling the artist distribution in playlists

The Echo Nest engineering team just pushed out a new feature giving you more control over the artist makeup in playlists.  There is a new parameter to the playlist/static API called distribution that can be set to wandering  or focused.   When the distribution is set to wandering the artists will appear with approximately equal distribution in the playlist. If the distribution is set to focused artists that are more similar to the seed artists will appear more frequently.  When combined with the variety parameter, you have excellent control over the number and distribution of artists in a playlist.  If you want to create a playlist suitable for music discovery, create a playlist with high variety and a wandering distribution.  If you want to create a playlist that more closely mimics the radio experience choose a low variety and a focused distribution.

I’ve put together a little demo that lets you create playlists with different levels of variety and distribution settings. The demo will create a playlist given a seed artist and show you the artist distribution for the playlist.  Here’s the output of the demo with distribution set to focused:

You can see from the artist histogram that the playlist draws more from artists that are very similar to the seed artist (Weezer).  Compare to these results from a wandering playlist with the same seed and variety:

You can see that there is flatter distribution of artists in the playlist.   You can use variety and distribution to tailor playlists to the listener.  For instance, you can give the Classic Rock Radio experience to a listener by setting variety to relatively low, setting the distribution to focused and seeding with a classic rock artist like Led Zeppelin.  Here’s the artist distribution for the resulting playlist:

That looks like the artist rotation for my local classic rock radio.

Give the demo a try to see how you can use variety and distribution to match playlists to your listener’s taste.  Then read the playlist API docs to see how to use the API to start incorporating these attributes into your apps.

The Demo:  Playlist Distribution Demo (source)

, , , , ,

Leave a comment

Using The Echo Nest to get the top 100 Twitter artists

This week Twitter and The Echo Nest announced a partnership where  Twitter IDs for verified accounts are incorporated into The Echo Nest’s ID mapping layer (aka Rosetta Stone).  This makes it easy for developers to get the Twitter handle for an artist.   To demonstrate just how easy it is, I wrote a little web app that displays the top 100 artists that have  verified Twitter accounts.    Here’s the core bit of code for the app:


function fetchTopTwitterArtists() {
    var url = 'http://developer.echonest.com/api/v4/artist/top_hottt?callback=?';

    $.getJSON(url, { 'api_key': 'GETYOUROWNAPIKEY', 'format':'jsonp',
      'results': 100, 'bucket': ['hotttnesss', 'id:twitter'], 'limit': true},
     function(data) {
       for (var i = 0; i < data.response.artists.length; i++) {
          var artist = data.response.artists[i];
          var elem = $("<li>");
          var link = $("<a>");
          var handle = artist.foreign_ids[0].foreign_id.replace('twitter:artist:', '');
          link.attr('href', 'http://twitter.com/' + handle);
          link.text(artist.name);
          elem.append(link);
          $("#results").append(elem);
       }
     }
  });
}

The key bits here are creating the artist/top_hottt request to The Echo Nest and  adding the id:twitter bucket and setting limit to true. This tells the Echo Nest to include the twitter handle information in the results and limit the results to only those artists that have twitter information.  After that it is just pulling the data out of the results and formatting it for the lovely display.   The Twitter ID info is returned in an ID block that looks like this:

            {"catalog": "twitter","foreign_id": "twitter:artist:LMFAO"}

Note that the Twitter ID is returned in a URN form. To get the actual Twitter URL for an artist we just need to replace the ‘twitter:artist:’ bits with ‘http://twitter.com/&#8217;.

You can see the app here:  Top 100 artists with verified Twitter accounts.  As you can see, I tried to make the web app as ugly as possible. The only thing it needs is some Comic Sansification.  The code is available for detailed study in this gist.

, ,

1 Comment

Building a Spotify App

On Wednesday November 30, Spotify announced their Spotify Apps platform that will let developers create Spotify-powered music apps that run inside the Spotify App.   I like Spotify and I like writing music apps so I thought I would spend a little time kicking the tires and write about my experience.

First thing, the Spotify Apps are not part of the official Spotify client, so you need to get the Spotify Apps Preview Version.  This version works just like the version of Spotify except that it includes an APPS section in the left-hand navigator.

If you click on the App Finder  you are presented with a dozen or so Spotify Apps including Last.fm,  Rolling Stones, We are Hunted and Pitchfork.  It is worth your time to install a few of these apps and see how they work in Spotify to get a feel for the platform. MoodAgent has a particularly slick looking interface:

A Spotify App is essentially a web app run inside a sandboxed web browser within Spotify.  However, unlike a web app you can’t just create a Spotify App, post it on a website and release it to the world. Spotify is taking a cue from Apple and is creating a walled-garden for their apps. When you create an app, you submit it to Spotify and if they approve of it, they will host it and it will magically appear in the APPs sections on millions of Spotify desktops.

To get started you need to have your Spotify account enabled as a ‘developer’. To do this you have to email your credentials to  platformsupport@spotify.com.    I was lucky, it took just a few hours for my status to be upgraded to developer (currently it is taking one to three days for developers to get approved).  Once you are approved as a developer, the Spotify client automagically gets a new ‘Develop’ menu that gives you access to the Inspector, shows you the level of HTML5 support and lets you easily reload your app:

Under the hood, Spotify Apps is based on Chromium so those that are familiar with Chrome and Safari will feel right at home debugging apps.  Right click on your app and you can bring up the familiar Inspector to get under the hood of your application.

Developing a Spotify App is just like developing a modern HTML5 app.  You have a rich toolkit: CSS, HTML and Javascript. You can use jQuery, you can use the many Javascript  libraries. Your app can connect to 3rd party web services like The Echo Nest.  The  Spotify Apps supports just about everything your Chrome browser supports with some exceptions: no web audio, no video, no geolocation and no Flash (thank god).

Since your Spotify App is not being served over the web, you have to do a bit of packaging to make it available to Spotify during development.   To do this you create a single directory for your app. This directory should have at least the index.html file for your app and a manifest.json file that contains info about your app.  The manifest  has basic info about your app.  Here’s the manifest for my app:

% more manifest.json
{
 "BundleType": "Application",
 "AppIcon": {
     "36x18": "MusicMaze.png"
  },
 "AppName": {
     "en": "MusicMaze"
  },
 "SupportedLanguages": [
     "en"
 ],
 "RequiredPermissions": [
     "http://*.echonest.com"
 ]
}

The most important bit is probably the ‘RequiredPermissions’ field. This contains a list of hosts that my app will communicate with. The Spotify App sandbox will only let your app talk to hosts that you’ve explicitly listed in this field. This is presumably to prevent a rogue Spotify App using the millions of Spotify desktops as a botnet.  There are lots of other optional fields in the manifest. All the details are on the Spotify Apps integration guidelines page.

I thought it would be pretty easy to port my MusicMaze to Spotify.   And it turned out it really was. I just had to toss the HTML, CSS and Javascript into the application directory, create the manifest, and remove lots of code.  Since the Spotify App version runs inside Spotify, my app doesn’t have to worry about displaying album art, showing the currently playing song, album and artist name, managing a play queue, support searching for an artist. Spotify will do all that for me. That let me remove quite a bit of code.

Integrating with Spotify is quite simple (at least for the functionality I needed for the Music Maze). To get the currently playing artist I used this code snippet:

var sp = getSpotifyApi(1);
function getCurrentArtist() {
   var playerTrackInfo = sp.trackPlayer.getNowPlayingTrack();
   if (playerTrackInfo == null) {
      return null;
   } else {
      return track.album.artist;
   }
}

To play a track in spotify given its Spotify URI use the snippet:

function playTrack(uri) {
     sp.trackPlayer.playTrackFromUri(uri, {
        onSuccess: function() { console.log("success");} ,
        onFailure: function () { console.log("failure");},
        onComplete: function () { console.log("complete"); }
    });
}

You can easily add event listeners to the player so you are notified when a track starts and stops playing. Here’s my code snippet that will create a new music maze whenever the user plays a new song.

function addAudioListener() {
    sp.trackPlayer.addEventListener("playerStateChanged", function (event) {
        if (event.data.curtrack) {
            if (getCurrentTrack()  != curTrackID) {
                var curArtist = getCurrentArtist();
                if (curArtist != null) {
                    newTree(curArtist);
                }
            }
        }
    });
}

Spotify cares what your app looks like.  They want apps that run in Spotify to feel like they are part of Spotify. They have a set of UI guidelines that describe how to design your app so it fits in well with the Spotify universe.   They also helpfully supply a number of CSS themes.

Getting the app up and running and playing music in Spotify was really easy. It took 10 minutes from when I received my developer enabled account until I had a simple Echo Nest playlisting app running in Spotify

Getting the full Music Maze up and running took a little longer, mainly because I had to remove so much code to make it work.  The Maze itself works really well in Spotify.   There’s quite a bit of stuff that happens when you click on an artist node.  First, an Echo Nest artist similarity call is made to find the next set of artists for the graph. When those results arrive, the animation for the expanding the graph starts.  At the same time another call to Echo Nest is made to get an artist playlist for the currently selected artist. This returns a list of ‘good’ songs by that artist.  The first one is sent off to Spotify for it to play.  Click on a node again and the next song it that list of good songs plays.  Despite all these web calls, there’s no perceptible delay from the time you click on a node in the graph until you hear the music.

Here’s a video of the Music Maze in action.  Apologies for the crappy audio, it was recorded from my laptop speakers.

[youtube http://youtu.be/finObm36V6Y]

Here’s the down side of creating a Spotify App.  I can’t show it to you right now.  I have to submit it to Spotify and only when and if the approve it, and when they decided to release it will it appear in Spotify.  One of the great things about web development is that you can have an idea on Friday night and spend a weekend writing some code and releasing it to the world by Sunday night.  In the Spotify Apps world, the nimbleness of the web is lost .

Overall, the developer experience for writing Spotify Apps is great.  It is a very familiar programming environment for anyone who’s made a modern web app. The debugging tools are the same ones you use for building web apps. You can use all your favorite libraries and toolkits and analytics packages that you are used to.   I did notice some issues with some of  the developer docs – in the tutorial sample code the ‘manifest.json’ is curiously misspelled ‘maifest.json’.  The JS docs didn’t always seem to match reality.   For instance, as far as I can tell, there’s no docs for the main Spotify object or the trackPlayer.  To find out how to do things, I gave up on the JS docs and instead just dove into some of the other apps to see how they did stuff . (I love a world where you ship the source when you ship your app).  Update – In the comments Matthias Buchetics points us to this Stack Overflow post that points out where to find the Spotify JavaScript source in the Spotify Bundle.  At least we can look at the code until the time when Spotify releases better docs.

Update2 – here’s a gist that shows the simplest Spotify App that calls the Echo Nest API.  It creates a list of tracks that are similar to the currently playing artist.  Another Echo Nest based Spotify App called Mood Knobs is also on github.

From a technical perspective, Spotify has done a good job of making it easy for developers to write apps that can tap into the millions of songs in the Spotiverse.   For music app developers, the content and audience that Spotify brings to the table will be hard to ignore.  Still there are some questions about the Spotify Apps program that we don’t know the answer to:

  • How quickly will they turn around an app?  How long will it take for Spotify to approve a submitted app? Will it be hours, days, weeks?  How will updates be managed?  Typical web development turnaround on a bug fix is measured in seconds or minutes not days or weeks.  If I build a Music Hack Day hack in Spotify, will I be the only one able to use it?
  • How liberal will Spotify be about approving apps? Will they approve a wide range of indie apps or will the Spotify App store be dominated by the big music brands?
  • How will developers make money?  Spotify says that there’s no way for developers to make money building Spotify apps. No Ads, no revenue share.  No 99 cent downloads.  It is hard to imagine why developers  would flock to a platform if there’s no possibility of making money.

I hope to try to answer some of these questions. I have a bit of cleanup to do on my app, but hopefully sometime this weekend, I’ll submit it to Spotify to see how the app approval process works.  I’ll be sure to write more about my experiences as I work through the process.

, , ,

36 Comments

Getting Ready for Boston Music Hack Day

hackday.1.1.1.1

Boston Music Hack Day starts in exactly 10 days.  At the Hack day you’ll have about 24 hours of hacking time to build something really cool.   If you are going to the Hack Day you will want to maximize your hacking time, so here are a few tips to help you get ready.

  • Come with an idea or two but be flexible – one of the really neat bits about the Music Hack Day is working with someone that you’ve never met before. So have a few ideas in your back pocket, but keep your ears open on Saturday morning for people who are doing interesting things, introduce yourself and maybe you’ve made a team.  At previous hack days all the best hacks seem to be team efforts.  If you have an idea that you’d like some help on, or if you are just looking for someone to collaborate with, check out and/or post to the Music Hack Day Ideas Wiki.
  • Prep your APIs – there  are a number of  APIs that you might want to use to create your hack. Before you get to the Hack Day you might want to take a look at the APIs, figure out which ones you might want to use- and get ready to use them.  For instance, if you want to build music exploration and discovery tools or apps that remix music, you might be interested in the Echo Nest APIs.   To get a head start for the hack day before you get there you should register for an API Key,  browse the API documentation then check out our resources page for code examples and to find a client library in your favorite language.
  • Decide if you would like to win a prize – Of course the prime motivation is for hacking is the joy of building something really neat – but there will be some prizes awarded to the best hacks.  Some of the prizes are general prizes – but some are category prizes (‘best iPhone /  iPod hacks’) and some are company-specific prizes (best application that uses the Echo Nest APIs).  If you are shooting for a specific prize make sure you know what the conditions for the prize are.  (I have my eye on the Ultra 24 workstation and display, graciously donated by my Alma Mata).

To get the hack day jucies flowing check out this nifty slide deck on Music Hackday created by Henrik Berggren:

 

    , , , , , , , ,

    Leave a comment