Update October 2018: I made a web scraping software to extract data from Apple Music with a speed of ~2 seconds per music album. I can provide Excel / CSV files of your favorite artists / bands in a matter of minutes, up to 10 artists free of charge. Watch the video!
If you are professional looking to pay for a large database are invited to discuss project requirements.
Original music database with songs rating
Shortly after connecting to internet in 2005 (I was 16 years old), I started creating a database in Excel with the MP3 songs downloaded, to review each song and give a rating from 0 to 16, and make top best artists, best albums, best genres, etc, using complex mathematical formulas, to show to friends exactly what music I like and how much.
This music database was NOT intended to contain every possible song released, but ONLY my favorite artists / bands, but also few artists / bands that I do not like, selected to be representative for each region of the world and each music genre.
Table format was defined by early iTunes version which was displaying songs in table format and I could copy-paste. Since iTunes was redesigned, I was no longer able to copy whole albums so had to copy songs name by one, manually, still took less than 1 minute per album… more time takes to listen songs and rate them. Only in 2018 I came with idea to make scripts to songs data automatically.
I added ONLY artists from which I downloaded MP3 songs, listened and rated them. Started in 2005, as 2016 I rated over 6000 songs, complete discographies of about 100 artists plus few random songs.
Database is useful for me to organize my music. How useful is for other people I don’t know… you can tell me!
Download music database:
Some local friends, the ones who know me in real life or the ones I sent my music to them via internet, blamed me for listening to the shittest music possible. What is the problem if I rarely listen music from my own country? if I can speak 4 languages and I listen music in languages that they do not understand, as well as other languages that I do not understand myself too, does not mean that the music is a fucking piece of shit. I love collecting music sung in as many languages possible and I do not always care about the lyrics.
Also the friends said that this Excel Music Database is the craziest thing made by me, or most useless thing they ever saw, and suggested me to STOP wasting time doing such things.
I published music database on my website in 2010 with a simple download link. While my other childhood hobbies such as car database and real estate databases gained interest along professionals paying big $$ for a database and allowed me to make a living, the music database turned to be one of most USELESS things that I ever made!
As 2016 I released a new edition with 6000 songs, together with writing this long article. I made a “free purchase” button that require visitors to enter an email in order to download files. Over 200 people downloaded it and I emailed a couple of them asking if my database helped their needs and how they use it. Only 2-3 replied saying that were needing an Excel spreadsheet for a school project. They were not even interested in music!
Starting from 2018 I put $1 but if you contact me and explain how do you use the database I can give you for FREE.
How my hobby for music started
Hobby for music started in 1999, after replacing our 486 computer with 200 MB hard disk with an AMD K6-2 333 Mhz with 4.3 GB hard disk, my dad brought some CDs with mp3 songs, mostly pop and rock songs from 1960s to 1990s, and made a selection of songs according his own preferences, unfortunately deleting many songs that I liked. The he burned the selected songs on 2 CDs. I had no rights to decide what to listen, only parents were putting music in our home, and for many years, my dad’s songs selection was the only music I was listening. I wonder, if I did not had these restrictive parents, were my music preferences different today?
Out of my dad selection, I also made own selection of few dozens songs which I was listening only when I was home alone.
After 2003 revolution and removal of restrictions imposed by family, I was able to listen my music anytime I wanted, and soon I got bored by my few dozens songs, I was desperate to get more music, and started recording via TV-Tuner (ending in having lots of songs in bad quality). I wanted more songs from certain artists, I went to music stores in the city but my parents did not agreed to pay money for music CDs.
In 2005 we connected to internet so I was able to download music freely for first time using DC++ file sharing network (Youtube was not yet launched). In November 2004 I accidentally turned on TV when was Junior Eurovision Song Contest, won by Maria Isabel. So, first songs that I downloaded after connecting to internet were Maria Isabel’s 2 albums and 2 more children artists discovered while looking for Maria Isabel: 3+2 and Danna Paola, additional songs from the artists I already had songs (Aqua, Shakira, Shakin Stevens, Thalia), also originals of about 100 songs recorded from TV in bad quality.
In 2006 while looking on Youtube for Danna Paola I found accidentally a video featuring Tatiana, so I started looking for Tatiana music too, on ARES (file sharing software popular in Latin America) and direct downloads (like MegaUpload) and by this way I downloaded also Fandango, Flans, Timbiriche, R.B.D, and got addicted to Mexican 1980s-1990s pop-rock. In 2008 I had 3000+ songs of which 30% being from Mexico. Tatiana remaining my all-time favorite even in 2013. In the same time I started watching Mexican TV shows and I learned Spanish.
I was looking for more diversity, so since 2009 I also downloaded music from other Latin American countries, and got addicted to Brazil country music as well as 3 big artists hosting children shows (Angelica, Eliana, Xuxa), which generated bad comments from my overseas friends (are you retarded? why do you listen to children music?), by this way in just one year I learned Portuguese to the level I am able to write lyrics of any song. I also downloaded American country music (Alan Jackson, Garth Brooks, Shania Twain, Taylor Swift, etc), British, French, German, Italian pop, rock and folk (ABBA, Al Bano & Romina Power, Alizee, Andrea Berg, Ricchi e Poveri, etc), Japanese and Chinese pop music (many small artists), I liked all them but none caused long-term addiction until 2013 discovery of Kyary Pamyu Pamyu. There is also music that I can’t tolerate: Arabic and Indian music.
The idea of creating an Excel music database
The idea of using Excel to make table with songs, and rate each song, dates back from 2002. I added all songs in WinAmp, generated HTML playlist and copied into Excel, then added a numerical rating. This means no complete discography of any artist, no year known, etc.
In 2005, thanks to the internet connection, access to internet music stores and filesharing networks, I could get information about artists and complete discographies, with album names and release date, I decided that is the time to start making a serious music database.
Source of data: primarily iTunes Music Store (copying album by album with just few clicks, so columns in my database were matching columns in iTunes until it was redesigned in 2010 thus I had to copy song by song), secondary Amazon Music Store. The the italic song titles are taken from MP3 filenames found on the internet so their spelling accuracy is lower.
I did not intended to make a database with ALL songs from my computer, also I did not intended to reach a certain number of songs at a specified deadline. I just added artist by artists at random basis, originally adding only my favorite artists (most of them having short music career), and since 2008 I paid attention to famous artists, adding in database a selection of artists representative for every region of world and every genre of music.
Music database evolution:
2000 songs, 37 artists in top – June 2007
3000 songs, 57 artists in top – February 2008
4000 songs, 71 artists in top – August 2009
5000 songs, 92 artists in top – April 2012
5437 songs, 97 artists in top – late 2013
6000 songs, 105 artists in top – June 2016
Note: the top include only artists with minimum 10 songs and at least half of their songs rated.
Songs rating system
Since the Music Database was started in the era I was fascinated by Base-16 numbering system, the songs are rated with numbers ranging from 0 to 16, originally lower values being better, but in 2016 I inverted the ratings, making higher values better. Total: 17 possible values, which is my birthday and my favorite number.
Rating is composed from 4 categories, each having value from 0 to 4.
Sound: I love instrumental diversity and guitars. Rock and country songs win rating 4 in this category, while hip-hop songs have rating 0.
Voice: I love nice voice and lyrics diversity, but I don’t care about the lyrics content. The songs sung in languages unknown by me or artificial languages can win rating 4 too. The rating drops if lyrics contains too many repeating words, or if the song is only instrumental, the rating is 0.
Mix: I love the songs which have a continuous and fast rhythm. Dance songs can win rating 4 in this category. Rock songs have rating 2-3, Pop songs have rating 1-3, slow songs or bad mixed songs gets rating 0.
Addiction: some songs attract me so much that I listen them again and again for hours, they win rating 4 in this category, they are bubblegum dance, Japanese pop as well as songs from children show of Latin America (this is what attract negative comments from my friends, that I listen childish music, music for retarded people, etc). Rock and country despite of winning in other categories, makes me bored after listening few times so they have rating 1-2, while the louder songs like hard rock which make pain for my ears that I cannot listen a song until its end have rating 0.
To rate each song, is enough to listen 30-second preview on iTunes, but I prefer to rate only when I download full songs. Addiction rating is hard to be decided initially and sometimes I modify it after days or months. The 17 ratings are distributed like Gaussian curve, but asymmetric, rating 8 having 10% of songs, rating 0 having 2% and rating 16 having 0.2%.
My everyday playlist is composed by songs rated from 12 to 16, including songs with rating 8-11 temporarily and keep them if rating is 3 or higher. This create a playlist of about 20% of songs included in database.
Artist ranking system
In 2005 I made a ranking based on average ratings of all songs of each artist. But this turned into a problem: the top places were occupied by small artists that produced just few but good songs, while the most famous artists occupied last places. Is natural that the artists with long career to not be able to make many songs good as the few good songs.
In 2006 I added a SCORE for each artist calculated by a more complex formula. I added columns for number of songs and the total value of songs. Song value is calculated like inverted binary logarithms: value 16 divided by every rating, exception for rating 1 which have value 12 and rating 0 which have value 16.
In 2008 I further improved the ranking by adding a multiply factor for song diversity, calculated like this: total value of songs divided by number of songs divided by average song rating, sum resulted square rooted and and multiplied by 2, resulting a multiply factor between 1 and 1.5. Artists having diversity, few good songs in a total of mostly bad songs, are helped by having higher multiply factor than the artists who have all songs at same medium rating.
How the score is calculated: average song rating (ranging 0 to 16) multiplied with 4 (I can increase this multiply factor to boost artists with one but good album or reduce the multiply factor to boost artists with long career), plus square root of total value of songs (ranging 4 for one-album rappers to 30 for Tatiana’s 20+ albums), sum of these 2 is multiplied with diversity factor between 1 and 1.5, them multiplied with 128 to get a nice-looking 4-digit score for all artists varying from 2500 to 9000+. This numerical value have no other meaning than classifying artist in top. Do not consider that an artist with score 8000 points is two times better than an artist with score 4000.