Web Analytics Made Easy -
StatCounter

Teoalida’s Music Database

Update October 2018: I made a web scraping software to extract data from Apple Music with a speed of ~2 seconds per music album. I can provide Excel / CSV files of your favorite artists / bands in a matter of minutes, up to 10 artists FREE of charge. Watch the video!

If you are professional looking to pay for a larger database (dozens, hundreds, thousands artists), you are invited to discuss project requirements.

Original music database with songs rating

Shortly after connecting to internet in 2005 (I was 16 years old), I started creating a database in Excel with the MP3 songs downloaded, to review each song and give a rating from 0 to 16, and make top best artists, best albums, best genres, etc, using complex mathematical formulas, to show to friends exactly what music I like and how much.

This music database was NOT intended to contain every possible song released, but ONLY my favorite artists / bands plus small a selection of artists / bands representative for each region of the world and each music genre.

Database is useful for me to organize my own music. How useful is for other people, I don’t know… your feedback is needed to make a better database for YOU and future customers!

Buy FULL music database

Buy & download

Free SAMPLE

Music Database

Some local friends, the ones who know me in real life or the ones I sent my music to them via internet, blamed me for listening to the shittest music possible. What is the problem if I rarely listen music from my own country? if I can speak 4 languages and I listen music in languages that they do not understand, as well as other languages that I do not understand myself too, does not mean that the music is a fucking piece of shit. I love collecting music sung in as many languages possible and I do not always care about the lyrics.

Also the friends said that this Excel Music Database is the craziest thing made by me, or most useless thing they ever saw, and suggested me to STOP wasting time doing such things.

Music database evolution & releases:
Jun 2007 edition – 2000 rated songs, 37 artists in top
Feb 2008 edition – 3000 rated songs, 57 artists in top
Aug 2009 edition – 4000 rated songs, 71 artists in top
Apr 2012 edition – 5000 rated songs, 92 artists in top
Late 2013 edition – 7523 total songs, 5467 rated songs, 97 artists in top
Jun 2016 edition – 8365 total songs, 6043 rated songs, 105 artists in top
Feb 2019 edition – 32078 total songs, 6839 rated songs, 110 artists in top

Note: until 2016 edition with 6000 rated songs, the database contained only artists from which I have downloaded mp3 files, listened them and rated at least a couple of their songs. The 2019 release included also the best-selling artists regardless I downloaded mp3 and listened them to add ratings. The top include only artists with minimum 10 songs and at least half of their songs rated.

How my hobby for music started

Hobby for music started in 1999, after replacing our 486 computer with 200 MB hard disk with an AMD K6-2 333 Mhz with 4.3 GB hard disk, my dad brought from friends some CDs with mp3 songs, mostly pop and rock songs from 1960s to 1990s, and made a selection of songs according his own preferences, unfortunately deleting many songs that I liked. The he burned the selected songs on 2 CDs. I had no rights to decide what to listen, only parents were putting music in our home, and for many years, my dad’s songs selection was the only music I was listening. I wonder, if I did not had these restrictive parents, were my music preferences different today?

Out of my dad selection, I also made own selection of few dozens songs which I was listening only when I was home alone.

After 2003 revolution and removal of restrictions imposed by family, I was able to listen my music anytime I wanted, and soon I got bored by my few dozens songs, I was desperate to get more music, and started recording via TV-Tuner (ending in having lots of songs in bad quality). I wanted more songs from certain artists, I went to music stores in the city but my parents did not agreed to pay money for music CDs.

In 2005 we connected to internet so I was able to download music freely for first time using DC++ file sharing network (Youtube was not yet launched). In November 2004 I accidentally turned on TV when was Junior Eurovision Song Contest, won by Maria Isabel. So, first songs that I downloaded after connecting to internet were Maria Isabel’s 2 albums and 2 more children artists discovered while looking for Maria Isabel: 3+2 and Danna Paola, additional songs from the artists I already had songs (Aqua, Shakira, Shakin Stevens, Thalia), also originals of about 100 songs recorded from TV in bad quality.

In 2006 while looking on Youtube for Danna Paola I found accidentally a video featuring Tatiana, so I started looking for Tatiana music too, on ARES (file sharing software popular in Latin America) and direct downloads (like MegaUpload) and by this way I downloaded also Fandango, Flans, Timbiriche, R.B.D, and got addicted to Mexican 1980s-1990s pop-rock. In 2008 I had 3000+ songs of which 30% being from Mexico. Tatiana remaining my all-time favorite even in 2013. In the same time I started watching Mexican TV shows and I learned Spanish.

I was looking for more diversity, so since 2009 I also downloaded music from other Latin American countries, and got addicted to Brazil country music as well as 3 big artists hosting children shows (Angelica, Eliana, Xuxa), which generated bad comments from my overseas friends (are you retarded? why do you listen to children music?), by this way in just one year I learned Portuguese to the level I am able to understand lyrics of any song. I also downloaded American country music (Alan Jackson, Garth Brooks, Shania Twain, Taylor Swift, etc), British, French, German, Italian pop, rock and folk (ABBA, Al Bano & Romina Power, Alizee, Andrea Berg, Ricchi e Poveri, etc), Japanese and Chinese pop music (many small artists), I liked all them but none caused long-term addiction until 2013 discovery of Kyary Pamyu Pamyu. There is also music that I can’t tolerate: Arabic and Indian music, and most of hip-hop music.

The idea of creating an Excel music database

The idea of using Excel to make table with songs, and rate each song, dates back from 2002. I added all songs in WinAmp, clicked “generate HTML playlist” and copied into Excel, then added a numerical rating. This means no complete discography of any artist, no year of release, etc.

In 2005, thanks to the internet connection, access to internet music stores and filesharing networks, I could get information about artists and complete discographies, with album names and release date, I decided that is the time to start making a serious music database.

I did not intended to include ALL songs from my computer, or to reach certain number of songs in database within specified deadline. I just added artist by artist at random basis, originally adding only my favorite artists (most of them having short music career), and since 2008 I paid attention to famous artists, adding in database a selection of artists representative for every region of world and every genre of music.

When I started database in 2005, iTunes was the biggest music store and I could copy-paste whole album’s table of song with just few clicks, so columns in my database matched columns in iTunes. iTunes app was redesigned in 2010 so I had to copy songs name one by one. Database also contains albums sourced from other websites if they are not available on iTunes, as well as names of mp3 files found on the internet (possible incorrect spelling).

In 4 years, the music database reached over 4000 rated songs, after which I continued to add new songs at slower rate.

I published music database on my website in 2010 with a simple download link. While my other childhood hobbies such as car database and real estate databases gained interest along professionals paying big $$ for a database and allowed me to make a living, the music database turned to be one of most USELESS things that I ever made!

As 2016 I released a new edition with 6000 songs, together with writing this long article. I made a “free purchase” button that require visitors to enter an email in order to download files. Over 200 people downloaded it and I emailed a couple of them asking if my database helped their needs and how they use it. Only 2-3 replied saying that were needing an Excel spreadsheet for a school project. They were not even interested in music!

Starting from March 2018 I put for sale at $1, inviting people to contact me IF you want to download it for free. By end of year 10 people paid $1 without any prior communication with me, meaning that people are willing to pay money for a music database, but what should contain the database to be usable for you?

A new era started in 2018: I discovered Spotify, a music streaming service where you can listen full songs free of charge. I also used for the first time a web scraping software to get data faster from Apple Music, at rate of 2 seconds per page (album), Amazon Music Store may be bigger than Apple, I found many artists on Amazon that do not exist on Apple, but due to inconsistencies between various albums on Amazon I need to spend extra time to clean up scraped data, so I prefer to source data from Apple Music unless it miss my favorite artists.

I expanded database with few extra columns such as Source of data or Record labels, and using scraping software I quickly added thousands new songs in database (also able to create custom music database of any artist / band at your choice) then using Spotify I listened full songs and rated them without having to dig for mp3 files on torrents and other pirated music download websites.

Songs rating system

Since the Music Database was started in the era I was fascinated by Base-16 numbering system, the songs are rated with numbers ranging from 0 to 16, originally lower values being better, but in 2016 I inverted the ratings, making higher values better. Total: 17 possible values, which is my birthday and my favorite number.

Rating is composed from 4 categories, each having value from 0 to 4.

Sound: I love instrumental diversity and guitars. Some rock and country songs can win rating 4 in this category, pop songs are around 1-3, while hip-hop songs have rating 0.

Voice: I love nice voice and lyrics diversity, but I don’t care about the lyrics content. The songs sung in languages unknown by me or artificial languages can win rating 4 too. The rating drops if lyrics contains too many repeating words, or if the song is only instrumental, the rating is 0.

Mix: I love the songs which have a continuous and fast rhythm. Some dance songs can win rating 4 in this category. most rock songs have rating 2-3, most pop songs have rating 1-3, slow songs or bad mixed songs gets rating 0.

Addiction: some songs attract me so much that I listen them again and again for hours, they win rating 4 in this category, they are bubblegum dance, Japanese pop as well as songs from children show of Latin America (this is what attract negative comments from my friends, that I listen childish music, music for retarded people, etc). Rock and country despite of winning in other categories, makes me bored after listening few times so they have rating 1-2, while the louder songs like hard rock which make pain for my ears that I cannot listen a song until its end have rating 0.

To rate each song, is enough to listen 30-second preview on iTunes, but I prefer to rate only when I download full songs. Addiction rating is hard to be decided initially and sometimes I modify it after days or months. The 17 ratings are distributed like Gaussian curve, but asymmetric, rating 8 having 10% of songs, rating 0 having 2% and rating 16 having 0.2%.

My everyday playlist is composed by songs rated from 12 to 16, including songs with rating 8-11 temporarily and keep them if addiction rating is 3 or higher. This create a playlist of about 20% of songs included in database.

Artist ranking system

In 2005 I made a ranking based on average ratings of all songs of each artist. But this turned into a problem: the top places were occupied by small artists that produced just few but good songs, while the most famous artists occupied last places. Is natural that the artists with long career to not be able to make many songs good as the few good songs.

In 2006 I added a SCORE for each artist calculated by a more complex formula. I added columns for number of songs and the total value of songs. Song value is calculated like inverted binary logarithms: value 16 divided by every rating, a song rated 0 have value 1, a song rated 8 have value 2, a song rated 12 have value 4, a song rated 14 have value 8, exception for rating 1 which have value 12 and rating 0 which have value 16.

In 2008 I further improved the ranking by adding a multiply factor for song diversity, calculated like this: total value of songs divided by number of songs divided by average song rating, sum resulted square rooted and and multiplied by 2, resulting a multiply factor between 1 and 1.5. Artists having diversity, few good songs in a total of mostly bad songs, are helped by having higher multiply factor than the artists who have all songs at same medium rating.

How the score is calculated: average song rating (ranging 0 to 16) multiplied with 4 (I can increase this multiply factor to boost artists with one but good album or reduce the multiply factor to boost artists with long career), plus square root of total value of songs (ranging 4 for one-album rappers to 30 for Tatiana’s 20+ albums), sum of these 2 is multiplied with diversity factor between 1 and 1.5, them multiplied with 128 to get a nice-looking 4-digit score for all artists varying from 2500 to 9000+. This numerical value have no other meaning than classifying artist in top. Do not consider that an artist with score 8000 is two times better than an artist with score 4000.

10 comments

5 pings

Skip to comment form

    • David Goldman on March 10, 2019 at 6:46 am
    • Reply

    Hi. Congrats on your hard work!
    I am doing a PhD in musicology where part of it involves compiling and rating songs I have collected in Spotify playlists.
    I am also a Spotify artist (David Goldmann)
    Your music scraping software knowledge is interesting to me.
    At first, I need an Excel layout that includes artist name, album, genre, duration, BPM, texture, timbre, tempo, vocal (y/n), rhythm track (y/n) etc, plus URL hypertext fields and jpeg support if poss.
    Do you think you may be able to assist me/give advice on where I might find suitable templates/software?
    Many thanks, David

    1. I can do web scraping from Apple Music and other similar music websites, they do have artist, album, song, duration, genre, URL of jpeg album art, but not the rest. You can buy the ready-made database with my personal selections of artists or I can do another database with artists at your choice. I also checked your music… slow and instrumental, not my favorite.

  1. What is wrong with this guy?

    Chat started on Wednesday, March 27, 2019 12:29:18 AM

    (12:29:18 AM) *** Visitor 2985373 has joined the chat ***
    (12:29:20 AM) Teoalida (site owner): Hi there, visitor from United Kingdom, are you looking for a specific database? Let me know if I can help you!
    (12:29:47 AM) Visitor 2985373: a csv file with a list of songs and their artists
    (12:29:58 AM) *** Teoalida has joined the chat ***
    (12:30:03 AM) Teoalida: of which artists?
    (12:30:12 AM) Visitor 2985373: the song artists
    (12:30:31 AM) Teoalida: I do scraping artist by artist
    (12:30:33 AM) Visitor 2985373: both the title of the track and the artist
    (12:31:50 AM) Teoalida: have you seen the video how it works?
    (12:32:36 AM) Visitor 2985373: um, no
    (12:32:42 AM) Visitor 2985373: but thanks for your time
    (12:32:52 AM) Visitor 2985373: I found out what I was searching for
    (12:32:55 AM) Teoalida: watch it and tell me which artists do you want CSV for list of songs
    (12:33:02 AM) Teoalida: or if you need something different let me know
    (12:33:12 AM) Visitor 2985373: I didn’t need just artist
    (12:33:21 AM) Visitor 2985373: I found out what I was looking for
    (12:33:28 AM) Teoalida: what did you found? give me link?
    (12:33:29 AM) Visitor 2985373: thank you for looking into
    (12:33:46 AM) Visitor 2985373: I am not too sure if I should share the link
    (12:34:10 AM) Visitor 2985373: Thanks for your help
    (12:34:15 AM) Teoalida: would help me to clear out confusion and offer better databases for future customers
    (12:34:26 AM) Visitor 2985373: I am ending the conversation
    (12:34:32 AM) Visitor 2985373: it’s basic google
    (12:34:43 AM) Visitor 2985373: to search for songs csv files
    (12:34:48 AM) Visitor 2985373 has rated the chat Bad
    (12:35:25 AM) Teoalida: if you find ready-made CSV files for download, probably you don’t need my service anymore
    (12:35:26 AM) *** Visitor 2985373 has left ***

Leave a Reply

Your email address will not be published.