As Gizmo from Skratchworx wonderfully illustrates in this graphic (see link in text to original), if digital music were actually vinyl, people would think twice about amassing such huge collections. This is 20,000 songs - less than half of what our reader is trying to get to grips with.
Reader Samet writes: “I would like to tidy up my three-year-old music collection. I’ve got a lot of compilations, monthly-separated single downloads and set mixes. My problem is there are a lot of duplicates between compilations and single downloads and also some of them are a poor quality because of downloading randomly.
“I want to get rid of them but there are 44,000 files and i dont know how to do it easily. There is a lot of software on the web for finding duplicate files but many of the file tags are written in different formats so they’re not that useful. Editing 40,000+ files is hard work for me. I wish there were an easier way.
“I need some software or technique to find duplicate files to delete, and to find poor quality files in order to download better quality ones.”
Digital DJ Tips says:
Lots of things here. I’ll bullet point for you some thoughts and actions. I’m assuming you’re using iTunes.
- Get rid of most of it – You have far too much music. 44,000 files is a ridiculous amount. You can’t possible know, use or play all of that. Look at this article from Skratchworx for a good argument as to why less than half the music you’re trying to grapple with is still far too much. Cut it down drastically before you do absolutely anything else. I have 600 tunes. Guess what? I don’t have any issues with file management at all
- Stop “downloading randomly” – If you’re stealing other people’s music, apart from it being wrong, you throw away any control on the quality you’re letting into your collection. Get it from reputable sources, and you’ll know it’s OK. An easy way to get rid of obviously poor quality files is to add the “Bit Rate” column in iTunes, sort by it, and block-delete everything below, say, 192 kbps. That’ll get rid of a lot of it, but if you’ve been taking music from disreputable sources, it won’t in any way guarantee to get rid of it all
- Take the time to understand “Artist”, “Album Artist” and “Part of compilation?” in iTunes – Read about these metadata functions in iTunes help; correctly using them will help you to keep compilations properly filed, and separate the DJs who perform your mixes from the artists who made the tracks
- Standardise your metadata – Decide what genres are important to you, how you want to use the “comments” field, where you want to list remixers (I do them in brackets after the title), and apply to your whole collection. There’s no fast way to do this, but you’ve got rid of tens of thousands of tunes, haven’t you? So it’s going to take a lot less time than it would have done. By far the best program for semi-automating much of this including getting rid of duplicates is beaTunes
Sorry if some of this sounded harsh, but 44,000 tunes is going to take anyone far to long to sort out and get to grips with, and as you admit you’re just “randomly downloading” half of this stuff, the root cause of your problem is that particular habit. Stop doing it, buy what you want, and then look after it carefully. Digital music deserves to be properly valued just like records and CDs are (and the artists deserve to be rewarded too). Good luck!
Do you have any advice to add to this? How have you got to grip with unwieldy MP3 collections? Please let us know your thoughts, experiences and epiphanies in the comments…
Now go to:
beaTunes 3.0 Review: No iTunes Required
Get Minimal With Your Digital Music Collection!
TuneUp Review: Can This Software Fix Your iTunes Library?
Look! How To Digital DJ Fast - Video Course
From Digital DJ Tips, four hours of online video to get you from beginner to first gig in just four weeks. Join the 1000s of DJs who've already benefited from this training. Click here!
Learn to DJ Free - email course plus bonus PDF book
Sign up for our weekly email course for beginners now...
Trouble choosing a controller? Get your free 162-page guide!
Grab "DJ Controllers: The Ultimate Buyer's Guide" at our Facebook Page.
Come and meet other DJs just like you!
The long-awaited Digital DJ Tips forums are now open. Come and chat!
Tags: mp3s, music library, tunes
I would say just start over. The only thing more difficult than organizing music, is organizing it after you failed to do so when you originally got it. The fact that you can snatch up 44,000 tracks within three years is impressive and you obviously wont have problems refilling your collection. So you should simply grab a couple hundred tracks that you know are good and tend to always use, organize those appropriately using the advise given here, and get rid of the rest. But when you start looking for and downloading your new music, make sure you organize it when you get it. you will be much happier.
Also, I would like to emphasize what was written above: Don’t simply download random songs. Your collection of music should be very intentional. And choose reputable sources. Anyone can export a track at 320kb/s, but what is more important is where that track came from. For all you know, someone could have stuck a microphone in front of their radio. Or more commonly, unintentionally imported a cd at the default 128kb/s and exported it somewhere else at 320kb/s. The point is, once you lose audio quality, you can’t get it back. And if you don’t know your source, you can’t trust the bit rate.
[ link ]There’s a good case for taking everything out of the collection, then putting stuff back in afterwards, so you’re starting with a blank slate. It’s the same as when you’re clearing out a wardrobe – best to empty it first and then replace what you want to keep, rather than trying to “prune” what’s in there. you end up with less stuff and you think harder about why you’re keeping what you do choose to keep.
[ link ]This exactly what I did with an iTunes library that had built up since the time the software was released. I’ve been adding stuff back in gradually, and purposefully. Most of it won’t go back in. But it’s still there if I think of it. I can go and explore the crap. I’ll delete the remainder eventually.
[ link ]I have more than that myself, but the difference is that mine mostly come from CDs with about 10,000 being legal downloads and digital promos.
My advice would be quite simple though…buy your music. If you are a DJ and expecting to make money then you should be buying at least 90% of your music (or obtaining is legally. There’s a lot of free music from big artists out there if you actually put a little effort in to find it). I understand downloading out of print records and/or a few tracks now and then that you can’t find on any download store, but if you’re downloading randomly and taking others music collections then you deserve to be in a mess.
Organisation is one of the key things to me when it comes to digital DJing. Knowing your tunes and knowing what fits with what and when to drop it is what separates professionals from amateur, and you display no interest in the actual music or becoming a pro. Sorry, but that’s just how it seems to me.
[ link ]I’m in agreement.
Even when I download free/legal stuff from sources like SoundCloud, I’ll go through a quick regimen of tweaking the ID3 tags, file names (Artist-Title_(Remix/Version).mp3), and file them away in selected spots.
Every year I’ll dump my whole hard drive and get stuff back from the archives. The goal really is to weed out stuff I don’t want. Most of this is generally rolling down file names and quickly grabbing what I call “essentials”. The rest stay archived and not on my general use hard drive.
Some music is timeless…most of the rest unfortunately are temporary. It’s not a sin to delete something or file it away and never use it again other than the few times you played it. The “sin” is carrying it all with you thinking you’ll actually use it. I’ll play 10-20 songs in a one-hour set. So why bring 44,000 if you only need perhaps 100-200 at the most?
I also reject low-quality. It’s not a snob thing, but a help in weeding out the crap. Better to reject that 96kbps file and buy a 320kbps than to end up with those duplicates you don’t want.
[ link ]Try a mp3 catalog of 150,000 songs….I will be taking your advice and drastically downsizing.
[ link ]I only have around 150 tunes and I still think I have too much. I find it hard to get rid of tracks I might only use once in a while that I sort of still like. I guess I have to be bold and just delete the damn things.
[ link ]There was nothing better for my djing than listening to my old 10.000 songs collections, and reducing it to 3.000
Still, 44.000 is ridiculous; I would order everything by play count, and delete everything that has 0. If it still holds a big number of songs, I would delete 1, 2, until I get a good number of songs.
[ link ]Nobody is really helping Samet. I have more files than he does but I keep them on my NAS, not my computer. My computer only has a modest dj’ing library & what I’m currently listening to on it.
With a library that large I don’t think you’ll ever get it 100% fixed but I would suggest taking a look at MediaMonkey. It has some pretty useful scripts that will identify duplicates, missing tags, missing artwork, etc. Most scripts are 3rd party but I’ve never had an issue with the ones on this page http://trixmoto.net/mm/search.php?all=scripts
I don’t trust those programs that claim to fix your library automatically. I’d be scared that they’d screw up.
I do agree with the other commenters though. You should tighten up your DJ’ing library. Dump all your files to an external drive & rebuild your main library from scratch. Use MediaMonkey (or whatever you prefer) to clean up the 40k+ files you own on the external but keep your dj library more modest.
[ link ]I second Media Monkey (if you’re a PC user, you can’t get it for Mac) – it is powerful music library management software.
[ link ]A great program to organize music is Media Monkey. I has all the features from Itunes and then some. I really helps find low quality files and duplicates. I also helps to move files around and getting everything into bigger groups and narrowing down the work load that way. I have about 41000 tunes on archive, but only have a much smaller library of active tunes on traktor.
[ link ]I am a mobile DJ with over 100,000 tracks that I have archived to my main computer. My mobile laptop library is dumped after most gigs and uploaded with tracks that I will more than likely play at the next gig. just in case I get an odd request for a jam that I have archived I bring a portable hard drive to dig from if it’s a must play for the party. The key to knowing your music is getting comfortable with tagging all you tracks for quick finding later. there is no way you will remeber all of your library especially if you just uploaded it from another source. I have archived all my vinyl, cassette, and cd’s(too bad I don’t have an 8-track player) to my library. I don’t remember all of what I have but I did tag everything before I uploaded it. Now all I do is search the genre or year and will make list accordingly. The genius function on I-tunes also works great for making a playlist that carries the same sound throughout. An example- I am not fluent in spanish nor know the difference between mariachi,salsa,rock espanol,Etc.. Nor do I speak or understand spanish. I have still pulled off gigs with an interptor with me, by using the genius playlist to generate a list that I will mix during the performance. I’ve invested in lots of compilation CD’s of the genre but I have no clue what is popular or proper for the occasion. The genius list does all that for me and I’ll have my interpetor MC the event. If anything do not take the advice of just starting over, that’s what a lazy DJ would do, just take the time to go through your songs and label them accordingly. At first it may be overwhelming but take it one song at a time. It took me many years to upload my library but now I get away with playing music that I don’t even understand the language to. When I say get away with it I mean get regular calls that I have to have my interpetor call back to see what they want. Being A DJ isn’t always about serving the crowd you want, it’s about being versitle and being able to make crowds move on the fly. Preperation is key. I’m sure some one who just does one genre will find themselves imprisioned by it,wheather they realize it or not but when a DJ broadens thier horizen they begin to make a new sound of thier own off of thier experiences.
[ link ]It’s very true that many mobile DJs find themselves in a different situation to club/creative DJs who are focusing on a sound or scene, in that they need to have a lot of music on hand for requests. Point taken.
[ link ]Just to say I was a little harsh in my reply, I apologize. But you seriously need to start implementing some quality control with your music. And I think D-Jam’s advice is good. Find a file structure that suits you and get into the habit of detagging and retagging files in a particular way. (I personally then set mine to read only to prevent certain software from undoing the work I’ve done. Traktor has a nasty habit of this.
I don’t think a large collection should be too much of an issue IF you keep them organized and create and maintain playlists within your software, that way you’ll always have certain tracks grouped. When you feel like playing something a little unexpected you can just browse through the entire collection during your sets.
[ link ]You may find yourself living in a shotgun shack.
And you may find yourself DJing your mother in laws 50th birthday.
And she asks you to play The Electric Slide, but you don’t d-d-d-don’t have it.
And your mother in law will turn to her daughter and say
My God, this DJ sucks.
My point being, the most important song of the night could be the one you don’t have. Lucky for me lastnight I have a terabyte drive w 64,000+ songs. And, The Electric Slide.
[ link ]Total agreement. As a mobile DJ I have to be ready to jump from Cumbia to top 40 and my arsenal of 48,000 songs allow me to do that.
[ link ]I think that’s fine if that’s what you do. Personally I only play the finest globally sourced tracks and the kids love it. I only have around 100 tracks otherwise it gets really complicated navigating Traktor for the tune you need at that particular time.
[ link ]Had to throw my hat in the ring, because I’m probably the worst offender here…lol!
I have over 200,000 MP3s…now, this may sound like overkill (understatement of the year lol), but there are many reasons for this.
This is a result of ripping records, CDs, as I moved to Scotland, and transporting vinyl across the Atlantic costs a small fortune. Putting them on an external hard drive? Not so much. Then there is exchanging with other music geeks like myself. Yes, I realize I should be paying for a lot of my music, but unfortunately, I’m not as wealthy as I’d like to be, and that is a reality for a lot of music lovers out there. I don’t sell any of my music to anyone. If someone wants some, I happily make a copy or have them plug their music player up to my computer.
I genuinely have a love for the music I collect. It’s not just randomly grabbing stuff for the sake of grabbing. I’ve listened to almost everything I own, since I watch very little TV. If I’m not socializing, I’m listening to music.
As idealistic or crazy as it sounds, I’d like to have music for ALL tastes. I’ve been able to build a rep in Glasgow for being the DJ who can’t be stumped…lol!
It is a bit obsessive, but I figure it’s a great way to channel my addictions. Everyone around me benefits in the process!
[ link ]I am a private collector with just over 45000 tracks. I am currently doing exactly what you are faced with, identifying and organising my iTunes library. I will admit, it’s a mammoth task and I wish I had been more conscientious when I first started buying and storing music digitally. That said I whole heartedly recommend you do it manually. Beatunes is good and it can help with some of the duplications but there is no substitute for doing it by hand. What you learn about your collection is worth far more than saving a few hours. And surely if you’re a DJ, having a good grasp of what music is out there is what it’s all about. A good mix is important and a pre-requisite to applying your trade but as a listener it’s all about the tunes and don’t let anyone tell you any different.
[ link ]If you are going to have a large collection then the most important thing you can do is structure it so that you can find the music you are looking for with ease.
I apply the following criteria to each track that I keep: Genre, Mood, Year, Rating and BPM
I only keep music that I feel I would want to listen to again. I do this by sampling each track, if it passes the test the whole album can stay. If it doesn’t it goes. I am fairly ruthless but surprisingly more albums stay than go.
You definitely should start from scratch. Copy your entire collection into a separate folder and import albums artist at time. In the advanced tab of iTunes preferences tick the options to ‘allow iTunes to organise your folder’ and ‘copy files to iTunes music folder when adding to library’. This means any changes you make to the tracks in iTunes will be replicated in your actual music folder and everything will be in one place. There is something very satisfying in looking at a folder perfectly organised and referenced! Did I say that out loud?
Artwork is also a must for me but maybe as a professional it’s not so much of a big deal.
I just want to go back to a point I made about moods. This was a late addition to my filing system. Basically I discovered genre’s and BPM don’t tell you enough about the music. I want to know when to play a track not just what it is. So by applying a mood I find it easier to pick out the music I want to listen to when I want to listen to it. Actually, I let iTunes do it for me using the smart playlist facility. The only reason I am able to allow iTunes to do this for me is simply because my filing system works. I use 7 different moods – dark, reflective, chilled, easing down, groovin, start rockin and having it!
It’s important to realise the mood is the mood of the music not the listener. Obviously this is subjective but at the end of the day the 80/20 rule applies pretty well. My iTunes folder is pretty good if I say so myself. It’s amazing how much music gets lost and forgotten. Having an eye on new music is one thing, having a really deep and broad collection is something else.
From a punters perspective – listening to the professionals, there’s nothing like hearing new and quality old tracks being played side by side. It’s very impressive!
Have fun
Great advice, Nathan. I also find using the star ratings helps in the “mood” category – 1 star is warm-up, three stars is medium energy, 5 is “havin’ it” – then you can organise smart playlists by rating and have an instant set order (roughly of course).
[ link ]I’ve found 3 things to help out a lot:
1 – Organization
[ link ]2 – Proper tagging of files
3 – Listen to your music everyday. Delete files everyday. Its what I do and I’ve weeded out so much unnecessary music.
My wife says I’m messy, but when it comes to my music I am extremely anal. In iTunes, I put ALL new downloads in a “sort” folder that I listen to whenever I have a chance. Before a song can get out of that folder, all the ID3 info has to be good and I rate the song (iTunes used 1-5 star system). That helps in the short run but also the long run—1 star songs go to the trash immediately, and once a month I trash all the 2 star songs (these are usually pop songs that have fallen out of favor). Keeps things lean and keeps my auto-playlists fresh. More info on my organization system here: http://reggaedelgado.blogspot.com/2011/04/records-and-organization.html
[ link ]1. Get some music management software, something that can write to the ID3 tags, I use MP3Tag.
2. Sort your music by Bit Rate. Anything below 192 kbps put into a folder called Home Listening. Don’t play these tunes in public the quality is too low for a large sound system – you WILL hear the difference, and so will the crowd.
3. Sort your files by Length. Move anything over 15 minutes into a folder called Albums. Anything under 1 minute move into a folder called Samples.
4. Sort your files by Type. Split each format up into separate folders. Uncompressed WAVs can be treated differently to 128 kbps MP3s. Delete any file in a format that your DJ software won’t play. Move or delete files that are not music.
5. Sort your music by Times Played. Delete any song you have never listened to. Do it. Now delete any song with less than 10 plays, if you don’t like it, get rid of it.
6. If there are any tracks you are sure you don’t like any more, delete them. They are just taking up space.
7. Now is the brave part…pick the Genre tent poles of whatever you listen to. I picked House, Trance, Breaks, Progressive, Techno, Mashup, Pop. Don’t get into sub genres like PsyTrance, Progressive Trance, Nu Skool yet. Make a folder for each major genre heading. Listen to each track and drop it into it’s genre folder, it doesn’t matter if they aren’t 100% right, this is just a start. If there is a track you don’t like…delete it. If you don’t want to listen to a track or tracks guess (or delete). There is no easy way around the listening part of the sort. You have to know your collection.
8. With a set of tracks in a major genre heading folder use the software to change the Genre tag. Then go through the folder again and start sub-genre grouping. Do this again…and again…and again. Change the Genre Tag when you need to so you have something like this: [House]: [French House]: [Tribal House]: [Vocal House] and so on and so on. If you don’t want to split it by Genre, spiting by Artist would probably be a good place to start.
9. I have a habit of using MP3Tag to arrange my song titles in a particular order that makes sense to me: Artist – Song Title – Remix. So the file name looks like this Laurent Garnier – Crispy Bacon – Original Mix.mp3. That’s it, I don’t need the track number, the album name, the release year or anything else, everything else is in the metadata. MP3Tag lets you set this up in a couple of clicks and 100s of tracks can be renamed in seconds.
10. If there is an extra bit of data that you want to have in the Tag use the Comment tag, its a catch all. Be careful using some of the Tags as your DJ software may or may not read them all (your mileage may vary).
11. Do all of the above for every tune you buy from now on. Every. Single. One.
12. Set aside time to do this, and back up everything as you go along.
13. If you are scared about deleting tracks you might have second thoughts about, firstly back up the originals and only move and alter copies, but also get a track list of ALL of your tracks. The management software should be able to handle it, although given the size of your collection it may have to be done in smaller chunks. With a track list you never really lose the song as you have all its details.
Good Luck!
[ link ]Again, this is great advice. Notice how it all comes down to listening to /knowing your music though…
[ link ]Great advice. I am oddly looking at just this problem myself, when i read this article by chance…
Anyway, does anyone know a mac-alternative to mp3tag?
[ link ]There’s no Mac alternative that even coms close TBH. I use mp3tag running Parallels!
[ link ]Like a few of the others here, I have music in the tens of thousands … but I work as a mobile and club/bar DJ and am not resident anywhere. My style changes depending on the venue and the event and play in a heavily tourism oriented Ski area, so the people change each week. SO I actually have 2-6 versions of a lot of songs. This helps with genre hopping and BPM transitions – I quite frequently jump from DNB to House/Electro to HipHop to Dubstep – having a selection that allows me do that smoothly is important for me. Having more music also gives me more chance of introducing something different in terms of live remixing old songs or live mash ups. So not all DJs are built the same – we don’t all have to or want to maintain small collections.
[ link ]Go into iTunes and open the help window (I know it’s kinda lame to not figure it out yourself but trust me it works) There you can find all sorts of stuff on how to locate duplicates, convert file types, consolidate files, create new libraries, sort by bit rate, BPM, and tons more stuff! You will be amazed at what just iTunes can do. You don’t need any fancy music organization programs! Also it is LOTS of hard work but do it in chunks (an hour a day, an artist a day, etc.) and you will eventually get it done.
[ link ]If this is legitimate… I’m with Phil on this one. Start over! But I honestly don’t believe that this is a dj we are talking about either. Obsessive compulsive downloading of 44 THOUSAND files in 3 years belongs on a “Hoarders” episode more than in a trade discussion. By his actions and utter disrespect for the music he supposedly planned to build his reputation on, ended up equating the digital dj community with the file sharing napster brats of years gone by.
So sad, so unbelievably f**king sad, it’s no wonder that every type of music that’s under ANY form of copy write protection is being pulled from the internet.
[ link ]Well all good advice. My experience as far as using software to organize my collection from iTunes to anything I find on download.com is it screws up anything I have customized. It wrecks my filing system that I try to implement. But they are great for someone who is a listener and not a DJ. They will re-write my “dj intro-outro” notations or consider it a double and delete it. I stay away from auto organizers now. Like mentioned earlier, there is no easy way, just put in the work day by day. I have narrowed my ‘DJ files” to around 1000 now but I have found it necessary to keep a folder I labeled “DJarchive” for the one’s that I may just need when I am not at my regular spots. It consists mostly of past music that was popular for specific genre’s.
[ link ]Good luck with that, It is worth the work. It is probably the most tedious of tasks a DJ must do.
I use a combination of Tag&Rename (amazing software), beatunes, and itunes.
When I get new tunes I sort them by genre into folders. No sub genres.
I then go folder by folder(genres) with Tag&Rename and fix all the ID3 info.
Then it is imported into itunes. All tags are already properly set so no need to do anything besides delete the source folders once iTunes copies to library.
Then I run a quick beatunes scan for duplicates, just making sure I didn’t add something I already had.
Now I load up the recently added playlist and listen to all the songs carefully and tag the comments with specific tags to identify picks and sub-genres.
Done. Currently I have about 6,000 songs all mainly drum and bass. I need to delete more but I get so nostalgic :/
[ link ]Disclaimer: This comes from a personal annoyance while trying to support the artists!
You mean reputable sources like Beatport that name their files
_.?
Added to that…Prices vary by *A* *LOT* between releases in both the same shop and different shops!
Imho, it’s not about the quantity of the collection, but the quality! If you’ve got a well organised collection, properly tagged, named and bagged; the quantity doesn’t matter, because you always know where to find what.
Just my 2 cents
[ link ]Forget to add that it’s all about how you’re in it as well.
I’ve digitised my CD collection a few years back; it seemed more effective. I’m a music lover first and that’s what brought me to djing. As for the latter, choose your weapon(s).
Ask yourself; do I *need* all this music for my performance?
[ link ]I use Phelix (http://www.download32.com/phelix-i2572.html) for cleaning my duplicates. Phelix uses the fingerprint to find duplicated tracks. That’s really accurate. I then look over the files that Phelix identified and unchecked those that are no duplicates (maybe a live album vs studio album). Phelix remembers these choices, so the next time I scan it does not flag these files again as duplicates. Really nice.
NOTE:
[ link ]The website posted above is not the publishers website. The official page seems to be dead. This way i can’t guarantee that the file is virus free.
If somebody is interested I can also post my installation file (from the original website) somewhere.
Phil Morse and James K. got it right just take it all out and put it all back in.. and ask yourself if you’d throw it down in a mix
I think it’d be dumb to erase your stuff. yah you want stuff over 192 kbs, but I play at a hip hop/ top 40 club and I pull it all out you may hear wheezer to ducksauce to tiesto to kid rock and all in the same set. You have to be your own dj. don’t worry about other people.
library: 12,000 .mp4
[ link ]