So I’m pretty good about securing my data and backing up. I have a RAID 1 setup on my main machine as well as an external NAS that is also setup as a RAID 1. The only thing I haven’t done yet but will soon is find an offsite backup solution in the cloud. I’ve been doing manual snapshot backups to my NAS from my local machine but I wanted a cleaner more automated solution that not only backed up a directory but also synced it.
So what do I mean by syncing? Well there are programs that will keep a mirror image of a source directory to a specified destination directory. The reason I want this is because after a backup I will often move directories around and delete them. When you setup a backup scenario these changes aren’t reflected, but if you setup a sync, they are. In the past I had used a program called Second Copy that did this well but I have recently discovered two new free apps that also provided sync functionality and wanted to give them a shot as it’s always nice to have a free solution to recommend to people. More on that in a minute, but first let me share my hell with you.
So I setup the first of these new backup apps which was called GFI Backup. I decided to use my MP3 collection as the source directory for these tests (huge mistake). It appeared to be a pretty nice program with a clean and simple interface. The backup process worked great, but the setup options related to synchronizing folders was a little confusing and I couldn’t get them to work properly. So I went on to the second app called Syncback which I read about in Lifehacker backup tools roundup. I decided to create a new source directory for this test with just a subset of my mp3 files in it. I did however place the destination directory in the same root directory as the one I setup for GFI Backup. I also could not get Syncback to properly synchronize the directory. So I gave up and went to bed that evening.
The next day I went to my machine and as I often do loaded Winamp and click the play all button for my collection. I noticed that it was skipping around from track to track in the player and suddenly felt my heart sink. I knew that behavior. It meant that the data from the loaded library wasn’t located in the directory. I quickly went to my MP3 folder. I saw all my folders in there but as I poked around within directories I found them all to be empty. I then went to the destination directory on my NAS. Same thing, no files. I went into panic mode. I double and triple checked the directories but no luck. At this point I realized that something in my synchronization setup of the backup apps had gone seriously wrong. I’ll say that I never went back to try and determine what exactly I did wrong, but if it was that easy to screw things up, I wanted nothing to do with either. To be fair, they both operated fine as backup programs, but synchronization can be a little trickier and neither made that setup work easily. Also, the fact that I shared a destination directory for my testing may have lead to the problems as well. So take my mistakes with a grain of salt and if you choose to try these programs do it with some unimportant data files first so that you don’t have to worry about losing anything important. I know better than taking the actions I did for these test. I can sometimes be a bonehead.
Anyways, at this point it was time to seek out a recovery program to attempt to undelete as much of my MP3 collection as I could possibly salvage. One thing to note is that whenever you delete data on a hard disk your best bet is to stop doing any file related actions on the machine and stop using it if possible. This will ensure the chances of recovering the data. When you delete files all the data is still on the system and the operating system just sets a flag to not show them in your file system. If you start moving files around and writing new data to your system then you risk overwriting the data that is on there and thus lowering your chances of recovery. So I had used a few recovery apps in the past and needed to refresh my memory as to which were good.
I found this great Lifehacker story on the five best free data recovery tools. I had used a few of these before and just wanted to determine which would be the best for the job. Of the five listed I decided to try the two which offered the most functionality which were Recuva and Undelete Plus. I’ll cut to the chase and tell you that Recuva was far better. It scanned the drive very quickly (less than a minute) to determine which deleted files where on my system. The same operation took about 20 minutes on Undelete plus. Also, recovering files using Recuva was very fast it appeared to be able to do a several within a second or so whereas Undelete Plus took several seconds just to recover a single file. Now this may not seem like a big deal but I had to recover over 15,000 files. Lastly, the ability to filter the results by a specific directory worked great in Recuva and I couldn’t find an easy way to do it in Undelete Plus. Needless to say I chose to use Recuva.
(Click image for full view with details)
Recuva also had a nice feature that showed the probability of being able to recover each file that it scanned as being deleted. In my case it appeared that over 95% of the files had an excellent chance of recovery. So before I started I got an external USB hard drive I had which also had a snapshot of my MP3 collection on it from 2 years ago so at least I had something but have obviously acquired quite a bit since then. I created a new directory on the drive, flagged all files in my MP3 directory and told Recuva to go to work. After ~2 hours I was able to recover over 15k files for a total of over 60Gb of data. That turned out to be over 95% of my lost files. I was pretty amazed by how lucky I was.
I had to take screenshots (because there is no facility to save the file info) of the remaining files Recuva identified as being poor or having no chance of recovery so I had a record of them. This way I had a record of what I lost. I did locate some of the files that they listed as having a poor chance of recovery and they worked so it could be a painstaking process to try and determine which are good and which need to be replaced but at least I have a record of them. Overall I’m extremely happy with Recuva and sending this awesome freeware a donation for what they’ve done for me.
The other issue is that there was some data that was still ok and on my local drive that didn’t get deleted. I knew this because showing properties on my local MP3 directory showed that there was 5Gb of data still on it. but there was no way I was going to click individually on the hundreds of folders many of them nested to determine where the data was. Ok, if push came to shove I would have done that but I was pretty sure there was a tool to do this for me. I remembered that there are some file / directory compare tools out there and was able to find WinMerge which was recommended on Lifehacker as well (Yes, I like Lifehacker). So I compared the original MP3 directory to the recovered one and WinMerge showed me that there were 7 directories on the original that weren’t on the recovered one. Nice, so now I was able to identify these folders with data in them that wasn’t deleted and could now merge them into the newly recovered MP3 directory.
But there were still more files to be found as that wasn’t everything. Apparently some of the folders in the source still had files in them. So the issue was that I would still have to click on every folder in WinMerge to try and find which ones had data on the original directory versus the newly recovered one. That was going to be a pain so I looked into a way to do a directory listing of the source to try and root them out. I found this great tip on how to create a text file of a directory using the command prompt. This worked perfectly. I opened the file in a text editor and then began searching the file for the value “.mp3” which then uncovered the only folders that had files in them. Bingo, that worked and now I finally was able to locate every possible file I could recover.
The last thing I want to mention is that I could have taken another course of action that would have probably insured a closer to 100% recovery. If you remember I had all the data on my NAS which also was subsequently deleted. I read about a process where if I would have taken one of the drives in my RAID 1 array out and placed it in an external case and then plugged it into my computer via USB and used a program called Diskinternals Uneraser which would mount the NAS ext2 file system and then undelete the files that way. But I figured I had done a pretty good job with my recovery process and really didn’t want to go through all the trouble and then risk any issues by putting that drive back in the NAS and losing any data on their after the process. Enough damage and time had been wasted already.
So as you can see this was a very agonizing process that I brought upon myself by not really being careful testing new software that can be very dangerous on unimportant data. I was very lucky to find such good tools to help with the recovery and hopefully this will be useful to someone else who unfortunately runs into a large data loss situation. So I’m going back to using Second Copy for backups and Synchronization which has worked perfectly in the past. I just hope I learned my lesson and can avoid this torture from happening again.