We’ve recently been contracted to move a client sites between servers. Well the job has been finished, sites were working correctly on the new server until recently, a few weeks after the move, when he discovered an old employee of his installed much to our surprise a lot of backdoor code inside the sites so that he could get unauthorized access at any time. By doing this, he managed to delete a lot of important database data as far as we checked.
While I do not have the details of their current relation, I’m inclined to say that his employee done this with specific actions in mind.
Needless to say, getting backdoor code is like finding needles in a haystack.
You would think by now, ok, why not recover the data from backups? Well that would be ideal, but here is what actually happen:
His hosting company were in charge of creating backups of his server, however what they actually forgot to specify their client is that those backups are in fact disaster recovery backups and will not protect him for malicious actions, so they setup a daily and weekly backup process, the daily one would refresh itself each day, and the weekly one, each week on Monday as you might guess!
Well, over the weekend, his employee was busy, he managed to delete the database data. Now, the actual business owner gets back to the office on Monday and gets the surprise, a lot of sites have been compromised! Ok, no worries, we have backups, but wait, it’s Monday the backup process just flushed the weekly backup and the old ones are gone, leaving … nothing in place!
The story might have a happy end as we kept the clients data after the initial move, so we are able to recover them from there, however any updates in the meantime might be lost!
There are a lot of things each of us can learn from this story:
- always make sure you keep proper backups of your websites and are able to recover
- taking a day off to test recovery scenarios will help
- try to always keep an ethical point of view and make sure you employee the right people
- and again, always keep a backup, specially when you dealing with impotant data
- nobody cares more about your site than you, so don’t take for granted everything, do a checkup yourself
Getting the law on your side, will help…, but it might be to late!