Automatic online backup with history

Six years ago I wanted to make an automatic online backup with history of all script files on the shared work computer. The caveats were:

  • It must be automatic. I can not relay on anyone performing any action.
  • It must be user friendly as in any user with basic IT knowledge must be able to find the desired version and restore from it. Changing the files and restoring old versions is quite common behavior.

And the benefits of the situation were, that the machine was running Linux (CentOS) and that the files were all text based scripts, so small file sizes. I made a separate backup system for measurements, but that is a story for another time.

When it comes to “backup” of versions of software, most programmers would jump to Git, as did I for this problem. After my initial attempts of trying to teach everyone Git were unsuccessful, I had to come up with another idea. So the TLDR is that I made a cron script which calls git add . , then git commit and finally git push every 5 minutes.

The longer story has some interesting details. I did not want to touch the original files for “just in case” reasons. So I first made a separate folder to which the script copies all files every 5 minutes and then runs git commands. So, you may be thinking: “This will make a lot of unnecessary commits, as most of the time, there are no changes!” Aha! Git to the rescue. Git automatically doesn’t commit if there is nothing to commit.

The last hurdle is, what to put as commit message? Well, not much can be done automatically, so the commit message is just the datetime stamp. Given the restoring process, it is usually finding the version that worked last Tuesday afternoon, so having the datetime in commit message enables some searching.

What I am most proud of is that it is still working after 6 years and 30 thousand commits. There have been only 3 instances, where I had to manually intervene. They were:

  • An unknown thing, that was fixed by restarting the computer.
  • Changing the ownership of the git project, as we needed to stop password sharing my account for obvious security reasons.
  • When bitbucket forced the use of application keys instead of account passwords.

P.S. I waited to publish this for a couple of weeks, to post it exactly on the 6th anniversary of the beginning of the service.