Hacking Together a Backup System

Sat Jul 23 17:10:40 EDT 2011

Tags: projects

Recently, my living room Mac mini's external drive, where it kept its iTunes library, met a horrible, clicking death. Naturally, I had devised a proper backup plan long before this happened - unfortunately, however, I had not yet implemented this plan. Crap.

So I decided to straighten out my backup system all around, ideally covering all of the machines in the apartment as well as my hosted Domino server. My ideal (at least for now) backup plan would fit within a couple attributes:

  • Cheap. I've been trying to spend less money overall lately, and picking up a new bill for hardware or services would counteract that somewhat. I wanted to come up with something that would work with the hardware I had on hand, plus one purchased replacement HD.
  • Automatic. I don't want to have to remember any manual backup process, since I most likely wouldn't.
  • Off-site backup isn't important. Sure, it would be nice to keep my data in the event of a physical catastrophe, but we're talking about TV shows and movies here, not anything vital.
  • Quick recovery or automatic failover aren't important. They'd be NICE, certainly, but I'm just looking for a basic "the data exists in at least one other place" setup. If a computer meltdown means that recovery will take a while or I'll have to rebuild the OS, that's fine.
  • Versioning isn't important. The occasions where I would want to restore intentionally-deleted files or modified documents are so few and far between that it's not worth going out of my way to achieve that.

My main laptop was far and away the easiest to set up. A while ago, my boss gave me an external USB drive which I've been using for Time Machine backups. Time Machine does pretty much everything I would want to, and even gets bonus points for ease of recovery and versioning. When I'm out of the office, it even kind of counts as off-site.

My Domino data was the next easiest, primarily since I work with it all the time. I set up a Parallels virtual machine to run a new Domino server, set up scheduled replication, and pointed my "create a replica of everything" agent at my production server. Voilà: up-to-the-hour backups without having to give it a second thought.

The hard drive I purchased to replace the failed one was a nice 2 TB one, giving me enough room to store my media library plus some Time Machine backups for the mini itself, the iMac in the bedroom, and the two other laptops floating around. It won't be enough space permanently, but it'll last me at least until I'm comfortable enough to buy another one. So that covers the other Macs themselves.

In addition to the media drive, the Mac mini also has a 750GB drive salvaged from my poor, video-card-exploded iMac. I cleaned off enough old crap from there that it will be able to serve as a mirror for the media files on its larger brother - again, at least for now. To implement that, I wrote a quick, two-line shell script:

rsync -aE --delete /Volumes/Tartaros/Movies /Volumes/Diaspar
rsync -aE --delete /Volumes/Tartaros/iTunes /Volumes/Diaspar

That basically mirrors the Movies and iTunes folders on Tartaros (the media drive) to equivalent folders on Diaspar (the iMac's old drive). The "-a" switch toggles "archive" mode, which enables a lot of useful behaviors for this case, "-E" enables support for HFS+ metadata like ACLs, forks, and extended attributes, while "--delete" removes any files in the target directory that no longer exist in the source. I added this to my crontab to run at 3 AM each day.

All in all, I think this setup should cover my basic needs pretty well. My next step, when I want to spend the money, will be to sign up for an online backup service like CrashPlan. There are some cheap options and they would hopefully be more reliable than my current scheme, which is still dependent on the fragile health of a handful of external USB drives.

Using PHP on a Domino Server via CGI

Sat Jul 16 21:34:21 EDT 2011

Mostly on a lark but partly for practical reasons, I wanted to set up PHP on my Linux-based Domino server. While an easy option would be to install Apache and have it use a non-80 port, I wanted to see if Domino's ancient CGI support still works in the modern world of today.

In short: yes. In fact, it's pretty easy; the bulk of the work I put into it was actually just because I refused to believe that the syntax on that IBM page and in Administrator's documentation wasn't a typo. On my Ubuntu-based server, the setup is quick:

  • Run "apt-get install php5-cgi"
  • Set "cgi.force_redirect = 0" in your php.ini file. In my case, this didn't exist, so I created it as /etc/php5/cgi/conf.d/php.ini with just that line. If you don't do this, you'll get errors about the CGI app not returning any output
  • Add a rule to your web site with the following setup (using the path to your PHP CGI binary as needed):
    • Type of rule: Directory
    • Incoming URL pattern: /*.php
    • Target server directory: /usr/bin/php5-cgi\*.php
    • Access level: Execute
  • Place your scripts in the domino/html directory in your server's data directory

    That backslash is actually there and there actually isn't any space in front of it. It's really weird, I know.

    If everything went well, you should be able to refresh HTTP and voilà: you can execute PHP scripts. I haven't investigated it super-thoroughly for edge-case gotchas, but it runs phpinfo() just fine, and you don't have to worry about making your PHP scripts executable or anything.

    One important thing to bear in mind is that this is just CGI. Your PHP code isn't going to suddenly have access to the Domino API in any new way - this isn't a new way to write Notes apps on the web. If you want some sort of API access, on Windows you could use OLE and on other platforms you could presumably compile a version of PHP with Java support, but that's about it. That said, you DO get the benefit of Domino handing off any authenticated user name (even using some crazy DSAPI method), so it's not COMPLETELY isolated from the rest of the server.

    Freaking Reader Fields

    Thu Jul 14 15:02:42 EDT 2011

    Tags: domino

    Periodically (read: every day), I wonder about switching from Domino to a SQL server for data storage on my guild web site. The primary reason for this is speed: I'm doing primarily relational things, so I've had to wrangle Domino quite a bit to do this with any amount of speed and code cleanliness. Additionally, while most of my documents are entirely distinct from each other, I had to make concessions here and there, such as storing the latest Post date in Topic documents so I can sort them that way, and, each time I have to do that, there's another little bit of code maintenance and clustering-unsafety.

    However, my ideas always come to a screeching halt when I remember Reader fields. They're simply too good, and the replacements I've found on the open source SQL databases have been, to put it kindly, lacking in comparison. They generally involve having some access level field or, best case, a multi-value field of names that are allowed to see the document, and then making sure that all of your queries or views honor that. Every method has some severe downside, ranging from inflexibility (access level) to nightmarish piles of code everywhere (multi-value name/group/role fields). Everywhere I accessed the database, I'd have to worry about security and document access, bloating up the code and just asking for data-leak bugs.

    Domino, for all of its faults, makes this something you just don't have to worry about. If you have a Reader field, you can toss names, groups, and roles in there with impunity, and the server will handle the rest like you'd want. You don't have to do your own directory lookups, security checks, or nested queries. If the current user isn't on the list, the document may as well not exist. Even if the user had the UNID of the document and designer access to the database, it'd be beyond their reach. This is enormously comforting. And even though it's just a guild web site and not a giant corporate database, I'd still rather deal with a bit of tricky code for performance than the headaches and drama involved with people seeing what they're not allowed to see.

    So, until I either get entirely fed up with Designer or I find an equivalent to Reader fields in a free SQL server, I'll be sticking with Domino.