Saturday, April 2nd, 2011

We’re a modern digital couple custom made for the new millennium (we met on twitter for crying out loud!). So, together, we have a lot of data. And, of course, hard drives are never really big enough. It’s a problem.

Our current solution involves a big 2TB external drive (aka BIG DRIVE) connected to Bran’s iMac (which itself is plugged in to the network via Cat5 instead of my own MBP’s WiFi connection). This drive is shared out on the network so we can access it from everywhere else in the house. Also connected to the iMac is a 4-disk Drobo. We back up BIG DRIVE to the Drobo using the excellent SuperDuper!.* This has served us well for a while now, though it has the normal problems: All these drives and storage robots were expensive. It all sucks down way too much power. The Drobo connects via USB while BIG DRIVE uses Firewire, so it causes a bit of port contention on Bran’s iMac. (Fortunately, Brans’ TimeMachine drive is also Firewire, so we can daisy chain it through BIG DRIVE. It works surprisingly well.)

And now, finally, after about a year, BIG DRIVE is starting to fill up. It’s not imminent (there’s still about 150GB free), but it’s definitely something to consider. I’m not sure what to do. All of the options I’ve come up with are bad or slightly-less-bad-but-really-expensive.

The most straightforward thing to do will be to buy another 2TB external drive, BIG DRIVE 2, and hook it up to the iMac. I can replace the smaller drives in the Drobo with some 2TB ones (only one of the 4 drives is a 2TB right now) and make sure that SuperDuper! knows how to back both drives up to the Drobo. There are obvious problems with this: It will be YET ANOTHER external drive hooked up to the iMac. We’ll have two drives and will have to come up with some way to know which files are on which drive. And I’ll probably have to figure out some way to do the backups to drive images since SuperDuper! is a whole-drive sort of deal. I’m not thrilled with this.

A similar option involves buying the second external hard drive and software RAIDing (RAID 0 or JBOD, probably) them together via OS X. This has the same port problem as the other; but since the drives would appear as a single volume, it would mitigate the complex backup issue as well as the “where are my files” issue? On the other hand, software RAID over USB or Firewire seems like it could end badly. I’m not much of a sysadmin, and I’m not sure that I’m smart enough to really handle this correctly. But that’s what the Drobo backup is for, right?

A third option would involve buying a Mac Mini and doing either of the first two options with it instead of Bran’s iMac. This would partially mitigate the port issue (at least from Bran’s perspective). And it would double as an HTPC if we hooked it up to the TV downstairs. But it involves buying an entirely new computer (as well as some new drives). So it’s a pretty expensive option while still maintaining most of the drawbacks of the previous ideas. Of course, since the problem isn’t imminent, we could buy the Mini now and buy the drives later, spreading the cost out a bit. And I have wanted a Mini hooked up to the TV for a while. So, with all the drawbacks, this is a surprisingly attractive option.

A fourth option is similar to the third, but involves cobbling together a Linux computer from Newegg. We could get something with room for lots of drive bays so the primary storage could end up being more internal than external. This has significant advantages (I’ve had a lot of drives mysteriously unmount (during copy operations!), sometimes requiring a full reboot in order to get them to mount again. I don’t really trust it.). We would probably still back this up to the Drobo. Similarly to my third option, this requires buying an entirely new computer. It would also require me to build a computer which I haven’t done since high school (when I started using laptops) and then figure out how to administer a Linux system. I’m comfortable as a Unix (or Linux) user. I’m not sure about administering a home server. I’m not really sure that this one is worth the headaches, but going with Linux (or BSD) could give me some advantages (like ZFS? I’d have to do lots of research) that using OS X might not have. At the same time, this seems to require a lot of work and study on my part. So I can’t see going this way without some really compelling reasons.

Another option we’ve explored is buying a Drobo-like device. This thing from Netgear has been well reviewed. And that’s an older device. I’m sure better ones are available now. This could also be pretty expensive; and after my experiences with the Drobo, I’m a little gunshy to drop that much cash on it. One reason I went with the Drobo originally is that it seems to handle varying drive sizes better than some of its competition (like this thing from Netgear); but maybe that’s not really an issue in practice? I’m also concerned about maxing out the storage.

Right now, the 2TB SATA drive seems to be where the industry has settled (though 3TB drives are finally available). What happens when I’ve filled all the drive bays with 3TB drives and still need more space? I buy another one and suddenly have two giant drives? This is starting to look like a generalization of option 1. Of course, all the other options have similar issues. I really wish I could plug a bunch of these different things together and have them all recognized as a single volume. I suppose software RAID + external drives helps with that. But, by that point, there are so many abstractions going on that I would be terrified of data loss.

So what do we do? Save up our pennies and build a SAN in the house? Live with dozens of external drives? Buy some fancy widget that will promise (and then fail) to solve all of our storage needs?

I might need some help on this one.

* Originally, we just had the Drobo and were relying on the Drobo’s internal mirroring for “safety”. Soon, the Drobo convinced me that the Drobo itself isn’t an entirely reliable device (one of the drives failed but the Drobo took months to tell me which one, during which it was in an unprotected state) which caused us to move to our current solution.

blog comments powered by Disqus