Saturday, September 10, 2011

Photos, memories... rare, medium or well done. How do you like your storage?

It's very interesting how in around a decade we have moved from film photography to almost fully digital. I personally started with digital at the end of the 90's and still I'm surprised by how quickly digital took over. The first cameras were kind of a novelty and wicked expensive. The Nikon D1 was the first true DSLR that started to turn things around. It was a beautiful brick to carry around, but its adoption by the press media signaled things to come. Then back in 2004 with the Nikon D70 that truly did it for me. From that point on, my 35mms went to rest, with the occasional let's take it for a spin kind of as you drive an old classic around the neighborhood. I still use my Nikon F100 every now and then, but digital is here to stay.


The part of digital that I still can't shake up is how people don't properly backup their photos. Maybe me working at Enterprise I'm used to backup absolutely everything and always be as redundant as possible, but my impression is that for most, they're just sitting in their Mac or PC's drive, no backups or optical media, no printed copies, waiting for something bad to happen. I see pictures as one important thing: Memories, hence they're priceless and irreplaceable.

Regardless of your system brand and what you have been told, your Mac basically has the same hardware components than those in an HP or Dell and vice versa. What changes is the software you're using to run it; the Operating System (OS). All of these hardware components have a limited lifetime (measured a MTBF or mean time between failures) and bottom line is that when they die, your memories will die with them (if you haven't properly backed them up).

Answer a simple question: Do you have an exact copy of all your memories in a different place and using different media, other than your main computer? If you answer No, then you're in trouble. If you answered Yes, you're on the right track. Now the question is how to do it better. Well I'm going to split it in several areas and tell you exactly how and why.

Simple, easy to set product recommendations if you don't want to read the whole thing:

I personally have tested and used several brands (Linksys, D-Link, WD, etc) but have found that the one with the best combination of quality, price, features, GUI are the ones made by Buffalo Tech. Pretty much can't go wrong with any of them. I have several different units and here's some recommendations:

  1. Single drive unit: Linkstation Pro LS-VL NAS (around $200, http://www.buffalotech.com/products/network-storage/home-and-small-office/linkstation-pro-ls-vl/). This is the best single drive unit you can get.
  2. RAID capable: Linkstation Duo Pro LS-WVL/R1 (around $275, http://www.buffalotech.com/products/network-storage/home-and-small-office/linkstation-pro-duo-ls-wvlr1/). This is a dual drive version of the previous.
  3. High performance RAID: Around $700 the Thecus it's literally a full blown server in a small box with 6 storage units and great performance, currently rules the performance and value charts and can easily handle a small business. (http://www.thecus.com/product.php?PROD_ID=47)

Figure 1: The Buffalo Tech Pro or "V" series offer high performance and a full rich set of features at a very reasonable price. Starting at around $200 there's no reason why you can't get one of these and make your data safe. The single drive version looks like any regular external HDD. The dual drive version offer expandability and RAID.

There are others at different price points. All depends on what you want to do and the level of complexity and expandability. Single drive units tend to be not user upgradable, at least easy to. RAID units tend to have open bays which allow for easy exchange of the drive units which it's really important specially if one of the units die and you need to replace it. All of the previous are sub $1K and easy to set with basically no maintenance required, and reliable enough to trust your data to them. For most professionals, the previous will serve them well for awhile.


Now let's talk details and cooking. How do you prefer it done?

Some people like food rare, others well done. I'm going to split this in rare, medium and well done, going from easiest to most complex.

1. Rare - Cheaper, simple but slow and limited:

1.A. Optical media: I know, nobody uses DVD's anymore. Well, bad news, you're very wrong. Optical media has a huge advantage over magnetic media: It's the closest thing to permanent you get!

You see the hard drive (HDD) or solid state drive (SDD) in your computer is made out semi-conductors and other magnetic media. The problem with magnetic media is that it does have a limited and unpredictable life span. Key word here is "unpredictable" and that's because we really don't know how long our HDD/SDD's are truly going to last. I still have some SCSI HDD from the 90's as a self experiment and they seem to be holding fine, but there's no way to know when they'll hit the bucket.

Optical media, at least has the theoretical potential of passing the 100 year mark and regardless if they last that long or not, the point is that you have plenty of time to look for an alternative to keep your photos or data safe, at least while you're alive. There's a second benefit and that's their lack of sensitivity to Electro Magnetic Pulses (EMP's). You see, optical media, ironically, works on the same principle as photo film: You have a media that's sensitive to a particular type of light and when it's exposed to it, that data is permanently etched in that film hence are resistant to magnetic pulses.

1.B. Magnetic media: After bad mouthing it, still a simple external or secondary HDD or SDD plugged into your computer, remains the easiest and cheapest backup storage option. The important thing you must remember is that such storage system should be a separate physical unit from your main unit. The reasoning is really simple; you want to avoid putting all the eggs in the same basket.

My ideal configuration is a second physical HDD in your computer so you're always running your applications and OS in a main HDD and then you put the data on the second HDD, then you use an external solution as the backup solution. Yeap, that was 3 physical units, that way when you're done backing up, simply pull it out and put it in a safe place. There's a side benefit from having a second HDD inside your computer for data storage only: You can always replace the main drive with a different OS or if you have a fatal crash requiring a full restore, your data remains safe in the data drive!


2. Medium - Independent, self contained, good performance and reasonable features:


2.A. BlueRay media: BD media is another type of optical, more expensive but with much larger capacities. Same benefits as all the ones for optical with the advantage of larger storage capacity and potentially longer life span as the media itself should be of higher quality in order for it to sustain a higher capacity. Again same side benefit of their resistance to EMP's. Pretty much magnetic media is sensitive to magnetic pulses and electricity; hence data held by such devices can be easily wiped by any of such. In theory, properly protected optical media like BD's or DVD's should survive the EMP of a atomic blast (of course assuming they're far away enough from that blast radius)!

2.B. RAID: Redundant Array of Inexpensive Drives is basically the use of multiple storage units to replicate the data you're storing in them. There are multiple classifications of RAID, but the simplest and most common is RAID 0 or a "mirror" in were at least two storage units have an exact copy of the same data. Most computers you can pop in two drives of the same size and set them up as a RAID 0. Better is if you have an external RAID storage unit plugged into your computer. That way the storage is redundant, and can be unplugged and stored safely at another location.

2.C. NAS: A Network attached storage is basically some sort of independent computer/storage unit which is accessed via a network connection and it's dedicated for storage purposes. These come on various flavors some of which are listed here:

2.C.1. USB to LAN adapters: Units such as the Pogo Plug or the Lynksys SLUG pretty much allow you to plug a USB device to it and give it access via a network connection. These are the simplest and cheapest but very often, painfully slow. I normally recommend staying away from these as you get what you pay for.

2.C.2. Router with USB adapter: Some routers allow you to plug in a USB device to their backs the same way as the previous, making it available to the network. With this kind of units you kind of kill two birds with one stone, but again, you pay the price is the capabilities and performance as often they're very simple but slow performers. One that I found to work really well is the Buffalo Tech Air Station WZR-HP-G300NH (http://www.buffalotech.com/products/wireless/wireless-n-routers-access-points/airstation-high-power-n300-gigabit-wireless-router-wzr-hp-g300nh/) which actually is pretty usable and simple to set offering a whole package solution in one unit.

Figure 2: Notice to the left the router, as any typical router unit you might already have. The advantage with this kind is that it allows you to plug an external USB HDD to it and turns it into a NAS.

2.C.3. NAS drive: Dedicated NAS units have the advantage of basically been a small server with a storage unit connected to it, normally in small packages. Depending on brand and features, some are really good performers that offer a good feature sets and capabilities other than storage. Best of all they don't cost much more than a typical external, direct connect unit and their power consumption is very low, for example the single drive Buffalo Pro unit listed in Figure 1 barely uses 17w and provides up to 2Tb of storage.


3. Well done - We're talking redundancy, top performance and even Enterprise level.


3.A. Small server: Do you have a old PC doing nothing sitting around? Good news, it's actually fairly easy to turn it into a dedicated storage unit or as a multi role server. There are plenty of free server OS that will allow you to do that. This is a higher level of complexity, but also a higher level of performance and features. Here are some good simple offerings for such a project:

3.A.1. Amahi Home Server (http://www.amahi.org): This OS and software tools combination will turn any old or new PC into a central computer allowing you to handle all your entertainment, storage, and computing needs. Most important it's FREE and expandable, allowing you to grow it as your needs change.

3.A.2. FreeNAS (http://www.freenas.org): Enough said, it's a NAS, it's FREE and most important simple and does what it has to do in an effective yet simple way. Actually it's a little bit more complex than that, the cool thing about it is that basically they prepared a whole Linux OS based on FreeBSD Linux (http://www.freebsd.org) and added NAS specific web based tools that are simple and easy to use and configure. You can literally run FreeNAS from a CD or USB thumb drive, allowing to almost intently turn any machine into a NAS, without actually having to change the core configuration. Pretty much a NAS on the fly...

3.A.3. Microsoft Home Server (http://www.microsoft.com/windows/products/winfamily/windowshomeserver/features.mspx): Actually this is a really simple to set and well featured system based on the powerful, Enterprise class Windows 2008 Server product. You have to either buy the software ($45 here http://dealnews.com/Microsoft-Windows-Home-Server-2011-for-45-free-shipping/497758.html) or buy a product already with it, but it does offer a powerful solution for your storage needs.

3.A.4. Apple Mac mini with Lion OS Server (http://www.apple.com/macmini/server/): This is a pretty cute little server that has a fully featured set. The only problem I have with it is that first it will set you $1000 back just to start and then it only has two 500Gb drives which is kind of limited specially in this day and age with huge media files. You can expand with an external unit, but that means more $ to shell out.

3.B. NAS/SAN/DLP/Optical/HDD dedicated backup units: These units are really expensive and mostly Enterprise class. Most are dedicated for a large number of users. For example, a well configured HP backup tape library storage unit (http://h71016.www7.hp.com/ctoBases.asp?oi=E9CED&BEID=19701&SBLID=&ProductLineId=450&FamilyId=2353&LowBaseId=21963&LowPrice=$5,660.00), well configured will easily set you thousands hence most of these are an overkill for home or a small office as we're talking hundreds of Terabytes of automated backups.

3.C. Dedicated Server running either Linux or a Windows Server OS: These can range from a single system running a dedicated server OS to a blade cluster. Either way, we're talking highest level of complexity and capabilities.

3.C.1. Ubuntu (http://www.ubuntu.com): The Ubuntu organization is probably the biggest hero of the freeware movement. It offers anywhere from a netbook OS to an Enterprise level server solution all based on Linux. This will be a bump in complexity, but also a huge level of expandability and feature sets that literally goes all the way up to Enterprise class. Did I mention FREE! The question becomes then what kind of hardware you will get to run this solution. Again a huge range of features and capabilities, but professional level of complexity and knowledge required.

3.C.2. Windows 2008 R2 (http://www.microsoft.com/en-us/server-cloud/windows-server/default.aspx): This is probably your more powerful and fairly expensive solution as it is Enterprise level. The software by itself will set you back almost $1000, and that's without the hardware but then we're talking Enterprise level and 99.99% uptime. There's a reason why this is the most used solution at Enterprise level. Same as for the previous, the question becomes then what kind of hardware you will get to run this solution. Again a huge range and high professional level of complexity and knowledge required.

3.C.3. Custom made solution, beyond belief technology: If you truly have money to burn, and I'm talking a lot of $ there are plenty of companies that can custom build a super infrastructure for you. For the particular case of arts and entertainment, the state of the art system is the infrastructure HP prepared for Paul McCartney. This dream system pretty much holds absolutely all media created by Paul and some of The Beatles. You can check this amazing system here (if you're not impressed by it there's something wrong with you): http://h10124.www1.hp.com/campaigns/enterprise/mccartney/us/en/paul-mccartney-digital-library-experience.html#


Summary:

If you don't have any of the previous, not only your photos, but any relevant documents you might have are at risk. From all the previous, the ones listed in the first two sections can be done by any hobbyist or with some tech skills. Feel free to ping me if you have any questions about the previous. Most important the cost of such systems is irrelevant compared with your memories... unless you have made none worth keeping...

No comments:

Post a Comment