Best way to store a large media library?
#61
Bongu Wrote:I am very lost with all of this but am trying to learn. I currently have about 4TB of media data split on three 1.5 TB USB drives. I then have two 2 TB USB drives backing up the data.

I don't know if it makes sense for me to switch to one of these server type solutions, but if I did, using something like unRaid how is this all set up? If I had three 1.5 TB drives how could one more 1.5 TB drive be a parity drive to backup 4.5 TB worth of media?

Sorry for posting what is probably a very basic question but I imagine many others are reading this thread scratching their heads as well.

Thank you.

Answered here

http://forum.xbmc.org/showpost.php?p=515...stcount=40

In your case, if you wanted to use Unraid you would use one of the 2TB drives as parity, the other as data, move the contents of one of your 1.5 drives to the data drive, then add the 1.5 drive to the array, move over another drives, rinse repeat etc. Each drive to be added needs to be formatted and the entire drive is written with 0's, so you cant add a drive with stuff on it into the array.
Reply
#62
Kode Wrote:Answered here

http://forum.xbmc.org/showpost.php?p=515...stcount=40

In your case, if you wanted to use Unraid you would use one of the 2TB drives as parity, the other as data, move the contents of one of your 1.5 drives to the data drive, then add the 1.5 drive to the array, move over another drives, rinse repeat etc. Each drive to be added needs to be formatted and the entire drive is written with 0's, so you cant add a drive with stuff on it into the array.

Ok. So basically the concept is that all the data is made up of 1s and 0s so if it could reconstruct all the 1,0 it can then replace the data. Is that correct?

How do you know if a drive fails and then how would you go about replacing it? Are there any instruction manuals to the whole process?

Thank you.
Reply
#63
Bongu Wrote:Ok. So basically the concept is that all the data is made up of 1s and 0s so if it could reconstruct all the 1,0 it can then replace the data. Is that correct?

How do you know if a drive fails and then how would you go about replacing it? Are there any instruction manuals to the whole process?

Thank you.
The unRaid manager will keep tabs on the status of the array. Basically, it uses the two working drives plus the prior calculations(parity) drive to reconstruct the failed drive once you replace it with a working drive. All you do is physically remove the dead drive from the PC and add a working drive. The unRaid manager will do all the work for you.
Reply
#64
I just wanted to say this topic has been a great read. I currently have 700GB of data mirrored across three 1TB drives. Each night the data is rsynced through my DNS-323 box, and then each week I swap out the backup disk with the offsite backup disk so there is an offsite copy of at most a week old should something happen.

I've been thinking about how to handle expansion though and this topic has sold me on unRAID. Using a single disk as a parity disk seems so simple yet so smart. Eventually now I'd like to move to an unRAID setup with a box handling at least eight drives to allow for expansion in the future. I'd also have an offsite backup of all data on 2TB drives as needed.
Reply
#65
As I've been reading more about NAS, RAID, Backups and Drive Failures it really is intimidating seeing how certain people are that drives will fail. Do drives fail because they get old or because they are used so much? I realize that anything is possible so I'm asking about generalities here.

Basically if I backup my data on a drive and place the drive off site and it is never touched, is there still a reasonable likelihood that when I first start it up that it will have failed? By reasonable I mean a similar percentage as if the drive was used regularly.

I think I read somewhere that a general number is that there is a 5% chance that a drive will fail in 3 years. If that is the case, I would expect that there would be a significantly smaller than 5% chance in a longer than 3 year time period (maybe 5 or 10 years) that the same drive sitting in a storage bin would fail. Is this correct? I'm obviously not talking about the drive being lost to external factors (fire, theft, etc), but simply when connecting it 5 years after it has been sitting in a box if it will start up.

Thanks for the help.
Reply
#66
Unraid has a preclear script which also allows you to "burn in" the drive, basically what happens is it runs the drive through intensive usuage for x amount of cycles and using smart you can see if any arrors occured, if there are any bad sectors, if data had to be reallocated, if a drive successfully completes this process you can be fairly confident using the drive.

HD's are mechanical devices and so the more they are used the quicker they are likely to run out, however, when a drive spins up it causes more stress than a drive constantly running. In a home setting though, this is likely to be less apparent, and the power savings are probably of more benefit.

All harddrives at some point will fail, its just a matter of when, 1 year, 5 years, 20 years? Also remember parity protected is not a replacement for backups, it should be used in conjunction.
Reply
#67
So based on your response, if a drive is working and then stored in a closet for 10 years (without ever having been powered on), it is very likely that it will work the next time you reconnect it.
Reply
#68
I'd probably run it through its paces before trusting important information to it, but yes, barring things like fire or water damage etc, it *should* be fine
Reply
#69
Bongu Wrote:So based on your response, if a drive is working and then stored in a closet for 10 years (without ever having been powered on), it is very likely that it will work the next time you reconnect it.

But remember that a healthy backup stragtegy accounts for timely updates on the backups, and most important, testing the restore process. Never trust a backup if you never succesfully restored it.

There is no reason to risk and let the disk "rotting" for ten years. Chances are that the software you used to backup the system may be the problem in 10 years.

My choice was to go for a non-RAID setup relying on a fully offsite backup which I frequently update and test.
Reply
#70
rbonon Wrote:But remember that a healthy backup stragtegy accounts for timely updates on the backups, and most important, testing the restore process. Never trust a backup if you never succesfully restored it.

There is no reason to risk and let the disk "rotting" for ten years. Chances are that the software you used to backup the system may be the problem in 10 years.

My choice was to go for a non-RAID setup relying on a fully offsite backup which I frequently update and test.

My backups are mainly for media (movies and pictures) so I won't use any software to do a backup.
Reply
#71
Excellent.
Now we have covered that backup is good, disks can break and some people have usb disks both for media and backup. Good know.
It is not my thread but I really like reading all the interesting information about multi TB storage, redundancy and backup solutions(or lack off). Perhaps we should skip the news about my new usb disk and return to the good stuff?Smile

BTW, I used to backup all my stuff on a 5,25 floppy but it doesn't fit in the asrockSad
Reply
#72
Bongu Wrote:As I've been reading more about NAS, RAID, Backups and Drive Failures it really is intimidating seeing how certain people are that drives will fail. Do drives fail because they get old or because they are used so much? I realize that anything is possible so I'm asking about generalities here.

For myself, I'm not certain that any of the drives will fail any time soon, but I know there's a possibility that they might fail. And for spending the money on an extra couple of drives to have it mirrored helps to minimize the chance I lose it all. It'd be an enormous pain to restore my media library and it makes the money spent now worth it!
Reply
#73
what is the best to buy ? an etrayz or sheevaplug?
Reply

Logout Mark Read Team Forum Stats Members Help
Best way to store a large media library?1