Unraid "Math"
#31
(2012-03-22, 17:29)Bstrdsmkr Wrote:
(2012-03-22, 16:44)TugboatBill Wrote:
(2012-03-22, 15:34)Tycho91 Wrote: Why don't you use RAID 5? RAID 5 places spreads the parity over all the disks

And it spins all the disk all the time and if 2 drives fail you lose the entire array and all the data on it. Raid 5 is an excellent solution for a business server. For a media NAS it isn't very good.

Worse then that, if it does survive a long time, then you're in even worse trouble. If they stop making your RAID controller, then it dies, better hope you can find that model on ebay or your whole array is blown =\
This is my biggest problem and why I'm very much OK with software RAID. For a consumer, I don't see any real need for a "proper" RAID. If you got money to blow, sure, there's nothing wrong with it, but I don't think most people need it.

Reply
#32
Thanks for reminding me why I chose unraid!

Another problem with RAID 5 is that it is a pain to expand your array. With unraid you just get another drive and just drop it in. It's laughable how easy it is.
HTPC: Win 7 Home 64-bit | MB | CPU | GPU | RAM | Case | PSU | Tuner | HDDs: OS, Media | DVD Burner | Remote
Media server: unraid 4.7 | CPU | MB | RAM | Case | PSU | HDDs: Parity-2TB, Data-2x2TB
Reply
#33
I've been using unraid for about 4-5 months and i find it really good.
although I dont understand how the parity can back up more data than it can store?
if i got 3x2tb, and drive 1 (full) fails - how can the parity have a back up if disk 2 is also full of data?
Reply
#34
Algebra in laymans terms..

For example... 1 + 2 = 3 (and if you want to get really crazy, 1 + 2 + 3 + 4 + 5 = 15)

1 is a data drive, 2 is a data drive and 3 is a parity drive.

You can solve x + 2 = 3, or 1 + 2 = x fairly easily.

Imagine x is a failed drive. Same applies to 1 + x + 3 +4 + 5 = 15. Now, it's WAY more complicated than that... I imagine a lot is done at the bit level, and multiple parity drives complicate it significantly I'm sure. But that's how I understand it.
Reply
#35
Unraid is pretty nice but i prefer flexraid
Image

If my replies help you, please click on my reputation Image below :) thanks :)
Reply
#36
(2012-03-22, 18:42)stiggs Wrote: I've been using unraid for about 4-5 months and i find it really good.
although I dont understand how the parity can back up more data than it can store?
if i got 3x2tb, and drive 1 (full) fails - how can the parity have a back up if disk 2 is also full of data?

Check out this from the Unraid Wiki:

http://lime-technology.com/wiki/index.ph...ty_work.3F
Reply
#37
(2012-03-22, 18:42)stiggs Wrote: I've been using unraid for about 4-5 months and i find it really good.
although I dont understand how the parity can back up more data than it can store?
if i got 3x2tb, and drive 1 (full) fails - how can the parity have a back up if disk 2 is also full of data?

The easiest way to understand this is that the parity drive doesnt actually have a 'back-up' of what is on each disk, but instead 'knows' what is on each disk if one fails or goes missing.

Say you have 3 2tb disks as in your system.
(A 2tb disk has 2 trillion 'slots' where either a 0 or 1 is written.)

The parity disk reads the info from each of the slots on each disk in your system and adds them all together and either leaves them alone, or adds a '1' to make it an even number.
So say slot 4 equals '4', its an even number and Unraid leaves it as is and remembers it is even. Now say slot 716 equals 3, unraid would add a '1' to it to make it even. It does this for each of the 2 trillion slots, or however many the HD's you are using contain.
Now say one of your data disks crash or you pull it out. You then put a blank disk in and tell unraid to 'rebuild' it.
UnRaid would then look at each of the 'slots' on the disk and add them up. If each location or slot equal an even number it would write a zero, if its an odd number it would write a '1'.

This way UnRaid calculates what 'should' be there and re-writes it without ever actually knowing what was there! (Unraid uses 'even parity'.)

At least this is my understanding........

Interesting fact: UnRaid is based on the ReiserFS files system...written by Hans Reiser...whos now in jail for killing his wife.
Reply
#38
(2012-03-05, 22:24)skyking376 Wrote: 4) With anybody's experience what is the percentage/average that 2 drives fail in the same day/week? 2%, 10%, 50%, What?

That depends how often you use your hard drive box as a coffee table...cause the odds of more than one drive failing in a week I will personally guarantee with my own money that unless you damage them either electrostatically or by water boarding your hard drives...and we all know water boarding is wrong...simply won't happen...seriously tho any mirrored raid setup is pretty damn hard to go wrong with...redundancy is in the name after all...I personally go the raid-0 route (i know...not really raid) but I'm pretty confidant in both my hard drives and my motherboards ability to tell me there is something wrong with my hard drive before anything goes wrong. (it correctly warned me twice already, but those were 6 year old HDDs and now all of mine are new now)
Reply
#39
Opening post pretty much had it right but I'll say that losing a Parity drive isn't any big disaster any more than losing any of the other drives. Nothing so special about it in other words. The posts above about the math to correct for a drive loss are correct - unRAID uses binary math to figure out what should have been in a spot that's been vacated. This is also why the Parity drive must be as big or bigger than any of the data drives. Standard RAID, in addition to the issues with losing the entire volume, has the added disadvantage of requiring you to use drives that are all the same size or losing data on larger ones when small ones are used. unRAID on the other hand can use all sorts of sizes so long as Parity is sized equal or larger.

As for a dual failure... I've been running unRAID for at least 5 years. I have had multiple drives fail but never ever two at one time. It's true that during a rebuild is the most risky time because all of the drives are working hard but I've yet to have a dual failure. Remember too that the F/S is standard ReiserFS which means that standard recovery tools can be used. There's a reason why disk recovery places advertise if they can handle RAID or not - $$$$$!

Last but not least - cache drives. Waaaay back when unRAID wasn't nearly as fast as it is now. Writes to the disk were far slower and a feature was added to allow full speed writes to a cache drive. then a script would run in the background and move the data onto one of the drives that was protected. Prior to the move the data wasn't protected. Then one day a revised version of SAMBA was added to unRAID and speeds doubled if not more! Now I can move data to my array as fast as I can to a cache disk. Cache disks no longer make sense IMO other than to use them as data storage for other kinds of programs. Sickbeard etc. is a good example of that but since I've had really rough issues trying to move to the new beta I'm not trying to run it on either of my unRAID machines right now.

Honestly my biggest fears are lightning, fire, or some punk stealing my servers. My requests for a full volume encrypted file system option for unRAID have been blown off so that's one thing you won't be getting from unRAID and no encfs isn't what I want :-(
Openelec Gotham, MCE remote(s), Intel i3 NUC, DVDs fed from unRAID cataloged by DVD Profiler. HD-DVD encoded with Handbrake to x.264. Yamaha receiver(s)
Reply
#40
Anyone know if it's possible or worth it to setup a ram disk for a cache drive? I know it's not the most practical thing in the world, but it seems like a fun idea
Reply
#41
Not worth it IMO, see my posting above. The cache drive adds little to no speed with the builds of unRAID that everyone is using. It seems to end up as a data drive for other programs these days and I wouldn't advise using one otherwise.
Openelec Gotham, MCE remote(s), Intel i3 NUC, DVDs fed from unRAID cataloged by DVD Profiler. HD-DVD encoded with Handbrake to x.264. Yamaha receiver(s)
Reply
#42
(2012-03-05, 22:36)TugboatBill Wrote: The odds are that if you have 2 drives fail at once one of them will be the parity drive. Why? Because it gets the most use as it is updated every time there's a change to any of the drives in the array.

The odds of a second drive failing at the same time are high, because often multiple DRIVES FAIL FOR THE SAME REASON. Just like the nuclear reactor meltdown in Fukushima, which had backups, but all the backups failed due to the tsunami, the same thing happens with other systems too.

Same thing with hard drives. If you power supply starts producing out of spec power, if there is a surge, if someone kicks over the server and it falls to one side, if someone spills coffee on it, if the operating temperature range of the drives is exceeded, or any number of a million possible things that could cause a drive to fail, in almost all of these cases it is highly likely that a second drive will be affected by the same thing that affected the first one.

The odds of an average, functional hard drive failing for no reason other than old age, on it's own, is less than 5% per year (some drives it's as low as 1%), so basically two drives failing at the same time for that reason is rarely going to happen. But this is far lower than the combined odds of all the other possible situations that could cause a failure. In reality two or more drives often do fail at the same time, like when rebuilding arrays due to bit errors. There are other situations also where a failure actually existed before, but you only discover it at the same time because you only try to access the data when you need it.

In a carefully, professionally, and well managed hosting environment, drive failures can be kept at below 5% per year. But for other types of users, the odds of two drive failures are sky high (more than 10%).

So yes a parity drive is being accessed at the time of drive failure slightly increases the risk of that drive failing also, but think of all the other risks combined that can cause data corruption or outright failure, they are even higher, much higher than you think.

We can easily predict the odds easily of a particular failure - but how well can you predict the overall risk, from all causes?

My point is without a second parity drive option, once you have lots of data more than 10TB, a single parity drive just isn't enough. When one drive fails and you need to rebuild the odds of a second failure are astronomical, it's practically guaranteed that you will lose data if you only have single parity.

So I really hope they add this feature, it's needed.
Reply
#43
Lots of stuff covered in here, just want to add my .02

Love my unRAID box, been going strong for a long while now.

unRAID is software raid, and raid in general only offers you fault tolerance. In this case, one hdd failings worth. As mentioned, at least if multiple drives go belly up at the same time, you only lose the ones that actually fail.

Also, its not a backup. If you want to make sure to not lose your data, you need a real backup. If you don't want to rerip tons of dvds/blurays/cds/etc you need a real backup.

Whats the best way to backup 6TB of data (say 3 2tb hdds in your array) ? thats up to you, but a pair of 4tb externals would work very nicely.

How do you backup 20TB of data (say 5 4tb hdds) ? i'd suggest you build a 2nd cheap server, load it up with hdds, and sync the two up on some basis you find acceptable.

Now, if you just DL all your media in the first place, well, just redownload what you ACTUALLY want if it all goes to hell.
Reply
#44
(2014-03-17, 09:55)gaffe Wrote:
(2012-03-05, 22:36)TugboatBill Wrote: The odds are that if you have 2 drives fail at once one of them will be the parity drive. Why? Because it gets the most use as it is updated every time there's a change to any of the drives in the array.

The odds of a second drive failing at the same time are high, because often multiple DRIVES FAIL FOR THE SAME REASON. Just like the nuclear reactor meltdown in Fukushima, which had backups, but all the backups failed due to the tsunami, the same thing happens with other systems too.

Same thing with hard drives. If you power supply starts producing out of spec power, if there is a surge, if someone kicks over the server and it falls to one side, if someone spills coffee on it, if the operating temperature range of the drives is exceeded, or any number of a million possible things that could cause a drive to fail, in almost all of these cases it is highly likely that a second drive will be affected by the same thing that affected the first one.

The odds of an average, functional hard drive failing for no reason other than old age, on it's own, is less than 5% per year (some drives it's as low as 1%), so basically two drives failing at the same time for that reason is rarely going to happen. But this is far lower than the combined odds of all the other possible situations that could cause a failure. In reality two or more drives often do fail at the same time, like when rebuilding arrays due to bit errors. There are other situations also where a failure actually existed before, but you only discover it at the same time because you only try to access the data when you need it.

In a carefully, professionally, and well managed hosting environment, drive failures can be kept at below 5% per year. But for other types of users, the odds of two drive failures are sky high (more than 10%).

So yes a parity drive is being accessed at the time of drive failure slightly increases the risk of that drive failing also, but think of all the other risks combined that can cause data corruption or outright failure, they are even higher, much higher than you think.

We can easily predict the odds easily of a particular failure - but how well can you predict the overall risk, from all causes?

My point is without a second parity drive option, once you have lots of data more than 10TB, a single parity drive just isn't enough. When one drive fails and you need to rebuild the odds of a second failure are astronomical, it's practically guaranteed that you will lose data if you only have single parity.

So I really hope they add this feature, it's needed.


And that is why you have a backup.
Reply

Logout Mark Read Team Forum Stats Members Help
Unraid "Math"0