[TugboatBill]

So if my XBMC client want a movie from a ZFS system does it spin up all the drives or just the one the movie is on?

I have a share "Movies" on a ZFS system. I rip to it and eventually I fill it up. Can I just add another drive to it (of a larger capacity)? Is it just a simple prep the drive (format) and tell ZFS that it's available to the Movies share?

[darkscout]

No, ALL the space isn't available. But if ALL the space is available then unRAID isn't backing things up. Meaning if you lose the drive, you lose data.

[TugboatBill]

No, ALL the space isn't available. But if ALL the space is available then unRAID isn't backing things up. Meaning if you lose the drive, you lose data.

? I'm sorry, is that a response to my query?

[fonzie]

http://www.youtube.com/watch?v=11BwLs3pHF4


jeez, this is much more information than I was expecting.

I think I'm more confused now than before I posted. haha.

[darkscout]

No.

But if you fill up all your drives. (meaning your parity drive too), and just toss in another drive. Anything you put on that drive isn't backed up.

You can add another 2 drives and add the 2 drives in mirror to the pool.

[gabbott]

No.

But if you fill up all your drives. (meaning your parity drive too), and just toss in another drive. Anything you put on that drive isn't backed up.

You can add another 2 drives and add the 2 drives in mirror to the pool.

The parity drive on unraid is never "full" so to speak. If you add another drive, it is protected by parity.

[TugboatBill]

The parity drive on unraid is never "full" so to speak. If you add another drive, it is protected by parity.

True. Adding another drive forces a rebuild of the parity as the parity is a checksum calculation of all the data drives. That's why the parity drive has to be at least as large as the largest data drive.

[bigdog66]

I could be reading this wrong but i am in the middle of gathering parts for my media server

Obviously everyone's concern is trying to protect this data the best they can.....

I dont have anything past some vital parts ....so no software loaded so I can say i am not biased about anything yet.......BUT....lol

if i pretty much have to have all of my data drives bought and paid for before i can build anything.........i would have to say thats doo doo....lol

and if i have to buy double the data drives to really have protection....then that is def doo doo

if i have to have double the data drives to have half or less of that space avail then i might as well save some money and reuse my old xp box and just use Acronis True Image and backup my true "data drives" with my other "data".....i mean "mirror"....i mean BACKUP drives

and in fact Acronis True Image Universal Restore works pretty well though sometimes it takes a little nudging .....if something like my mobo fried and i needed to restore my os data on the new mobo

so considering the advantages and disadvantages I found i decided on unraid

but that day I build a data center in my house then im sure i will go in a diff direction......

[darkscout]

Disadvantages include slower performance than any single disk in both read and write, slow drive rebuild, filesystem overhead (additional checksums are required to avoid querying the other disks to check the data disks in use), scaling problems, much larger IO burden on the parity drive than other drives, bottlenecking when multiple drives are used concurrently. The parity drive must be at least as large as the largest data drive to provide protection. UnRAID is implemented as an add-on to the Linux MD layer.

I've said it once. I'll say it again.

If you want just a dedicated Media Server. unRAID is great. If you lose a disk or have bit rot. Then you will have to re-download or re-rip your media. Even worse ReiserFS's main developer was arrested in 2008. v4 hasn't even been merged into Linux main.

If you want to build a home server for everything, including stuff you can't afford to lose (always backup, offsite, too): Pictures, Documents, and also your media. ZFS with raidz2 or raidz3 can't be beat. I scrub weekly. Snapshot daily. Snapshot every time I rsync from my MacBook all my important documents.

[bigdog66]

it sounds like you think that if you lose a disk in unraid you lose data....

if you lose a data disk or parity disk or even your mobo you dont lose data...

and of course your stuff you couldn't replace (family photo's,etc) you would make multiple copies of and even an off site copy

[darkscout]

What happens when you get bit-rot? What happens when your drive thinks that it is good but you're just replicating and taking parity of bad data? What happens when you lose 2 drives?

[PANiCnz]

Put that down as a disadvantage to ZFS then (unless you like to waste the hours away fiddling).

It doesn't take time to configure, with a product like FreeNAS you can have the whole thing configured and operational in under 15 minutes.

What I was trying to allude to is that some of the ideas behind ZFS are a bit complicated e.g. its not a file system, its not RAID etc

The whole idea of an combined file system and logical volume manager is something that's relatively new to the tech world and I think it scares some people. I think that might be why they like unRAID its easy to understand how it works. I know when I was first researching my NAS terms such as zpools and vdev's were pretty confusing.

[poofyhairguy]

What happens when you get bit-rot? What happens when your drive thinks that it is good but you're just replicating and taking parity of bad data? What happens when you lose 2 drives?

Well, then you re-download the re-rip the stuff I guess. As you said, Unraid is best for a Mediaserver, something we agree on.

I will say that on Unraid's forum there are many users that have used the system for over 4 years (just upgrading the arrays one hard drive and mobo at a time) and not once have I read about any "bit rot." Just lots of happy customers with no data loss.

What I think you are referring to (in other comparisons I have read about ZFS vs traditional RAID) is the infamous RAID 5/6 "write hole." This problem does not affect Unraid, because Unraid is based on RAID 4. In fact, its safer than RAID 4 because its not stripped, and in most cases of data corruption in normal RAID comes from the striping of the data.

You are correct that Unraid does not verify the parity to avoid "bit rot" if it did occur, and again I would never use it for a business system for that and other reasons.

But for media collections, Unraid really fits the bill. No stripped RAID (ZFS or otherwise) system allows you to mix and match hard drive sizes, power down not used hard drives, or pull the hard drives out and read them in another computer.

I personally am not scared of having more than one HD fail at a time because I always watch the SMART data my Unraid box emails to me and because I mix and match drives to avoid bad batches. In fact in my newest Unraid box I am trying to collect every 1.5tb and 2tb drive on the market into a single array (and I am a WD Black 2TB away from succeeding) to avoid any one model or batch being a dude. You can't do that with ZFS.

For stuff I care about (pictures/documents/etc) I personally use a 2 disk NAS in RAID 1 backed up by a external HD I keep at work. I personally think that the data that matters is so small compared to my media collection, that I would rather use different solutions for each. Some people might not agree and want a single server. Those people should use ZFS. Just as you said.

[bigdog66]

i was wondering if it was just my noobtasticness that had never heard of bit rot....haha
thanks for making me feel better......though i still don't question my noobness in all of this still......

oh.....and send me some of those extra drives when you get through testing...lol

[darkscout]

ZFS is a very "abstract" concept.

You have Pools. "Pools" are just massive sets of data.

To pools you can add "vdevs", virtual devices.

To virtual devices, you can add disks.

When adding disks, you can add them alone. You can add them mirrored. You can add them in raidz1, raidz2, raidz3.

It is designed for enterprise settings, but it is easily adapted for the home.
----------
A 'typical' setup:

I have 4 drives. I combine them as a ZFS raidz1 vdev. This means that any 1 drive can fail and I still have backup.

I add that vdev to pool "tank". "tank" is now a massive pool that contains all 4 of my drives in raidz1.

Out of this I can create [nearly] infinite file systems: Such as "tank/Movies" "tank/TV" "tank/Pictures", etc. Every single one of those can be created. I can create virtual devices that can be formatted as UFS, ext2, ext3. Literally anything.

Somehow I manage to fill my vdev, and [and since my first vdev is the only vdev in the pool], the entire pool.

So "tank" is entirely full. That means every "zfs" that is created from that is full. Movies, TV, Pictures, etc.........

With ZFS, I can create a new vdev. So my budget is limited, so I can only add 2 new drives. I add "drive1" and "drive2" to a new vdev in mirror. I can add this new vdev to the pool. So I will add the size of one disk to the pool (since they are mirrored).

You could also add another 2, 3, 4, 5, 100 drives in raidz1, raidz2, raidz3 to the mirror. Without doing anything, tank/Movies just got larger by the size of the drives we added to it.

ZFS also checks for bit rot. It checks the integrity of every single bit. If any byte of a copy is duplicated, ZFS will make sure that you don't needlessly duplicate files. Say I copy 1MB tank/Pictures/1.jpg to tank/Pictures/2.jpg. On normal RAIDs (and unRAID included), you will use 2MB of data. With ZFS you still only use 1MB.

Again. I'll repeat everything I said in this post.

* Sorry if I screwed the terms up, it's been a while since I've studied them. Literally set and forget.
** Glad to see the ZFS advocates finally come out. I felt like I was a loner for a while