2010-03-20, 00:20
Just re-deploy an old redundant PC, install Ubuntu (whatever flavour, my favourite is Mythbuntu for my specific use case), don't spend any cash on anything but a bunch of huge hard drives.
Have fun.
Have fun.
2010-03-20, 07:59
Intel Scaleo home server
Add 4 1.5 or larger drives
Cheap, fast, quiet.
aka Intel SS4200
Raid, DOC Linux OS, upgradeable to 2gb ram, e-sata
Low power usage
4.5 TB of storage in Raid 5 for ~$500
http://www.legendmicro.com/store/7560_IN...msp?RID=23
Add 4 1.5 or larger drives
Cheap, fast, quiet.
aka Intel SS4200
Raid, DOC Linux OS, upgradeable to 2gb ram, e-sata
Low power usage
4.5 TB of storage in Raid 5 for ~$500
http://www.legendmicro.com/store/7560_IN...msp?RID=23
XBMC Flicks - Netflix for XBMC http://forum.xbmc.org/showthread.php?tid=87552
XBMC Guide for adding Movies/Media
Universal Media Manager - http://universalmc.sourceforge.net/
Media Info Plus -(old product) http://sourceforge.net/projects/movieinfoplus/
2010-03-20, 18:50
I would caution everyone about RAID5. Keep in mind that if you lose 2 drives you lose the entire array and all the data on it. The next thought is "I'll replace a bad drive immediately if one fails". That may not save you.
I work in IT and recently lost a RAID5 array. This array was on a server that ran 24*7*365 (linux so no shutdowns). A drive failed. Up to that time we had no SMART indications anything was wrong. We rebooted the server into maintenance mode to access the raid controller. Another drive then failed and refused to come back up. Without a backup (which we had) the entire array was lost including all the data on it.
The important point to consider is that with a RAID 5 array (as well as RAID0) you still need backups to protect your data. If you have all the original media then you have this backup. If you don't, backups get to be very expensive. Online services would probably be expensive (I assume they charge by the MB) as is tape or additional hard drives.
If you don't have a way to do backups then Unraid and FlexRaid become a better solution. They use a single parity drive. If a drive is lost you can replace it and not lose any data (as RAID5). If you lose 2 drives you lose only some of your data.
Unraid and Flexraid are designed as media storage systems. They are a lot easier to expand. Many RAID5 systems cannot be expanded or if they can require a drive that matches the existing drives. So if your array (RAID5) is made up of 4 - 500GB drives and you want to expand you have to buy another 500GB drive, even if that 1.5Tb is cheaper. Unraid and Flexraid (if I understand flexraid correctly) allow you to just add another drive, no matter what size it is. Unraid does require the parity drive be as large as the largest drive in the array but making that change is well documented and doesn't require moving any data off the array.
Also know that unraid and flexraid (I think) will automatically spin down drives if they're not being accessed. This cuts down your power usage and extends the drive's life. RAID5 keeps them spinning.
RAID 5 is faster for reads and writes. You'll be able to store that BR rip much faster. Reads don't really matter as all of them are plenty fast for 1080p playback.
I work in IT and recently lost a RAID5 array. This array was on a server that ran 24*7*365 (linux so no shutdowns). A drive failed. Up to that time we had no SMART indications anything was wrong. We rebooted the server into maintenance mode to access the raid controller. Another drive then failed and refused to come back up. Without a backup (which we had) the entire array was lost including all the data on it.
The important point to consider is that with a RAID 5 array (as well as RAID0) you still need backups to protect your data. If you have all the original media then you have this backup. If you don't, backups get to be very expensive. Online services would probably be expensive (I assume they charge by the MB) as is tape or additional hard drives.
If you don't have a way to do backups then Unraid and FlexRaid become a better solution. They use a single parity drive. If a drive is lost you can replace it and not lose any data (as RAID5). If you lose 2 drives you lose only some of your data.
Unraid and Flexraid are designed as media storage systems. They are a lot easier to expand. Many RAID5 systems cannot be expanded or if they can require a drive that matches the existing drives. So if your array (RAID5) is made up of 4 - 500GB drives and you want to expand you have to buy another 500GB drive, even if that 1.5Tb is cheaper. Unraid and Flexraid (if I understand flexraid correctly) allow you to just add another drive, no matter what size it is. Unraid does require the parity drive be as large as the largest drive in the array but making that change is well documented and doesn't require moving any data off the array.
Also know that unraid and flexraid (I think) will automatically spin down drives if they're not being accessed. This cuts down your power usage and extends the drive's life. RAID5 keeps them spinning.
RAID 5 is faster for reads and writes. You'll be able to store that BR rip much faster. Reads don't really matter as all of them are plenty fast for 1080p playback.
2010-03-20, 19:23
Quote:Unraid and Flexraid are designed as media storage systems. They are a lot easier to expand. Many RAID5 systems cannot be expanded or if they can require a drive that matches the existing drives. So if your array (RAID5) is made up of 4 - 500GB drives and you want to expand you have to buy another 500GB drive, even if that 1.5Tb is cheaper. Unraid and Flexraid (if I understand flexraid correctly) allow you to just add another drive, no matter what size it is. Unraid does require the parity drive be as large as the largest drive in the array but making that change is well documented and doesn't require moving any data off the array.
Also know that unraid and flexraid (I think) will automatically spin down drives if they're not being accessed. This cuts down your power usage and extends the drive's life. RAID5 keeps them spinning.
This is very interesting. Thank you.
There is also freeNAS / software raid and an ongoing project to create something like freeNAS but on linux - OMV.
I would like a software solution for the reasons you mentioned but I am concerned about os failure. Do you have any understanding about how the data is stored in the different solutions? If the server breaks how easy/possible is it to retreive the data and/or rebuild.
From what I can understand flexraid would not have to be rebuild in that sense since it seems to be a client/server solution.
2010-03-20, 20:19
yup, no raid, or raid-like solution is a replacement for backups, and backups won't help from fire/flood/theft/etc if they are all stored in one location.
I went with raid-6 (x-raid2/dual redundancy) on my new nas, I lose a 2nd drive worth of usable space, but the array can lose 2 drives before dataloss.
eventually my plan is move my old nas to a different location and setup a vpn to sync the most important stuff (not all will fit on old).
media files, while large and a pain in the butt to rip, are replaceble.. home videos and other important data is not, choose your backup strategy wisely.
I went with raid-6 (x-raid2/dual redundancy) on my new nas, I lose a 2nd drive worth of usable space, but the array can lose 2 drives before dataloss.
eventually my plan is move my old nas to a different location and setup a vpn to sync the most important stuff (not all will fit on old).
media files, while large and a pain in the butt to rip, are replaceble.. home videos and other important data is not, choose your backup strategy wisely.
2010-03-20, 23:19
Unraid uses its own linux OS. If for some reason the usb drive the OS is on fails you can recover all the data. It would require setting up a new USB boot drive and the telling it which drive is the parity and which HD is which drive in the array (may not have to be accurate as far as data drives - not sure about this).
I'm not familiar with flexraid and freenas to give any advice - it's best to check their forums.
RAID5 is usually hardware based. If the OS dies you're ususally ok. If the raid controller fails there's a good chance you will lose everything. It depends on the controller. If the controller is old and they don't make it any more then it's nearly certain you lost everything.
I'm not familiar with software RAID, but I'd guess that if the OS goes the raid is probably gone with it.
The best method is to pick one (or two) that are close to what you're looking for and then determine if they have the tools, features, and data recovery tools that work for you. Each has its strengths and weaknesses.
I'm not familiar with flexraid and freenas to give any advice - it's best to check their forums.
RAID5 is usually hardware based. If the OS dies you're ususally ok. If the raid controller fails there's a good chance you will lose everything. It depends on the controller. If the controller is old and they don't make it any more then it's nearly certain you lost everything.
I'm not familiar with software RAID, but I'd guess that if the OS goes the raid is probably gone with it.
The best method is to pick one (or two) that are close to what you're looking for and then determine if they have the tools, features, and data recovery tools that work for you. Each has its strengths and weaknesses.
2010-03-20, 23:25
Another thing I should mention - No matter what system you use, plan out what you'll do if you have a minor (IE 1 drive) or major (OS/multiple/controller) failure. What I see on the forums are people posting where they have a failure, try to blindly recover, and then (after they've hosed most or all of their data) look for help. You don't want to share their experience. 
2010-03-21, 02:37
what about using raid6?
Looking at unraid and Lime Tech has anyone used there RB-1200, how is it for XBMC?
Looking at unraid and Lime Tech has anyone used there RB-1200, how is it for XBMC?
2010-03-21, 02:54
software raid can be safer than hardware raid, in that if a controller fails and you can't find an exact or compatible replacement, then the drives (while still have good data) can/may not be read by other controllers.
with software raid (either windows or linux etc), all you have to is re/load the OS and configure the raid back, if its not automatically detected. The hardware doesn't matter at all for software raid.
raid-6 uses 2 parity disks, so it can handle 2 simultanious disk failures (lose data on 3rd fail), but obviously has less usable space vs raid-5.
with software raid (either windows or linux etc), all you have to is re/load the OS and configure the raid back, if its not automatically detected. The hardware doesn't matter at all for software raid.
raid-6 uses 2 parity disks, so it can handle 2 simultanious disk failures (lose data on 3rd fail), but obviously has less usable space vs raid-5.
2010-03-21, 06:02
I haven't seen it mentioned yet, but I'm using Openfiler. I have it installed in a DIY server loaded up with Promise SATA TX4 cards. 12TB (usable) RAID-6 array with 2 hot spares.
The main draw to Openfiler for me was that it (unofficially) supported growing your Software RAID, unlike FreeNAS. It was cake to setup, and it uses mdadm (which basically the Linux standard for RAID management).
My uptime on that machine is approaching a year, and I couldn't find a problem with it if I wanted to. The hardware it's running on is an older Socket-939 FX60 and a DFI MB in a Coolmaster tower with IcyDock 4-in-1 hotswap SATA cages.
All of my drives are Samsung 1TB EcoGreen F1s (5400RPM). Low noise, low power and low heat. The server is never off, always chugging along and sending me daily heath status reports via e-mail.
Keep in mind, it's strictly a NAS. There are no non-essential services running. I have a separate Atom 330 running Debian which handles all my torrent, NZB and the like.
* Edit: As for redundancy, if the OpenFiler OS fails for whatever reason, you can recover your array simply by restoring the mdadm.conf in a new OpenFiler installation. The mdadm configuration is regularly backed up to my offsite storage via the OpenFiler web interface.
The main draw to Openfiler for me was that it (unofficially) supported growing your Software RAID, unlike FreeNAS. It was cake to setup, and it uses mdadm (which basically the Linux standard for RAID management).
My uptime on that machine is approaching a year, and I couldn't find a problem with it if I wanted to. The hardware it's running on is an older Socket-939 FX60 and a DFI MB in a Coolmaster tower with IcyDock 4-in-1 hotswap SATA cages.
All of my drives are Samsung 1TB EcoGreen F1s (5400RPM). Low noise, low power and low heat. The server is never off, always chugging along and sending me daily heath status reports via e-mail.
Keep in mind, it's strictly a NAS. There are no non-essential services running. I have a separate Atom 330 running Debian which handles all my torrent, NZB and the like.
* Edit: As for redundancy, if the OpenFiler OS fails for whatever reason, you can recover your array simply by restoring the mdadm.conf in a new OpenFiler installation. The mdadm configuration is regularly backed up to my offsite storage via the OpenFiler web interface.
2010-03-21, 06:26
In case it's not already been made clear, RAID5 or RAID6 (or any RAID setup) are just fine as long as you don't anticipate losing more than the number of drives allowed (eg RAID5 == 1 drive, RAID6 == 2 drives). If you do lose more, you lose them all.
That is the one main advantage of the "unRAID" style setups (I think WHS is similar to this?) - if you lose 5 out of your 6 drives then the other one is recoverable - there is no striping - all the data is sitting on the disk in a standard filesystem. Whether this is something you care about is up to you - I think you'll find plenty of stories where more than one disk went AWOL at or around the same time.
In the end, I think it's fair to say that commercially produced media is replaceable - it costs only time. Thus, the amount of effort you put in to ensuring it doesn't get wiped out depends on how much you value your time.
Ensure that, no matter what you do, your unreplaceable data (photos, home videos, documents, raid config etc.) is backed up well - i.e. somewhere other than the RAID.
I personally rsync my stuff to a mates place, and he does the same here - works nicely as we both have offsite backups of critical data. If either of our servers goes belly up we have a backup. The chance of both of 'em going belly up at once is pretty small.
Cheers,
Jonathan
That is the one main advantage of the "unRAID" style setups (I think WHS is similar to this?) - if you lose 5 out of your 6 drives then the other one is recoverable - there is no striping - all the data is sitting on the disk in a standard filesystem. Whether this is something you care about is up to you - I think you'll find plenty of stories where more than one disk went AWOL at or around the same time.
In the end, I think it's fair to say that commercially produced media is replaceable - it costs only time. Thus, the amount of effort you put in to ensuring it doesn't get wiped out depends on how much you value your time.
Ensure that, no matter what you do, your unreplaceable data (photos, home videos, documents, raid config etc.) is backed up well - i.e. somewhere other than the RAID.
I personally rsync my stuff to a mates place, and he does the same here - works nicely as we both have offsite backups of critical data. If either of our servers goes belly up we have a backup. The chance of both of 'em going belly up at once is pretty small.
Cheers,
Jonathan
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
2010-03-21, 17:41
Also evaluate the array technologies for expandability. RAID5/6 aren't always expandable, it depends on the controller. This could be an issue if your data needs expand past the initial design so do some research into the controller capabilities. If your motherboard has available slots you can add another array - there's no rule that says you can't have multiple RAID arrays on one OS.
2010-03-21, 21:10
I have an HP EX485 MediaSmart Server with 4.5TB of storage and absolutely love it. It offers drive pooling, folder redundancy, and a backup solution.
WHS uses drive pooling to expand the size of the data drive. To the user and the OS, all of the drives you have added to your storage pool appear as a single drive. If one of the drives fail you will lose all of the data that was stored on that physical drive, however the rest of your data will be intact. Even if you lose the OS, the drives can be connected to another computer and the data read from there.
WHS allows you to turn on data duplication to mitigate the damage done by drive failure. Data duplication ensures that each of your folders is present on 2 physical drives. While this can greatly increase your storage needs, you have the option to turn off duplication on folders if you need to.
WHS also gives you the option to, instead of adding a new HDD to the drive pool, to configure it as a backup drive. In this case WHS will sync the data to the backup drive. You can then store the backup drive elsewhere/offsite for disaster recovery.
WHS uses drive pooling to expand the size of the data drive. To the user and the OS, all of the drives you have added to your storage pool appear as a single drive. If one of the drives fail you will lose all of the data that was stored on that physical drive, however the rest of your data will be intact. Even if you lose the OS, the drives can be connected to another computer and the data read from there.
WHS allows you to turn on data duplication to mitigate the damage done by drive failure. Data duplication ensures that each of your folders is present on 2 physical drives. While this can greatly increase your storage needs, you have the option to turn off duplication on folders if you need to.
WHS also gives you the option to, instead of adding a new HDD to the drive pool, to configure it as a backup drive. In this case WHS will sync the data to the backup drive. You can then store the backup drive elsewhere/offsite for disaster recovery.
2010-03-22, 06:45
I'm a fan of a dedicated home built server. As several have mentioned, any pre-built 'server' labelled machine is going to carry a premium, but you can piece together a nice solid platform for a reasonable price.
I have a 20bay hot swap SATA case with a few year old server class motherboard and 2 3ware 12-port RAID5 cards. I run linux, so OS cost is zilch. Its not fully populated yet, but I have 14Tb spread over 2 arrays right now. I chose to use WD's green power drives specifically to keep the power overhead lower.
this approach is not for everyone - you need somewhere to put it, you need the extra OS or linux expertise, etc.
I'm not a big fan of software RAID, but it can be very powerful and expandable, especially with the new SATA port multiipliers
I have a 20bay hot swap SATA case with a few year old server class motherboard and 2 3ware 12-port RAID5 cards. I run linux, so OS cost is zilch. Its not fully populated yet, but I have 14Tb spread over 2 arrays right now. I chose to use WD's green power drives specifically to keep the power overhead lower.
this approach is not for everyone - you need somewhere to put it, you need the extra OS or linux expertise, etc.
I'm not a big fan of software RAID, but it can be very powerful and expandable, especially with the new SATA port multiipliers
