NAS upgrade

[luckylinux]

I'm currently planning on upgrading my Home NAS / server on ZFS.

Current configuration:
- FreeBSD 9.1
- 1 x E5-2620 6C/12T CPU
- 64GB ECC registered RAM
- 2x2.0TB Mirror (ROOT) + 6x3.0TB RAIDZ-2 (DATA) -> 12TB usable space

I'm planning on leaving FreeBSD because I don't really feel at ease with managing it (I don't have enough experience probably). Furthermore it doesn't provide any virtualization solution (other than Virtualbox which is what I'm using). A possible solution would be to make an all-in-one ESXi host, leaving storage to openindiana/omnios + napp-it. Another one would be to use ZFSOnLinux and use a GNU/Linux distribution and virtualize using KVM.

Next configuration:
- Gentoo GNU/Linux + KVM virtualization
- 1 x E5-1650 (v1/v2 depending on availability)
- 64GB registered ECC RAM or 128GB
- 2x240GB SSD (ROOT) + 2 x (6x3.0TB RAIDZ-2) (DATA) -> 24TB usable space

For the SSD I settled and already bought 2x240GB Crucial M500 thanks to their integrated power outage protection.

The currently used HDDs are a mix of WD RED, WD Black and TOSHIBA DT01ACAxxx, hence all Desktop drives.

For the new NAS I'm not sure if I should go with entreprise drives (Such as WD SE or RE4, HGST/Hitachi Ultrastars, TOSHIBA entreprise) or stick with desktop drives.
From what I understood (and I'm not an expert in this field, so help me if you can ) entreprise drives have the following advantages over desktop drives:
- Longer warranty (5 years instead of 2/3 years in most cases)
- TLER/CCTL (Error Recovery Time limits)
- ECC memory buffer (Eccor Correcting Codes)

From these TLER/CCTL doesn't seem much of an issue to me since I'd be using software RAID (with ZFS). Longer warranty is interesting but I could get the same with WD Blacks for a lower price.

What really worries me is ECC: is it really so powerful / interesting or are we kind of talking about 1 bit flip in a day (same as ddr3 ECC)?
What really determines if entreprise drives are worth is how your data is worth. Well, while I admit I have some important data (personal photos, documents, ...), it's not some million dollars business or anything like that.

What would you therefore reccomend: entreprise or consumer drives

And would you agree in mixing different manufacturers / models in a vdev in order to lower the risk of simultaneous failures? I always heard it is bad idea to build huge pools of disks in RAID environments from the same batch of drives because if the batch was flawed many drives would fail before the array could be rebuilt .

 

[MrCaffeineX]

I'm currently planning on upgrading my Home NAS / server on ZFS.

Current configuration:
- FreeBSD 9.1
- 1 x E5-2620 6C/12T CPU
- 64GB ECC registered RAM
- 2x2.0TB Mirror (ROOT) + 6x3.0TB RAIDZ-2 (DATA) -> 12TB usable space

I'm planning on leaving FreeBSD because I don't really feel at ease with managing it (I don't have enough experience probably). Furthermore it doesn't provide any virtualization solution (other than Virtualbox which is what I'm using). A possible solution would be to make an all-in-one ESXi host, leaving storage to openindiana/omnios + napp-it. Another one would be to use ZFSOnLinux and use a GNU/Linux distribution and virtualize using KVM.

Next configuration:
- Gentoo GNU/Linux + KVM virtualization
- 1 x E5-1650 (v1/v2 depending on availability)
- 64GB registered ECC RAM or 128GB
- 2x240GB SSD (ROOT) + 2 x (6x3.0TB RAIDZ-2) (DATA) -> 24TB usable space

For the SSD I settled and already bought 2x240GB Crucial M500 thanks to their integrated power outage protection.

The currently used HDDs are a mix of WD RED, WD Black and TOSHIBA DT01ACAxxx, hence all Desktop drives.

For the new NAS I'm not sure if I should go with entreprise drives (Such as WD SE or RE4, HGST/Hitachi Ultrastars, TOSHIBA entreprise) or stick with desktop drives.
From what I understood (and I'm not an expert in this field, so help me if you can ) entreprise drives have the following advantages over desktop drives:
- Longer warranty (5 years instead of 2/3 years in most cases)
- TLER/CCTL (Error Recovery Time limits)
- ECC memory buffer (Eccor Correcting Codes)

From these TLER/CCTL doesn't seem much of an issue to me since I'd be using software RAID (with ZFS). Longer warranty is interesting but I could get the same with WD Blacks for a lower price.

What really worries me is ECC: is it really so powerful / interesting or are we kind of talking about 1 bit flip in a day (same as ddr3 ECC)?
What really determines if entreprise drives are worth is how your data is worth. Well, while I admit I have some important data (personal photos, documents, ...), it's not some million dollars business or anything like that.

What would you therefore reccomend: entreprise or consumer drives

And would you agree in mixing different manufacturers / models in a vdev in order to lower the risk of simultaneous failures? I always heard it is bad idea to build huge pools of disks in RAID environments from the same batch of drives because if the batch was flawed many drives would fail before the array could be rebuilt .

I was always told the opposite about RAID disks: use the same drives from the same series/family to ensure that they are all operating at the same performance level. This may have been more important with older RAID controllers or for fake RAID setups and may no longer be an issue. Now you have me curious...

When building RAID arrays, I have always used enterprise drives when reliability was important. I have a number of WD RE2, 3, and 4 drives that are still going strong, some for a number of years. I have had nothing but bad luck with the lower-end consumer grade drives (WD Blue, Seagate Barracuda) in general and my experience has been mixed with the prosumer level drives (WD Black, Hitachi Deskstar). Samsung and Maxtor were good for me when they were around as well.

 

[MrGuvernment]

there was a really good article on TLER/CCTL and how that isnt the only factor that matter for raid drives as everyone thinks it does and it doesnt play as much into drives dropping from arrays, now to find the darn thing!

i love Hitchi drives, have i think now about 30 of them in servers and only had 4 fail over the last..3-4 years?

 

[luckylinux]

I was always told the opposite about RAID disks: use the same drives from the same series/family to ensure that they are all operating at the same performance level. This may have been more important with older RAID controllers or for fake RAID setups and may no longer be an issue. Now you have me curious...

Well surely my configuration is not optimal for performance. If a drive batch was flawed I think you'de be more likely to have issues than me.

When building RAID arrays, I have always used enterprise drives when reliability was important. I have a number of WD RE2, 3, and 4 drives that are still going strong, some for a number of years. I have had nothing but bad luck with the lower-end consumer grade drives (WD Blue, Seagate Barracuda) in general and my experience has been mixed with the prosumer level drives (WD Black, Hitachi Deskstar). Samsung and Maxtor were good for me when they were around as well.

So you'd take WD SE / RE over WD RED/BLACK any day?

I could get:
- WD RED 3.0TB ~ 145$ 3 year warranty
- WD Black 3.0TB ~ 215$ (+50%) 5 years warranty
- WD SE 3.0TB ~ 215$ (+50%) 5 years warranty
- WD RE 3.0TB ~ 245$ (+70%) 5 years warranty

Not sure about WD SE vs RE series though ...
I'd say WD SE is still affordable (same as WD Black) although quite a bit more expensive than WD RED (with the plus of a longer warranty though).

 

[luckylinux]

I'm also wondering about another thing which would allow me to reuse some hardware (Chassis, CPU, motherboard and RAM) but with less HDDs.

What would be the performance impact of using 8 HDDs in a raidz2 configuration compared to "better" configuration such as 6 HDDs or 10 HDDs? If I lose 10% in performance I wouldn't care much about it.

That way I could setup 2x240GB mirror for ROOT and 8x3.0TB or 4.0TB for DATA.
If the performance impact is very severe (-50% for instance) then I should go with 2x(4xHDDs in raid2 each).

Since these are mainly photos, videos and documents and I'd be bottlenecked by the network speed (1gbit) anyway I don't think the performance impact of a sub-optimal configuration would be severe ...

What would you think?