Blade Server SAN partitioning

G

gerbiaNem

2[H]4U
Joined
Mar 6, 2005
Messages
2,169
So my company is getting a new Blade server in with a DS4200 IBM SAN.

My question is this.. since no OS is installed on the SAN, how do I create raid arrays? Anyone with specific knowledge would be helpful. I was thinking about running the storage software from a linux boot usb. But if I can get into the firmware somehow, that would be much easier.
 
The DS4200 appears to be a SATA/FATA only version of the DS4500 when it comes to management from what I can see. That being said you should be able to install the storage software on any workstation for initial setup and management of the SAN. I'm guessing you are thinking you need to layout a RAID set for each blade server (correct me if I'm wrong here).

The typical best practice for most newer SAN devices when space is being presented primarily to windows boxes is to create several larger arrays out of all your disks and then span a volume across multiple arrays for optimal speed (Although it appears IBM recommends using software RAID to bundle raid sets on AIX/Linux Boxes I've been told varying information by their own techs). The exact size of the arrays is a function of the type of drive, raid type, and level of risk you are comfortable with (see SAN documentation). For instance on our ds4500s we create raid sets of 8 drives each with one drive being on a separate disk drawer. Our configuration is slightly different as far as spanning with volumes goes as we use IBM SAN Volume Controllers to front end our DS4500s. Once your raid sets and volume is created you can present it out in smaller luns to multiple machines rather than creating a dedicated raid set for each machine as that method would create hotspots and waste much of the performance of the SAN as other RAID sets would sit idle.

Ultimately I would strongly recommend you read through the IBM redbooks on DS4200/DS4000 Best Practices and performance tuning: www.redbooks.ibm.com/abstracts/sg246363.html

You want to make sure you implement your SAN setup correctly as with these lower end devices it is very difficult to move things around or resize volumes or RAID sets properly later. The DS4000 series units don't support online restriping of volumes/raid sets once they have been created. You also can cripple the performance of the device and end up purchasing far more drives down the road to compensate.
 
Thanks for the useful information.

The first project which will be implemented on the system will be a desktop backup system used to do nightly backups onto disk, with a weekly tape backup. Performance is not as great of a concern so much as reliability and redundancy.

There is one ds4200 enclosure with 16x1TB SATA drives connected via fiber to the bladecenter. The backup is the first project I'm implementing, and my current plan is as follows:

- Create a raid 6 array out of 8 drives for 6TBs of data storage (more than enough for the time being).

- With the remaining 8 drives I'm going to assign 2 as global hot spares and create another raid 6 (or perhaps raid 10 for added performance) out of the remaining 6 to store operating system data.

- I will partition the 4TBs so that each OS on the blade has its own partition. If the need for more storage arises (shouldn't be for a few years), I'll order an expansion drawer and span a volume across.

Once again, speed isn't the greatest of concerns.

Since the Blade will boot from OS's stored within the SAN, I need to create the arrays before installing any operating systems. Taking a second look at the DS4200, it has an ethernet based management port I can use to connect a laptop with the storage manager, so problem solved.

If you see anything glaringly wrong with my plan, any suggestions are taken into account.
 
That should work. Though I'd definitely stay away from RAID6 for the operating system volume as it takes a significant hit to write performance vs RAID10 or even RAID5. Though if you were going to go with RAID6 for both I would just create two 7 drive RAID6 arrays and span a volume across both arrays if it will let you do that and then carve that volume up into LUNs to present out as needed for OS and data Storage. My memory is a bit fuzzy on weather the DS4000s will allow a volume to span multiple RAID sets without a virtual gateway front ending it/ That configuration (two raidsets with one volume spanning over both) would give you the best performance by allowing the OS to utilize the I/O capability of 14 drives while sharing with the data storage which probably isn't pushing heavy I/Os. Then leave two drives for global spares as you originally suggested. The additional spindles would help negate the write performance hit of RAID6 while allowing you to burn less space by not having to use RAID10.

Another way of achieving the same result would be to create a RAID60 array if that particular SAN supports it (Two RAID6 arrays with a RAID0 stripe across both of them) or for that matter a RAID50 array would probably meet any redundancy requirements you are looking to achieve while being roughly 30% faster on writes vs RAID60. Definitely lots of potential options there just really depending on your requirements. You should also be able to tune the cache on the controller head for the SAN to allow more to be used for write to offset some of the RAID parity write performance hit though if there is an auto setting it will likely already be taking care of that.
 
Data security is the primary concern in this situation, and I need to ensure that the data storage LUNS are only accessible via 1 host server. To accomplish this I will use storage partitioning. Separating the raid 6 from all other arrays and LUNS should ensure that any errors in the shared array don't corrupt the data in the volume. Reading up more on IBM hotspares, IBM recommends 1 hotspare per 28 drives, so I will reduce my number to 1 (IBM support arrives within 4hrs of failure anyway).

I like the idea of raid 60, but the ds4200 doesn't support it. Spanning both arrays to one volume is a great idea, but the speed increases do not justify the increased complexity when the applications which will be running are mainly stored in RAM (16gb or ram per host unless virtualized or 32-bit) and perform few disk writes. So for the time being this is what I'm planning:

- Raid 6 w/ 9x1TB for 7TBs capacity and increased speed

- 1 global hotspare

- raid 10 with the 6 remaining drives (size not as important as speed with increased redundancy)

This works well I believe due to the fact that and need for storage upgrades down the line will come with a 16x1TB expansion unit. When that time comes I can create two 8 drive raid 6 arrays in the expansion unit for 12TBs of faster redundant storage, migrate the data from the old storage array, and use the 4200 16 drives for two hotspares and a 14 drive raid 10 which would be blazing fast for several virtualized operating systems. I could of course save space and span 2 raid 5 arrays in the future, but the storage needs are not there yet, and the write speeds (used most) would never compare to the pure raid 10.

Sound like a plan?
 
First of all I highly recommend not using RAID6 arrays for backups to write to. Their peformance is bad. Really bad. As in never do it again bad.

Is the OS data you are talking about the OS images for virtual machines, or the actual OS which machines are running on, live? The reason I ask is this:

I would never put OS data on expensive SAN disk. I also know that blades have their own individual disk to install OS on as an option, which is much cheaper than SAN disk. Secondly, OS load puts a lot of load on drives because they are typically always being written to. You should never have your application and your OS writing to the same drives, especially RAID6. Here is why:

Say your OS writes once to the OS disks.

No Raid=write once. 1 iop for your SAN to handle.
RAID 1= write twice, once for each disk.
RAID5 (Say 5 disk)= write 5 times.
RAID6 Way more IOs. Depends on number of disks. Could be upward of 15 plus. That is 15 times the writes for a single page file write an OS would do.

When you have it set up on it's own disk, it writes to a mirrored pair and doesn't effect application peformance. When you write it to an array which all of your machines share, All of their peformance is degraded.

SAN Disk is typically for high performance, high availability application data. In order to keep the OS safe, people build high availability clusters.

As for your creating a special partition for your data to keep it separate, when you pay for a professional SAN you expect it to keep the data safe..there is no need for a special partition to limit access. Your interface should include which computers are allowed to see what. Typically a LUN is bound to the world wide ID of a FC HBA. The whole point of a SAN is to be able to do operations like this. If you are running an appication which needs to be that highly segregated, you should be running a external enclosure instead of a SAN.
 
You must log in or register to reply here.
Back
Top