Dave Veness

Cisco UCS and vSwitch load balancing

Dave V — Thu, 10 Feb 2011 11:06:55 +0000

Wow, long time since a post, I’ve been very slack.

I’ve been working on a very large Cloud project lately involving Cisco UCS, Nexus 1Kv and 7k’s and NetApp disk.

In the traditional server world, we’re used to setting up our ESX hosts with etherchannel aggregation to the host and of course using IP Hash as the load balancing policy on the vSwitch.

Don’t do this for UCS, or you’ll have the fun experience of your NIC’s flapping up and down!

See this link for more information – http://kb.vmware.com/kb/1013094

IBM DS4700 Model 70, VMware ESX 3.5 and Site Recovery Manager

Dave V — Fri, 09 Jan 2009 12:55:54 +0000

Just finished up a project for a customer using the above products, and it was definately an interesting (frustrating) experience. I’ve put together some quick and dirty notes on implementing this in a hope it will help someone else out there who has to set this up. The documentation is really really lacking and it was a lot of trial and error to get this all working properly.

Although the ending setup is not ideal (should use a Model 72 minimum), this does work. The documentation for this is non-existent apart from a small readme which comes with the IBM SRA itself.

Software pre-requisites

Full fabric activation key for FC switches (we were using the IBM SAN16B-2 Brocade switches).
Flash copy license for DR side DS4700.
Enhanced remote mirror license for both DS4700’s.
Ensure you’re running latest firmware on the DS4700’s (must be version 7 point something).
Ensure you have the latest SRM and SRA from Vmware.com. Versions up until the very latest SRA from IBM had typo’s in the script – unbelievable.

Configuration Notes

DS4700 Model 70 only has 2 FC ports per controller. A single port from each controller is required for ERM (we used port 2), which means you have to run a singular meshed fabric across your FC switches (we had 2 switches per side) to provide any level of redundancy, rather than the best practice of two separate fabrics. What you end up with is 1 port per controller carrying host traffic, and one for ERM replication. The Model 72 has 4 FC ports per controller, so this allows separated fabrics (and is therefore recommended over the Model 70).
You will need to setup your SAN fabric to allow all your Production side ESX hosts to see each other, as well as one of the ports (Port 1 in our case, Port 2 is used for ERM) from each controller on the SAN – repeat the process for the DR side. You can have separate zones for each individual FC port on the ESX hosts to see the port on the SAN, but group the hosts together. Having a typical 1 ESX host port to 1 SAN port zoning setup gave us pathing issues on the SAN, the hosts need to see each other for whatever reason.
Setup your VMFS LUNs and configure ERM to replicate these to the DR side. You can use different RAID levels on each end to maximize disk space if required. For example we had a mix of RAID 10 and RAID 5 on our production side with these replicating to a RAID 5 array on the DR side.
On the DR side SAN, leave as many unconfigured disks (ie. do not include them in an array) as you have SRM protected LUNs. The IBM SRA uses an unallocated physical disk to configure Flashcopy for SRM protection plan testing – having all disks allocated to arrays even with free space available will not work, even though this is sufficient for a normal flash copy to be run through Storage Manager.
After installing the IBM SRA, ensure the path to the Perl binaries is in your PATH system variable, otherwise you will get errors when SRM tries to call the IBM SRA scripts.

I believe those were the main road blocks we ran into. Please leave a comment if you find this useful or have anything to add!

Thanks!

Been busy!

Dave V — Thu, 02 Oct 2008 12:12:32 +0000

Sorry for the lack of updates lately, been very busy working on some very interesting solutions for customers. Will have some new info here shortly!

Got my VCP!

Dave V — Mon, 23 Jun 2008 02:26:56 +0000

I passed my VCP exam last Friday! I must say I was not very confident going into it.. There is a huge amount of material to cover when studying for it, and the exam expects you to know it all backwards…

I wouldn’t say the exam was of high quality either, there were a few obvious mistakes in my exam, which does make things a little more difficult.

Tips for anyone looking to go for this exam should would be;

1. Don’t rely on what you learnt during the I&C class, it’s no where near enough..
2. Read all of the PDF’s here – http://www.vmware.com/support/pubs/vi_pages/vi_pubs_35.html
3. Really know your iSCSI, Fibre Channel and NFS storage.
4. Know the capabilities of all the VMWare products well – including Workstation, Fusion etc..
5. Have a lab environment to configured to really learn all the features capabilities and limits

I might do up an article soon on how I built my home VMWare lab on a budget..

IBM N-Series (NetApp) default disk sparing

Dave V — Fri, 13 Jun 2008 16:53:00 +0000

Had an issue recently with a customers new N3300 NAS and the amount of hot spare disks OnTAP wants to keep. This particular customer ordered a clustered model (A20), along with an EXN4000 disk shelf containing 14 x 300GB disks.

When we fired it up, the controllers had taken ownership of all the disks evenly (7 per controller), set each of its vol0 to a 266GB usable RAID-DP configuration (3 disks gone) and with two hot spares each. Now from the original 14 physical disks, we were now down to 4 disks left out of 14 to work with! No problem, lets jump into FilerView and add one of the hot spares into aggr0 to get some more space… no bingo… “The number of spare disks remaining is too little for the requested parity”…

It seems this hot sparing change to OnTAP only came in recently (we’re running version 7.2.4L1), and I must say it seems a little crazy to force 2 hot spares for each controller set in such a small configuration, the crazier part was how difficult it was to find out how to reduce the hot spare count.

After trying many different command combinations sent to us by IBM & NetApp we finally found one which did the trick, and allowed us to drop the hot sparing back to a single disk per controller…

This command needs to be entered in the CLI to reduce the minimum amount of hot spares required…

options raid.min_spare_count

All good? Sort of – The FilerView still does not allow you to expand the aggregrate any further, it still kept reporting “The number of spare disks remaining is too little for the requested parity” error when you try to add in an additional disk to aggr0. Jumping back into the CLI however, would let us add another disk into aggr0, expanding our usable space by 300GB per controller. The command for doing this;

aggr add aggr0 1@300
*300 being the size of my disk available

Has anyone else encountered this? I must admit I’m still very new to these devices, so this may be common knowledge to many more experienced NetApp pilots out there.