joeware - never stop exploring... :)

A while ago I mentioned buying the parts and putting together SFVM – http://blog.joeware.net/2007/02/17/808/

I have received lots of emails and quite a few comments posted on an update… So here it goes.

First I apologize for the delay, I have been immensely swamped, I keep saying that but that is only because it is keeps being true. 🙂

First off, let me say it wasn’t a flawless implementation. Lots of fun issues.

Issue #1 Bad hard drives. I ordered 12 x 500GB Western Digital drives. I love these drives, I have them in all of my machines throughout my house, I have several Terabytes online in the house and it is all on these drives. However three of the drives were not functioning properly when I received them. They “worked” but they were very slow and the RAID controller didn’t like them one bit. I sent the drives back, got new ones delivered and they were fine.

Issue #2 Bad RAID Card. Well really it was probably a poor quality card with poor x64 drivers, I expect they all worked the same. The boot sequence was extremely slow with the card and the driver kept getting “dorked up” and forcing rebuilds. Obviously this isn’t a good position when you are looking for good throughput. I sent the card back and in its place ordered a nice shiny new Adaptec 2820SA (2169900-R). The things I liked about this new card besides it having a solid driver and much better boot times were that it didn’t use proprietary cables and it was MUCH faster overall than the last RAID card.

Issue #3 This one took me a while to work out but I had some bad memory. I realized that once the machine was really loaded up cooking along and available Physical Memory dropped below about 500MB (from the 8GB it is loaded with) and the machine would then get a little flakey and odd errors would get thrown in the VMs. So I took SFVM offline and ran memory tests for 3 days with the Microsoft Memory Diags and found no errors. I fired it back up and it still ran into issues when I forced it into a corner with its available physical memory. So I tried memtest86 and it didn’t find anything. Then on a whim, I downloaded a new version of memtest86 (updated on 2007-01-14) and it immediately found errors in the memory… It popped several thousand errors in the first 5 seconds of testing. Very odd. After an hour of mucking around I narrowed the problem down to one of the 2GB chips. I contacted G.Skill and they said send the chip and its matched partner to them and they would test them and if the memory was indeed bad I would get a replacement. I think it was five days later I had two brand new memory chips. During that time the server ran flawlessly, just ran out of RAM about twice as fast. 😉 I popped the new chips in, let memory tests run overnight and everything came out perfect so fired up the OS and things were good. 🙂 The customer service and the speed of the memory and the lifetime warranty means I will be looking to buy G.Skill again in the future.

I am still setting things up on this machine, it isn’t yet what it will eventually be, I would say I am at about 60% setup at the moment which means there could still be quite a bit of change going on.

So how did I config the DASD? That is a really popular question. Easy answer. I set up three RAID partitions.

One set of 4 disks are running in a RAID-5 on the onboard Intel RAID. That space is broken broken into C 195GB / D 195GB / E 195GB / F 811GB. C has the OS, D and E have tools, utilities, programs, ISO Images, a downloads folder for everything installed on the machine. F has backups of the virtual machines I really really care about – say like the backup for my virtual Exchange server that handles the mail for joeware.net.

The remaining 8 disks are set up as two RAID-0 stripe sets with 4 disks each, I wanted IOPS – period. Each RAID set has one logical disk and is 1,862GB in size. I load the up the virtuals on one drive or the other and keep a backup of it on the other drive.

Several folks were expecting me to do mirrors all over, I am not a mirror guy on any machines (production or test). If I need IOPS, I run stripe sets. If I need redundancy I run RAID-10/0+1 or RAID-5. The biggest fastest production DCs I have built for Fortune 50+ size companies have all been RAID-10/0+1 the last 3-4 years and RAID-5 prior to that.

Here is a pretty picture:

Another popular question has been about overclocking of the processor and the RAM? Did I? Did I not? The RAM I am running at the highest rating voltage specified for the RAM 1.9v. The processor is not currently overclocked. Do I need to overclock? At this point, honestly? No.

What am I doing with it and how busy is it you ask??? The system is running Windows Server 2003 Enterprise x64 R2 SP2 with VMWare Virtual Server 1.0.1. The normal running load on this machine during the week at the moment is ten (10) Windows Server 2003 R2 Virtuals (mixture of x86 and x64) . Of those, 8 are domain controllers hosting some 75,000 or so users, the other two are member servers, one running SQL Server 2005 Enterprise and Active Roles Server (from Quest). The last is simply a machine to run tests from. All of the machines have between 256MB-512MB of RAM assigned. On top of that there are 4 Longhorn B2 Enterprise Servers running as Domain Controllers (one RODC) for various AD tests. All of the Longhorn Servers have 512MB RAM assigned. CPU load ranges from 3% to about 46%. I have about 1.5GB available physical memory. All of the VMs are quite responsive. I have about 8 other virtuals that get fired up and shut down throughout the day depending on what kinds of things I have to test out.

Here is another pretty picture:

I am going to let things run stable for about another month or two to see how the warmer months impact the temps and then I expect I will start playing with amping up the processor a bit if things look ok.

  joe