Monday 26 March 2012

vCenter server service won't start event id 1000

I was at a customer the other day. We needed to reboot the VC VM the customer hadn't done this in ages, and typically , after the reboot the vCenter server service wouldn't start, I check the eventvwr and found event id 1000. After some Googling I would appear that something was already running on port 80. I checked the VPXA logs in this location %ALLUSERSPROFILE%\Application Data\VMware\VMware VirtualCenter\Logs\ and found this error:


[VpxdReverseProxy] Failed to create http proxy: Resource is already in use: Listen socket: :80 


Futher Googling suggested either IIS has been installed, or the SQL reporting service was installed. Some reports suggest that WinRM takes port 80 but this was later found to be incorrect.


I even tried upgrading the customer the vCenter 4.1 U2 this still did not work.


NETSTAT -ANO reported that java.exe was taking the port so I presumed that Tomcat was stuffed.


Fix: In the end I thought I would stop some services to see if that was the cause. Sure enough when I stopped VMware Update Manager, port 80 was then free. I set the VUM service to delayed start and this sorted the issue. A couple of reboots proved this worked! What I can't work out is that know one else had blogged this issue.


Troubleshooting.
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1015101
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003971

IOP values for harddisks

Device Type IOPS Interface Notes
5,400 rpm SATA drives HDD 50-80 IOPS SATA 3 Gb/s
7,200 rpm SATA drives HDD 75-100 IOPS SATA 3 Gb/s
10,000 rpm SATA drives HDD 125-150 IOPS SATA 3 Gb/s
10,000 rpm SAS drives HDD 140 IOPS SAS SAS is full duplex
15,000 rpm SAS drives HDD 175-210 IOPS SAS SAS is full duplex

In a mirrored configuration:
Disk IOPS = Read IOPS + (2 * Write IOPS)
In a parity (RAID5) configuration:
Disk IOPS = Read IOPS + (4 * Write IOPS)

Example calculations

Now let's look at an example. If you estimate that you need to support 40 Read IOPS (40 reads/sec) and 80 Write IOPS (80 writes/sec).
If you want to use a mirrored configuration of drives:
Disk IOPS = Read IOPS + (2 * Write IOPS) = 40r/s + (2 * 80w/s) = 200 Disk IOPSUsing 7200 rpm drives, you need: 200 / 50 = 4 disk drives
Using 10k rpm drives, you need: 200 / 130 = 2 disk drives (always round up)
If you want to use a parity (RAID5) configuration of drives:
Disk IOPS = Read IOPS + (4 * Write IOPS) = 40r/s + (4 * 80w/s) = 360 Disk IOPSUsing 7200 rpm drives, you need: 360 / 50 = 8 disk drives (always round up)
Using 10k rpm drives, you need: 360 / 130 = 3 disk drives

An excellent article on how how RAID levels effect IOPS 
http://skabelka.com/node/125
http://sudrsn.wordpress.com/2010/12/25/iops-raid-penalty-and-workload-characterization/