Deploying OVA fails with "Failed to Deploy OVF/OVA package: The operation is not supported on the object"

We recently had an issue deploying either an OVA/OVF file that was created from one of the virtual machines in our environment.  Deploying this OVA failed instantly (after entering cluster locations / datastore) with the following error: "The Operation is not supported on the object".  After much investigation and examining the /var/log/hostd.log file on the ESXi host we were trying to deploy too, we found the line:  "Video Ram size edit is not supported when auto-detect is True."  Editing the Virtual Machine hardware settings and editing the Video Card and changing the setting from Auto-detect to Specify custom settings (This can be anything) and then exporting the virtual machine as an OVA allowed the deployment to work! Very interesting that this setting caused a problem!  Previously we have also encountered this when the Virtual Machine has an ISO mounted when to converting to OVA.

PowerCLI script to check VIBs on a host

I work in an environment where ESXi hosts require a lot of additional software to be installed.  Examples include a specific Cisco VEM module to be installed, the Trend DSA Filter driver etc.  A simple way to check what version or even if the software has been installed on the host is as follows.  This example will check if the Trend DSA Filter is installed.

(get-esxcli -vmhost hostname).software.vib.list() | where {$_.Name -eq "dvfilter-dsa"}

This will return information if the module is installed and give important information such as Versions etc.

VMware vSphere Storage Appliance (VSA) Installation and Removal Tricks

I've recently had the pleasure of playing around with VMware's vSphere Storage Appliance.  I would like to quickly point out some of the troubles I had with the installation so that if you guys come across the same problems they can quickly be sorted.  The installation I was using at the time was "VMware-vsa-all-5.1.3.0-1090545.iso"

 

During the installation if you receive an Error 2896 during the VSA Manager installation it is because of a script problem that doesn't like servers with multiple drives, specifically if you are also trying to install the product on for example D:\ rather than C:\. 

 

Browse to %temp%\sva\VSAManager/installTool and edit the runtool.bat file.  Edit the 9th line down and change it from cd %BUILDDIR% to cd /d %BUILDDIR% this will ensure that the installation will occur.

 

If you receive the error during setup through VSA Wizard (from within the VI Client) "Cannot create VSA cluster:  Adding hosts to HA Cluster failed."  Looking at the VSA Configuration log you will see that EVC mode failed to get configured.  I was running a pair of AMD hosts, it was trying to set the EVC mode to amd-rev-e.  If i manually created a cluster, there was no issue.  This hack allows the install to complete, you can then turn on EVC manually on the cluster afterwards.   Browse to the installation directory of vCenter on which you installed the VSA manager, e.g. "C:\program files\vmware\infrastructure\tomcat\webapps\VSAManager\WEB-INF\classes\" and wtih an administrative notepad, edit "dev.properties" file.  Change the evc.config=true  line to evc.config=false  NOTE that this option will REQUIRE a restart of the "VMware VirtualCenter Server" and "VMware VirtualCenter Management Webservices" services.  You should then be able to run through the VSA setup wizard without a hitch.

 

Remember, your hosts must have 4 NIC's in order for the setup to complete, if you have only a single quad port card, you will have to do a "Brownfield" network setup, otherwise you will recieve the following error:  Cannot create VSA cluster:  Failed to complete network configuration for specified host:  <hostname> , hence revered all other hosts, Could not find 2 NICs on the system.

The setup can be done from this documentation:  VSA Brownfield Setup

 

One final thing to note, is that if you are trying to run the cleanup.bat file (when removing the VSA) edit the file so that any line that reads cd %BUILDDIR% to cd /d %BUILDDIR% and the system I was on had no JAVA_HOME variable set.  So I set it to the following "set JAVA_HOME=D:\vmware\Infrastructure\jre" and that allowed a nice clean up.

 

PowerCLI to enable LockDown Mode on all hosts

Here is a quick and easy way to make sure that LockDown Mode is enabled on all your hosts using PowerCLI (if your environment dictates it)

(get-vmhost * | get-view) | foreach-object -process {$._EnterLockdownMode()}

That will go through every host and enable LockDown mode.  If you see the following error while running the script:

Exception calling "EnterLockdownMode" with "0" argument(s):  "The administrator permission has already been dsiabled on the host (except for the vim user)"

 It just means that the host already has that setting enabled and it can be ignored.

vCenter VPXD.exe process using 100% memory

We recently had an issue in an environment that I am looking after that caused the VPXD.exe (vcenter server service) to use 100% of the memory allocated to a machine, plus all the disk space on the server (as the page file was set to system managed and just kept growing) until the server stopped responding.  After much investigation, the following lines were noted in the vCenter server log (appearing a lot).  The error is as follows:

[05444 error 'Default'] SSL StreamImpl::DoServerHandshake (0000000028a34600) SSL_accept failed with Unexpected EOF
[05444 warning 'ProxySvc'] SSL Handshake failed for stream TCPStreamWin32(socket=TCP(fd=5470) local=<vcenter IP>, peer=<remote client connection>), error=SSL Exception: Unexpected EOF

That error message basically translates to a VI Client instance connecting in to vCenter, when this occurs, each connection consumes additional memory which isn't released.  In our case, we had an incident where the VM Tools instance was ripped out of a management server, this management server used the VMXNET 3 driver.  In its unusual state it was polling the vCenter server a large number of times a minute (like it was flapping). Once VMware Tools was reinstalled on the management server, vCenter calmed down and returned back to a normal amount of memory usage.

VMware VRM Server not able to connect to local vCenter Server

If you install the VRM Server as part of vSphere Replication (this problem is from version 5.0.2) you may see this error in the hms log file.  The HMS logs are located on the VRM itself under /opt/vmware/hms/logs.  You can see how to get these logs from KB2013091.

The error is as follows:

2013-04-12 11:01:14.526 WARN hms.configuration [mai]
(..hms.cfg.VcExtensionChecker) | Extension login failed:java.lang.RuntimeException:

java.util.concurrent.ExecutionException: com.vmware.vim.binding.vim.fault.InvalidLogin:
Inheritied from com.vmware.vim.binding.vim.fault.VimFault:
Inheritied from com.vwmare.vim.fault.InvalidLogin:  Cannot complete
login due to an incorrect user name password.

Now, the problem is that your VRM Server needs to not only communicate to the vCenter on HTTP (TCP 80) it also needs to do the initial connection when the services start on HTTPS (TCP 443).  This was the case in the environment I was working in due to firewalls between the two.  If the VRM Server can only talk to vCenter on 80, it will fail to fully communicate to vCenter.  Once you open up HTTPS between vCenter and the VRM Server, login to the admin page https://<ip of your vrm server>:5480 to restart the services.  It should then connect.  If this is the case for your environment, you will need to follow KB2019120 in order for vCenter to show the health status correctly on the health page for the VRM.  You will also note that KB1009562 which is the Port Number reference for SRM and its' components do not state that this port is required.  I could not get communication functional between the same site vCenter Server and VRM Server until both TCP 80 & 443 were allowed through the firewall to the vCenter server.  I would also allow the VRM server to talk to the remote site vCenter server on HTTP and HTTPS (the kb article only states TCP 80).

Resetting a Local Administrator Account Password in Windows 2008 R2

Had trouble using the NT Password utility or any other recover tool to reset a local administrator account for Windows 2008 R2?   Well here's a simple way to do that which uses the Vista Backdoor trick.  You should get yourself a copy of Hiren's Boot CD to do this.  One thing to note is that if you are doing this on a virtual machine the VMware Paravirtual SCSI device will not show any disks when you boot off this tool.  The other thing to note is that the "Offline NT/XP/2000/Vista/ Windows 7 Password Changer" is a use at your own risk tool, I've also not had it work on fully patched Windows 2008 R2 machines:

• Burn the ISO to CD (if no iLO/RMC or the machine is not virtual)
• Boot off the Hiren's Boot CD
• Select MiniXP Environment

• Once the environment has booted, fire up Explorer

• Browse to the Drive (if you have multiple, check each one) with the Windows installation on it

• Browse to <Drive>\Windows\System32\

• Rename "Magnify.exe" to "Magnify.exe.old"

• Copy "CMD.exe" and paste it in the same directory (you should end up with a file called 'CMD_2.exe' at the very bottom of the explorer list of files) rename the copy to "Magnify.exe"

• Reboot the Server (with Hiren's now unmounted)
• At the login screen, down the bottom left click the "Ease of Access" icon (circled in the image)

• Select "Make items on the screen larger (Magnifier)" then click OK

• You'll now have a Command Prompt running (if you type whoami, you'll see it is running as NT System)

• Type "net user" and hit enter (you'll see a list of local users)

• Type "net user <username> <password>" and the user of your choice's password is now changed!!!!  Note * change the password to a password that would match the local/domain security policy of your machine

• You can also use standard net commands to add a user to the local administrators group, create a new user, etc! Close down the Magnify (well command prompt window) for the tidy up.

 

• Once done, delete the "Magnify.exe' and replace it by renaming "Magnify.exe.old" back to "Magnify.exe"  Remember to take ownership of Magnify.exe.old first to Administrators and give full access before you do this!

• The last thing to do is to make sure that Magnify doesn't launch every time you log in to the server.  Go to "Ease of Access" from control panel

• Click "Optimize Visual Display"

• Uncheck "Turn on Magnifier"

• Click "Apply" then "OK"

There you have it, a simple and "SAFE" way of changing a local administrators password on a Windows Server 2008 R2 machine