Skip to main content

 

Druva Documentation

Troubleshoot common VMware issues

This topic lists the common workarounds of issues that you might encounter while backing up or restoring virtual machines. 

Backup fails because Backup Proxy cannot take snapshots 

Sometimes, a backup fails because Backup Proxy cannot take a snapshot of virtual machines that are configured for backup. You might see an error like this in your log file: 

An error occurred while saving the snapshot: Change tracking target file already exists

The log files are created at /var/log/Phoenix

Resolution

  1. Consolidate the snapshots. For more information, see this article in the VMware Knowledge Base.  
  2. Wait for the next scheduled backup, and check the backup status after the backup completes.
    -or-
  3. Start a backup, and check the backup status after the backup completes.

Backup fails because the Small Computer System Interface (SCSI) Controller is in bus sharing mode

A backup fails if the Small Computer System Interface (SCSI) Controller is in sharing mode. This is because VMware does not support snapshots of virtual machines that are in bus-sharing mode. 

The log file displays the following error: 

Virtual machine is configured to use a device that prevents the snapshot operation: Device '' is a SCSI controller engaged in bus-sharing

The log files are created at /var/log/Phoenix

Resolution

  1. Ensure that the SCSI mode is not set to bus-sharing. 

Backup fails if you enabled Changed Block Tracking for VMs

Backups might fail if you enabled Changed Block Tracking (CBT) for virtual machines. 

The log files are created at /var/log/Phoenix

Resolution

1. Login to the backup proxy.

2. Disable the CBT of VM using vmcontrol utility.

3. Enable the CBT of VM using vmcontrol utility.

Note: This utility deletes all the snapshots in the VM.

Backup fails if restore of a snapshot is in progress

A backup which starts after the VM has been reverted to some snapshot from the vSphere Client, fails with the following message:

Error caused by file /vmfs/volumes/55708983-391f41a1-eaa8-00224db1d33b/Rhel6.4-MD-SDS-cbtEnable-MixProvision/Rhel6.4-MD-SDS-cbtEnable-MixProvision.vmdk', faultCause = <unset>, faultMessage = (vmodl.LocalizableMessage) [], file = '/vmfs/volumes/55708983-391f41a1-eaa8-00224db1d33b/Rhel6.4-MD-SDS-cbtEnable-MixProvision/Rhel6.4-MD-SDS-cbtEnable-MixProvision.vmdk' }

The log files are created at /var/log/Phoenix

Resolution

You must disable and enable CBT on the associated virtual machines to resolve this issue. 

1. Login to the backup proxy.

2. Disable the CBT of VM using vmcontrol utility.

3. Enable the CBT of VM using vmcontrol utility.

Note: This utility deletes all the snapshots in the VM.

Backup Fails on a Standalone ESX

Backup fails with the following error message:

[2015-07-29 17:21:59,807] [ERROR] Error <class 'agents.vmware.vixDiskLibHelper.vddk_lib.VDDKException'>:Vddk Error [code = 5643587026947, msg = None]. Traceback -Traceback (most recent call last):

File "agents/vmware/vixDiskLibOCManager.py", line 118, in handle_oc_work

File "agents/vmware/vixDiskLibHelper/vddk_lib.py", line 111, in open_disk

File "agents/vmware/vixDiskLibHelper/vddk_lib.py", line 197, in __raise_vddk_error

VDDKException: Vddk Error [code = 5643587026947, msg = None]

Resolution

The password contains ‘@’. Change the password and retry backup. The backup should work fine after this.

Backup fails due to timeouts

Backup fails with the following error message:

[2015-07-20 18:40:20,487] [ERROR] Error <class 'agents.vmware.vixDiskLibHelper.vddk_lib.VDDKException'>:Vddk Error [code = 10041633552057, msg = None]. Traceback -Traceback (most recent call last):

File "agents/vmware/vixDiskLibOCManager.py", line 118, in handle_oc_work

File "agents/vmware/vixDiskLibHelper/vddk_lib.py", line 111, in open_disk

File "agents/vmware/vixDiskLibHelper/vddk_lib.py", line 197, in __raise_vddk_error

VDDKException: Vddk Error [code = 10041633552057, msg = None]

[2015-07-20 18:40:20,487] [ERROR] Failed to open the Disk [local SAS] Win7 VCenter/Win7 VCenter.vmdk

[2015-07-20 18:40:20,487] [ERROR] [VixThread (id=0, pool=vmdk, parent_id=None)] Backup of VMDK Failed. Fault: Vddk Error [code = 10041633552057, msg = None]

[2015-07-20 18:40:20,487] [INFO] [VixThread (id=0, pool=vmdk, parent_id=None)] In finally to close the connection

[2015-07-20 18:40:20,488] [INFO] [VixThread (id=0, pool=vmdk, parent_id=None)] Closed the connection successfully

Resolution

On the backup proxy there is a vddk config file: /etc/Phoenix/vddk.cfg. It is empty by default. In this you can configure NBD timeouts.

The exact config can be read at this link.

Restore Fails

[2015-07-23 12:22:38,491] [ERROR] Exception: (vmodl.fault.ManagedObjectNotFound) {

  dynamicType = <unset>,

  dynamicProperty = (vmodl.DynamicProperty) [],

  msg = 'The object has already been deleted or has not been completely created',

  faultCause = <unset>,

  faultMessage = (vmodl.LocalizableMessage) [],

  obj = 'vim.VirtualMachine:vm-6055'}

Resolution

Sometimes the agent is not able to query the vCenter properly. Retry the restore in some time.

Backup Fails

[2015-07-29 19:24:08,367] [ERROR] (vim.fault.ApplicationQuiesceFault) {

  dynamicType = <unset>,

  dynamicProperty = (vmodl.DynamicProperty) [],

  msg = "An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.",

Resolution

Retry the backup.

Backup and restore fails for virtual machines and data stores that have special characters in its names.

Backup and restore will fail with a "Phoenix Internal error", if the virtual machine or data stores name has the following special characters.

  • Hash  ("#")
  • Left Curly Brace ("{") 
  • Right Curly Brace ("}") 
  • Vertical Bar/Pipe ("|") 
  • Backslash ("\") 
  • Caret ("^") 
  • Tilde ("~") 
  • Left Square Bracket ("[") 
  • Right Square Bracket ("]") 
  • Grave Accent ("`") 
  • Forward slash/Virgule ("/") 
  • Left Bracket ("(") 
  • Right Bracket (")"

Resolution

Do not add the above listed special characters to the virtual machine and data store name. 

HotAdd Troubleshooting

Test to see if the disk of the VM to be backed up can be manually HotAdded to the proxy

Performing the following steps will help determine if the infrastructure is present to allow the Proxy to attach the disks of the VM to be processed in HotAdd.

1.  Create a snapshot on the VM to be processed by Backup Proxy(Source VM to be backed up).

2.  Within a vSphere client, attach the base disk from the VM in Step 1 to the Backup Proxy.

  • Edit the Druva Backup Proxy that will be processing the VM from Step 1.
  • Click Add.
  • Choose “Hard Disk”.
  • Chose “Use an existing virtual disk”.
  • Navigate to the location of the base disk from the VM in step 1.
  • Specify that the disk should be attached Independent - Nonpersistent.
  • Click Finish.

3.  If this task completes this means the infrastructure is present to allow the Proxy to HotAdd the VMs base disk.

4.  Detach the disk that was attached to the Backup Proxy.

  • Edit the Druva Backup Proxy from Step 2.
  • Highlight the attached disk.
  • Click Remove (DO NOT change the radio option).
  • Click Finish.

​ 5. Remove the snapshot created in Step 1.

Note: Remember to complete steps 4 and 5 above.

Removing Stuck VMDK’s from the Backup Proxy

Sometimes the VMware API call to remove the VMDK is not received or did not complete properly. So the VMDKs are stuck to the backup proxy. In order to remove those disks perform the following steps.

Note: Kindly perform these steps only when there is no backup going on in the Backup Proxy.

  1. Through vSphere, you should be able to right-click the Druva Proxy VM and select edit settings. Look for the Hard disks attached to the VM. Normally the stuck disks will be listed at the bottom. You should also be able to identify the disks by the name and the datastore they are in under the disk file box (top right).

  1. To remove them, select the disk by clicking on it and select remove at the top right. Under removal options choose “remove from virtual machine”. DO NOT choose the option that deletes it from disk (the second option).

Original restore fails

  1. Restore fails, if the vRDM disk is converted to independent vRDM or pRDM.

Resolution

Revert from the independent vRDM disk/pRDM to vRDM disk. Then try original restore.

  1. ​Restore fails, if the new uuid does not match to old uuid


Resolution

If the uuid of the new created vmdk could match to the old uuid restore will fail. 

 

Backup fails of VM having vRDM disks

If the same lun is mapped to two different disks, the backup fails.

Resolution

Unmap the lun from both disks and then map to only one of the disks.

Restore fails on a standalone disk

Restore fails on the standalone disk with the following error message:

[2015-08-21 16:15:54,708] [ERROR] (vim.fault.FileAlreadyExists) {
  dynamicType = <unset>,
  dynamicProperty = (vmodl.DynamicProperty) [],
  msg = 'Cannot complete the operation because the file or folder /vmfs/volumes/5575befd-d1e64fa5-f919-00224db1d2b5/Test-rhel-vm-saranya_5/Test-rhel-vm-saranya_5.vmx already exists',
  faultCause = <unset>,
  faultMessage = (vmodl.LocalizableMessage) [],
  file = '/vmfs/volumes/5575befd-d1e64fa5-f919-00224db1d2b5/Test-rhel-vm-saranya_5/Test-rhel-vm-saranya_5.vmx'
}

Resolution

Rename the folder or vmx file that creates the conflict. Restore work fine after this.

VM with EFI firmware cannot boot after restore

Perform the following steps if a virtual machine is configured with EFI firmware, and cannot boot after restore.  For more information on EFI, see Using EFI/UEFI firmware in a VMware Virtual Machine.

Note: The following steps are applicable if the Backup Proxy version is below 4.6.4.

Resolution

  1. Log into vSphere. 
  2. Select the VM that is unable to boot. 
  3. Right-click the VM and select PowerPower Off.
  4. On the Confirm Power Off screen, click Yes.
  5. Right-click the VM and select Edit Settings, the Edit Settings window is displayed.
  6. Under the Options tab, select EFI option.
  7. Click OK.
  8. Right-click the VM and select PowerPower On.
    The VM should boot automatically.
    ​​​​​​

If the VM does not boot, perform the following steps:

  1. Right-click the VM and select Edit Settings, the Edit Settings window is displayed.
  2. Under the Options tab, select The next time the virtual machine boots, force entry into the EFI setup screen option.
  3. Click OK.
  4. Right-click the VM and select Power> Reset.
  5. Right-click the VM and select Open Console. The Boot Maintenance Manager is displayed.
  6. Select Enter setup.
  7. Select Boot from file and then on the next screen highlight the boot partition and press Enter.
  8. Select the EFI > <Operating system> <.efi file>.
    The VM will boot automatically.

Restore stalled and the subsequent restore requests for the same client are queued?

If the restore is stalled and the subsequent restore requests for the same client are queued,  refer to the VMware Knowledge Base article.

If the issue still persists, contact Druva Support.

 


​​​​

 

  • Was this article helpful?