Skip to main content
druva_logo.pngDruva
|
Documentation

How can we help you?

 

Trending articles:
Druva Documentation

Troubleshoot common VMware issues

  1. Last updated
  2. Save as PDF

This topic lists the common workarounds of issues that you might encounter while backing up or restoring virtual machines. 

Backup fails because backup proxy cannot take snapshots 

Sometimes, a backup fails because backup proxy cannot take a snapshot of virtual machines that are configured for backup. You might see an error like this in your log file: 

An error occurred while saving the snapshot: Change tracking target file already exists

The log files are created at /var/log/Phoenix. 

Resolution

  1. Consolidate the snapshots. For more information, see this article in the VMware Knowledge Base.  
  2. Wait for the next scheduled backup, and check the backup status after the backup completes.
    -or-
  3. Start a backup, and check the backup status after the backup completes.

Backup fails because the Small Computer System Interface (SCSI) Controller is in bus sharing mode

A backup fails if the Small Computer System Interface (SCSI) Controller is in sharing mode. This is because VMware does not support snapshots of virtual machines that are in bus-sharing mode. 

The log file displays the following error: 

Virtual machine is configured to use a device that prevents the snapshot operation: Device '' is a SCSI controller engaged in bus-sharing

The log files are created at /var/log/Phoenix. 

Resolution

  1. Ensure that the SCSI mode is not set to bus-sharing. 

Backup fails if you enabled Changed Block Tracking for virtual machines

Backups might fail if you enabled Changed Block Tracking (CBT) for virtual machines. 

The log files are created at /var/log/Phoenix. 

Resolution

1. Login to the backup proxy.

2. Disable the CBT of the virtual machine using vmcontrol utility.

3. Enable the CBT of the virtual machine using vmcontrol utility.

Note: This utility deletes all the snapshots in the virtual machine.

Backup fails if restore of a snapshot is in progress

A backup which starts after the virtual machine has been reverted to some snapshot from the vSphere Client, fails with the following message:

Error caused by file /vmfs/volumes/55708983-391f41a1-eaa8-00224db1d33b/Rhel6.4-MD-SDS-cbtEnable-MixProvision/Rhel6.4-MD-SDS-cbtEnable-MixProvision.vmdk', faultCause = <unset>, faultMessage = (vmodl.LocalizableMessage) [], file = '/vmfs/volumes/55708983-391f41a1-eaa8-00224db1d33b/Rhel6.4-MD-SDS-cbtEnable-MixProvision/Rhel6.4-MD-SDS-cbtEnable-MixProvision.vmdk' }

The log files are created at /var/log/Phoenix. 

Resolution

You must disable and enable CBT on the associated virtual machines to resolve this issue. 

1. Login to the backup proxy.

2. Disable the CBT of VM using vmcontrol utility.

3. Enable the CBT of VM using vmcontrol utility.

Note: This utility deletes all the snapshots in the virtual machine.

Backup Fails on a Standalone ESX

Backup fails with the following error message:

[2015-07-29 17:21:59,807] [ERROR] Error <class 'agents.vmware.vixDiskLibHelper.vddk_lib.VDDKException'>:Vddk Error [code = 5643587026947, msg = None]. Traceback -Traceback (most recent call last):

File "agents/vmware/vixDiskLibOCManager.py", line 118, in handle_oc_work

File "agents/vmware/vixDiskLibHelper/vddk_lib.py", line 111, in open_disk

File "agents/vmware/vixDiskLibHelper/vddk_lib.py", line 197, in __raise_vddk_error

VDDKException: Vddk Error [code = 5643587026947, msg = None]

Resolution

The password contains ‘@’. Change the password and retry backup. The backup should work fine after this.

Backup fails due to timeouts

Backup fails with the following error message:

[2015-07-20 18:40:20,487] [ERROR] Error <class 'agents.vmware.vixDiskLibHelper.vddk_lib.VDDKException'>:Vddk Error [code = 10041633552057, msg = None]. Traceback -Traceback (most recent call last):

File "agents/vmware/vixDiskLibOCManager.py", line 118, in handle_oc_work

File "agents/vmware/vixDiskLibHelper/vddk_lib.py", line 111, in open_disk

File "agents/vmware/vixDiskLibHelper/vddk_lib.py", line 197, in __raise_vddk_error

VDDKException: Vddk Error [code = 10041633552057, msg = None]

[2015-07-20 18:40:20,487] [ERROR] Failed to open the Disk [local SAS] Win7 VCenter/Win7 VCenter.vmdk

[2015-07-20 18:40:20,487] [ERROR] [VixThread (id=0, pool=vmdk, parent_id=None)] Backup of VMDK Failed. Fault: Vddk Error [code = 10041633552057, msg = None]

[2015-07-20 18:40:20,487] [INFO] [VixThread (id=0, pool=vmdk, parent_id=None)] In finally to close the connection

[2015-07-20 18:40:20,488] [INFO] [VixThread (id=0, pool=vmdk, parent_id=None)] Closed the connection successfully

Resolution

On the backup proxy there is a vddk config file: /etc/Phoenix/vddk.cfg. It is empty by default. In this you can configure NBD timeouts.

The exact config can be read at this link.

Restore fails

[2015-07-23 12:22:38,491] [ERROR] Exception: (vmodl.fault.ManagedObjectNotFound) {

  dynamicType = <unset>,

  dynamicProperty = (vmodl.DynamicProperty) [],

  msg = 'The object has already been deleted or has not been completely created',

  faultCause = <unset>,

  faultMessage = (vmodl.LocalizableMessage) [],

  obj = 'vim.VirtualMachine:vm-6055'}

Resolution

Sometimes the agent is not able to query the vCenter properly. Retry the restore in some time.

File level restore fails with FLR02 error

Error message logs:

{snippet}
[2017-04-17 16:04:05,764] [ERROR] Error <type 'exceptions.OSError'>:[Errno 13] Permission 
denied: '/mnt/unc_svr-8600-li_277064/svr-8600-li-3353'. Traceback -Traceback (most recent 
call last):
  File "agents/flr/flrUtils.py", line 906, in mount_and_restore
OSError: [Errno 13] Permission denied: '/mnt/unc_svr-8600-li_277064/svr-8600-li-3353'
{snippet}

Resolution

Ensure that restore location has write permission. If Druva cannot write to a folder, file level restore fails with FLR02.  In addition, you can try to restore your files to a folder with write permission. 

Backup fails

[2015-07-29 19:24:08,367] [ERROR] (vim.fault.ApplicationQuiesceFault) {

  dynamicType = <unset>,

  dynamicProperty = (vmodl.DynamicProperty) [],

  msg = "An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.",

Resolution

Retry the backup.

Backup and restore fails for virtual machines and data stores that have special characters in its names.

Backup and restore will fail with a "Phoenix Internal error", if the virtual machine or data stores name has the following special characters.

  • Hash  ("#")
  • Left Curly Brace ("{") 
  • Right Curly Brace ("}") 
  • Vertical Bar/Pipe ("|") 
  • Backslash ("\") 
  • Caret ("^") 
  • Tilde ("~") 
  • Left Square Bracket ("[") 
  • Right Square Bracket ("]") 
  • Grave Accent ("`") 
  • Forward slash/Virgule ("/") 
  • Left Bracket ("(") 
  • Right Bracket (")"

Resolution

Do not add the above listed special characters to the virtual machine and data store name. 

HotAdd Troubleshooting

Test to see if the disk of the virtual machine to be backed up can be manually HotAdded to the proxy

Performing the following steps will help determine if the infrastructure is present to allow the proxy to attach the disks of the VM to be processed in HotAdd.

1.  Create a snapshot on the virtual machine to be processed by backup proxy(Source virtual machine to be backed up).

2.  Within a vSphere client, attach the base disk from the virtual machine in Step 1 to the backup proxy.

  • Edit the Druva backup proxy that will be processing the virtual machine from Step 1.
  • Click Add.
  • Choose “Hard Disk”.
  • Chose “Use an existing virtual disk”.
  • Navigate to the location of the base disk from the virtual machine in step 1.
  • Specify that the disk should be attached Independent - Nonpersistent.
  • Click Finish.

3.  If this task completes this means the infrastructure is present to allow the proxy to HotAdd the virtual machine s base disk.

4.  Detach the disk that was attached to the backup proxy.

  • Edit the Druva backup proxy from Step 2.
  • Highlight the attached disk.
  • Click Remove (DO NOT change the radio option).
  • Click Finish.

 5. Remove the snapshot created in Step 1.

Note: Remember to complete steps 4 and 5 above.

Removing Stuck VMDK’s from the backup proxy

Sometimes the VMware API call to remove the VMDK is not received or did not complete properly. So the VMDKs are stuck to the backup proxy. In order to remove those disks perform the following steps.

Note: Kindly perform these steps only when there is no backup going on in the backup proxy.

  1. Through vSphere, you should be able to right-click the Druva proxy virtual machine and select edit settings. Look for the Hard disks attached to the virtual machine. Normally the stuck disks will be listed at the bottom. You should also be able to identify the disks by the name and the datastore they are in under the disk file box (top right).

  1. To remove them, select the disk by clicking on it and select remove at the top right. Under removal options choose “remove from virtual machine”. DO NOT choose the option that deletes it from disk (the second option).

Original restore fails

  1. Restore fails, if the vRDM disk is converted to independent vRDM or pRDM.

Resolution

Revert from the independent vRDM disk/pRDM to vRDM disk. Then try the original restore.

  1. Restore fails, if the new uuid does not match to old uuid


Resolution

If the uuid of the new created vmdk could match to the old uuid restore will fail. 

 

Backup fails of virtual machine  having vRDM disks

If the same lun is mapped to two different disks, the backup fails.

Resolution

Unmap the lun from both disks and then map to only one of the disks.

Restore fails on a standalone disk

Restore fails on the standalone disk with the following error message:

[2015-08-21 16:15:54,708] [ERROR] (vim.fault.FileAlreadyExists) {
  dynamicType = <unset>,
  dynamicProperty = (vmodl.DynamicProperty) [],
  msg = 'Cannot complete the operation because the file or folder /vmfs/volumes/5575befd-d1e64fa5-f919-00224db1d2b5/Test-rhel-vm-saranya_5/Test-rhel-vm-saranya_5.vmx already exists',
  faultCause = <unset>,
  faultMessage = (vmodl.LocalizableMessage) [],
  file = '/vmfs/volumes/5575befd-d1e64fa5-f919-00224db1d2b5/Test-rhel-vm-saranya_5/Test-rhel-vm-saranya_5.vmx'
}

Resolution

Rename the folder or vmx file that creates the conflict. Restore work fine after this.

Virtual machine with EFI firmware cannot boot after restore

Perform the following steps if a virtual machine is configured with EFI firmware, and cannot boot after restore.  For more information on EFI, see Using EFI/UEFI firmware in a VMware Virtual Machine.

Note: The following steps are applicable if the Backup Proxy version is below 4.6.4.

Resolution

  1. Log into vSphere. 
  2. Select the virtual machine that is unable to boot. 
  3. Right-click the virtual machine and select Power> Power Off.
  4. On the Confirm Power Off screen, click Yes.
  5. Right-click the virtual machine and select Edit Settings, the Edit Settings window is displayed.
  6. Under the Options tab, select EFI option.
  7. Click OK.
  8. Right-click the virtual machine and select Power> Power On.
    The virtual machine should boot automatically.
    ​​​​​​

If the virtual machine does not boot, perform the following steps:

  1. Right-click the virtual machine and select Edit Settings, the Edit Settings window is displayed.
  2. Under the Options tab, select The next time the virtual machine boots, force entry into the EFI setup screen option.
  3. Click OK.
  4. Right-click the virtual machine and select Power> Reset.
  5. Right-click the virtual machine and select Open Console. The Boot Maintenance Manager is displayed.
  6. Select Enter setup.
  7. Select Boot from file and then on the next screen highlight the boot partition and press Enter.
  8. Select the EFI > <Operating system> > <.efi file>.
    The virtual machine will boot automatically.

Restore stalled and the subsequent restore requests for the same client are queued?

If the restore is stalled and the subsequent restore requests for the same client are queued,  refer to the VMware  Knowledge Base article.

If the issue still persists, contact Druva Support.

No staging VM is listed in the staging VM drop-down on the browse and restore window

Staging VM is used for both file-level browse and restore for dynamic GPT disks, and SQL PIT restores.

Resolution

  1. Deploy a staging VM. For more information, see staging VM prerequisites.
  2. Assign credentials to the desired staging VM on the All Virtual Machines listing page on the Management Console. For more information, see Manage credentials for VMware servers.