VMware Backups fails with error VMWARE VMOMI1
Problem description
Phoenix VM backups fail with VMWARE_VMOMI1: An error occurred while saving the snapshot: Failed to quiesce the virtual machine.
Traceback
The VMware.log shows that the error has occurred while quiescing the virtual machine.
2020-01-03T16:44:46.963Z| vmx| I125: SnapshotVMX_TakeSnapshot start: 'Test', deviceState=0, lazy=0, logging=0, quiesced=1, forceNative=0, tryNative=1, saveAllocMaps=0 cb=BDDB500, cbData=32476790
2020-01-03T16:44:46.984Z| vmx| I125: DISKLIB-LIB_CREATE : DiskLibCreateCreateParam: vmfsSparse grain size is set to 1 for '/vmfs/volumes/6edcdf5b-b46dc6d5/fics8/fics8-000001.vmdk'
2020-01-03T16:44:46.988Z| vmx| I125: DISKLIB-LIB_CREATE : DiskLibCreateCreateParam: vmfsSparse grain size is set to 1 for '/vmfs/volumes/6edcdf5b-b46dc6d5/fics8/fics8_1-000001.vmdk'
2020-01-03T16:44:46.988Z| vmx| I125: SNAPSHOT: SnapshotPrepareTakeDoneCB: Prepare phase complete (The operation completed successfully).
2020-01-03T16:44:47.165Z| vcpu-3| I125: ToolsBackup: changing quiesce state: IDLE -> STARTED
2020-01-03T16:44:52.917Z| vmx| I125: [msg.snapshot.quiesce.vmerr] The guest OS has reported an error during quiescing.
2020-01-03T16:44:52.917Z| vmx| I125+ The error code was: 5
2020-01-03T16:44:52.917Z| vmx| I125+ The error message was: 'VssSyncStart' operation failed: IDispatch error #8454 (0x80042306)
2020-01-03T14:14:59.707Z| vmx| I125: Msg_Post: Warning
2020-01-03T14:14:59.707Z| vmx| I125: [msg.snapshot.quiesce.vmerr] The guest OS has reported an error during quiescing.
2020-01-03T14:14:59.707Z| vmx| I125+ The error code was: 3
2020-01-03T14:14:59.707Z| vmx| I125+ The error message was: Quiesce operation aborted
Verification
Manually take a snapshot using vSphere client or using web client with the help of the following steps:
Select VM in vSphere Client → Actions →Snapshots →Take Snapshot |
|
|
|
Resolution
If the snapshots have failed, perform the following steps in sequence:
- Older version of VMware tools
- Check whether the latest version of VMware tools is installed. If not, upgrade to the latest version of the VMware tools.
- Expired Licenses
- Ensure that the vStorage API and vCenter licenses are active.
- Orphaned snapshots
- Verify that there are no orphaned snapshots in the datastore folder. Occasionally, the snapshot manager may show zero snapshots. However, you may find orphaned snapshot files in the datastore folder.
- Review the VMware.log file for Error 3 or Error 5.
- Ensure that the following services are running:
- Microsoft VSS
- VMware Tools service
- VSS Writers - Run VSSADMIN LIST WRITERS to list all of the writers that are currently available on the machine and display the state of each. To bring back the writers that are failed to a stable state, reboot the machine.
- Insufficient space for snapshots
- Check if there is sufficient space available for the VSS snapshots on all drives. Create a shadow copy for the drives. Ensure that the snapshot size has No limit option checked.
- If the issue persists, uninstall the VMware Tools and reinstall them.
- VSS errors
- Review the events in the event viewer for VMware provider errors for VSS. The VMware provider for VSS may conflict with Microsoft VSS provider or other third-party VSS providers. In such cases, it is recommended to uninstall or disable third-party VSS providers and use the Microsoft VSS provider.
- If the snapshot fails and the VM is a Windows Server 2019 with GPT partition type,
- Review the VMware.log for the following error:
[2020-01-03 09:39:30,287] [ERROR] Error in Deleting TL Files from streamset: Fileset/Streamset does not exist. (#1000000a2) (Error Code : PHOENIX162)
- To resolve the snapshot issue with Windows 2019, refer to the following article: https://kb.vmware.com/s/article/60395
- To fix the issue, download and install the security update MS KB kb4534321.
- Review the VMware.log for the following error:
- When the manual snapshot with Quiescing checked from the vSphere webclient also failed with this error
An error occurred while saving the snapshot: msg.snapshot.error – QUIESCINGERROR
Then login to that VM and disable the VMware snapshot provider service as mentioned in this VMware article:VMware Knowledge Base