File server backups completing with errors - PhoenixFSSnapshot executable timed out
Problem description
Windows File server backups complete with errors (show the status as ‘Successful with Errors’). Progress logs show ‘Failed to take snapshot of “C:” or “D:”. Check event log for detail error. Backing up live data.’
Cause
Snapshot creation for disk volumes (C:\, D:\ etc) fails because of VSS writers not working as expected
Traceback
level=info ts=2023-02-28T22:00:10.914441+05:30 filename=snapshot_windows.go:126 message="Executing PhoenixFSSnapshot" path="C:\\Program Files\\Druva\\Phoenix Agent\\PhoenixFSSnapshot.exe"
level=info ts=2023-02-28T22:30:10.967287+05:30 filename=snapshot_windows.go:140 message="PhoenixFSSnapshot executable timed out"
Resolution
- Connect to the File Server
- Open Task manager, Background processes show 3 Phoenix Processes
- Kill those processes (Phoenix.exe , PhoenixCPHwnet64.exe and PhoenixFSBackupAgent.exe) (Right click on the process and End task)
- Stop the Hybrid Workloads Agent Client Service
- Open command prompt and run the below command
vssadmin list writers - Shows no output (keeps running in the background)
- Cancel the command (Press Control+C on the keyboard)
- Restart the Volume Shadow Copy service (If it times out with a message that it did not respond in timely fashion, try to start it again)
- Start the Hybrid Workload Agent Client Service
- Run the
vssadmin list writerscommand again, should show the writers in stable state - Trigger a manual backup of the affected server. This time it should complete successfully
Verification
Observe the next scheduled backup to ensure that it runs successfully without any errors