Skip to main content
Druva Documentation

VMware backups stay in the queued state with IO error Error 28 no space left on device

 

Problem description

VMware backups or log requests stay in the queued state. The system does not generate any error messages related to the backup proxy. The following error is displayed in the logs IOError: [Error 28] No space left on device.

Cause

This can happen if you have enabled Phoenix debugging on a backup proxy and are backing up a virtual machine that is fragmented.  Backups of fragmented virtual machines cause excessive processing on the proxy and generate large amounts of log data. In this situation, the /var/log volume runs out of allocated space.

Traceback

You can view the error message in the /var/log/messages log

Resolution

  1. Determine the size of the following directory
    /var/log/Phoenix/VMWARE/backup/
    Command: du -sh /var/log/Phoenix/VMWARE/backup/
    This gives us the exact size of the directories under the backup directory
  2. Review and delete the logs that are not required to free up space.  Use the following command to delete directories that are not required:
    rm -rf “backup log directory name” without quotes
  3. Use the instructions in the How to enable or disable debug logging for Phoenix backup proxy article to disable VMware logging.

Note:  Druva recommends defragmenting virtual machines whose backup can generate a large amount of logs. Contact Druva Technical Support if you need help in determining which folders/logs are safe to delete.