Problem Description

Application-aware transaction log (T-Log) backups may fail with exit code VMware21 when there is insufficient free space on the system drive.

During T-Log backups, the logs are temporarily copied to the system drive before being uploaded. The temporary storage location depends on the agent version.

Agent Version 6.x.x: %ProgramData%\Phoenix\VMWare\SQL\TlLogs

Agent Version 7.x.x: %ProgramData%\EnterpriseWorkloads\VMWare\SQL\TlLogs

Agent Version 7.0.5 and later %ProgramData%\Druva\EnterpriseWorkloads\VMWare\SQL\TlLogs

If the available free space on the system drive drops below 1 GB, the backup process fails and the job exits with the VMware21 error.

Cause

During application-aware T-Log backups, SQL transaction log files are temporarily written to the system drive before they are uploaded to the Druva Cloud.
If the system drive does not have enough free space, the backup process cannot complete this temporary write operation. When the available space falls below 1 GB, the application-aware processing stops and the backup job fails.
This situation commonly occurs when:
- The system drive is nearly full, or
- The transaction log files are large, causing the temporary storage location to run out of space during the backup process.

Traceback

Follow the instructions in the Logs article to obtain the job logs.

After extracting the logs, open the following file:

Phoenix<YYYYMMDD>_<timestamp>.log

Check whether the log contains an error similar to the following:

[2020-03-18 10:41:18,319] [ERROR] Error <class 'inSyncLib.inSyncError.SyncError'>:Failed to get TL backup status. error code: 4026 error msg: App-Aware processing failed since there is not enough space on the system volume <1 GB. Please ensure there is enough free space for app-aware processing to succeed (#100010015) (Error Code : VMWARE21). Traceback -Traceback (most recent call last):

File "agents/vmware/appaware_sql_tl_backup_restore.py", line 399, in upload_trans_log_file

SyncError: Failed to get TL backup status. error code: 4026 error msg: App-Aware processing failed since there is not enough space on the system volume <1 GB. Please ensure there is enough free space for app-aware processing to succeed (#100010015) (Error Code : VMWARE21)
Next, open the following log file to review disk space usage during the backup:

Phoenix-SQL-<YYYYMMDD>_<timestamp>_sqlguestplugin.log

This log shows the available free space on the system drive during the backup process. As the backup progresses, the free space decreases. When the available space falls below 1 GB, the backup fails.

Example log entries:

[2020-03-18 10:22:52,903] [INFO] Creating and opening virtual device for db[<DBname>] backup
[2020-03-18 10:22:52,997] [DEBUG] free system drive size: 40419921920
[2020-03-18 10:41:27,279] [DEBUG] free system drive size: 1074597888
[2020-03-18 10:41:27,303] [DEBUG] free system drive size: 1073549312
[2020-03-18 10:41:27,305] [ERROR] App-Aware processing failed since there is not enough space on the system volume <1 GB. Please ensure there is enough free space for app-aware processing to succeed
[2020-03-18 10:41:27,305] [ERROR] Low System Drive space

Note: The free system drive size shown in the logs is displayed in bytes.

Resolution

Determine the Total Size of .LDF Files
1. Open SQL Server Management Studio (SSMS) and connect to the SQL Server instance.
2. Run the following query to determine the size of log files for each database:
```
SELECT name AS DatabaseName, size * 8 / 1024 AS SizeMB FROM sys.master_files WHERE type_desc = 'LOG';
```
3. To calculate the total size of all transaction log files, run:
```
SELECT SUM(size * 8 / 1024) AS TotalSizeMB FROM sys.master_files WHERE type_desc = 'LOG';
```
Free Up Space on the System Drive
1. Identify unnecessary files on the system drive.
2. Move non-essential files or data to another drive.
3. Verify the available disk space using Windows Explorer or the dir command in Command Prompt.
Expand the System Drive
1. Open Disk Management on the server.
2. Expand the system drive using available unallocated space or a third-party disk management tool.
3. Confirm that the system drive size has increased.
Shrink SQL Transaction Log Files (If Required)
Consult the database administrator before performing this operation.
1. Option 1 – Using SSMS
  1. Right-click the database.
  2. Select Tasks > Shrink > Files.
  3. Set File type to Log.
  4. Select the appropriate log file.
  5. Click OK.
2. Option 2 – Using SQL Command
  1. Run the following command:
    DBCC SHRINKFILE (LogFileName, TargetSizeInMB);
    
    Replace:
    LogFileName with the actual log file name.
    TargetSizeInMB with the desired file size.
    
    Example:
    DBCC SHRINKFILE (Database1_log, 100);
Verify the New Log File Size
1. Re-run the following query to confirm the updated log size:
  SELECT SUM(size * 8 / 1024) AS TotalSizeMB FROM sys.master_files WHERE type_desc = 'LOG';