Problem description
Unable to perform the VMWare FLR Restore as Druva UI shows error as "Browse time out" and “Could not load configuration file.”
Cause
FLR Browse request creates a PhoenixStats.cfg file at “/var/Phoenix/VMWARE/stats/browse” location with the browse request number directory and the “PhoenixStats.cfg” file in it. One of the stats files has “null bytes” entry Issues due to which it was unable to load config files.
Traceback
FLR browse logs: 2023-05-30 14:18:11,074] [DEBUG] Error <type 'exceptions.TypeError'>:compile() expected string without null bytes. Traceback -Traceback (most recent call last): File "agentPartnerProcess.py", line 338, in _load_agent_config File "roboLib/roboConfigParser.py", line 1534, in load File "roboLib/roboConfigParser.py", line 174, in load File "roboLib/roboConfigParser.py", line 200, in _load File "roboLib/roboConfigParser.py", line 128, in loadstr File "ast.py", line 37, in parse TypeError: compile() expected string without null bytes
Mainservice.log: [2023-05-30 14:22:42,988] [ERROR] Fuse Mount timeout occured [2023-05-30 14:22:42,988] [ERROR] SyncError: Error in disk browse. Fuse failed. [2023-05-30 14:22:42,988] [ERROR] Error: Error in disk browse. Fuse failed. (#100070005) (Error Code : FLR5) [2023-05-30 14:22:42,989] [ERROR] Error <class 'inSyncLib.inSyncError.SyncError'>:Error in disk browse. Fuse failed. (#100070005) (Error Code : FLR5). Traceback -Traceback (most recent call last): File "client/vmware/vmware_client_rpcs.py", line 280, in vmware_command_disk_browse File "client/vmware/vmware_client_rpcs.py", line 725, in wait_for_mount File "inSyncLib/inSyncError.py", line 32, in raise_error SyncError: Error in disk browse. Fuse failed. (#100070005) (Error Code : FLR5)
[2023-05-30 14:22:42,989] [ERROR] SyncError occured in disk browse. Cleaning up...
Resolution
Identify and Login to the Backup Proxy server.
Identify the Proxy (If customer have multiple proxy):
Log in to the proxy and goto “/var/log/Phoenix/VMWARE/browse” directory.
Use command “ls -ltr” to find recent updated directory by time and “cd” to the latest modified directory.
Use “ls -ltr” again to sort and find latest log file to identify if the proxy is being used for browse job.
Go to “/var/Phoenix/VMWARE/stats/browse”
Identify the backup set ID which is having issues. (eg 131640)
Identify the backup set ID:
Navigate to “/var/log/Phoenix/VMWARE/browse” directory.
Open the browse job log, It will have the backup set id.
Eg: u'vm_backupset_id': 182164
Delete problematic backupset ID directory/s and file available under the “/var/Phoenix/VMWARE/stats/browse” directory.
Note:Before deletion please take a backup of the /var/Phoenix/VMWARE/stats/browse at some other location in the proxy.
Restart the Phoenix services
eg:
[root@RJP-BDR-02 browse]# ls 131640 131641 [root@RJP-BDR-02 browse]# rm -r 131640 rm: descend into directory ‘131640’? y rm: remove regular file ‘131640/PhoenixStats.cfg’? y rm: remove directory ‘131640’? y [root@RJP-BDR-02 browse]# service Phoenix restart Restarting Phoenix (via systemctl): [ OK ] [root@RJP-BDR-02 browse]# ls
Note:You might need to delete multiple directories if the issue is with multiple backup set.