Problem description:
The CloudCache service encounters "access forbidden" errors when attempting to bind to TCP port 443. This port is essential for the CloudCache service to receive incoming backup requests. The error message typically indicates that the socket access is forbidden due to permissions.
Cause:
The root cause of this issue is another service already listening on TCP port 443, preventing the CloudCache service from acquiring the necessary binding. On Windows servers, this often involves the "System" process (PID 4) holding the port via the HTTP.sys kernel driver. Common services that utilize HTTP.sys and can conflict on port 443 include:
Remote Desktop Web Access (RDWebAccess): A component of Remote Desktop Services that provides a web portal for accessing remote applications and desktops.
Internet Information Services (IIS): Particularly the "Default Web Site," which commonly binds to ports 80 (HTTP) and 443 (HTTPS) by default.
Other Microsoft services such as SQL Server Reporting Services (SSRS) or Windows Admin Center (WAC) can also bind to port 443.
Traceback:
When the CloudCache service attempts to start, Entries similar to below will get logged:
level=debug ts=2025-05-29T14:32:13.4474749Z filename=main.go:274 component=main restServerExit="listen tcp :443: bind: An attempt was made to access a socket in a way forbidden by its access permissions."
level=debug ts=2025-05-29T14:32:13.4490803Z filename=main.go:251 component=main msg="Exiting cache service" errorStack="Service internal error: listen tcp :443: bind: An attempt was made to access a socket in a way forbidden by its access permissions.
Logs for the CloudCache service can be found at: C:\ProgramData\Druva\EnterpriseWorkloads\logs\cloudcache\mainservice\ with the primary log file named cloudcache.
This error indicates that the port is in use or the application lacks the necessary permissions to bind to it, due to another process holding the port.
Resolution:
The resolution involves identifying and disabling/reconfiguring the service(s) currently occupying TCP port 443 to free it for the CloudCache service. CloudCache service cannot be reconfigured to use an alternative port.
Identify the conflicting process on Port 443:
Open Command Prompt as Administrator.
Run the command: netstat -ano | findstr :443
Look for a line with LISTENING status for :443 and note the Process ID (PID) associated with it. If PID 4 is listed, it confirms that HTTP.sys is in use.
Screenshot Example (netstat output showing PID 4 listening on 443):
Identify the specific service(s) using HTTP.sys on Port 443:
In the same Administrator Command Prompt, run: netsh http show servicestate
Examine the output carefully for URL groups that have HTTPS://*:443/ or more specific HTTPS://*:443/ paths listed under "Registered URLs."
Note the "Request queue name" associated with these entries. Common names are DefaultAppPool (indicating IIS) or RDWebAccess.
Screenshot Example (netsh http show servicestate output showing RDWebAccess):
Screenshot Example (netsh http show servicestate output showing DefaultAppPool):
Disable/Stop the Conflicting Service(s):
If DefaultAppPool (IIS) is found:
Open IIS Manager (search for "IIS Manager" in Windows Start).
In the "Connections" pane, expand your server name, then "Sites."
Right-click on "Default Web Site" and select "Stop."
This action will release the port 443 binding associated with IIS.
If RDWebAccess (Remote Desktop Web Access) is found:
Open Server Manager.
Go to "Manage" -> "Remove Roles and Features Wizard."
On the "Server Roles" page, locate and uncheck "Remote Desktop Services." Follow the wizard prompts to remove the role. This may require a server restart.
Note: Removing the RDWebAccess role will disable the web-based portal for accessing RemoteApps and desktops, but it will not impact direct RDP access to the server on port 3389.
If other services are identified:
Open Services (search for services.msc in Windows Start).
Locate the service corresponding to the "Request queue name" identified in step 2 (e.g., SQL Server Reporting Services, Windows Admin Center).
Right-click the service and select "Stop." To prevent it from starting automatically after a reboot, set its "Startup type" to "Manual" or "Disabled" (use caution with system services).
Restart the CloudCache Service:
Open Services (search for services.msc in Windows Start).
Locate the "Druva-EnterpriseWorkloads" service.
Right-click on it and select "Restart."
Verification:
After performing the resolution steps:
Verify Port 443 is Free:
Open Command Prompt as Administrator.
Run: netstat -ano | findstr :443
Confirm that there are no LISTENING entries for port 443.
Check CloudCache Service Status:
Open Services (services.msc).
Ensure the "Druva-EnterpriseWorkloads" service is running.
Test Backup Operations:
Initiate a test backup to confirm that the CloudCache service is correctly receiving incoming requests on port 443.
Review CloudCache logs (at C:\ProgramData\Druva\EnterpriseWorkloads\logs\cloudcache\mainservice\cloudcache) for any new errors or successful connection indications.