Problem Description
An Oracle RMAN backup job may fail during the data transfer phase while processing a datafile. The job may run for several hours before terminating with media manager–related errors.
This issue typically occurs when the connection between the Oracle RMAN process and the Phoenix SBT/IOServer is interrupted during a long-running backup session.
Traceback
The following errors may appear in the RMAN logs:
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of backup command on ch1 channel at 12/04/2025 06:00:48
ORA-27192: skgfcls: sbtclose2 returned error - failed to close file
ORA-19511: non RMAN, but media manager or vendor specific failure, error text:
RPC invalid client session request error.
ORA-19502: write error on file "Phx_DN-APPCDB1_DBID-256270233_DT-20251203-220957_2q4affei_71770_1_1", block number 1539073 (block size=8192)
ORA-27030: skgfwrt: sbtwrite2 returned error
ORA-19511: non RMAN, but media manager
These errors indicate that the RPC client session between RMAN and the media manager became invalid during the backup process.
Cause
The failure occurs when the RPC session between Oracle RMAN and the Phoenix SBT/IOServer is interrupted.
This usually happens when a firewall, load balancer, or network device terminates long-running or idle sessions. Since Oracle backups can run for several hours, network devices with aggressive idle timeout or session timeout settings may close the connection, causing the backup to fail.
Resolution
1. Rerun the Backup Job
Rerun the failed backup job.
In many cases, the failure may be transient, and the backup may complete successfully when retried.
2. Verify Network Stability
Validate the network path between the Oracle host and the Phoenix Media Agent.
Ensure the following:
No aggressive idle session timeouts
No aggressive connection timeout policies
Network devices support long-running sessions
Check configurations on:
Firewalls
Load balancers
Network gateways
3. Validate Keepalive Configuration
Ensure that the environment supports TCP keepalive or similar mechanisms to maintain long-running sessions during backups.
This helps prevent network devices from closing idle connections.
4. Validate Port Connectivity
Test connectivity between the Oracle host and the Phoenix IOServer.
Verify that port 20000 remains open and stable during long-running connections.
Example test:
Initiate a connection from the Oracle host to the Phoenix IOServer on port 20000.
Keep the connection open for several hours.
Confirm that the connection is not terminated by network devices.