Problem Description
This article addresses a specific failure during Disaster Recovery (DR) restore operations in an AWS environment using Druva AWS proxy. The failure occurs due to IAM role credential propagation issues, even when the IAM role appears attached in the AWS Console.
Traceback
[ERROR] AWS was not able to validate the provided access credentials (#100080080)
(Error Code : AWS_DR128)
Service logs contain errors like:
unable to get aws infra details: failed to execute function: get_vpc_details
Metadata command returns no IAM credentials.
AWS proxy instance shows IAM role in Console, but APIs return credential errors.
Cause
The proxy EC2 instance failed to retrieve valid IAM role credentials due to one of the following:
Stale IAM instance profile attachment β the role is attached but not active internally.
Metadata service (IMDS) not returning credentials.
The proxy service (like Druva) using expired or uninitialized credentials.
Role policies or SCPs not allowing necessary permissions.
This leads to AWS rejecting the signed requests, causing restore jobs to fail with AWS_DR128.
Verification
Step 1: Check Metadata Service for IAM Role
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/
Expected: Role name (e.g., DruvaIAMRolePL)
Actual:Empty output or 404
Step 2: Fetch Role Credentials
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/DruvaIAMRolePL
Expected: JSON with AccessKeyId, SecretAccessKey, and Token
Actual:Timeout, 403 error, or no output
This confirms the EC2 instance has no valid credentials, even though the IAM role is attached.
Resolution
1. Detach and Reattach IAM Role
Go to EC2 > Actions > Security > Modify IAM Role.
Detach the role (wait 10β15 seconds).
Reattach the same role.
2. Restart the Proxy EC2 Instance
Reboot ensures:
Re-initialization of metadata services.
Instance profile binding is refreshed.
3. Verify Metadata is Returning Valid Credentials
After reboot:
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/DruvaIAMRolePL
This must return temporary credentials with a valid expiration time.
Best Practices for IAM on Druva Proxy
Best Practice | Description |
π Always validate via metadata | Donβt rely solely on Console β use curl to confirm role detection. |
β» Reattach IAM role + reboot | This resets instance profile associations cleanly. |
π§ Understand IMDSv2 if enabled | If IMDSv2 is enforced, ensure your software requests tokens before accessing metadata. |
π Check for SCPs or permission boundaries | They may silently override role permissions even if they appear correct. |
π§° Restart backup/restore agent post reboot | Always reboot/restart the service after IAM changes. |