Problem description
M365 backup fails showing error ‘ETHROTTLE ’ or ‘Throttling error’ on Druva management console.
Cause
Throttling limits the number of concurrent calls to a service to prevent overuse of resources. Microsoft Graph is designed to handle a high volume of requests. If an overwhelming number of requests occurs, throttling helps maintain optimal performance and reliability of the Microsoft Graph service.
When a throttling threshold is exceeded, Microsoft Graph limits any further requests from that client for a period of time.
When throttling occurs, Microsoft Graph returns HTTP status code 429 (Too many requests), and the requests fail.
A suggested wait time is returned in the response header of the failed request. Throttling behaviour can depend on the type and number of requests.
The most common causes of throttling of clients include:
A large number of requests across all applications in a tenant.
A large number of requests from a particular application across all tenants
This commonly occurs if you have configured M365 newly and are running backups for the first time.
Traceback
The failures will be intermittent.
📝 Note
+snip+2022-11-12 1922:47,133] [INFO] Microsoft 365 group backup finished successfully. For group: JXXX[2022-11-13 19:22:48,336] [DEBUG]HttpError 429 when requestingfor user returned 'Response :{'error': {'code': 'activityLimitReached', 'message': 'The request has been throttled', 'innerError': {'code': 'throttledRequest', 'innerError': {'code': 'quota'}, 'date': '2022-11-13T19:22:48', 'request-id': '1a0257b9-02b5-454c-9b2e-23d260fd84c2', 'client-request-id': '1a0257b9-02b5-454c-9b2e-23d260fd84c2'}}}'Retrying 2 after sleeping 20 secs.+snip++snip+[2022-11-15 09:52:59,785] [DEBUG] API HttpError 500 when requestingfor user returned 'Response :{'error': {'code': 'UnknownError', 'message': '', 'innerError':[2022-11-16 07:23:16,685] [INFO] Microsoft 365 group backup finished successfully. For group: XXXX[2022-11-10 09:53:59,618] [DEBUG] API HttpError 429 when requestingfor user returned 'Response :{'error': {'code': 'activityLimitReached', 'message': 'The request has been throttled', 'innerError':+snip+
Resolution
The following are best practices for handling throttling
Decrease the frequency of backups. For example if the frequency is set to 'Twice a day' then change it to 'Once a day'.
Steps to change the Backup frequency.
Navigate to Profile and open the profile which the user belongs to.
Click on Edit>SaaS Apps>Backup Schedule & Retention.
Reduce the backup frequency from the current value.
Verification
After increasing the backup frequency, Number of requests would be reduced as changes would be backed up frequently. Backups will complete successfully.
Monitor the backups for a couple of days, If you still observe throttling please contact Druva support for deploying Auxiliary app.
See also