Overview

DRaaS failback failed with error DR16419. The DR failback jobs stuck at 6%

Issue

In the job logs, we see the below error

Traceback

[2021-08-19 09:21:53,474] [ERROR] Error <class 'inSyncLib.inSyncError.SyncError'>:Timeout occurred in attribute collection (#1000b4000) (Error Code : DR16384). Traceback -Traceback (most recent call last):
File "agents/failback/fbconversion.py", line 1215, in do_vm_failback
File "agents/failback/fbconversion.py", line 955, in run_attribute_collection
File "agents/failback/tw_commands.py", line 83, in attribute_collection
File "agents/failback/tw_commands.py", line 75, in _wait_for_attribute_collection
SyncError: Timeout occurred in attribute collection (#1000b4000) (Error Code : DR16384)
[2021-08-19 09:21:53,495] [INFO] failback: InstanceId:[17901] status updated to:[Request Failed]
[2021-08-19 09:21:53,513] [INFO] DR Failback Finished with failure for instance:[17901]
[2021-08-19 09:21:53,520] [ERROR] Failback failed for all instances.
[2021-08-19 09:21:53,520] [ERROR] Failback Finished with failure: Failback failed for all instances (#1000b4023) (Error Code : DR16419)
[2021-08-19 09:21:53,521] [ERROR] Error <class 'inSyncLib.inSyncError.SyncError'>:Failback failed for all instances (#1000b4023) (Error Code : DR16419). Traceback -Traceback (most recent call last):
File "agents/failback/drFailback/failback.py", line 452, in perform_failback
File "agents/failback/drFailback/failback.py", line 391, in _calculate_and_set_fb_job_status
SyncError: Failback failed for all instances (#1000b4023) (Error Code : DR16419)

Cause

We were not able to connect to \\ip-of-EC2-instance\c$ from source network.

Resolution

To override this setting for Built-in admins without changing Domain policy, please add reg key/value on the EC2 instance:

1. Execute the below command on EC2 instance:

reg add "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System" /v

"LocalAccountTokenFilterPolicy" /t REG_DWORD /d 1 /f

2. Then reboot EC2 instance.

3. Kindly verify SMB connection to a public IP again.

4. Re-run the failback and it should be successful now.