Skip to main content
Druva Documentation

Backup fails with SSL or certificate error during certificate validation

This article applies to:

  • OS: All supported operating systems    
  • Product edition: inSync Cloud and On-Premise

Problem description

Backups fail in the networks having proxy servers. The following error is found in the logs:

[WARNING] Backup failed. Error: Network not reachable. (#100000022)
[ERROR] SSL/certificate error while validating the cloud server's certificate. Error: [('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')]

Cause

An SSL termination proxy in the network can cause this failure.

The SSL Termination Proxy handles the incoming SSL/TLS connections, decrypt the SSL/TLS, and passes on the unencrypted requests to the destination.

SSL/TLS termination proxy reduces the load on the main server by offloading the cryptographic processing to another system and supports the servers that do not support SSL/TLS. During this operation, the SSL termination proxy server, decrypts the “Server Hello” packet and changes the “Issuer” attribute of the Druva certificate located in the cloud or the master server. The “Server Hello” packet is the network packet in which inSync Server sends its public certificate information to client.
inSync Client is designed to trust those certificates which have been issued by a known CA. Druva cloud certificates are issued by: USERTrust RSA Organization Validation Secure Server CA, The USERTRUST Network,  Jersey City , New Jersey, US. 

If any certificate issued by any other issuer reaches the inSync Client, it will not be able to continue the backup process.

Traceback

inSyncClient.log

[2018-11-29 07:15:06,747] [INFO] Trying to connect to cloud.druva.com:443.
[2018-11-29 07:15:06,951] [ERROR] SSL/certificate error while validating the cloud server's certificate. Error: [('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')]
[2018-11-29 07:15:06,953] [ERROR] Connect2: error while connecting to server: cloud.druva.com:443 Error: Network not reachable. (#100000022).
[2018-11-29 07:15:06,953] [INFO] Trying to connect to cloud.druva.com:6061.
[2018-11-29 07:15:07,115] [INFO] Connection successful with cloud.druva.com:6061.
[2018-11-29 07:15:17,331] [ERROR] SSL/certificate error while validating the cloud server's certificate. Error: [('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')]
[2018-11-29 07:15:17,334] [ERROR] Error during activationPostlude. Error : Network not reachable. (#100000022)
[2018-11-29 07:15:17,690] [ERROR] SSL/certificate error while validating the cloud server's certificate. Error: [('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')]
[2018-11-29 07:15:17,691] [ERROR] Error <class 'inSyncLib.inSyncError.SyncError'>:Network not reachable. (#100000022). Traceback -Traceback (most recent call last):
  File "inSyncLib\inSyncSyncer.pyc", line 4407, in sync
  File "inSyncLib\inSyncSyncer.pyc", line 4899, in dosync
  File "inSyncLib\inSyncSyncer.pyc", line 4028, in connect
  File "inSyncLib\inSyncRPCHelper.pyc", line 307, in Connect3
  File "inSyncLib\inSyncRPCHelper.pyc", line 34, in validate_server
  File "inSyncLib\inSyncRPCClient.pyc", line 305, in srvcert_invalid
  File "inSyncLib\inSyncRPCBase.pyc", line 1275, in connect
  File "inSyncLib\inSyncRPCClient.pyc", line 390, in sslwrap
SyncError: Network not reachable. (#100000022)
 
[2018-11-29 07:15:17,753] [WARNING] Backup failed. Error: Network not reachable. (#100000022)
[2018-11-29 07:15:17,871] [ERROR] SSL/certificate error while validating the cloud server's certificate. Error: [('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')]

Resolution

Take a network trace with the help of tools like Microsoft Network Monitor or Wireshark, while reproducing the issue. Look for the incoming  "Server Hello" packet to get the information about the certificate that is passed through the entire network and eventually reaching the inSync Client. Check the 'Issuer' attribute in the certificate section of the "Server Hello" packet.

Example 

In the trace snippet below, the issuer of the certificate is the proxy server, as the packet has been examined and processed by the proxy server before it reaches the inSync Client.

BackupFailureScenario.png

Use one of the following options to resolve the backup failure:

  • Whitelist *.druva.com in the proxy server with the help of the in-house networking team. This will exclude any SSL/TLS connection established by Druva. 
  • Turn off SSL/TLS termination proxy feature from the proxy server or router.

BackupSuccessScenario.png