Skip to main content

 

Druva Documentation

inSync Disaster Recovery process

Overview

Disaster recovery is a set of policies and procedures to overcome the failure and recover the infrastructure and data. This topic explains you how to plan and take necessary steps to recover from inSync, in case of a disaster. The process includes:

  1. Plan for Disaster Recovery
  2. Implement Disaster Recovery Plan
  3. Backup the Edge Server
  4. Backup inSync Master and Storage Node Server
  5. Expected frequency of backing up inSync components
  6. Resolve inSync Share issues on User Devices
  7. Restore inSync after a disaster
  8. After performing a restore
    1. Restore inSync Edge Server
    2. Restore inSync Master and Storage Node Server

Plan for Disaster Recovery

Overview

The components of your inSync Private Cloud deployment - inSync Master, inSync Storage Nodes, and the Edge Servers (optional) - are interconnected.

It is prudent to put in place disaster recovery procedures for your inSync Private Cloud set up. It will help you recover from the unfortunate disruptions in inSync’s functioning, such as follows:

  • The inSync Master fails
    • Embedded MySQL database fails
  • inSync Storage Nodes fails.
  • inSync Edge Server fails.

Failure of inSync Master or any Storage Node which hosts the inSync Share data, leads to failure of inSync Share.

To prepare for disasters, you must set up processes to back up all components of your inSync Private Cloud, together (in the single backup session/snapshot), at regular intervals. In case, you are unable to recover from failure of one or multiple components, you must restore all components from the latest recovery snapshot.

The inSync Master contains the configuration details of the following:

  • inSync users
  • User devices running inSync
  • inSync Server

inSync Master accesses the Config DB and Share DB and retrieves the following information:

From Config DB

  • Users
  • Devices
  • Profiles
  • Configurations

From Share DB

  • User to Share mapping details
  • Collaborators
  • Share details
  • Owners
  • Links
  • Notifications

The impact of a failure depends on the component affected. The following table describes the impact of failure for each component of your inSync Private Cloud set up.

Component inSync Users affected because of failure
inSync Master  All users are affected.
inSync Storage Node All users backing up their data and inSync Share data on the inSync Storage Node are affected.
inSync Edge Server

If the failed inSync Edge Server is for the inSync Master, all users backing up over WAN from outside your organization’s premises are affected.

If the failed inSync Edge Server is for a inSync Storage Node, only users backing up to the inSync Storage Node over WAN from outside your organization’s premises are affected.

Note:

  • It is recommended that you consult with Druva Support when preparing for disaster recovery.
  • Always have dry runs to ensure consistency of the backed up data and prepare for disaster recovery.
  • Recovery processes require you to start the inSync services after restoring the data. DO NOT START inSync services until the restore process is complete.

Implement Disaster Recovery Plan

Overview

Your disaster recovery plan must include backing up of all the components of your inSync Private Cloud deployments:

  • inSync Master
  • inSync Storage Nodes
  • inSync Edge Server (optional)

Because of the interlinked nature of the components, backing up and restoring components in isolation does not work.

Back up the Edge Server

Overview

If you have set up the inSync Edge Server, you must back it up when you install and configure it for the first time. You must have atleast one backup copy of the inSync Edge Server in order for recovery to work.

Procedure

To back up the Edge Server configuration files

  1. Stop the inSync Services on the Edge Server.
  2. Back up the following folders from the Edge Server.
    • Windows: C:\ProgramData\Druva\inSyncCloud
    • Linux: /etc/inSyncCloud, /var/log/inSyncCloud
  3. Start the inSync Services on the Edge Server.

Back up the inSync Master and Storage Node Server

 Overview

You must backup the inSync Master and inSync Storage Nodes when you install and configure it for the first time. You must have atleast one backup copy of the inSync Master and inSync Storage Nodes in order to recover the failure.

When backing up inSync components, Druva recommends either of the following approach:

  • Simultaneously backup inSync Master and all the inSync Storage Nodes situated across multiple locations.
    OR
  • Take complete backup of all the inSync Storage Nodes situated across multiple locations one at a time maintaining an order. After taking backup of the storage nodes, you must backup inSync Master.
    Example: Take a backup of storage node situated at X location first, followed by those situated at Y location and then inSync Master.

For any recovery to work, inSync Storage Node backup snapshot should be one behind the inSync Master snapshot.

Procedure

To back up your inSync Private Cloud deployment

  1. Stop the inSync services on the inSync Master.
  2. Stop the inSync services on each of the inSync Storage Nodes servers.
  3. Back up the following folders on the inSync Master.
    • Windows: C:\ProgramData\Druva\inSyncCloud
    • Linux: /etc/inSyncCloud, /var/log/inSyncCloud
    • If your organization uses external MySQL database, back it up manually using a MySQL utility.
  4. For each storage node server back up the following folders on the inSync Storage Nodes.
    • Windows: C:\ProgramData\Druva\inSyncCloud
    • Linux: /etc/inSyncCloud, /var/log/inSyncCloud
  5. Back up the Data folder of the storages. To locate the data folder:
    1. On the inSync Master Management Console menu bar, click Manage > Storage List.
    2. Under the Storage List tab, click the on the storage you want to backup.
    3. Under the Summary tab, in the Data Storage Details section, you can view the folder location for data folder.
  6. Back up the Database and Database Logs folder of the storages.
    1. On the inSync Master Management Console menu bar, click Manage > Storage List.
    2. Under the Storage List tab, click the storage to which you want to backup.
    3. Under the Performance and Compaction tab, in the Performance area you can view the location of Database and log folders.
  7. Start the services on the inSync Master.
  8. Start the services on all inSync Storage Nodes server.

To see the entire list of inSync services running on various inSync components and for instructions on how to start and stop them, see inSync Logs, Configuration files, and Services.

Expected frequency of backing up inSync components

Overview

The following table describes the frequency at which each inSync component should be backed up.

Component Frequency

inSync Master and inSync Storage Nodes

Both these components must be backed up together at regular intervals.

Druva recommends that you:

  • Backup inSync Master and inSync Storage Nodes every week.
  • Simultaneously backup all the inSync Storage Nodes situated across multiple locations and then inSync Master.
inSync Edge Server Only when you change the Edge Server configuration.

Resolve inSync Share issues on User Devices

Overview

Even after restoring inSync Master and inSync Storage Nodes, users can still come across issues with inSync Share on their devices.

Below are some scenarios with their resolution.

Scenario: Users see multiple conflict files in their inSync Share folder

Cause

If local copies of the files exist that have not been synced to the server, inSync will create conflict copies of the same files.

Resolution

Ask users to delete the local file and rename the file in inSync Share with the local file name.

Scenario: An already shared folder is available in "All Content" and in "Shared Content" with different name.

Cause

If a shared folder was renamed after the last backup of the inSync Storage Node, then the renamed shared folder entry is not available in the restored snapshot of the inSync Storage Node. This causes the original shared folder to be displayed as unshared in 'All Content' on inSync web and renamed folder to be displayed in 'Shared Content' listing.
 

Resolution

User who owns the shared folder must do the following:

  1. Create a new folder.
  2. Copy the files in the new folder from the previously shared folder.
  3. Share this new folder with the same collaborators.

Scenario: Files inside a shared folder are not getting synced to the server

Cause

If a folder was created and shared with other users after last backup of the inSync Storage Node, then files inside the shared folder will not get synced to the server.

Resolution

User who created and shared the original folder must do the following:

  1. Create a new folder.
  2. Copy the files in the new folder from the previously shared folder.
  3. Share this new folder with the same collaborators.

Restore inSync after a disaster

Overview

In case, a particular component fails try to identify the issue and recover the component. If the recovery fails, you must restore all components of your inSync deployment.

Disaster Recovery Scenarios

Any of the following disaster scenarios can occur

Scenario Impact
One or more inSync Storage Nodes have crashed, while the inSync Master and inSync Client are both residing at the latest timestamp.
  • All the users residing on those inSync Storage Nodes are impacted
  • All inSync Share users are impacted.
inSync Master has crashed, while inSync Storage Node and inSync Client are both residing at the latest timestamp. All the users residing on all the inSync Storage Nodes are impacted.
Both, inSync Storage Node and inSync Master have crashed, while the inSync Client is residing at the latest timestamp. All the users residing on all the inSync Storage Nodes are impacted.

Scenario: Only inSync Storage Node has crashed

Perform a recovery from the last snapshot of the inSync Storage node. For detailed steps, see Restore inSync Master and Storage Node Server.

inSync Share: In addition to restoring the inSync Storage Node, you must perform series of steps on the end user devices to resolve issues with inSync Share. For more information, contact Druva Support.

Scenario: Both inSync Master and inSync Storage Node have crashed

Perform a recovery from the last snapshot of both, inSync Master and inSync Storage node. For detailed steps, see Restore inSync Master and Storage Node Server.

inSync Share: After restoring inSync Master and inSync Storage Node, you must perform series of steps on the end user devices to resolve issues with inSync Share. For more information, contact Druva Support.

After performing a restore

This topic lists the things you should verify after you have performed a restore.

After you perform a restore, verify the following:

  • Verify that the restore has been successful and all the data has been restored successfully. If restore has failed with errors or skipped files identify the issue and restart the restore job.
  • The IP address/Host name of the source (crashed machine) and target machines (recovery machine), should be same. This ensures that when the target machines are up and running, inSync Clients are able to connect to it.
  • Check if the storage folders (Database, Logs, and Data) are restored on the correct drives and accessible.
    Set the following permissions for all the folders restored (configuration and storage data, database and logs)
    • Windows:  Set full access permissions for system account for all the restored data.
    • Linux: The inSync server user and group should have full permissions on the restored data.
  • After you have set the permissions install the inSync component (Master/Storage node) software.
  • Login to the inSync Admin console and verify if the Storage shows a healthy status.

To see the entire list of inSync services running on various inSync components and for instructions on how to start and stop them, see inSync log files, configuration files, and services.

Restore inSync Edge Server

Procedure

To restore the Edge Server configuration file

  1. Stop the inSync services on the Edge Server.
  2. Copy the backed up configuration files to their original location on the Edge Servers.

Note: After restores on the respective servers are completed, you need to proceed with the installation. For detailed instructions to install inSync Edge Server, see Set up inSync Edge Servers.

Restore inSync Master and Storage Node Server

Before you begin

  • In case of a disaster scenario, restore the backed up inSync data on a fresh server.
  • Before you install the inSync components (Master or node installers) you must restore the configuration and the actual storage data to the original locations.
Procedure

To restore your inSync Private Cloud deployment

  1. Restore the configuration (including Embedded MYSQL database) and the log files of the inSync Master to their original location.
    • Windows: C:\ProgramData\Druva\inSyncCloud
    • Linux: /etc/inSyncCloud, /var/log/inSyncCloud
    • If your organization uses external mySql database, restore it manually.
  2. Restore the configuration and the log files of the inSync Storage Nodes to their original location.
    Note:Ensure that the snapshot from which you are restoring inSync Storage Node is an older or of the same version of the snapshot using which inSync Master is restored.
  3. For each storage, restore the following:
    • Database folder
    • Database Logs folder
    • Data folder