Backup

The backup operation enables you to save the current node data state. 

There are several types of backup operations available:

  • Cold backup – Perform a backup on a stopped node.
  • Hot backup – Perform a backup on a running node.

On a primary/replica cluster, you only need to back up the primary node. 

To backup a specific checkpoint (that is a specific data state), use the bin/tnd-backup-checkpoint script which copies all the files composing a checkpoint into another directory.

Usage: <install directory>/bin/tnd-backup-checkpoint <checkpoint TT> <output directory>

where:

  • <checkpoint TT> is the checkpoint identifier (for example, 1549260116593737729).
  • <output directory> is the directory into which the files will be written.

(warning) On a running node, you may want to lock the checkpoint before calling the script. This ensures all checkpoint files are available until the script is executed.

Example:

  Click here to expand...
qatest@lptxdiqa03:~/platform$ bin/tnd-backup-checkpoint.sh 1549260116593737729 /home/qatest/backup
-----------------------------------------------------------------------------
Collecting files...
Checkpoint contains 28 files totaling 1464494 bytes (~=1 MB)
-----------------------------------------------------------------------------
Copying files...
        Copying file 'calcium/checkpoint/tt01549260116593737729/checkpoint.bin' (1/28)
        Copying file 'calcium/checkpoint/tt01549260116593737729/filelist.txt' (2/28)
        Copying file 'calcium/checkpoint/tt01549260116593737729/metadata.txt' (3/28)
        Copying file 'calcium/checkpoint/tt01549260116593737729/sha1sum.txt' (4/28)
        Copying file 'titanium-temporal/configuration/checkpoint/tt01549260116593737729/alive.bin' (5/28)
        Copying file 'titanium-temporal/configuration/checkpoint/tt01549260116593737729/amemtable.bin' (6/28)
        Copying file 'titanium-temporal/configuration/checkpoint/tt01549260116593737729/catalog.xml' (7/28)
        Copying file 'titanium-temporal/configuration/checkpoint/tt01549260116593737729/filelist.txt' (8/28)
        Copying file 'titanium-temporal/configuration/checkpoint/tt01549260116593737729/hvmemtable.bin' (9/28)
        Copying file 'titanium-temporal/configuration/checkpoint/tt01549260116593737729/hvopen.bin' (10/28)
        Copying file 'titanium-temporal/configuration/checkpoint/tt01549260116593737729/metadata.txt' (11/28)
        Copying file 'titanium-temporal/configuration/checkpoint/tt01549260116593737729/rmemtable.bin' (12/28)
        Copying file 'titanium-temporal/configuration/checkpoint/tt01549260116593737729/sha1sum.txt' (13/28)
        Copying file 'titanium-temporal/configuration/checkpoint/tt01549260116593737729/sstable.bin' (14/28)
        Copying file 'titanium-temporal/configuration/config.xml' (15/28)
        Copying file 'titanium-temporal/configuration/sha1sum.txt' (16/28)
        Copying file 'titanium-temporal/partition-0/checkpoint/tt01549260116593737729/alive.bin' (17/28)
        Copying file 'titanium-temporal/partition-0/checkpoint/tt01549260116593737729/amemtable.bin' (18/28)
        Copying file 'titanium-temporal/partition-0/checkpoint/tt01549260116593737729/catalog.xml' (19/28)
        Copying file 'titanium-temporal/partition-0/checkpoint/tt01549260116593737729/filelist.txt' (20/28)
        Copying file 'titanium-temporal/partition-0/checkpoint/tt01549260116593737729/hvmemtable.bin' (21/28)
        Copying file 'titanium-temporal/partition-0/checkpoint/tt01549260116593737729/hvopen.bin' (22/28)
        Copying file 'titanium-temporal/partition-0/checkpoint/tt01549260116593737729/metadata.txt' (23/28)
        Copying file 'titanium-temporal/partition-0/checkpoint/tt01549260116593737729/rmemtable.bin' (24/28)
        Copying file 'titanium-temporal/partition-0/checkpoint/tt01549260116593737729/sha1sum.txt' (25/28)
        Copying file 'titanium-temporal/partition-0/checkpoint/tt01549260116593737729/sstable.bin' (26/28)
        Copying file 'titanium-temporal/partition-0/config.xml' (27/28)
        Copying file 'titanium-temporal/partition-0/sha1sum.txt' (28/28)
-----------------------------------------------------------------------------
Backup of checkpoint 1549260116593737729 is available in directory '/home/qatest/backup'

Cold backup

You can do a cold backup only if the node is properly stopped as described in this guide or if at least one checkpoint was created before the node crash occurred.

  1. Stop the node if it is not already stopped. In the node command shell. Type the exit 0 command and wait for the node to stop.

  2. Remove any existing orphaned directory from all stores. Navigate to <install directory>/var/data/titanium-temporal/ <partition name>/orphaned.
  3. Make a file copy of all stores. Copy the  <install directory>/var/data/titanium-temporal/ directory.
  4. Make a file copy of engine states. Copy the  <install directory>/var/data/calcium/ directory.
  5. Optionally, create an archive file of the previously copied directories (for instance, create a .zip file).  

Cold backup with Docker

You can perform a cold backup with Docker only if:

  • The node in the container is properly stopped as described in this guide.
  • Or if at least one checkpoint was created before the node crash occurred.

Let us assume your container is named adi_XXXXXXXX-XX.

  1. Stop the container if it is not already stopped.

    $ docker stop --time 60 adi_XXXXXXXX-XX

    You can check that the container is stopped using :

    $ docker ps -a
    CONTAINER ID        IMAGE                               COMMAND                CREATED              STATUS                       PORTS                     NAMES
    b1018a5699ce        axway/decisioninsight:XXXXXXXX-XX   "/opt/adi/platform/b   15 minutes ago       Exited (143) 9 seconds ago                             adi_XXXXXXXX-XX
  2. Start a temporary container with all volumes from adi_XXXXXXXX-XX mounted on it: 

    • Remove any existing orphaned directory from all stores ( /opt/adi/node/var/data/titanium-temporal/<partition name>/orphaned ).
    • Make a file copy of all stores. Copy the /opt/adi/node/titanium-temporal/ directory into the backup directory.
    • Make a file copy of engine states. Copy the /opt/adi/node/var/data/calcium/ directory into the backup directory.
    • Optionally, create an archive file of the previously copied directories (for instance, create a .tar.gz file).  
    $ docker run --rm --tty --interactive --user adi-admin --entrypoint=/bin/sh --volumes-from=adi_XXXXXXXX-XX -v ~/backup:/backup axway/decisioninsight:XXXXXXXX-XX
    / # rm -rf /opt/adi/node/var/titanium-temporal/<partition name>/orphaned
    / # cp -R /opt/adi/node/var/data/titanium-temporal /backup/
    / # cp -R /opt/adi/node/var/data/calcium /backup/
    / # cd /backup
    / # tar -czvf /backup/backup_decisioninsightXXXXXXXX-XX.tar.gz ./titanium-temporal ./calcium
    / # exit

Hot backup

Choose an available checkpoint for backup or create a new one.

To list the available checkpoints in the node:

  1. Type the following command in the node command shell:

    calcium:listCheckpoints

    The command displays an output similar to this:

    Checkpoints found:
    1394714001032609793 (locked)
    1394714012096135169
  2. Write down the transaction time of the checkpoint you wish to use for backup.
  3. Lock the selected checkpoint if it is not already locked.

     calcium:lockCheckpoint <checkpointTransactionTime>

    Where <checkpointTransactionTime> is the transaction time given by the previous createCheckpoint command (for instance: lockCheckpoint 1394691903529680897 in the above example)

If you do not find any available checkpoint or you want to create a new one:

  1. Type the following command in the node command shell:

    calcium:createCheckpoint <checkpoint name>

    The command will display a similar output:

    Checkpoint created with transaction time 1394691903529680897
  2. Write down the transaction time of the created checkpoint and go to step 3. The newly created checkpoint is automatically locked.

To create a backup for your checkpoint:

  1. For each store directory in <install directory>/var/data/titanium-temporal/<store name>/, make a file copy of all files referenced in the  <install directory>/var/data/titanium-temporal/<store name>/checkpoint/<checkpointTransactionTime>/filelist.txt  file.
    where  <checkpointTransactionTime> is the transaction time provided by the previous createCheckpoint command.
  2. Make a file copy of the <install directory>/var/data/calcium/checkpoint/tt<checkpointTransactionTime> directory.
  3. Optionally, create an archive file of the previously copied files (for instance, create a .zip file). 
     

Example: backup the "1394691903529680897" checkpoint.

  1.  For each titanium store checkpoint, copy all the files listed in filelist.txt: (keep the path from "plaform/.." for each file).This means, all files listed in:
      • var/data/titanium-temporal/configuration/checkpoint/tt01394691903529680897/filelist.txt
      • var/data/titanium-temporal/partition-0/checkpoint/tt01394691903529680897/ filelist.txt
  2. Copy the calcium checkpoint directory, that is var/data/calcium/checkpoint/tt01394691903529680897.

Once you're done, unlock your checkpoint. In the node command shell, type the calcium:unlockCheckpoint <checkpointTransactionTime> command,

where <checkpointTransactionTime> is the transaction time provided by the previous createCheckpoint command. For example, type unlockCheckpoint 1394691903529680897 to follow the above scenario. 

Hot backup with Docker

To backup data from a node running inside a docker container, follow these steps.

Let us assume your container is named adi_XXXXXXXX-XX.

  1. Attach to the process standard input and output  with the docker attach command:

    $ docker attach adi_XXXXXXXX-XX
  2. Choose an available checkpoint for backup or create a new one. 
    To list the available checkpoints in the node, type the following command in the node command shell:

    calcium:listCheckpoints

    The command displays an output similar to:

    Checkpoints found:
    1394714001032609793 (locked)
    1394714012096135169

    Write down the transaction time of the checkpoint that you wish to use for backup and proceed to step 3.

    If you do not find any checkpoint available in the node or want to create a new one, type the following command in the node shell:

    calcium:createCheckpoint <checkpoint name>

    The command will display an output similar to:

    Checkpoint created with transaction time 1394691903529680897

    Write down the transaction time of the created checkpoint and proceed to step 4. The newly created checkpoint is automatically locked.

     

  3. Lock the selected checkpoint  if not already locked:

     calcium:lockCheckpoint <checkpointTransactionTime>

    Where <checkpointTransactionTime> is the transaction time you selected in the previous step.

  4. Detach the process.

    (warning) If you press Control+C , the node is stopped, and then the attach command will terminate. To detach without stopping the node, you must press Control+PQ (that is press and hold the Control key, then press and release P, then press and release Q, then release the Control key).

  5. Start a temporary container with all volumes from adi_XXXXXXXX-XX mounted on it.

    1. Execute the command tnd-backup-checkpoint.sh  to copy the checkpoint into the backup directory.
    2. Optionally, create an archive file of the previously copied directories. For instance, create a .tar.gz file. 
    $ docker run --rm --tty --interactive --user adi-admin --entrypoint=/bin/sh --volumes-from=adi_XXXXXXXX-XX -v ~/backup:/backup axway/decisioninsight:XXXXXXXX-XX
    / # /opt/adi/node/bin/tnd-backup-checkpoint.sh <checkpointTransactionTime> /backup
    / # cd /backup
    / # tar czvf /backup/backup_decisioninsightXXXXXXXX-XX.tar.gz ./titanium-temporal ./calcium
    / # exit
  6. Attach to the process standard input and output with the docker attach command again and unlock the previously created checkpoint.

    $ docker attach adi_XXXXXXXX-XX

    In the node command shell, type the following command:

     calcium:unlockCheckpoint <checkpointTransactionTime>

    Where <checkpointTransactionTime> is the transaction time you selected in step 2.

     

Related Links