Upgrade Apache Cassandra

This topic explains how to upgrade Apache Cassandra from version 2.2.5 to version 2.2.8. We recommend that you upgrade your Cassandra version after you have upgraded your API Gateway installation to version 7.5.3.

Why upgrade?

API Gateway 7.5.3 supports Apache Cassandra versions 2.2.5 and 2.2.8. This means that you do not have to upgrade Cassandra when you are upgrading to API Gateway 7.5.3, however, we recommend that you upgrade Apache Cassandra to version 2.2.8 to avail of the 160 improvements (in particular stability improvements) in that version relative to 2.2.5. Cassandra version 2.2.8 is the recommended version for multi-node high availability (HA) environments.

Best practice

We recommend the following when upgrading your Apache Cassandra version:

  • Upgrade your API Gateway installation before upgrading your Apache Cassandra version.
  • In multi-datacenter clusters, upgrade every node in one datacenter before upgrading another datacenter. Upgrade and restart the nodes one at a time. Other nodes in the cluster continue to operate at the earlier version until all nodes are upgraded.
  • When upgrading a cluster on a single-datacenter or multi-datacenter setup, you must avoid any schema changes until the entire cluster has been upgraded to the same version.
  • Running nodetool repair on a Cassandra node will affect performance on a system running live traffic. We recommend that you perform the Cassandra upgrade in the evening or during a maintenance window when the load is minimal.

Cassandra upgrade steps – Single-node

The following steps give an example of how to upgrade Cassandra in a single-node setup.

Step 1 – Install Cassandra 2.2.8

Follow these steps to install Apache Cassandra 2.2.8 using the API Gateway installer in default GUI mode:

  1. Select the Custom option in the installer.
  2. When prompted to select components to install, select only the Cassandra component.
  3. When prompted for the Cassandra configuration, enter the following settings:
Operating System Installation Directory JRE Location
UNIX/Linux /opt/db/cassandra-228 /opt/jre
Windows c:\cassandra-228 C:\Program Files\Java\jre8
Caution   On Windows, do not use the default 32-bit JRE included in the API Gateway installer. Instead, you must install a 64-bit JRE and enter its location in the JRE Location field.
  1. Do not start Cassandra 2.2.8 when the installation completes. (Cassandra 2.2.5 should still be running.)

You can also install Cassandra in unattended mode, for example:

UNIX/Linux:

./APIGateway_7.5.3_Install_linux-x86-64_BN<n>.run --mode unattended 
--setup_type advanced 
--enable-components cassandra
--disable-components apigateway,analytics,qstart,policystudio,
configurationstudio,apitester,apimgmt,packagedeploytools
--cassandraInstalldir /opt/db/cassandra-228
--cassandraJDK /opt/jre
--startCassandra 0

Windows:

./APIGateway_7.5.3_Install_linux-x86-32_BN<n>.run --mode unattended 
--setup_type advanced 
--enable-components cassandra
--disable-components apigateway,analytics,qstart,policystudio,
configurationstudio,apitester,apimgmt,packagedeploytools
--cassandraInstalldir c:\cassandra-228
--cassandraJDK c:\Program Files\Java\jre8
--startCassandra 0

For more information on installing Cassandra, see Install Apache Cassandra in the API Gateway Installation Guide.

Step 2 – Run nodetool drain on Cassandra 2.2.5

Run the following commands to drain the node and flush the memTables to the SSTables before copying data to the Cassandra 2.2.8 installation. Writes are not accepted while this is happening.

$ cd /opt/db/cassandra-225/bin
$ ./nodetool drain

Step 3 – Stop Cassandra 2.2.5

Run the following commands to stop Cassandra.

$ ps –ef | grep cassandra
$ sudo kill -9 <cassandra_pid>

For more information on starting Cassandra on UNIX/Linux or Windows, see Manage Apache Cassandra on UNIX/Linux and Windows in the API Gateway Installation Guide.

Step 4 – Copy data from Cassandra 2.2.5 to Cassandra 2.2.8

Run the following command to copy the data folder and all subfolders from your Cassandra 2.2.5 installation to your new Cassandra 2.2.8 installation.

$ cp –R /opt/db/cassandra-225/data/* /opt/db/cassandra-228/data/

For example, after running this command, you should have the following directories:

  • /opt/db/cassandra-228/data/commitlog
  • /opt/db/cassandra-228/data/data
  • /opt/db/cassandra-228/data/saved_caches
Tip   By default, all Cassandra data is stored in the data directory. However, in a production environment you should store the commit log (for example, /opt/db/cassandra-228/data/commitlog) on a separate disk partition, or a separate physical device from the data file directories (for example, /opt/db/cassandra-228/data/data). You can change the default locations in cassandra.yaml (commitlog_directory, data_file_directories, and saved_caches_directory properties). For more information, see http://docs.datastax.com/en/archived/cassandra/2.2/cassandra/configuration/configCassandra_yaml.html.

Step 5 – Copy SSL certificates from Cassandra 2.2.5 to Cassandra 2.2.8

To copy the SSL certificates, copy the following files to /opt/db/cassandra-228/conf/:

  • /opt/db/cassandra-255/conf/.truststore
  • /opt/db/cassandra-255/conf/.keystore

Step 6 – Update Cassandra 2.2.8 configuration files

Update your Cassandra 2.2.8 configuration files with the relevant settings from your Cassandra 2.2.5 installation. The following files must be updated:

  • /opt/db/cassandra-228/conf/cassandra.yaml
  • /opt/db/cassandra-228/bin/cassandra.in.sh
  • /opt/db/cassandra-228/conf/cassandra.rackdc.properties
  • /opt/db/cassandra-228/conf/cassandra-topology.properties
  • /opt/db/cassandra-228/conf/cassandra-env.sh (This file only needs to be updated if you changed your JMX configuration after Cassandra 2.2.5 installation)

You can do a diff on the files to see a complete list of the differences. The following are the values in each file that must be updated:

/opt/db/cassandra-228/conf/cassandra.yaml:

start_rpc: true
rpc_address: <Set to the IP address of this Cassandra node>
listen_address: <Set to the IP address of this Cassandra node>
seed_provider:
    # Addresses of hosts that are deemed contact points. 
    # Cassandra nodes use this list of hosts to find each other and learn
    # the topology of the ring.  You must change this if you are running
    # multiple nodes!
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          # seeds is actually a comma-delimited list of addresses.
          # Ex: "<ip1>,<ip2>,<ip3>"
          - seeds: "<IP seed1>,<IP seed 2>"
server_encryption_options: <Set to same values as your 2.2.5 installation>
client_encryption_options: <Set to same values as your 2.2.5 installation>

/opt/db/cassandra-228/bin/cassandra.in.sh:

JAVA_HOME=<Set to the location of a 64-bit JRE, same value as your 2.2.5 installation>

/opt/db/cassandra-228/conf/cassandra.rackdc.properties:

Perform a diff on this file to identify the rack settings to update.

/opt/db/cassandra-228/conf/cassandra-topology.properties:

Perform a diff on this file to identify the rack settings to update.

/opt/db/cassandra-228/conf/cassandra-env.sh

If you changed your JMX configuration after Cassandra 2.2.5 installation, perform a diff on this file to identify the JMX settings to update.

Step 7 – Start Cassandra 2.2.8

Run the following commands to start Cassandra.

$ cd /opt/db/cassandra-228/bin
$ ./cassandra

For more information on starting Cassandra on UNIX/Linux or Windows, see Manage Apache Cassandra on UNIX/Linux and Windows in the API Gateway Installation Guide.

Step 8 – Run nodetool upgradesstables on Cassandra 2.2.8

Run the following commands to rewrite SSTables that are not on the current version and upgrade them to Cassandra version 2.2.8.

$ cd /opt/db/cassandra-228/bin
$ ./nodetool upgradesstables

Step 9 – Run nodetool repair on Cassandra 2.2.8

Run the following commands to repair the tables.

$ cd /opt/db/cassandra-228/bin
$ ./nodetool repair

Cassandra upgrade steps – Multi-node single datacenter

The following steps give an example of how to upgrade Cassandra in a single datacenter HA setup. In this example the datacenter has 2 API Gateways, 3 Cassandra nodes, and 2 groups.

The following steps give an example of how to upgrade a Cassandra cluster on a three-node HA setup with the following topology:

  • Node 1 – Admin Node Manager, Cassandra 2.2.5
  • Node 2 – Admin Node Manager, API Gateway, API Manager, Cassandra 2.2.5
  • Node 3 – Node Manager, API Gateway, API Manager, Cassandra 2.2.5

There are two API Gateway groups, therefore two keyspaces. The groups are as follows:

  • Group1 has two API Manager-enabled API Gateway instances (one running on Node 2 and another on Node 3).
  • Group2 has two API Manager-enabled API Gateway instances (one running on Node 2 and another on Node 3).

Multi-node HA topology

Cassandra is set up as follows:

  • Cassandra read/write consistency is set to Local Quorum
  • Cassandra authentication is set (user name and password set to cassUser/cassPasswd)
  • SSL encryption is enabled on Cassandra

Node 1

Perform the following steps on Node 1.

Step 1 – Install Cassandra 2.2.8 on Node 1

Follow these steps to install Apache Cassandra 2.2.8 using the API Gateway installer in default GUI mode:

  1. Select the Custom option in the installer.
  2. When prompted to select components to install, select only the Cassandra component.
  3. When prompted for the Cassandra configuration, enter the following settings:
Operating System Installation Directory JRE Location
UNIX/Linux /opt/db/cassandra-228 /opt/jre
Windows c:\cassandra-228 C:\Program Files\Java\jre8
Caution   On Windows, do not use the default 32-bit JRE included in the API Gateway installer. Instead, you must install a 64-bit JRE and enter its location in the JRE Location field.
  1. Do not start Cassandra 2.2.8 when the installation completes. (Cassandra 2.2.5 should still be running.)

You can also install Cassandra in unattended mode, for example:

UNIX/Linux:

./APIGateway_7.5.3_Install_linux-x86-64_BN<n>.run --mode unattended 
--setup_type advanced 
--enable-components cassandra
--disable-components apigateway,analytics,qstart,policystudio,
configurationstudio,apitester,apimgmt,packagedeploytools
--cassandraInstalldir /opt/db/cassandra-228
--cassandraJDK /opt/jre
--startCassandra 0

Windows:

./APIGateway_7.5.3_Install_linux-x86-32_BN<n>.run --mode unattended 
--setup_type advanced 
--enable-components cassandra
--disable-components apigateway,analytics,qstart,policystudio,
configurationstudio,apitester,apimgmt,packagedeploytools
--cassandraInstalldir c:\cassandra-228
--cassandraJDK c:\Program Files\Java\jre8
--startCassandra 0

For more information on installing Cassandra, see Install Apache Cassandra in the API Gateway Installation Guide.

Step 2 – Run nodetool drain on Cassandra 2.2.5

Run the following commands to drain the node and flush the memTables to the SSTables before copying data to the Cassandra 2.2.8 installation. Writes are not accepted while this is happening.

$ cd /opt/db/cassandra-225/bin
$ ./nodetool drain

Step 3 – Stop Cassandra 2.2.5

Run the following commands to stop Cassandra.

$ ps –ef | grep cassandra
$ sudo kill -9 <cassandra_pid>

For more information on starting Cassandra on UNIX/Linux or Windows, see Manage Apache Cassandra on UNIX/Linux and Windows in the API Gateway Installation Guide.

Step 4 – Copy data from Cassandra 2.2.5 to Cassandra 2.2.8

Run the following command to copy the data folder and all subfolders from your Cassandra 2.2.5 installation to your new Cassandra 2.2.8 installation.

$ cp –R /opt/db/cassandra-225/data/* /opt/db/cassandra-228/data/

For example, after running this command, you should have the following directories:

  • /opt/db/cassandra-228/data/commitlog
  • /opt/db/cassandra-228/data/data
  • /opt/db/cassandra-228/data/saved_caches
Tip   By default, all Cassandra data is stored in the data directory. However, in a production environment you should store the commit log (for example, /opt/db/cassandra-228/data/commitlog) on a separate disk partition, or a separate physical device from the data file directories (for example, /opt/db/cassandra-228/data/data). You can change the default locations in cassandra.yaml (commitlog_directory, data_file_directories, and saved_caches_directory properties). For more information, see http://docs.datastax.com/en/archived/cassandra/2.2/cassandra/configuration/configCassandra_yaml.html.

Step 5 – Copy SSL certificates from Cassandra 2.2.5 to Cassandra 2.2.8

To copy the SSL certificates, copy the following files to /opt/db/cassandra-228/conf/:

  • /opt/db/cassandra-255/conf/.truststore
  • /opt/db/cassandra-255/conf/.keystore

Step 6 – Update Cassandra 2.2.8 configuration files

Update your Cassandra 2.2.8 configuration files with the relevant settings from your Cassandra 2.2.5 installation. The following files must be updated:

  • /opt/db/cassandra-228/conf/cassandra.yaml
  • /opt/db/cassandra-228/bin/cassandra.in.sh
  • /opt/db/cassandra-228/conf/cassandra.rackdc.properties
  • /opt/db/cassandra-228/conf/cassandra-topology.properties
  • /opt/db/cassandra-228/conf/cassandra-env.sh (This file only needs to be updated if you changed your JMX configuration after Cassandra 2.2.5 installation)

You can do a diff on the files to see a complete list of the differences. The following are the values in each file that must be updated:

/opt/db/cassandra-228/conf/cassandra.yaml:

start_rpc: true
rpc_address: <Set to the IP address of this Cassandra node>
listen_address: <Set to the IP address of this Cassandra node>
seed_provider:
    # Addresses of hosts that are deemed contact points. 
    # Cassandra nodes use this list of hosts to find each other and learn
    # the topology of the ring.  You must change this if you are running
    # multiple nodes!
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          # seeds is actually a comma-delimited list of addresses.
          # Ex: "<ip1>,<ip2>,<ip3>"
          - seeds: "<IP seed1>,<IP seed 2>"
server_encryption_options: <Set to same values as your 2.2.5 installation>
client_encryption_options: <Set to same values as your 2.2.5 installation>

/opt/db/cassandra-228/bin/cassandra.in.sh:

JAVA_HOME=<Set to the location of a 64-bit JRE, same value as your 2.2.5 installation>

/opt/db/cassandra-228/conf/cassandra.rackdc.properties:

Perform a diff on this file to identify the rack settings to update.

/opt/db/cassandra-228/conf/cassandra-topology.properties:

Perform a diff on this file to identify the rack settings to update.

/opt/db/cassandra-228/conf/cassandra-env.sh

If you changed your JMX configuration after Cassandra 2.2.5 installation, perform a diff on this file to identify the JMX settings to update.

Step 7 – Start Cassandra 2.2.8

Run the following commands to start Cassandra.

$ cd /opt/db/cassandra-228/bin
$ ./cassandra

For more information on starting Cassandra on UNIX/Linux or Windows, see Manage Apache Cassandra on UNIX/Linux and Windows in the API Gateway Installation Guide.

Step 8 – Run nodetool upgradesstables on Cassandra 2.2.8

Run the following commands to rewrite SSTables that are not on the current version and upgrade them to Cassandra version 2.2.8.

$ cd /opt/db/cassandra-228/bin
$ ./nodetool upgradesstables

Node 2

Repeat steps 1 to 8 on Node 2.

Node 3

Repeat steps 1 to 8 on Node 3.

Final step – Run nodetool repair on Cassandra 2.2.8 on each node

Run the following commands, one at a time on each node, to repair the tables.

$ cd /opt/db/cassandra-228/bin
$ ./nodetool repair

Cassandra upgrade – Multi-datacenter

To upgrade Apache Cassandra in a multi-DC setup, you can follow the same steps as for the multi-node single-DC setup:

  • Repeat the steps on each node in the cluster
  • Upgrade all of the nodes in one DC before moving to the next DC

Related Links