Perform essential Apache Cassandra operations

This section describes the minimum essential operations that are required to maintain a healthy Apache Cassandra HA cluster.

Perform anti-entropy repair

Apache Cassandra is inherently HA, so when a node is absent from a cluster for a time, it can be brought back into the cluster and becomes eventually consistent by design. Node repair is required after reintegration into the cluster.

The nodetool repair command repairs inconsistencies across all Cassandra replica nodes for a given range of data. Most importantly, it ensures that deleted data remains deleted in a distributed system. You should execute this command at least weekly, at off-peak times, and stagger execution on different nodes.

For example, the following crontab commands execute nodetool repair on specified schedules in test and production environments:

# Every 10 mins. NB: TEST ONLY, not for production.
$ crontab
*/10 * * * * /home/cassandra/bin/nodetool repair >> /home/cassandra/logs/repair.log

# Every week at 3am on a Sunday
$ crontab
0 2 * 0 /home/cassandra/bin/nodetool repair >> /home/cassandra/logs/repair.log

Replace dead nodes

If a node is down for more than 10 days, it should be replaced. For details on replacing dead Cassandra nodes, see Replacing a dead node or dead seed node documentation.

Reconfigure an existing Apache Cassandra installation from scratch

There is no need to reinstall Cassandra from scratch. You can just move Cassandra data files and restore the cassandra.yaml configuration file if necessary. Preform the following steps:

  1. Stop Cassandra. For details, see Manage Apache Cassandra.
  2. Move CASSANDRA_HOME/data to CASSANDRA_HOME/data/OLD-DATA-DATE.
  3. Restore cassandra.yaml in CASSANDRA_HOME/conf if necessary.

Enable Apache Cassandra debug logging

You can enable Cassandra debug logging using any of the following approaches:

  • logback.xml
  • You can specify a logger in the cassandra/conf/logback.xml configuration file as follows:

<logger name "org.apache.cassandra.transport" level=DEBUG/>"

  • nodetool
  • You can use the nodetool setlogginglevel command as follows:

nodetool setlogginglevel org.apache.cassandra.transport DEBUG

  • JConsole
  • The JConsole tool enables you to configure Cassandra logging using JMX. For example, you can invoke setLoggingLevel and specify org.apache.cassandra.db.StorageServiceMBean and DEBUG as parameters. For more details, see the next section.

For more details on enabling logging, see the following, which also applies to Cassandra 2.2:

Monitor a Cassandra cluster using JMX

You can use Java Management Extensions to monitor and manage performance in a Cassandra cluster. For details, see Monitoring a Cassandra cluster documentation.

Upgrade your Cassandra version

For details on upgrading your Cassandra version, see Upgrade Apache Cassandra in the API Gateway Upgrade Guide.

Related Links