Configure API Gateway with the metrics database

This topic explains how to configure an API Gateway instance and Node Manager to store metrics on historic traffic in a relational database used to store metrics. For example, you can configure monitoring in API Gateway Analytics or API Manager to view data stored in the metrics database, or write custom SQL queries to retrieve metrics data as required.

Note   This topic explains how to configure API Gateway with a metrics database. This topic assumes that you have already created your metrics database using the steps described in Install and configure a metrics database.

This topic explains how to perform the following tasks:

  • Use Policy Studio to configure an API Gateway instance to write audit logging events to the metrics database, and to write metrics data to the transaction event log.
  • Use the managedomain command to configure the Node Manager to process event logs and update the metrics database.

API Gateway metrics data streams

The following data streams are used to populate the metrics database:

  • Transaction and system data: Transaction data includes clients, services, remote hosts, and protocols. System data includes CPU, memory and disk usage, and SLA breaches. The API Gateway writes this data to a transaction event log, with a new log file automatically created every 5 minutes. The Node Manager parses completed event logs and updates the metrics database.
  • Transaction audit log events: These are written directly to the metrics database by the API Gateway instance.

Connect to the API Gateway in Policy Studio

To connect to the API Gateway in Policy Studio, perform the following steps:

  1. Ensure the Admin Node Manager and API Gateway are running.
  2. Create a new project or open an existing project based on a running API Gateway instance. For more details, see the API Gateway Policy Developer Guide.

Configure the metrics database connection

To configure the API Gateway connection to the metrics database, perform the following steps:

  1. Expand the Environment Configuration > External Connections > Database Connections node in the Policy Studio tree.
  2. Right-click the Default Database Connection tree node, and select Edit.
  3. Configure the database connection to point to your metrics database. For details on connection settings, see the API Gateway Policy Developer Guide.
  4. Verify that your database connection is configured correctly by clicking the Test Connection button on the Configure Database Connection dialog.
Tip   You can troubleshoot your database connection by viewing the contents of your server .trc file in the INSTALL_DIR/apigateway/trace directory. For more details, see Configure API Gateway diagnostic trace in the API Gateway Administrator Guide.

Configure transaction audit logging to the metrics database

To configure the API Gateway instance to write transaction audit log data to the metrics database, perform the following steps:

  1. In the Policy Studio tree, select the Server Settings node, and select Logging > Transaction Audit Log in the window on the right.
  2. Select the Database tab, and select Enable logging to database.
  3. Select the Default Database Connection from the drop-down list if appropriate. Alternatively, select a database connection that you have configured. You must ensure that your database connection points to your metrics database.

For more details, see Configure API Gateway logging and events in the API Gateway Administrator Guide.

Tip   To write the content of message transactions to the database, you must also configure the Log Message Payload filter in your policies (for example, at the start and end of the policy). For more details, see the API Gateway Policy Developer Guide.

Configure the API Gateway to write to the transaction event log

To configure the API Gateway instance to write transaction data to the transaction event log, perform the following steps:

  1. In the Policy Studio tree, select the Environment Configuration > Server Settings node, and select Logging > Transaction Event Log in the window on the right.
  2. Ensure Writing to Transaction Event Log is selected.
  3. To enable monitoring of protocol and remote host metrics, select the Monitoring > Traffic Monitor node, and ensure the following settings are selected:
    • Enable Traffic Monitor
    • Record inbound transactions
    • Record outbound transactions

For more details, see Configure API Gateway logging and events in the API Gateway Administrator Guide.

Deploy the updated configuration to the API Gateway

You must deploy these configuration changes to the API Gateway. Click the Deploy button in the toolbar, or press F6.

The API Gateway now sends transaction audit logging to the metrics database, and writes transaction data to the transaction event log. The final step is to configure the Node Manager to read the transaction event logs and write system and transaction metrics to the metrics database.

Configure the Node Manager to process event logs and update the metrics database

If you have not already done so, you must use the managedomain tool to enable the Node Manager to process event logs from your API Gateway host, and to write metrics data to the metrics database.

All API Gateway instances running on the host node generate transaction event log files. These files are all written to the same folder, and are collectively processed and aggregated by the Node Manager on the host, and then written to the metrics database. The metrics database provides the data for the graphical charts in the monitoring views in API Gateway Analytics and API Manager.

Note   The Node Manager on each host in the domain must be configured to write metrics data to the same database that API Gateway Analytics reads from. The API Gateway can write to the same database for transaction audit logging if required.

Use the managedomain interactive menu

You can enable metrics using the interactive managedomain --menu command. The following shows an example:

Use the managedomain command options

Alternatively, you can use managedomain command options to enable metrics when initializing a host, adding a host other than the Admin Node Manager, or editing a host.

The following example shows enabling metrics when initializing a host machine:

The following example shows enabling metrics when adding a host machine other than the Admin Node Manager:

The following example shows enabling metrics when editing a host machine in the domain:

The managedomain metrics options are described as follows:

Option Description
--nm_entitystore_pass Specifies the encryption passphrase used to access the API Gateway instance configuration. If no passphrase has been set, omit this argument. For more details, see API Gateway Administrator Guide.
--metrics_enabled Specifies whether writing of metrics data is enabled. Enter y or n .
--metrics_dburl Specifies the JDBC URL for the metrics database (for example, jdbc:mysql://127.0.0.1:3306/reports).
--metrics_dbuser Specifies the metrics database user (for example, root or metrics DB user name).
--metrics_dbpass Specifies the password for the metrics database user.
Note   When the managedomain command has finished, you must restart the Node Manager.

For more details on managedomain, see Configure an API Gateway domain in the API Gateway Administrator Guide.

Configure additional options for event log processing in the Node Manager

The parameters described in this section specify how transaction event logs are processed in the Node Manager. You can configure these optional settings by editing the Node Manager configuration using the esexplorer tool.

For example, perform the following steps:

  1. Change to the following directory:
    INSTALL_DIR/apigateway/posix/bin
  2. Enter the esexplorer command.
  3. Select Store > Connect.
  4. Browse to INSTALL_DIR/apigateway/conf/fed/configs.xml.
  5. Select System Components > Metrics Generation Configuration in the tree on the left.
  6. Configure the appropriate fields in the window on the right:
  7. Option Description
    sourceEventLogDir Specifies the folder in which the Node Manager looks for event log files. This should match the API Gateway transaction event log directory set in Policy Studio (see Configure API Gateway logging and events in the API Gateway Administrator Guide). Defaults to ${environment.VDISTDIR}/events.
    retainProcessedEventLogs Specifies whether processed event logs should be deleted or retained in a separate directory. By default, event logs are deleted when their contents are written to the metrics database. Logs can be retained if they are needed for audit purposes or as input to a custom analytics process. Defaults to false.
    processedEventLogDir When retainProcessedEventLogs is true , specifies the directory to which event files are moved after being processed by the Node Manager. Defaults to ${environment.VDISTDIR}/events/processed.
    dirSizeMb If retainProcessedEventLogs is true, specifies the maximum size of the processedEventLogDir. When the configured size is reached, the oldest log files in the directory are deleted. Defaults to 1024 MB.
    processCustomMessageAttributes Specifies whether message attributes contained in the transaction event log, are written to the database transaction_data table. Defaults to true. For more details, see Configure API Gateway logging and events in the API Gateway Administrator Guide.
    processCustomMetrics Specifies whether custom metrics generated by the API Gateway Java Metrics API and written to the transaction event log are written to the database. Defaults to true. For more details, see the following:
Note   When making changes using esexplorer , ensure that you open the latest configuration. For example, you could overwrite changes made using managedomain if an old version of the configuration was loaded into esexplorer and then updated.
  1. Stop and restart the Node Manager after editing its configuration using esexplorer.

Related Links