Customize your API Management topology in Docker

This topic describes how to customize your Docker-based API Management topology to suit your environment. For example, you can do this by creating Docker data volumes for persistence and editing your docker-compose.yml file to suit your environment.

Create Docker data volumes for persistence

You must create Docker data volumes to persist API Management and Cassandra configuration data. For example, this includes transaction, audit, trace, and user data. This is required for the API Gateway, Node Manager, and Cassandra on each node. This section shows how to do this for the recommended sample 3-node HA system.

Perform the following steps:

  1. Create a Docker data volume for each configuration to persist. For example:

cd src/util

 

# Create volumes for node1, node2, and node3

./volume.py --create --nodeName node1
./volume.py --create --nodeName node2
./volume.py --create --nodeName node3

 

# Create volumes for the Cassandra cluster

docker volume create --name cassandra-s1_DATA
docker volume create --name cassandra-s1_LOGS
docker volume create --name cassandra-s2_DATA
docker volume create --name cassandra-s2_LOGS
docker volume create --name cassandra-m_DATA
docker volume create --name cassandra-m_LOGS

  1. This creates the following data volumes:

docker volume ls
DRIVER VOLUME NAME
local cassandra-m_DATA
local cassandra-m_LOGS
local cassandra-s1_DATA
local cassandra-s1_LOGS
local cassandra-s2_DATA
local cassandra-s2_LOGS
local node1_CONF_DATA
local node1_EVENTS_DATA
local node1_GROUP_DATA
local node1_LOGS_DATA
local node1_TRACE_DATA
local node1_USER_DATA
local node2_CONF_DATA
local node2_EVENTS_DATA
local node2_GROUP_DATA
local node2_LOGS_DATA
local node2_TRACE_DATA
local node2_USER_DATA
local node3_CONF_DATA
local node3_EVENTS_DATA
local node3_GROUP_DATA
local node3_LOGS_DATA
local node3_TRACE_DATA
local node3_USER_DATA

  1. Enable the volumes for each API Gateway node in the generated docker-compose.yml file. For example:

version: '2'
services:

  node1:
  image: gwlatest-3nodeha-apimgr_node1
  links:
  cassandra-m
  cassandra-s1
   cassandra-s2
   volumes:
   # Configuring volume for Node Manager conf (modified using API Gateway Manager web application)
  - node1_CONF_DATA:/opt/Axway/apigateway/conf
  # Configuring volume for Node Manager trace (modified using API Gateway Manager web application)
  - node1_TRACE_DATA:/opt/Axway/apigateway/trace
   # Configuring volume for Node Manager logs (includes Domain and Transaction logs modified using API Gateway Manager web application)
   - node1_LOGS_DATA:/opt/Axway/apigateway/logs
   # Configuring volume for Node Manager events (modified using API Gateway Manager web application)
   - node1_EVENTS_DATA:/opt/Axway/apigateway/events
   # Configuring volume for API Gateway Group Configuration, data, logging and trace.
  - node1_GROUP_DATA:/opt/Axway/apigateway/groups
   # Configuring volume for API Gateway user data eg. ActiveMQ, File-based filters, Ehcache, etc.
   - node1_USER_DATA:/custom_path/to/userdata/
   hostname: node1
   environment:
   - START_BACKOFF_SECS=180 # starts after Cassandra cluster and 30s after previous instance
  - CASSANDRA_HOSTS=cassandra-m,cassandra-s1,cassandra-s2
   privileged: true


  node2:
   image: gwlatest-3nodeha-apimgr_node2
    links:
    - cassandra-m
    - cassandra-s1
    - cassandra-s2
   volumes:
   # Configuring volume for Node Manager conf (modified using API Gateway Manager web application)
   - node2_CONF_DATA:/opt/Axway/apigateway/conf
   # Configuring volume for Node Manager trace (modified using API Gateway Manager web application)
   - node2_TRACE_DATA:/opt/Axway/apigateway/trace
   # Configuring volume for Node Manager logs (includes Domain and Transaction logs modified using API Gateway Manager web application)
   - node2_LOGS_DATA:/opt/Axway/apigateway/logs
   # Configuring volume for Node Manager events (modified using API Gateway Manager web application)
   - node2_EVENTS_DATA:/opt/Axway/apigateway/events
   # Configuring volume for API Gateway Group Configuration, data, logging and trace.
   - node2_GROUP_DATA:/opt/Axway/apigateway/groups
   # Configuring volume for API Gateway user data eg. ActiveMQ, File-based filters, Ehcache etc
   - node2_USER_DATA:/custom_path/to/userdata/
   hostname: node2
   environment:
   - START_BACKOFF_SECS=210 # starts after Cassandra cluster and 30s after previous instance
   - CASSANDRA_HOSTS=cassandra-m,cassandra-s1,cassandra-s2
   privileged: true


  node3:
   image: gwlatest-3nodeha-apimgr_node3
   links:
   - cassandra-m
   - cassandra-s1
   - cassandra-s2
   volumes:
   # Configuring volume for Node Manager conf (modified using API Gateway Manager web application)
   - node3_CONF_DATA:/opt/Axway/apigateway/conf
   # Configuring volume for Node Manager trace (modified using API Gateway Manager web application)
   - node3_TRACE_DATA:/opt/Axway/apigateway/trace
   # Configuring volume for Node Manager logs (includes Domain and Transaction logs modified using API Gateway Manager web application)
   - node3_LOGS_DATA:/opt/Axway/apigateway/logs
   # Configuring volume for Node Manager events (modified using API Gateway Manager web application)
   - node3_EVENTS_DATA:/opt/Axway/apigateway/events
   # Configuring volume for API Gateway Group Configuration, data, logging and trace.
   - node3_GROUP_DATA:/opt/Axway/apigateway/groups
   # Configuring volume for API Gateway user data eg. ActiveMQ, File-based filters, Ehcache etc
   - node3_USER_DATA:/custom_path/to/userdata/
   hostname: node3
  environment:
   - START_BACKOFF_SECS=240 # starts after Cassandra cluster and 30s after previous instance
   - CASSANDRA_HOSTS=cassandra-m,cassandra-s1,cassandra-s2
   privileged: true

Note   You should store all user data in a centralized location for convenience. In the volumes section of docker-compose.yml, update <custom_path/to/userdata> as appropriate.
  1. Enable the volumes for each Cassandra node in the generated docker-compose.yml file. For example:

   cassandra-m:
   image: cassandra
   environment:
   - START_BACKOFF_SECS=0 # prevents race between nodes joining the cluster
   - SEEDS=cassandra-m
   - MAX_HEAP_SIZE=1G
   - HEAP_NEWSIZE=400m
  volumes:
   # Cassandra data volume
   - cassandra-m_DATA:/opt/cassandra/data
   # Cassandra logs volume
   - cassandra-m_LOGS:/opt/cassandra/logs
   image: cassandra
   hostname: cassandra-m
   restart: on-failure:2

  cassandra-s1:
   image: cassandra
   environment:
  - START_BACKOFF_SECS=60 # prevents race between nodes joining the cluster
  - SEEDS=cassandra-m
  - MAX_HEAP_SIZE=1G
  - HEAP_NEWSIZE=400m
   volumes:
   # Cassandra data volume
   - cassandra-s1_DATA:/opt/cassandra/data
   # Cassandra logs volume
   - cassandra-s1_LOGS:/opt/cassandra/logs
   image: cassandra
   hostname: cassandra-s1
   restart: on-failure:2

  cassandra-s2:
   image: cassandra
   environment:
   - START_BACKOFF_SECS=120 # prevents race between nodes joining the cluster
   - SEEDS=cassandra-m
   - MAX_HEAP_SIZE=1G
   - HEAP_NEWSIZE=400m
   volumes:
   # Cassandra data volume
   - cassandra-s2_DATA:/opt/cassandra/data
   # Cassandra logs volume
   - cassandra-s2_LOGS:/opt/cassandra/logs
   image: cassandra
   hostname: cassandra-s2
   restart: on-failure:2


volumes:
  node1_CONF_DATA: external: true
  node1_TRACE_DATA: external: true
  node1_LOGS_DATA: external: true
  node1_EVENTS_DATA: external: true
  node1_GROUP_DATA: external: true
  node1_USER_DATA: external: true
  node2_CONF_DATA: external: true
  node2_TRACE_DATA: external: true
  node2_LOGS_DATA: external: true
  node2_EVENTS_DATA: external: true
  node2_GROUP_DATA: external: true
  node2_USER_DATA: external: true
  node3_CONF_DATA: external: true
  node3_TRACE_DATA: external: true
  node3_LOGS_DATA: external: true
  node3_EVENTS_DATA: external: true
  node3_GROUP_DATA: external: true
  node3_USER_DATA: external: true
  cassandra-m_DATA: external: true
  cassandra-m_LOGS: external: true
  cassandra-s1_DATA: external: true
  cassandra-s1_LOGS: external: true
  cassandra-s2_DATA: external: true
  cassandra-s2_LOGS: external: true

Example: Persist user-defined directories for API Gateway filters

If your policy uses a file-based API Gateway filter that allows a user-defined directory such as /home/user/Downloads (for example, File Download, File Upload, or Save to File), you can specify this in the Docker container. For example:

  1. Create a volume for the user data (for example, GatewayNODE2_userdata) on the host:

./volume.py --create --nodeName GatewayNODE2_userdata

  1. Get the full path of the volume created on the host. For example:

docker volume inspect --name GatewayNODE2_userdata /var/lib/docker/volumes/GatewayNODE2_userdata/_data

  1. Update the volume entry in the docker-compose.yml file. For example:

volumes:
# Configuring volume for API Gateway user data (eg., ActiveMQ, File-based filters, Ehcache, etc.)
- /var/lib/docker/volumes/GatewayNODE2_userdata/_data:/home/user/Downloads

Example: Persist user-defined directories using the Directory Scanner

Alternatively, you can configure persistence using the Directory Scanner in Policy Studio. This enables you to specify multiple directory locations on disk (for example, Input, Processing and Response directories). This is available under the Environment Configuration > Listeners > API GatewayDirectory Scanner node in the Policy Studio tree.

You can use either of the following approaches:

  • Create a directory inside the container (for example, /home/userdata/DirectoryScanner), and set it as the directory path for all. You must update the entry in docker-compose.yml as follows:

volumes:
# Configuring volume for API Gateway user data (eg., ActiveMQ, File-based filters, Ehcache, etc.)
- /var/lib/docker/volumes/GatewayNODE1_userdata/_data:/home/userdata/DirectoryScanner

  • Create nested directories inside the container, for example:

/home/userdata/DirectoryScanner/Input
/home/userdata/DirectoryScanner/Processing
/home/userdata/DirectoryScanner/Response

Add a corresponding entry in docker-compose.yml:

- /var/lib/docker/volumes/GatewayNODE1_userdata/_data:/home/userdata/DirectoryScanner

And create multiple volumes with a one-to-one mapping with each container path in docker-compose.yml:

volumes:
- /var/lib/docker/volumes/GatewayNODE1_userdata1/_data:/home/userdata/DirectoryScanner/Input
- /var/lib/docker/volumes/GatewayNODE1_userdata2/_data:/home/userdata/DirectoryScanner/Processing
- /var/lib/docker/volumes/GatewayNODE1_userdata3/_data:/home/userdata/DirectoryScanner/Response

Finally, to bring the system up, see Step 3—Run and test your API Management system.

Delete data volumes

You can delete data volumes as needed. For example:

./volume.py --delete --nodeName node1
./volume.py --delete --nodeName node2
./volume.py --delete --nodeName node3
docker volume rm cassandra-s1_DATA
docker volume rm cassandra-s1_LOGS
docker volume rm cassandra-s2_DATA
docker volume rm cassandra-s2_LOGS
docker volume rm cassandra-m_DATA
docker volume rm cassandra-m_LOGS

Test your API Management system

This section explains how to verify that your customized configuration is working correctly.

Configure hosts and ports

To configure specific host names and expose specific ports for test purposes, edit the docker-compose.yml file in compose-generated/servers. For example:

version: '2'
 services:
  node1:
   image: gwlatest-2nodeha-apimgr_node1
  hostname: node1 privileged: true
  ports:
    - "8075:8075"
    - "8090:8090"
    - "8080:8080"

  node2:
   image: gwlatest-2nodeha-apimgr_node
   links:
    - node1:node1
    hostname: node2
   privileged: true
...

Connect to API Management services

You can perform the following tests to check that your system is running correctly:

  • Health check policy
  • Enter the following command to connect to the API Gateway Health Check policy:
  • curl http://<docker-host-name>:8080/healthcheck
    <status>ok</status>
  • API Gateway Manager
  • Enter the following URL in a browser to launch the API Gateway Manager web console:
  • https://<docker-host-name>:8090
  • API Manager
  • Enter the following URL to launch the API Manager web console:
  • https://<docker-host-name>:8075

Related Links