Integration node pattern

What is the integration node pattern?

This pattern enables you to split the integration and the application part of Decision Insight into two different nodes in order to achieve higher performance, cross-firewall and prevent data loss.

For instance, let's say you want to monitor a large amount of payments in a resilient way. For your implementation, you need:

  • One Decision Insight node dedicated to integrating the payments. 
  • Another Decision Insight node dedicated to the analysis. 
  • A Decision Insight Messaging System (DIMS) cluster in-between to keep your data safe while the processing is done.

This pattern requires:

  • One server for integration, see Decision Insight Prerequisites.
  • 3 servers for the Decision Insight Messaging System cluster, see Prerequisites for their requirements.
  • One server for the application, see Decision Insight Prerequisites.

Besides performance, the great advantage of this deployment is that the data is acknowledged to the external source only once it has been secured in the DIMS cluster.

For more information about the types of deployment architecture, see Deployment architectures.

Integration node

Node used by administrator, application builder, data integration specialist

This node:

  • Extracts information by connecting to production systems. 
  • Transforms, normalizes, and filters data before forwarding that data to the application node(s) using Decision Insight Messaging System.

The node typically does not store information in the database, apart from states (see How to use set states in routes). To ensure resiliency and crash recovery, it can rely on local disk space.

The user interface of that node is mostly used for administration of the integration.

Decision Insight Messaging System

No end user interacts directly with DIMS. DIMS mediates communication between the integration and application nodes.

This middleware:

  • Stores processed data as soon as possible in a resilient manner.
  • Acknowledges data from external systems only after the data is persisted.
  • Ensures high availability of processed data.
  • Replays recorded data from any point specified.
  • Replays the same data to multiple application nodes.

Application node

Node used by administrator, application builder, application user

This node:

  • Hosts the monitoring of your product itself.
  • Receives data from an integration node using Decision Insight Messaging System.
  • Executes queries.
  • Serves the dashboards.

This node uses all the main capabilities of the product.

This node stores data into the database using the application model.

How to deploy integration node pattern?

Deploy DIMS

In order to be resilient, you need to deploy three DIMS each on a different server. Each server must have an orchestrator and a messaging server.

Setup configuration

For information about installing DIMS in your environment, see Install a Decision Insight Messaging System node.

Server Server A Server B Server C
Orchestrator cluster
  • Port
2181 2181 2181
  • Id
1 2 3
  • Connection string (1)
2=server-b.acme.int:2181,3=server-c.acme.int:2181 1=server-a.acme.int:2181,3=server-c.acme.int:2181
1=server-a.acme.int:2181,2=server-b.acme.int:2181
Messaging server
  • Port
9092 9092 9092
  • Default replication factor
3 3 3
  • Minimal synchronized replicas
2 2 2

(1) The connection string is a comma-separated list of the other orchestrators of the cluster. Each orchestrator is referenced by its identifier, host and port.

TLS configuration

TLS must be configured on all servers, for more information, see Messaging Server - TLS configuration.

Application node

This node absorbs the data that DIMS has already processed.

Mapping definition

 First, create a mapping of the data you want to absorb. Let's assume you're absorbing payments with the following parameters:

  • id : identifier of the payment
  • amount : payment amount
  • accountName : name of the emitter account
  • instant : when the payment update occurred (used as resolution time and operation begin time)

For more information about configuring mappings, see Configure mappings.

Route definition

Now you can create the route that reads from DIMS and absorbs data using the mapping. Follow the instructions at Decision Insight Messaging System - Receiving a message.

Integration Node

This node:

  • Produces data or receive it from external systems.
  • Processes it in order to extract only the data required for absorption.
  • Forwards it to the DIMS cluster.

Route definition

Now,  create the route that reads from the external source, processes and sends it to the messaging system. Follow the instructions at Decision Insight Messaging System - Sending a message.



Related Links