Decision Insight Messaging System

What is Decision Insight Messaging System ?


The Decision Insight Messaging System is the only recommended way to send messages between two Decision Insight applications in a safe and reliable way.

Data is sent from one application to the other via messages.

Messages are the fundamental unit of processing through by Decision Insight Messaging System.

  • Messages consist of a key, a timestamp, and a value whose format (a Java Map) is suitable for Decision Insight absorption.
  • Messages are independent data payloads sent over the network.
  • Messages are stored into topics, which are logical collections of messages dedicated to a common processing.

Usage

There are two possible ways to configure connection to a DIMS cluster:

  • using a DIMS connector: used to configure DIMS cluster's servers and TLS configuration (this is the recommended way to configure a DIMS connection)
  • directly in the uri of the routing context: DIMS cluster's servers and TLS configuration reference are set directly in routing context.

Create a connector

Select DIMS class type to create a DIMS connector.

Available properties

Property Description
servers Comma separated list of Decision Insight Messaging Server members of the targeted cluster. The format is serverName:port
sslContextParameters Refers to the SSL configuration from a connector. See How to configure SSL on a component for Mutual Authentication .

You can create a data integration property named DIMS_SERVERS with the list of messaging servers to interact with.

"DIMS_SERVERS" property for a DIMS cluster of 3 nodes with default port configuration
server-a.acme.int:9092,server-b.acme.int:9092,server-c.acme.int:9092

Sending a message

To publish messages to Decision Insight Messaging System, you will create a route similar to:

Integration Node
<routes xmlns="http://camel.apache.org/schema/spring" xmlns:u="http://www.systar.com/aluminium/camel-util">
    <route>
        <!-- Read from JMS -->
        <from uri="jms:queue:test?connectionFactory=activemq"/>
        <!-- Process the message -->
        <!-- ... -->
        <!-- Set the body -->
        <setBody>
            <u:map-create>
                <u:map-entry key="id">
                    <!-- Extract the payment id -->
                </u:map-entry>
                <u:map-entry key="amount">
                    <!-- Extract the amount-->
                </u:map-entry>
                <u:map-entry key="begin">
                    <!-- Extract when this payment update occurred -->
                </u:map-entry>
                <u:map-entry key="accountName">
                    <!-- Extract the account name-->
                </u:map-entry>
            </u:map-create>
        </setBody>
        <!-- Set the message key -->
        <setHeader headerName="DIMS.KEY">
            <simple>${body[accountName]}</simple>
        </setHeader>
        <to uri="dims:DimsConnector?topic=payments"/>
        <!-- Equivalent configuration without connector usage
        <to uri="dims:{{DIMS_SERVERS}}?topic=payments&amp;sslContextParameters=#sslcp"/>
        -->
    </route>
</routes>

Line 5, read from JMS. You can replace this with any source.

Line 9, the message body must be a Map.

Line 26, we provide a message key. If you don't need to enforce message order, then you can remove this <setHeader> block.

Line 29, forward the message to the Messaging System.

When using JMS, the message is automatically acknowledged at the end of the processing (here after the <to uri="dims..." />). Most sources work the same way.

Where the following URI options are available:

Name Required Default value Description
uri
yes none

or

  • Comma separated list of Decision Insight Messaging Server members of the targeted cluster (format is serverName:port).
topic
yes none Topic on which to publish messages
sslContextParameters
yes (if not provided by DIMS connector) none

Refers the SSL configuration from a connector.

See How to configure SSL on a component with mutual authentication .

retries
no 0 Setting a value greater than zero will cause the client to resend data for which send fails.
maxRequestSize no

1048576

(Advanced) The maximum size a request (message or batch of messages) can have. Tune this setting if you expect to send huge messages.

Messages are ordered within a partition. Messages with the same DIMS.KEY will go to the same partition, so their order will be preserved, while messages with different keys received in rapid succession (in a 5 second time-frame) may not preserve their order. Omitting the DIMS.KEY header allows messages to be distributed randomly across partitions.

Receiving a message

To receive messages from Decision Insight Messaging System, you will create a route similar to:

Application Node
<routes xmlns="http://camel.apache.org/schema/spring" xmlns:u="http://www.systar.com/aluminium/camel-util">
    <route>
        <!-- Read from DIMS -->
        <from uri="dims:DimsConnector?topic=payments&amp;offsetRepository=#payments&amp;groupId=application"/>
        <!-- Equivalent configuration without connector usage
        <from uri="dims:{{DIMS_SERVERS}}?topic=payments&amp;sslContextParameters=#sslcp&amp;offsetRepository=#payments&amp;groupId=application"/>
        -->
        <!-- Absorb directly -->
        <to uri="tnd-absorption:updatePayment"/>
    </route>
</routes>

Line 7, send data to a mapping for absorption. See Configure mappings for information on how to configure mappings. If absorption is not your goal, you may replace this with any destination.

Where the following URI options are available:

Property Mandatory Default value Description
uri
yes none

or

  • Comma-separated list of Decision Insight Messaging Server members of the targeted cluster (format is serverName:port).
topic
yes none Topic to use
sslContextParameters
yes (if not provided by DIMS connector) none

Refers the SSL configuration from a connector.

See How to configure SSL on a component with mutual authentication.

groupId
yes none Id of consumer group. Each consumer group will receive a copy of the message.

Offset management

offsetRepository
yes none Offset repository to use to store the current progress. Set the name of a map state to use it and resume when re-starting the route. See How to configure states.
          startFrom
        
no latest

Position to start consuming message if

  • the last known offset was purged
  • there is no last known offset
Can be set to consume the earliest or the latest available messages. To fail consumption and throw an exception, use none.
consumersCount
no 1 The number of consumers that connect to the Decision Insight Messaging Server.

Related Links