Sequential message delivery

This topic includes the following sections:

About sequential message delivery

In Activator message-handling, sequential delivery is the ability to deliver messages:

  • In the order they were originally consumed on a specific pickup,
  • and,
  • Filtered for delivery by the resolved destination application or partner delivery.

This is sometimes referred to as first-in-first-out (FIFO) behavior.

How Activator manages sequential delivery

A Activator internal sequence manager ensures that messages are delivered in the order in which they are consumed, per pickup and per resolved destination partner or application delivery.

Supported protocols

You can currently configure sequential message delivery on the following application and partner pickups:

  • AS/2
  • FTP and FTPS client
  • FTP and FTPS embedded server
  • PGP over FTP, SFTP, email
  • POP
  • MQSeries native
  • SFTP Client
  • SFTP Embedded Server
  • SMTP Server

The behavior of Activator for the sequential delivery of messages is determined by processes in both the trading engine.

Sequential delivery behavior

Message consumption (as client)

  • Each pickup with the sequential message delivery option activated has a single consumer per cluster (no multiple parallel message consumption).
  • For MQ and POP3, messages are presented in a FIFO manner. The trading engine assigns the order (generating sequential IDs) based on this FIFO order.
  • For FTP and SFTP clients, the trading engine assigns the order (generating sequential IDs) according to the order of the files in the list returned by the FTP LIST command.
Note   The FTP LIST results can vary from one FTP server to another. No Axway Activator configuration options are available to change this behavior.

Message consumption (as server)

For FTP and SFTP embedded servers:

  • The trading engine assigns the order (generating sequential IDs) once the payload starts to be received by the trading engine.
  • The order can only be fully controlled from the side of the client who is uploading the files (and not by Activator). If messages must be processed in sequence, the client must send them in sequence over a single session.
  • Note: Activator cannot limit the number of allowed sessions for a client. Such a limitation would otherwise result in connection errors for clients due to connection limitations.

Message production (as client)

In general, sequential delivery can only work if one and only one output message is generated for each input message.

For MQ, SMTP, FTP, and SFTP clients:

  • Each delivery has one producer per cluster (no multiple parallel production) for sequentially delivered flows. For all deliveries (partner and application), Activator orders messages according to the message order assigned by the pickup. Messages are released one at a time by the delivery in sequential order.

Message production (as server)

For the FTP and SFTP embedded servers:

Activator reorders messages according to the order of message assigned by the pickup process for all deliveries.

  • The consummation order can only be fully controlled on the client side (and not by Activator). If messages must be processed in sequence, the client must send them in sequence over a single session.
  • If a delivery receives both sequenced and non-sequenced messages, the non-sequenced messages are delivered immediately, and the sequenced messages are ordered according to their sequential IDs.

When the trading engine assigns a new sequence ID and number (derived from consummation order and destination delivery exchange) it adds the previous SequenceId and SequenceCount to the metadata (PreviousSequenceId and PreviousSequenceNumber respectively). You can then use this metadata in Message Tracker searches to track messages in their previous sequence.

Timeout on sequential delivery

You can set a timeout that determines how long the Activator server waits for missing messages for sequential delivery in case of:

  • Message in error
  • Processing that takes too long

Configuration in the user interface

You can set the Timeout to apply for out of sequence messages limit on each pickup, when sequential delivery is activated for that pickup. This setting is expressed in seconds and has a default value of 60 seconds. It defines how long Activator waits for missing messages that are part of a sequence before taking the appropriate action.

Trading engine

By default, message-handling timeout is set to 60 seconds. The trading engine starts the timeout timer for an entry as soon as the trading engine is ready to produce the message. When an entry expires, the trading engine checks if the previous entry is available or has been delivered. If the previous entry is available, the timer is ignored.

Recovery on shutdown

Trading engine

Activator sequential delivery behavior is identical for clean shutdowns and forced shutdowns. In both cases, Activator stops processes abruptly. There is no concept of a graceful shutdown. When the trading engine starts up again up, the fail-over coordinator analyzes all work that was in-process at the moment of the shutdown. The fail-over coordinator re-introduces the work in the system, resetting timers to the original configured value of xx seconds. Reintroduction of the “old remaining workload” consumes 50% of the system capacity.

For example:

Messages 1, 2, and 3 are ready to be sent. Message 2 is in the actual process of being sent. Messages 2 and 3 are in the queue. Then the system stops abruptly. On system restart, messages 1, 2, and 3 are all pushed back to the queue. Their sequence is maintained and their timers are reset (in this example to default = 60 seconds).

Before restart:

Message number Status Timeout status (seconds)
1 In transfer
2 Available 5
3 Available 10

After restart:

Message number Status Timeout status (seconds)
1 In transfer (reintroduced)
2 Available 60
3 Available 60

A side effect of this pattern is that any message that timed out will get a new opportunity for delivery after the restart. For example, Message 1 is missing and 2 and 3 are still waiting for this message. When the system restarts, messages 2 and 3 are reintroduced by the failover coordinator and the timers for messages 2 and 3 are re-started., so message 1 is allotted an additional x seconds to arrive.

Message number Status Timeout status (seconds)
1 Not available
2 Available 60
3 Available 60

Existing workload vs new workload

After a shutdown, the new messages that the trading engine picks up from either the application side or the trading side end up back in the list of to-do work. As reload tasks are limited to 50% of the capacity, it can take a while to achieve full recovery.

The trading engine does not reintroduce work in sequential order, so the likelihood of timeouts and gaps occurring is much higher when there are a lot of messages being processed at shutdown. For example, if the trading engine has 3,000 sequenced messages to restart, it could end up reintroducing the messages in random sequence order (299, 1, 1967, 556, etc). Since this occurs at 50% of processing capacity, timeouts are likely.

With larger numbers and gaps between messages, this logic will cause out of sequence message delivery. For example, if you have a large quantity of messages in the backlog and new messages are consumed for processing, the trading engine sets timers for this “new” work. After the timeout expires, there will most likely be a gap between the “expired” entry and the last processed entry. The result is that particular entry (and everything after it) is delivered out of sequence.

Manual resubmission and sequential delivery

When Activator rejects messages because the processing failed, it is the responsibility of the sending user to resubmit the messages in the proper sequenced order.

If sequencing is activated on the corresponding pickup, Activator guarantees that the order in which the messages are re-submitted is kept.

Related Links