Sequential message delivery

In Activator message-handling, sequential delivery is the ability to deliver messages in the order they were originally consumed on a specific pickup, and also filtered for delivery by the resolved destination application or partner delivery. This is sometimes referred to as first-in-first-out (FIFO) behavior.

How Activator manages sequential delivery

An Activator internal sequence manager ensures that messages are delivered in the order in which they are consumed, per pickup and per resolved destination partner or application pickup delivery.

Supported protocols

You can currently configure sequential message delivery on the following application and partner pickups:

  • AS/2
  • FTP and FTPS client
  • FTP and FTPS embedded server
  • PGP over FTP
  • POP
  • MQSeries native
  • SFTP Client
  • SFTP Embedded Server
  • SMTP Server

Sequential delivery behavior

Message consumption (as client)

  • Each pickup with the sequential message delivery option activated has a single consumer (no multiple parallel message consumption).
  • For MQ and POP3, messages are presented in a FIFO manner. Activator assigns the order (generating sequential IDs) based on this FIFO order.
  • For FTP and SFTP clients, Activator assigns the order (generating sequential IDs) according to the order of the files in the list returned by the FTP LIST command.
  • Note: The FTP LIST results can vary from one FTP server to another. No Axway Activator configuration options are available to change this behavior.

Message consumption (as server)

For FTP and SFTP embedded servers:

  • Activator assigns the order (generating sequential IDs) once the payload starts to be received by Activator and Activator is ready to send the message for processing.
  • The order can only be fully controlled from the side of the client who is uploading the files (and not by Activator). If messages must be processed in sequence, the client must send them in sequence over a single session.
  • Note: Activator cannot limit the number of allowed sessions for a client. Such a limitation would otherwise result in connection errors for clients due to connection limitations.

Message processing

There is no sequential message handling during the processing phase. The sequential IDs that are assigned on consumption (pickup) are only taken into account at production (delivery).

Message production (as client)

In general, sequential delivery can only work if one and only one output message is generated for each input message.

For MQ, SMTP, FTP and SFTP clients:

  • Each delivery has one producer (no multiple parallel production) for sequentially delivered flows. For all deliveries (partner and application) Activator orders messages according to the message order assigned by the pickup. Messages are released one at a time by the delivery in sequential order.

Message production (as server)

For the FTP and SFTP embedded servers:

Activator reorders messages according to the order of message assigned by the pickup process for all deliveries.

  • The consummation order can only be fully controlled on the client side (and not by Activator). If messages must be processed in sequence, the client must send them in sequence over a single session.
  • If a delivery receives both sequenced and non-sequenced messages, the non-sequenced messages are delivered immediately, the sequenced messages are ordered according to their sequential IDs.

Previous message sequence saved as metadata

When Activator assigns a new sequence ID and number (derived from consummation order and destination delivery exchange) it adds the previous SequenceId and SequenceCount to the metadata ( PreviousSequenceId and PreviousSequenceNumber respectively). You can then use this metadata in Message Tracker searches to track messages in their previous sequence.

Empty message handling

In the case where Activator allocates a sequence ID/Number but fails to produce the message to the integration engine, it produces an empty message. This can occur when Activator starts to consume a payload, assigns the sequence ID/Number, but then payload transfer fails. In this case, the allocated sequence ID/Number is canceled in Activator and an empty message is sent to the integration engine indicating that no message will be received for that ID/Number.

Examples

Normal sequence:

Sequence ID Sequence number
1 1
1 2
1 3
1 4

Sequence with failed message number 3.

Sequence ID Sequence number
1 1
1 2
1 3 (failed)
1 4

Sequence with dummy message replacing failed message number 3.

Sequence ID Sequence number
1 1
1 2
1 3 (dummy/empty)
1 4

Out of sequence behavior

In the case of multiple out of sequence instances, the sequence is not guaranteed.

For example:

Message number Status Timeout status (seconds)
1 Not available
2 Not available
3 Available 60
4 Available 5
5 Available 0

In this example, messages 3, 4 and 5 wait until 1 and 2 are available. When 5 expires, Activator checks the availability of 4. If 4 is available, Activator ignores the timer status. This behavior cascades upstream to generate the new status:

Message number Status Timeout status (seconds)
1 Not available
2 Not available
3 Available 55
4 Available
5 Available

If message number 3 arrives at the time-out limit and messages 1 and 2 are still not available, Activator sends messages 3, 4 and 5 in sequence.

In the case of multiple out of sequence gaps, the situation is different. Consider the following example:

Message number Status Timeout status (seconds)
1 Not available
2 Available 60
3 Not available
4 Available 55
5 Available 50

If message 5 times out first, Activator checks the availability of message 4. Because message number 4 is available, Activator ignores the timer.

Message number Status Timeout status (seconds)
1 Not available
2 Available 5
3 Not available
4 Available 0
5 Available

When message number 4 times out, Activator detects that message 3 has not yet been received. Activator sends message 4, despite the fact that number 2 did not time out yet. Activator sends messages number 4 and 5 before message number 2. The order between 4 and 5 is maintained.

In the case of a larger number of queued messages and multiple sequence gaps between messages, this logic causes out of sequence message delivery: Activator sets timers on the various messages. When the time-outs expire there will likely be a gap between the “expired” entry and the last processed entry. This causes the expired entry, and everything after it, to be delivered out of sequence.

Recovery on shut down

Activator sequential delivery behavior is identical for clean shutdowns and forced shutdowns. In both cases, Activator stops processes abruptly. There is no concept of a graceful shutdown. When Activator starts up again up, the fail-over coordinator analyzes all work that was in-process at the moment of the shutdown. The fail-over coordinator re-introduces the work in the system, resetting timers to the original configured value of XX seconds. Reintroduction of the “old remaining workload” consumes 50% of the system capacity.

For example:

Messages 1, 2 and 3 are ready to be sent. Message 2 is in the actual process of being sent. Message 2 and 3 are in the queue. Then the system stops abruptly. On system restart, messages 1, 2 and 3 are all pushed back to the queue. Their sequence is maintained and their timers are reset (in this example to default = 60 seconds).

Before restart:

Message number Status Timeout status (seconds)
1 In transfer
2 Available 5
3 Available 10

After restart:

Message number Status Timeout status (seconds)
1 In transfer (reintroduced)
2 Available 60
3 Available 60

A side effect of this pattern is that any message that timed out will get a new opportunity for delivery after the restart. For example, Message 1 is missing and 2 and 3 are still waiting for this message. When the system restarts, message 2 and 3 are reintroduced by the fail-over coordinator and the timers for message 2 and 3 are re-started., so message 1 is allotted an additional X seconds to arrive.

Message number Status Timeout status (seconds)
1 Not available
2 Available 60
3 Available 60

Existing workload vs new workload

After a shut down, the new messages that Activator picks up from either the application side or the trading side, end up back in the list of to-do work. As reload tasks are limited to 50% of the capacity, it can take a while achieve full recovery.

Activator does not reintroduce work in sequential order, so the likelihood of time-outs and gaps occurring is much higher when there are a lot of messages being processed at shutdown. For example, if the trading engine has 3000 sequenced messages to restart, it could end up re-introducing the messages in random sequence order (299, 1, 1967, 556, etc). Since this occurs at 50% of processing capacity, timeouts are likely.

With larger numbers and gaps between messages, this logic will cause out of sequence message delivery. For example, if we have a large quantity of messages in the backlog and new messages are consumed for processing, Activator sets timers for this “new” work. After the time-out expires, there will most likely be a gap between the “expired” entry and the last processed entry. The result is that particular entry (and everything after it) is delivered out of sequence.

The failover coordinator stops injecting old workload if Activator throttle is invoked due to new work.

Manual resubmission and sequential delivery

When Activator rejects messages because the processing failed, it is the responsibility of the sending user to resubmit messages in the proper sequenced order.

If sequencing is activated on the corresponding pickup, Activator guarantees that the order in which messages are re-submitted is kept.

Unique identities in metadata for sequenced messages

The following metadata attributes are now added to messages that are handled in Activator

  • SequenceId
  • SequenceCount

Together these attributes provide a unique sequence identity for messages consumed by Activator pickups, when sequential messaging is activated for the pickup.

Sequential delivery limitations

  • Sequential delivery is only provided for the transport protocols listed. See Supported protocols.
  • When using sequential delivery with a Metadata Profile, you cannot change the format type in the associated Metadata Service. The new format type is not recognized and the document is routed based on the original format type.

Related topics

Related Links