Handling fake stuck inflights

There might be unexpected situations where SecureTransport is not reporting the transfer events to Embedded Analytics / Decision Insight. As a result, you might end up with transfers sitting in their initial state because the closing event, for example, an event of type Sent or Received or Failed, is never received.

If the root-problem – for example, the SecureTransport server unavailable,  or you have potential event reporting issues – has not been solved in a timely manner, the situation might get worse with live transfers piling up day by day. The situation may skew indicators and raise the server memory consumption,  which would in turn overload the Decision Insight instance and negatively affect its performance.

To prevent this, you can:

  • Clean up the 'stuck' transfers using an automated route based on a configurable threshold for monitoring the age of transfer instances; the route then sets the status of stuck transfers to UNKNOWN.
  • List the transfers that have been processed by this route for further investigation.
  • Perform a manual transfer update to catch up with the actual status of the transfers on SecureTransport.

Keep in mind that the procedure below is intended to set up a temporary solution to a problem that still needs a permanent resolution on SecureTransport's side.

Clean up fake stuck inflights

When a transfer is sitting in the same initial state for a relatively too long period, it is to be considered as a fake stuck inflight and needs to be cleaned up so as to prevent it from polluting  your overall data and creating performance issues.

To clean up the fake stuck inflights, run a route that will select transfer instances older than a certain maximum age, and then close them with the status 'UNKNOWN'. 

You can modify the route properties to suit your needs by navigating to  Data Integration > Properties > In space 04-ST-Integration.

Property Description Default value
TRANSFER_MAX_AGE  Maximum threshold value on transfer's age in minutes 1440 (24 Hrs)
CLEANCRONEXP Cron expression for scheduling the clean-up route 0 30 6 1/1 * ? * (everyday, at 6:30 AM)
CLEANTZ Time zone for scheduling the clean-up route America/New-York
UNKNOWN_DT Date of the first clean-up treatment 2018-04-22T00:00:00.000

Once you've made your modifications, go to Data Integration > Routes > In space 04-ST-Integration and run the 10_CleanLiveTransfersScheduled route.

Monitor 'UNKNOWN' transfers

To see the list of all 'UNKNOWN' transfers, on the amin menu, click the All dashboards button, then the Search unknown transfers dashboard. 

To get an overview of the volume of 'UNKNOWN' transfers over the last 30 days, from the All dashboards screen, click the Last 30 days unknown transfers dashboard.

Both views are accessible to users with the Global admin role.

Fix the transfers

You may want to fix the transfers manually to avoid keeping the 'unknown' instances for too long, and avoid performance-heavy recomputing later on to correct the events. 

  1. From the Search unknown transfers dashboard, start by exporting the 'unknown' transfers into a .csv file. 
  2. Open the file. The first line corresponds to the header of the file and each subsequent line corresponds to a transfer. For each transfer, input the actual correct values for status and closing time
  3. Run the corrective route to inject the corrected values.

  1. Put the corrected .csv file onto the directory. Herein the needed steps to update unknown transfers :
    Note: You can specify which directory to use by modifying the CSVDIR property:

    Property Description Default value
    CSVDIR Server directory where the csv file is located
  2. In Data Integration > Routes > In space 04-ST-Integration, run the 11_UpdateCleanedTransfers route.
  3. You can trigger manual recomputing from Decision Insight computing console if needed

Related Links