Handling fake stuck inflights
There might be unexpected situations where SecureTransport is not reporting the transfer events to Embedded Analytics / Decision Insight. As a result, you might end up with transfers sitting in their initial state because the closing event, for example, an event of type Sent or Received or Failed, is never received.
If the root-problem – for example, the SecureTransport server unavailable, or you have potential event reporting issues – has not been solved in a timely manner, the situation might get worse with live transfers piling up day by day. The situation may skew indicators and raise the server memory consumption, which would in turn overload the Decision Insight instance and negatively affect its performance.
To prevent this, you can:
- Clean up the 'stuck' transfers using an automated route based on a configurable threshold for monitoring the age of transfer instances; the route then sets the status of stuck transfers to UNKNOWN.
- List the transfers that have been processed by this route for further investigation.
- Perform a manual transfer update to catch up with the actual status of the transfers on SecureTransport.
Keep in mind that the procedure below is intended to set up a temporary solution to a problem that still needs a permanent resolution on SecureTransport's side.
Clean up fake stuck inflights
When a transfer is sitting in the same initial state for a relatively too long period, it is to be considered as a fake stuck inflight and needs to be cleaned up so as to prevent it from polluting your overall data and creating performance issues.
To clean up the fake stuck inflights, run a route that will select transfer instances older than a certain maximum age, and then close them with the status 'UNKNOWN'.
You can modify the route properties to suit your needs by navigating to Data Integration > Properties > In space 04-ST-Integration.
||Maximum threshold value on transfer's age in minutes
||1440 (24 Hrs)
||Cron expression for scheduling the clean-up route
||0 30 6 1/1 * ? * (everyday, at 6:30 AM)
||Time zone for scheduling the clean-up route
||Date of the first clean-up treatment
Once you've made your modifications, go to Data Integration > Routes > In space 04-ST-Integration and run the 10_CleanLiveTransfersScheduled route.
Monitor 'UNKNOWN' transfers
To see the list of all 'UNKNOWN' transfers, on the amin menu, click the All dashboards button, then the Search unknown transfers dashboard.
To get an overview of the volume of 'UNKNOWN' transfers over the last 30 days, from the All dashboards screen, click the Last 30 days unknown transfers dashboard.
Both views are accessible to users with the Global admin role.
Fix the transfers
You may want to fix the transfers manually to avoid keeping the 'unknown' instances for too long, and avoid performance-heavy recomputing later on to correct the events.
- From the Search unknown transfers dashboard, start by exporting the 'unknown' transfers into a .csv file.
- Open the file. The first line corresponds to the header of the file and each subsequent line corresponds to a transfer. For each transfer, input the actual correct values for
- Run the corrective route to inject the corrected values.
he needed steps to update unknown transfers :
||Server directory where the csv file is located
- In Data Integration > Routes > In space 04-ST-Integration, run the 11_UpdateCleanedTransfers route.
- You can trigger manual recomputing from Decision Insight computing console if needed