Housekeeping for catalog and output files

Transfer CFT provides a set of services to track activity related to transfers and their status as well as evaluating the system health.

This section describes how to manage the log, catalog, and output files to keep your system running smoothly.

Catalog management

Transfer CFT records all file transfers in its local database, the catalog. The size of the Catalog is defined to store a maximum number of file transfer records.

Automatic catalog expansion

The auto-expand catalog option lets you enlarge the catalog by a preset percentage when an alert is sent that the catalog is reaching its threshold. Additionally you can indicate a script to execute if this expanded limit is exceeded.

Overview

Once the catalog exceeds the minimum alert level, it can be expanded to the amount defined in the auto-expand option. For example, if you set the auto-expand to 100%, then the catalog doubles. If you define it as 50%, the catalog resizes to 1.5 times the original number of records. After the auto-expand limit is reached, a predefined script can be executed.

If you defined a timer for catalog alerts, TLVCLEAR, once the usage surpasses the allotted time value one of the following occurs:

  • The catalog is expanded and a message sent to the log, but no script is executed
  • The catalog is expanded, a message is sent to the log, and the TLVCEXEC script executes.

Steps

To enable the auto-expand option, with Transfer CFT running:

  1. Set the uconf values for:
    • cft.cftcat.auto_expand_percent
    • cft.cftcat.auto_expand_max_size
  2. To activate the new values, run the command: CFTUTIL reconfig type = uconf
    • If Transfer CFT is stopped when setting uconf values, you do not need to execute the reconfig command.
Parameter Default Description
cft.cftcat.auto_expand_percent 0

This value indicates the factor increase, as a percentage, that the catalog will automatically expand.

The value 0 disables the automatic expansion feature.

Tip   We recommend that you set this to a relatively high value, at least 50. When repeatedly expanded, the catalog's internal structure may become fragmented and, consequently, catalog access less efficient.
cft.cftcat.auto_expand_max_size 1M

The maximum number of records for the automatic catalog expansion option.

Related parameters:

  • TLVCLEAR: Level below which the alert stops.
  • TLVCEXEC: Batch to execute when the alert ends.
  • TLVWRATE: The minimum amount of time, in seconds, to wait before resending an alert.
  • TLVWEXEC: Batch to execute when CFTCAT/TLVWARN is reached.
  • TLVWARN: Catalog usage limit before issuing an alert. When this limit is reached, the CFTCAT/TLVWEXEC is executed.

Catalog purge policies

Purge using the CFTCAT command

The local file transfer internal datafilerules include standard purge rules that you can modify using the catalog command CFTCAT.

There are 6 parameters that manage the purge, depending on the transfer status and direction. For each of the following you can set the number, in days, for the purge to occur. In our example, the purge is set for 10 days.

CFTCAT ID = 'CAT0',

FNAME = '$CFTCATA',

WSCAN = '1',

TIMEP = '23595999',

UPDAT = '1',

SH = '10',

ST = '10',

SX = '10',

RH = '10',

RT = '10',

RX = '10',

The first letter indicates the transfer direction (S for SEND or R for receive ).

The second letter refers to the state (CFTSTATE).

Normal mode

  • (H) Transfer phase and hold phasestep, or Transfer phase and kill phasestep
  • (T) Ack phase and all phasesteps
  • (X) Done phase and Done phasestep
Note Transfers in phase (Y) and phasestep (D) are not purged when purging the catalog. To purge these records, you must execute an END command that modifies the state to (X).

Compatibility mode

  • (H) Hold, keep, or pre-processing status
  • (T) Completed status
  • (X) Executed status

Purge using UCONF settings

You can use the unified configuration to perform the same sort of purges as with the CFTCAT command, except that UCONF offers a more granular scheduling, i.e. hours, minutes, or seconds.

For example, set the following where the sx represents 10 days for an executed SEND transfer:

UCONFSET id=cft.purge.sx, value=10D

 

NOTE:

The amount of time is entered in days (x or xD), in hours (xH) or in minutes (xM). If set to -1, the CFTCAT value is used.

To schedule a periodic purge every 30 minutes:

CFTUTIL uconfset id=cft.purge.periodicity,value=30M

To apply the dynamic configuration parameters change:

CFTUTIL reconfig type=UCONF

For information on the reconfig command, please see Manage configuration updates.

Purge per template definitions

When defining CFTSEND or CFTRECV templates you can set a parameter to purge the records after the transfer completes.

CFTSEND DELETE=YES

Purge records with the keep status

Delete all file transfer records in keep status (K) due to a file creation error.

CFTRECV RKERROR=YES

Manually delete catalog records

The DELETE command is used to delete one or more catalog entries. A transfer in process is interrupted and its transfer request disappears from the catalog.

This file is automatically deleted for an R, Receive, type request in non terminated state, a state other than T and X, if the receive file is a temporary file.

For any selected set of transfers, transfers are processed in batches of 20 transfers every 5 seconds.

For example, delete all executed transfers from the catalog.

DELETE STATE=X

Manage transfer events sent to Sentinel

A file transfer event is sent as an XFB.Transfer event to the Sentinel server each time the file transfer status is updated.

You can set filters to reduce the number of messages (more than a 50% reduction) sent to Sentinel, for example:

UCONFSET id=sentinel.xfb.transfer,value=SUMMARY

Archive Transfer CFT standard output files

All output files are stored in the <runtime/run> directory. Among these are the standard output files for Transfer CFT and the Copilot processes. The copui.trc is the standard output for the Copilot server, which continues to grow in size but cannot be rotated.

However, Transfer CFT processes use a standard output file <runtime/run>/cft.out to log internal system messages. You can define the number of archive files you want to rotate using the command:

UCONFSET id=cft.output.backup_count,value=n

Delete completed transfer files

There are multiple ways to clear out completed transfer files. Let's begin with a simple example of deleting files that have successfully been sent.

CFTSEND ID=CLEANUP,FNAME=<FILENAME>,FACTION=DELETE

If you would additionally like to delete the catalog records, as well as the file after it's transfer (defined according to the transfer state).

  • The FDELETE option removes the file once the catalog record is deleted.
  • You can use the DELETE=YES option in conjunction with FDELETE to remove both the file and the record.

For example, to remove both the file and the record when sending a file.

CFTSEND ID=CLEANUP,FNAME=<FILENAME>,DELETE=YES,FDELETE=CDKHTX
Note The DELETE/FDELETE options are also valid for a CFTRECV, but note that setting FDELETE=CDKHTX deletes the file regardless of the state at the end of the transfer.

Related Links