Job filters

Files may be included in or excluded from transfers based on metadata attributes, file size, item name patterns, and extensions. This feature is useful for customers who need to transfer only a subset of items based on certain file characteristics.

Filters are applied by adding a filter block within the transfer block of a transfer job body when calling the REST API (POST .../v1/jobs).

A source or destination block or both may be defined. Each refers to the source and destination connections for the job. Note that the filter block is placed within the transfer block.

The action value is set to either exclude or include.

Include items that match any rule: Transfer items that match any of the defined rules in this filter. Exclude all other content.

Exclude items that match any rule: Exclude items that match any of the defined rules in this filter. All other content will be transferred.

General Format - DataHub API

If the action attribute is missing or set to null, the default action is set to exclude

{
    ...
    "transfer": {
        ...
        "filter": {
            "[source|destination]": [{
                    "action": "[exclude|include]",
                    "rules": [{...}],
                    "type": "filter_rule"
                }
            ]
        }
    }
    ...
}

Job Filter Options

Related Links