Filter by date range or age

Items may be filtered by dates using the filter_date or date rule type. Date filter rules have two attributes: targets and range. Date filters are applied based on either a fixed date range or a sliding range.

Below is a description of the attributes.

Attribute

Description

targets An array of date targets to which the filter applies. These targets include: none, created_on, modified_on, accessed_on, and all.
range

The range attribute is defined a subset of the attributes below depending on whether items are intended to be filtered by a fixed range of dates or a sliding range. For fixed range filters, set type to range and set the to and from attributes. See Example 1 on the right for this type of filter. For sliding range filters, set the type to any other value and the value attribute to the amount to use for the scale. See Example 2 on the right for this type of filter.

type: Allowed values include day (default), week, month, year, and range. The day, week, month, and year values are used in conjunction with value to define a sliding range (to/from not used). Use range with to and from attributes to define a fixed range.

from: The date and time the filter begins - a date-time offset in UNIX time format (e.g. 1508544000 for October 21, 2017 12:00 AM UTC)

to: The date and time the filter ends; see the from attribute for format details

value: For a sliding range, this value represents the quantity of the range as an integer (e.g. for a type "day" and value 5 with targets set to created_on, the filter will apply to files created within the last five days).

type filter_date or date are the only allowable values for this filter type. Both values signify the same filter.

Example 1 - DataHub API

The following example excludes files created between June 25, 2017 and August 25, 2017 from being transferred from the source.

{
    ...
    "transfer": {
        ...
        "filter": {
            "source": [{
                    "action": "exclude",
                    "rules": [{
                            "targets": "created_on",
                            "range": {
                                "type": "range",
                                "from": 1508457600,
                                "to": 1508544000
                            },
                            "type": "filter_date"
                        }
                    ],
                    "type": "filter_rule"
                }
            ]
        }
    }
    ...
}

Example 2 - DataHub API

The following example will only transfer files modified within the last five days.

{
    ...
    "transfer": {
        ...
        "filter": {
            "source": [{
                    "action": "include",
                    "rules": [{
                            "targets": "modified_on",
                            "range": {
                                "type": "day",
                                "value": 5
                            },
                            "type": "filter_date"
                        }
                    ],
                    "type": "filter_rule"
                }
            ]
        }
    }
    ...
}

Related Links