Advanced scripting - additional configuration options

The advanced scripting feature option available in the Create New Job user-interface allows you to access features without needing a REST API tool such as Postman. These features will be implemented in upcoming releases

Transfer Features List

Related:

Configure Additional Transfer Feature Options

Paste the following into the JSON block

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

{
"transfer": {
"audit_level": "trace",
"batch_mode": "always",
"failure_policy": "continue",
"item_inspection_policy": "all",
"large_item": "skip",
"lock_propagation": "ignore",
"versioning": {
            "preserve": "native",
            "select": "published"
        },
"performance": {
       "parallel_writes": {
            "requested": 2
          }
      }, 
"timestamps": true,       
"empty_containers": "create",
"max_items_per_container": 10000,
"duplicate_names": "rename",
"item_overwrite": "overwrite",
"trust_mode": true,
"tags": "none"
 }
}

Configure a Custom Job Schedule 

  • There are more job schedule configurations than what is available in the DataHub user-interface (UI)
  • To leverage these options, see Scheduling a Job

The following example is scheduled to run:

  • everyday
  • every hour
  • between the hours of 6 AM GMT - 9 PM GMT
  • No schedule to stop, will run indefinitely
  • The next job run will be 1 hour after the last run has completed

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

{
"transfer": {
 },
  "schedule": {
    "mode": "auto",
    "days": "everyday",
    "repeat_interval": {
        "value": 3600000,
        "unit": "ms"
    },
    "start_window": {
        "hr": 6,
        "min": 0,
        "ms": 0,
        "sec": 0
    },
    "end_window": {
        "hr": 21,
        "min": 0,
        "ms": 0,
        "sec": 0
    }
  }
}

Related Links