Reconciliation tools

Outlines the best calls to use to reconcile your job

Reports - Useful Calls

Related:

Review Job - Verify all Requested Features

When creating your job using JSON, a wide variety of features may have been used. To verify all were implemented as intended, we recommend verifying this by performing a Job Details call

  • Create your JSON with desired features
  • Submit your job
  • Before you start your job, run the following:

GET {{url}}v1/jobs/{{job}}?fields=all

Expected response:

  • Will include each feature you requested

If your requested feature is not listed, it is likely due to an incorrect JSON (note, not an invalid JSON)

  • While your JSON may appear to be valid, your requested feature likely falls outside of the transfer block
  • Review and update your request so the feature falls within the transfer block

Review Job - Warnings

After reviewing your Job Audit History, you may observe warnings that occurred after your job completes successfully. 

Here is how you can investigate these warnings

Job Audit History

GET {{url}}v1/jobs/{{job}}/history/?include=all
 
Sample response:
{
    "status": 200,
    "meta": {
        "links": {
            "self": {
                "href": "http://000.000.0.250:9090/v1/jobs/{{job_id}}/history/?offset=0&limit=100&fields=all"
            },
            "download": {
                "href": "http://000.000.1.250:9090/v1/jobs/{{job_id}}/history/.csv"
            }
        },
        "offset": 0,
        "limit": 100,
        "fields": [
            "all"
        ],
        "total_count": 1
    },
    "type": "job_executions",
    "job_executions": [
        {
            "job_id": "{{job_id}}",
            "progress": 1,
            "phase": "complete",
            "id": 108,
            "start_time": 1522232339,
            "end_time": 1522232358,
            "duration": {
                "value": 18.91,
                "unit": "s"
            },
            "status": "completed",
            "node_address": "000.000.1.250",
            "stats": {
                "destination": {
                    "folders": 5,
                    "import": {
                        "metadata": 9
                    },
                    "new": {
                        "bytes": 116070,
                        "files": 10,
                        "versions": 10
                    },
                    "pending": {
                        "batches": 0,
                        "files": 0,
                        "versions": 0
                    },
                    "rate_limits": 0
                },
                "source": {
                    "pending": {
                        "batches": 0,
                        "files": 0,
                        "versions": 0
                    },
                    "rate_limits": 0
                },
                "warnings": 56
            }
        }
    ]
}

Run the call below then copy the download link from the response body

  • Example: http://000.000.1.250:9090/v1/transfers/{{job_id}}/auditing.csv?audit_levels=error&jobs={{job_id}}
  • Paste this link into your Chrome browser
  • Note: You may need to log into the application again
  • Your browser will download the audit log with just the warnings as a .csv

Job Audit History | By Job, Show Warnings Only

GET {{url}}v1/transfers/{{job}}/auditing?level=warn
 
Sample response:
{
    "status": 200,
    "meta": {
        "links": {
            "self": {
                "href": "http://000.000.1.250:9090/v1/transfers/{{job_id}}/auditing?offset=0&limit=100&audit_levels=warn&jobs={{job_id}}"
            },
            "download": {
                "href": "http://000.000.1.250:9090/v1/transfers/{{job_id}}/auditing.csv?audit_levels=warn&jobs={{job_id}}"
            }
        },
        "offset": 0,
        "limit": 100,
        "audit_levels": [
            "warn"
        ],
        "jobs": [
            "{{job_id}}"
        ]
    },
    "item": [
        {
            "target": {
                "id": 3828
            },
            "execution_id": 108,
            "event": "metadata_import_fail",
            "id": 4516,
            "level": "warn",
            "message": "The destination path could not be determined from the mapped source path. (line=2)",
            "recorded_on": 1522232351
        },
.......
.....................

Review Job - Errors

Run the call below then copy the download link from the response body

  • Example: http://000.000.1.250:9090/v1/transfers/{{job_id}}/auditing.csv?audit_levels=error&jobs={{job_id}}
  • Paste this link into your Chrome browser
  • Note: You may need to log into the application again
  • Your browser will download the audit log with just the warnings as a .csv

Job Audit History | By Job, Show Errors Only

GET {{url}}v1/transfers/{{job}}/auditing?level=warn
 
Sample response:
{
    "status": 200,
    "meta": {
        "links": {
            "self": {
                "href": "http://000.000.1.250:9090/v1/transfers/{{job_id}}/auditing?offset=0&limit=100&audit_levels=error&jobs={{job_id}}"
            },
            "download": {
                "href": "http://000.000.1.250:9090/v1/transfers/{{job_id}}/auditing.csv?audit_levels=error&jobs={{job_id}}"
            }
        },
        "offset": 0,
        "limit": 100,
        "audit_levels": [
            "error"
        ],
        "jobs": [
            "{{job_id}}"
        ]
    },
    "item": [
        {
            "target": {
                "id": 3828
            },
            "execution_id": 108,
            "event": "fail",
            "id": 4515,
            "level": "error",
            "message": "Specified method is not supported.",
            "recorded_on": 1522232351
        },
.......
.....................

Related Links