Use the Datastore Runtime command tool

Datastore provides a set of commands:

Use the batch tool

  1. Navigate to the [Installation_Path]/Datastore/DatastoreRuntime directory.
  2. To start the command tool and display the list of available commands, type:
    dstools help
  3. Enter or select a command.

When the execution is terminated without error, the return code of dstools is 0 , otherwise it is different from 0.

Priority rules for setting values to dstools parameters

For information on the priority rules, see Priority rules of encoding during injection.

Global parameters

These parameters are available for all the dstools services.

Language

Description

The language used for getting the extraction parameters from the dictionary defined in the Repository: english (default value), french, or other languages defined by the user.

Syntax: --language myLanguage

Example

dstools extract --file --query folder/myquery --format VariableFlatFile --language french

Profile

Description

The parameter profile allows you to choose which configuration profile will be used. Profiles are defined in Administration.

Syntax

--as “myprofile”

Parameter

-a | --as

Example

dstools inject --collection AUDIT_COLLECTION --as FLOW1 auditTrail.dat

Encoding

Description

The parameter Encoding allows you to choose which encoding will be used in input/output files. The default value is from the System charset default.

Syntax

--encoding “UTF-8”

Parameter

-e | --encoding

Example

dstools inject --collection AUDIT_COLLECTION --encoding ISO-8859-1 auditTrail.dat

Note   The dstools help --encodings command provides the list of available encodings

Parameter file

Description

The parameter @ allows you to launch the dstools command with a set of named parameters that have been previously defined in a text file.

Note that the PATHS parameter (defining the files to be injected) is not a named parameter, and thus cannot be specified inside the text file.

Parameter

@

Syntax

dstools inject @params.txt ../sample/dat/period.dat

Note   It is possible to change some of the parameters in the file by adding a new definition in the command line. For example, the following syntax changes the injection date defined in the previous example:
dstools inject @params.txt --date 20160203 ../sample/dat/period.dat

Extension parameters

Description

The parameter -exit-customization allows you to define the parameters that can be used in the exits using the ParameterizedExtension interface.

Syntax

--exit-customization key=value, key=value

Parameter

--exit-customization

Example

dstools inject [ … ] --collection AUDIT_COLLECTION --file auditTrail.dat

Verbose

Description

The parameter --verbose allows you to output more information in the console.

Syntax

--verbose

Parameter

-V | --verbose

Clean the database

Description

Clean the Datastore Runtime database.

This operation removes all information and functional structure from the database. There is not recovery solution from the product.

Caution   In case of error, a database restoration is mandatory.

Syntax

dstools clean

Configure from the Repository

Description

Configure from the Repository.

This command is used to synchronize Datastore with last configuration updates.

Note   This functionality is available from the Designer when using Deploy.

Syntax

dstools configure

Delete the collection

Description

Delete a collection by the collection identifier.

This command is used to delete one collection and all objects attached to this collection.

The collection identifier must be given in the argument.

Syntax

dstools delete-collection

-i | --id COLLECTION_ID

Parameters

collectionId - Collection Identifier which is displayed in the log file after injecting the collection

Example

dstools delete-collection --id C.20111130.122.20111130.122.20111130.122

Extract data

Description

Extract data selected by a query from a database into a file.

The query is defined by the folder and the name defined in the Datastore GUI.

Note   The query and its parameters can be defined using command line parameters or in the context file.

Syntax

dstools extract

-f|--file FILE

-q|--query QUERY

[-m|--format FORMAT]

[-sd|--startDate START_DATE]

[-ed|--endDate END_DATE]

[--collectionStartDate COLLECTION_START_DATE]

[--collectionEndDate COLLECTION_END_DATE]

[-l|--language LANGUAGE]

[-t|--timezone TIMEZONE]

[-h | --header]

[-label|--label]

[-F|--force]

[-ctxFile|--contextFile CONTEXT_FILE]

[-qCtxArgs|--queryContextArguments CONTEXT_ARGUMENTS]

[-qArgs|--queryArguments ARGUMENTS]

[--output-name NAME [--output-version VERSION]]

Parameters

  • format - Extraction file format, the possible values are: VariableFlatFile, FixedFlatFile
  • Default value is FixedFlatFile
  • file - File where extracted events are written
  • query - Complete query path
  • startDate - Query Start Date [yyyyMMdd] parameter
  • endDate - Query End Date [yyyyMMdd] parameter
  • collectionStartDate - Collection Start Date [yyyyMMdd] parameter
  • collectionEndDate- Collection End Date [yyyyMMdd] parameter
  • output-name - name of the objectType used to structure the output file
  • output-version - version if the objectType used to structure the output file
  • language - The language used for getting the extraction parameters from the dictionary defined in the Repository
  • Default value is English
  • If no language is defined in the dictionary, then these default extraction parameters are:
    • Column delimiter = ,
    • Decimal format = en
    • Date format = MM/dd/yyyy
    • DateTime format = MM/dd/yyyy HH:mm
  • timezone - A full time zone name such as America/Los_Angeles
  • header - The property type names are extracted in the first line. Available only for VariableFlatFile format
  • label - The labels are extracted in the second line, first line if header is not asked. The label is defined in the dictionary where the key is the property type name. Available only for VariableFlatFile format.
  • force - Overwrite action on the output file if it already exists
  • contextFile - Path to the file that contains the values for the missing command line arguments, except force and file
  • queryContextArguments - Query context arguments for the input values separated by commas
  • queryArguments - Query arguments for the missing values separated by commas

Using Dictionary

If you want to export Variable flat file (CSV format), you can use the Designer to add some extraction parameters or labels in the Datastore dictionary and deploy them into the Administration.

Example

Extraction based on a query:

dstools extract --file data.txt --query folder/myquery --format FixedFlatFile

dstools extract --file data.txt --query folder/myquery --format VariableFlatFile

dstools extract --file data.txt --query folder/myquery --format VariableFlatFile --language french

Using context file:

dstools extract --file data.txt --contextFile ctxFile.txt

with ctxFile.txt:

query=path/myquery

Set query arguments values

dstools extract --file data.txt --query folder/queryWithArgs --queryArguments IBAN1

Note   queryArguments is used when a query was defined with a set of parameters. For such query execution in Datastore GUI, the user has to set those parameters values.

Using queryContextArguments to set queries input values

dstools extract --file data.txt --query folder/queryWithArgs --queryArguments IBAN1

Note   queryContextArguments is used when a query was defined as a query with input.

Set Output objectType

If you want to extract query results following an ObjectType structure, use the output-name and output-version parameters:

dstools extract --file data.txt --query folder/myquery --format FixedFlatFile --output-name ACC_ME

dstools extract --file data.txt --query folder/myquery --format FixedFlatFile --output-name ACC_ME --output-version 1

When output-version is omitted, the last valid object type version is used. The validity of the Object type is determined by comparing its validity period with the extraction date.

Set Collection date range

collectionStartDateand collectionEndDate are used to limit the query process to the collections that match a specific collection date range.

The following command selects records from the query that were injected on the October, 2016, 1st only:

dstools extract --file 20161001.txt --query folder/myquery –-collectionStartDate 20161001 -–collectionEndDate 20161001

Get version number

Description

Displays:

  • the last version of the configuration pushed to the Datastore
  • the last available configuration version.

Syntax

dstools version

Inject data

Description

Inject data from a file into a database.

Syntax

dstools inject

[-c|--collection COLLECTION

[-v|--version VERSION]]

[-d|--date DATE]

[-t|--timezone TIMEZONE]

[-H|--header]

[-L|--label]

[-x|--properties-extension PROPERTIES_EXTENSION]

[--continue-on-error]

[-n|--display-only]

[-a|--as PROFILE]

[--exit-customization PARAM=VALUE[,...]]

[-e|--encoding]

[--verbose]

[PATHS...]

Parameters

  • collection - Name of the Collection Type attached to the input file
  • version -Version of the Collection Type attached to the input file
  • properties-extension - File extension of collection properties files. Default value: .properties
  • By default, the collection properties file should have the same name as the data file with the extension set in parameter properties-extension. If the parameter properties-extension is not set, the default value .properties is used.
  • Examples when properties-extension is not set:
    • If the data file name is IEventTrace then the collection properties file name should be IEventTrace.properties.
    • If the data file name is IEventTrace.txt, then the collection properties file name should be IEventTrace.txt.properties.
  • Example when properties-extension is set with .xml: If the data file is OSegtGL.seq then the collection properties file name should be OSegtGL.seq.xml.
  • header - Ignores header line. Variable flat file only.
  • label - Ignores label line. Variable flat file only.
  • date - Date associated with the injection [yyyyMMdd]. If the value is not entered, the current date is used.
  • file - File that contains the events to inject (use - to inject from standard input)
  • timezone ID - Time zone used to compute DateTime information. This information must be entered when the injected file is produced in a different time zone.
  • display-only - Display processing information without performing injection
  • continue-on-error - In case of injection error:

    When omitted: dstools stops when an injection raises an error.
    When set:
    current file injection error is logged and stopped,
    dstools injects other input files
    at the end of the processing, dstools returns a dedicated error code

  • as - Defines the profile used during injection
  • paths - Paths of files to be injected. When a path defines a folder, all files from this folder and its children are injected.
  • @ - File containing default command line parameters.

Collection properties file

During injection, collection properties can be injected with data properties. The collection properties are set in a collection properties file. The name of the collection property file must follow the rules described previously in the description of the properties-extension parameter.

You can use XML or properties file types for the collection properties file.

  • Example of XML file:
    • <?xml version="1.0" encoding="UTF-8" ?>
    • <properties>
      • <property>
        • <name>RECONCILIATION_ID</name>
        • <value>4</value>
      • </property>
    • </properties>
  • Example of properties file: RECONCILIATION_ID=4

Dynamic settings

The dynamic settings are managed in the Administration user interface.

Input Event trace version identification - This setting drives the identification of the object type version when injecting input event audit traces. There are two possible values:

  • date (default value) : Use the injection date to select the valid object type version
  • eventversion: The version is given by the audit property type EventVersion

Examples

Inject period:

dstools inject --collection PERIOD_COLLECTION --date 20120625 ../getStarted/withAccountingIntegration/data/period.dat

Inject file:

dstools inject --collection AUDIT_COLLECTION --date 20120625 ../../getStarted/withAccountingIntegration/data/ap.IEvent_trace

Inject folder:

dstools inject --date 20140712 ../folder1

Inject files using a file pattern:

UNIX:
./dstools inject -c AUDIT_COLLECTION *IEvent*

Windows:
dstools inject -c PERIOD_COLLECTION *IEvent*

Inject files using regex (UNIX) :

/dstools inject -c AUDIT_COLLECTION `find /tmp/TC -regex "Test_0[0-9][0-9]\."`

Inject files using regex (Windows, powershell)

$files = Get-ChildItem . | Where-Object {$_.Name -match "Test_0[0-9][0-9]\."} | select -expand FullName
dstools inject -c PERIOD_COLLECTION $files

Manage settings

Description

List and validate settings. The validation includes the check of the database connectivity and its configuration.

Syntax

dstools setting list|validate

Parameters

  • list - List the settings and their values
  • validate - Validate settings values

Examples

dstools setting list

dstools setting validate

Run a reconciliation

Description

You can use dstools to run a reconciliation.

Syntax

dstools reconcile

-q|--query QUERY

--id RECONCILIATION_ID

--date DATE

[--source1-startDate START_DATE]

[--source1-endDate END_DATE]

[--source2-startDate START_DATE]

[--source2-endDate END_DATE]

[--ignore-commented-records]

Parameters

  • query - Complete reconciliation function path
  • id - String, number or date [yyyyMMdd] reconciliation identifier
  • A reconciliation identifier is used to select source collections involved in the reconciliation. This identifier is defined as a collection property type in each reconciliation source collection. It is set when injecting data sources using collection properties files.
  • date - reconciliation date [yyyyMMdd]. If the value is not entered, the current date is used.
  • source1-startDate - Source1 collection start date [yyyyMMdd]
  • source1-endDate - Source1 collection end date [yyyyMMdd]
  • source2-startDate - Source2 collection start date [yyyyMMdd]
  • source2-endDate - Source2 collection end date [yyyyMMdd]
  • ignore-commented-records - Exclude commented records from the reconciliation sources

Example

dstools reconcile --query ./GETSTARTED_RECO --id 201608 –-date 20160815
--source1-startDate 20160801 --source1-endDate 20160831
--source2-startDate 20160801 --source2-endDate 20160831

Note   When the optional parameters are omitted, the command uses the values defined in the reconciliation request (these may be empty).
Note   The batch command runs the reconciliation only: the request to display the reconciled records is not executed.

Related Links