Define partitions for Datastore

Datastore partitions are used to store records following the space configuration implemented. Partitioning is used in order to:

  • Improve injection and query performance.
  • Ease the management of online partitions and retention periods, and implement a partition lifecycle management strategy.

Partitioning policy

At injection time, records that match the criteria of a given partition depending on Object/Element (Date & Range/ Period & Range / Range) are stored in this partition.

A possible data distribution is:

  • Day - One partition is dedicated for each functional date and designed for huge numbers of injections (100 000 000 events per day).
  • Period - One partition is dedicated for each functional period and designed for large numbers of injections (100 000 000 events by period (~/month)).
  • None - All records are stored within a single partition. It is designed for small volumes such as referential data.
Note   You can also define a custom partition service and control the names of partitions for any given dates. For more information, see Create Datastore exits. See also the "Update the partition service" section in the Capacity Planning Guide.

Partition lifecycle

Due to legal requirements, huge volumes of information (such as audit trails) must be available during long periods.

Available for partitioned read-only spaces, the partition lifecycle is used to manage information depending on its age and activity around it:

  • activity on recent information is high because of daily requests of consolidation
  • activity on less recent information is medium
  • activity on older information is low and - in general - linked with legal audits

Information lifecycle

Offline or purged partitions

When a query requests records that are located in offline partitions, the request is rejected and a warning message displays the list of offline partitions. Copy this list and send it to your administrator. The administrator will have to mount the corresponding partitions. Once all needed partitions are available, the query can be re-executed.

Queries that request records located in purged partitions are not rejected and no warning message is displayed.

Related Links