Precomputings 101

Precomputings are an optimization to efficiently compute all aggregations that meet the eligibility criteria. They reduce the load of the server(s) and split the data process between eligible computations.

Precomputing aggregates data on the fly as it is being stored in the product database and produces elementary aggregation contexts. As a result, the final computation of a precomputed aggregate is simply a decrease of the right contexts.

Goal and principles of precomputing

In Axway Decision Insight, configured attributes (vs. collected ones) are all computed by the deployment and stored. When an attribute is displayed in a dashboard or used as an input to compute another attribute, the value of this first attribute has already been computed and stored previously. This way, long computation chains are avoided.

Computations are scheduled by batch, frequently enough for dashboard content to always be available in live mode. Nevertheless, a batch still requires scanning every input data necessary for the computation, which may involve a huge amount of data to process. 

The overhead induced by batches is usually high for aggregations, whether they are structural or temporal (aka. consolidation). What's more, several computed attributes often share most of their configuration, and sometimes only the aggregation function changes (SUM, AVG, STDDEV, COUNT, etc...).

Precomputings are an optimization targeting exactly this latter kind of computed attributes. By processing data on the fly as it is being stored and by sharing aggregation contexts when only the aggregating function changes, precomputings make batches less sensitive to data volume and smoothen the batches runtime load.

The precomputing optimization relies on a structure internally called cube. Cubes hold and maintain aggregation contexts on the fly. For more information, see the section about settings.

Eligibility of a computed attribute for precomputing

To be eligible for precomputings, an attribute must meet the following criteria:

  • It must be:
    •  an aggregation – many inputs on an entity to one output on another. 
    • or a consolidation – many inputs along time for one entity to one output on the same entity.
  • Its function must be one of the following: AND, AVG, COUNT, OR, RATIO, STDDEV, SUM, COUNT.
  • Indicator input: 
    • Value (monovalued only), Age (for age, only AVG, STDDEV and SUM are eligible) or Instance (only COUNT is available).
    • Carried by one entity only.
    • Arrhythmic.
    • The input must be on the same entity as the dimension or on a remote entity provided that the path length to the dimension is 1 and the relations are carried by the same entity as the input, and they are not multivalued-head.
  • Time configurations:
    • Either at reference time or on time range, but with the same configuration for every time configuration.
    • Or correlated with one of the relations from the input to aggregate's dimensions
  • Filters: 
    • Only by attribute, using a boolean attribute, eventually compared with a boolean constant (EQUALS or NOT EQUALS)
    • The boolean attribute must be on the same entity as the input
    • The constraints on the boolean are the same as those on the input
  • The computed attribute is rhythmic.

Monitoring

Manage precomputings from the Computings and Precomputings screens.

Computings

In this screen, precomputings information is in the precompute column. The values for an attribute may be:

  • N/A – the computed attribute is not eligible for precomputings.
  • Stop – the computed attribute is already plugged on a precomputing. (warning) Clicking on this action will trigger the recomputation of all already computed values.
  • Precompute – the computed attribute is eligible but not plugged on a precomputing. (warning) Clicking on this action will trigger the recomputation of all already computed values.

Precomputings

This screen provides monitoring for active precomputings:

  • Attribute names – the attributes eligible for the cube
  • Entity – the dimensions of the aggregates managed by the precomputing
  • Status – what is the cube doing (idle, bootstrapping, processing events)
  • Pending – number of transactions remaining to be processed by the cube 
  • Processing – number of transactions processed by the cube and remaining to be flushed (precomputed values are only available once flushed)
  • Actions – the trash bin is to delete the precomputing. It is equivalent to the Stop action in the Computings screen.

Behavior of precomputings

  • When an indicator configuration is created/updated, if an attribute is eligible for precomputing, a new cube will be created unless an eligible one already exists and the attribute computation will be plugged on it.
  • On application import, cubes will be automatically created for imported attributes that are eligible for them.
  • Precomputing is fully compatible with recomputing on correction feature.

Related Links