Module Callback API - Data Control API

JobInfoT has a member struct m_Callbacks that contains a set of function pointers that provide modules with access to table structural information and the results of a cross tabulation request.

The full set of callback functions accessible to a module, and associated documentation for parameters and return values, can be found in the supplied header file dcmodulecallback.h (located in C:\ProgramData\STR\SuperSERVER SA\etc\include\dcmodulecallback.h).

Many of these functions return pointers to data describing attributes of the cube currently being processed.

The API implementation is responsible for managing these resources, client programmers must not call free or delete on pointers returned by the API.
The memory associated with these resources is guaranteed to be maintained by the SuperSERVER for the duration of the job being processed. If values obtained from the API need to be stored between job invocations then the client programmer must manage this storage.

Data Cube Functions

The callback functions provide access to the cross tabulation data cube data structure.

The data cube is an n-dimensional array (cube) that holds counts for each cell in the cross tabulation request. The data cube structure differs from the structure presented to the SuperSTAR end user in SuperCROSS.

The following diagram shows a table in SuperCROSS with three fields (two fields in the row and one in the column):

When represented by a data cube, each field is represented in its own unique dimension that can be referenced by an integer value:

You can find the number of dimensions a data cube contains by calling the function:

CPP

GetDimensionCount(JobInfoT *JobInfo)

The number of fields for a given dimension can be found by calling the function:

CPP

GetDimensionFieldCount(JobInfoT *JobInfo, int Dimension)

Information about the fields contained in a dimension can be found by calling the following functions:

CPP

GetDimensionFieldBaseIdentifier(
    JobInfoT *JobInfo,
    int Dimension,
    int FieldOffset)

GetDimensionFieldBaseLabel(
    JobInfoT *JobInfo,
    int Dimension,
    int FieldOffset)

GetDimensionFieldIdentifier(
    JobInfoT *JobInfo,
    int Dimension,
    int FieldOffset)

GetDimensionFieldLabel(
    JobInfoT *JobInfo,
    int Dimension,
    int FieldOffset)

Identifier Functions

The GetDimensionFieldBaseIdentifier and GetDimensionFieldIdentifier functions return an identifier.

This is the name of the column in the source database that this field is associated with. For SXV4 databases, field IDs are only useful internally within SuperSTAR and are generated when the SXV4 is built in SuperCHANNEL.

Label Functions

The GetDimensionFieldBaseLabel and GetDimensionFieldLabel functions return labels. These are the "display names" shown to end users in the clients (e.g. "Gender"/"Area").

GetDimensionFieldBaseLabel will always return the label as defined at build time even if the user has renamed or recoded the field.
GetDimensionFieldLabel will return a modified label if it has been renamed or recoded.

Dimension Field Items

Each dimension contains a number of DimensionFieldItems. For example, the Area field has field items called New South Wales, Victoria etc.

You can find the number of DimensionFieldItems contained in a dimension by calling GetDimensionFieldItemCount for a particular dimension.

Each DimensionFieldItem contained in a dimension consists of an identifier and a label. For example, for the dimension containing the Gender field:

GetDimensionFieldItemIdentifier will return codes M, F.
GetDimensionFieldItemLabel will return labels Male, Female.

Data Cube Associations

Tabulation results are stored in a data cube. A data cube can associate with more than one other data cube. The associated data cubes might contain frequency of contributors and first N largest contributors.

For example:

Dimension Type

A data cube consists of at least one dimension. The type of a dimension can be summation or classification.

Dimension Item Type

Each dimension has at least one dimension field item. A dimension field item has its own item type. The Data Control API callback interface provides a function to query the item type of a dimension type:

CPP

CategoryTypeE GetDimensionFieldItemType(
    JobInfoT *JobInfo,
    int Dimension,
    int FieldOffset,
    int Item);

The Data Control API identifies the following item types:

Name	Value	Description
CAT_TYPE_INVALID	-1	An invalid dimension index or dimension item index was supplied.
CAT_TYPE_PLAIN	0	Item represents a normal classification field.
CAT_TYPE_HIERARCHY	1	Item represents an hierarchic classification field.
CAT_TYPE_GRAND_TOTAL	2	Item represents a recode total that contains all classification values within the field.
CAT_TYPE_SUB_TOTAL	3	Item represents a recode total that contains a strict subset of classifications values within the field.
CAT_TYPE_OVERLAPPED	4	Place holder (not currently supported). Overlapped totals can occur in hierarchical fields whose definition is a directed acyclic graphs (DAG) rather than a tree. In a DAG the same leaf item may be counted multiple times depending on which higher level nodes in the hierarchy are present in the recode.
CAT_TYPE_STORED_TOTAL	5	Item represents a stored total computed by the server and stored in an SXV4 cube.

Plain items

A dimension item is a plain item when this dimension item is a single/grouping recode.

Totals and total additivity

A dimension item is a total when this dimension item is recode total. All sub-items contributing to the dimension item recode total must be in the table.

For the item type CAT_TYPE_GRAND_TOTAL, the totals may be additive or non-additive. The Data Control API callback interface provides a function to identify the totals as additive or non-additive:

CPP

int AreAllCubeTotalsAdditiveT( JobInfoT *JobInfo);

This function returns 1 if all totals in the cube are additive and 0 if they are not.

Accessing Fact Table Values

The Data Control API callback interface provides functions to query the size of a Fact table and the sum of its main weight

Function	Description	Arguments	Returns
CPP `int GetSumMainWeightTable( JobInfoT JobInfo, const char tableName, double* result)`	Retrieve the sum of the main weight in the specified table.	JobInfo (Input) - Information about the current job. tableName (Input) - Specifies the table. result (Output) - Specified pointer to receive the resulting sum value.	1 if the values are retrieved successfully 0 otherwise
CPP `int GetNumberRecordsInTable( JobInfoT JobInfo, const char tableName, double* result)`	Retrieve the number of records in the specified table.	JobInfo (Input) - Information about the current job. tableName (Input) - Specifies the table. result (Output) - Specified pointer to receive the resulting number of records.	1 if the values are retrieved successfully 0 otherwise

Accessing Cell Values

The Data Control callback interface provides functions to assist in navigating cells in the data cube:

Function

Description

CPP

FirstCell (JobInfoT *JobInfo)

Moves to the first cell in the data cube.

For a 3 dimensional cube, resets the location to [0][0][0].

CPP

NextCell (JobInfoT *JobInfo)

Moves the iterator to the next cell of the cube.

When calling NextCell for the first time after FirstCell, the iterator location will move to [0][0][1].

Once all cells in a particular dimension have been visited, that dimension's index is reset to zero and the following dimension's index will be incremented by one. For example [0][1][0].

When all cells have been visited (the last cell of the data cube is passed), this function returns 0.

CPP

EndCell (JobInfoT *JobInfo)

Allows client programmers to test whether the current cell is the last in the iteration of the cube.

Returns 1 to indicate that this is the last cell in the iteration.

The Iterator will visit all cells in the data cube. Using the iterator functions is the recommended method of navigating the data cube as it allows SuperSERVER to implement data cube navigation efficiently.

The following functions are also supplied for scenarios where an implementation only requires direct access to a subset of well-known cells, rather than iterating through the entire cube.

CPP

GetCellValue (
    JobInfoT *JobInfo, 
    int *DimensionItems,
    double *Value)

SetCellValue (
    JobInfoT *JobInfo,
    int* DimensionItems,
    double Value)

GetCellAssociatedValue ( 
    JobInfoT *JobInfo,
    char* AssociationName,
    int* DimensionItems,
    double *Value)


SetCellAssociatedValue ( 
    JobInfoT *JobInfo, 
    char* AssociationName, 
    int* DimensionItems, 
    double Value)

Logging and end user notification

The following functions are provided for logging and end user notification.

Function	Description	Arguments	Returns
CPP `LogInfoMessage( JobInfoT JobInfo, const char Message)`	Deprecated. Replace with `LogMessage` (see below). Logs a message to the SuperSERVER log.
CPP `LogErrorMessage( JobInfoT JobInfo, const char Message)`	Deprecated. Replace with `LogMessage` (see below). Logs a message to the SuperSERVER log.
CPP `SetUserMessage( JobInfoT JobInfo, const char Message)`	Deprecated. Replace with `SendUserMessage` (see below). Displays a message to the end user in the client.
CPP `const char* GetLogOutputLocation( JobInfoT *JobInfo)`	Get the output location for logging. This is useful if the API wants to write other output to a separate file (use `LogMessage` for normal cases).		`const char*` - The location of the path, or NULL for no location
CPP `int IsLoggingEnabledFor( JobInfoT *JobInfo, LogMessageLevelE level)`	Checks if the specified logging level is enabled for output (which severity messages are output to log files). Use for conditionally avoiding doing expensive debug operations or to log to a separate location (use `LogMessage` for normal cases).	level - The logging level: LOG_LEVEL_ERROR LOG_LEVEL_WARN LOG_LEVEL_INFO LOG_LEVEL_DEBUG LOG_LEVEL_TRACE	1 - Logging is enabled 0 - Logging is disabled
CPP `void LogMessage( JobInfoT JobInfo, LogMessageLevelE level, const char Message)`	Log a message that has occurred in the module, at a certain level. For example, Info, Warning, Error. This usually logs to the log file and the stdout of the process.	JobInfo (Input) - Information about the current job. level (Input) - The information level: LOG_LEVEL_ERROR LOG_LEVEL_WARN LOG_LEVEL_INFO LOG_LEVEL_DEBUG LOG_LEVEL_TRACE Message (Input) - The message to log.
CPP `int SendUserMessage( JobInfoT JobInfo, UserMessageLevelE level, const char Message)`	Send a message to the user that has occurred in the module. These messages are processed in the SuperSTAR UI applications.	JobInfo (Input) - Information about the current job. level (Input) - The information level: LEVEL_ERROR LEVEL_WARN LEVEL_INFO Message (Input) - User message	1 if successful 0 otherwise

Configuration Values

Configuration values (properties) may be stored in the confidentiality or method catalogue of SuperADMIN.

When invoked in 'local mode', the production system can also be configured to access the underlying XML configuration data for the confidentiality catalogue.

When a Data Control module is invoked these configuration values may be accessed by the module using this function:

CPP

GetResultPropertyValues (
    JobInfoT *JobInfo, 
    const char* Key, 
    const char*** Values,
    int* Len)

Facilities are also provided to allow external modules to dynamically add (SetResultPropertyValue()) and remove (RemoveResultPropertyValue(), RemoveResultProperty()) properties.

These calls do not affect the underlying entries in the catalogue and are only present for the duration of the tabulation request. You can use these facilities to allow a simple form of 'inter module' communication when multiple modules are configured for a particular tabulation request.

The PropertyValue interface may associate multiple values with a single key.

Standard Properties

The following standard properties are defined for use by SuperSERVER:

Property	Type	Description
FREQ	boolean (true/false)	The module requires SuperSERVER to produce an associated data cube containing contribution count information for each cell. Data is retrieved using the `GetCurrentCellAssociatedValue()` callback passing the association name as 'FREQ'. The FREQ property (hence the FREQ cube) is affected by null handling. The FREQ cube contains the number of contributors. If there is a null field value that is contributing to a specific cell in the cube, then it may, or may not (depending on null configuration) be counted. This will mean a different number of contributors hence different values in the FREQ cube.
TOPN	integer (1+)	The module requires SuperSERVER to produce an associated data cube containing the values for the top n contributors for each cell. Data is accessed by client programmers using the `GetCurrentCellNthAssociatedValue()` callback.
CONCEALMENT		This is a special, well known, association name. Client programmers programmatically create this associated data cube by calling the `AddAssociation()` callback function and then set the associated cell value to 1 to indicate a cell should be hidden by using the `SetCurrentCellAssociatedValue()` callback passing the association name as "CONCEALMENT".
ConfidentialityModule	boolean (true/false)	Indicates to SuperSERVER that this module applies confidentiality rules. SuperSERVER may block record view requests based on the setting of this property and the values of the 'CONCEALMENT' associated data cube.
GrandTotalRequire		Indicates to SuperSERVER that grand totals for all fields in the tabulation request be computed and present in the data cube prior to execution of an external module. During module execution, grand totals can be identified using the `GetDimensionFieldItemType()` callback function. Grand Totals added to the cube by SuperSERVER in this fashion will be removed from tabulation data prior to returning results to the User Interface client application.
CELL_ERRORS	boolean (true/false)	The module requires SuperSERVER to produce an associated data cube containing a count of errors (or Nulls) for each cell. Data is retrieved using the `GetCurrentCellNthAssociatedValue()` callback passing the association name as 'CELL_ERRORS'.

Job Identification

The following functions are provided to allow client programmers access to details identifying the current job being processed:

CPP

GetJobName (JobInfoT* JobInfo)

GetJobType (JobInfoT* JobInfo)

GetJobOwnerID (JobInfoT* JobInfo)

GetJobOwnerGroups (
    JobInfoT* JobInfo,
    const char*** Groups,
    int* Len)

GetJobDBCatalogueID (JobInfoT* JobInfo)

GetJobDBFileName (JobInfoT* JobInfo)

These functions may be of use when writing logging code in a client implementation.

In addition, client programmers may use the function GetAPIVersion() to determine the connected server's implementation version of the Data Control API itself.

Annotations

SuperSERVER maintains a separate database for storing annotation information. For details of the database schema and facilities supported by the SuperSERVER annotation subsystem, see Annotations - Data Control API.

The following functions allow access to the annotation facilities within SuperSERVER.

Functions	Used For
CPP `AddSymbolDescription ( JobInfoT* JobInfo, const char* Symbol, const char* Description) GetALLSymbolDescription ( JobInfoT* JobInfo, const SymbolDescriptionT** Values, int* Len)`	Manipulation of symbols and their descriptive text.
CPP `AddDatabaseAnnotation ( JobInfoT JobInfo, const char Symbol) DeleteDatabaseAnnotation ( JobInfoT JobInfo, const char Symbol) GetDatabaseAnnotation ( JobInfoT JobInfo, const char** Values, int* Len)`	Manipulation of annotations associated with the current database.
CPP `AddFieldAnnotation ( JobInfoT JobInfo, int Dimension, int FieldOffset, const char Symbol) DeleteFieldAnnotation ( JobInfoT JobInfo, int Dimension, int FieldOffset, const char Symbol) GetFieldAnnotation ( JobInfoT JobInfo, int Dimension, int FieldOffset, const char** Values, int* Len)`	Manipulation of annotations associated with a nominated field.
CPP `AddFieldValueAnnotation ( JobInfoT JobInfo, int Dimension, int FieldOffset, int WhichItem, const char Symbol) DeleteFieldValueAnnotation ( JobInfoT JobInfo, int Dimension, int FieldOffset, int WhichItem, const char Symbol) GetFieldValueAnnotation ( JobInfoT JobInfo, int Dimension, int FieldOffset, int WhichItem, const char** Values, int* Len)`	Manipulation of annotations associated with a nominated field value.
CPP `AddCurrentCellAnnotation ( JobInfoT JobInfo, const char Symbol) DeleteCurrentCellAnnotation ( JobInfoT JobInfo, const char Symbol) GetCurrentCellAnnotation ( JobInfoT JobInfo, const char* Values, int* Len) AddCellAnnotation ( JobInfoT JobInfo, const int CellLocation, int Len, const char* Symbol)`	Manipulation of annotations associated with the Cell currently selected by the iteration callbacks.

Thread Safety

The Data Control API does not support multiple threads in module code.

SuperSERVER is a multi-threaded application. Client programmers must take care when implementing modules to ensure that code that they write is thread safe.

In particular, ensure the use of 'static' storage within a module's implementation, and access to external systems, is protected against the impact of operating in a multi-threaded environment.

Memory Management

When using the Data Control API all resources allocated (malloc, fopen, ...) by a module must be cleaned up by that module.

Resources allocated for each job should be cleaned up in cleanupJob() at the latest.

If resources are allocated and stored statically in the module they should be cleaned up in destroy().

All pointers returned from SuperSERVER via callback functions will be cleaned up by SuperSERVER and must not be freed or deleted by any modules.