Change History

The Open Data API now automatically caches data to make it more efficient to serve multiple requests for the same table or schema elements. This means that, for example, if multiple requests are received for the same table, the API will return the results from the cache, rather than re-querying SuperSERVER.

The API will also cache data from the metadata server (such as dataset labels).

• Configure the Cache

It is now possible to query the API for tables that have been saved to the User Data Repository (as long as the account doing the query has permission to access the saved table).

The ID of the table can be retrieved from the /schema endpoint, and this can then be passed to the /table/saved endpoint to retrieve the table query itself.

• Saved Tables

• Schema Endpoint

It is now possible to choose whether to explicitly set the decimal places for a saved table (TXD or SCS file). This can be particularly useful when loading TXDs into SuperWEB2, as any TXD decimal place settings will take precedence over SuperWEB2's precision settings. This means that you can now easily choose whether to explicity set the precision for your saved TXDs, or use the default SuperWEB2 settings.

In previous releases, the decimal places setting from SuperCROSS was always saved in the TXD file.

This setting is controlled by the SetDecimalPlaces command in the super.ini configuration file, and can also be changed via Edit > Options > Format > Decimal Places > Specify in the SuperCROSS GUI.

• super.ini
• Options

A change has been made to the default behaviour when a field containing a field derivation is removed from the table. In previous releases, field derivations remained in memory and could be added back to the table. The downside to this behaviour is that the Add Total With Default Recodes option did not apply in cases where a field with a recode was added back to the table.

The new default behaviour is that field derivations are automatically removed from memory when the field is removed, which means that the Add Total With Default Recodes option will now apply if selected when the field is added back to the table.

A new configuration option, RemoveFieldDerivationWhenFieldRemovedFromTable has been added to the cross.ini configuration file. You can adjust this setting if you wish to revert to the previous behaviour.

• cross.ini

The setting to enable Single Sign On is now stored in the ssii.ini configuration file. Previously it was stored in the Windows Registry.

If you are upgrading from an earlier version and you use Single Sign On you will either need to add this setting to the ssii.ini file manually, or reselect the single sign on option in the SuperADMIN Connection dialog.

• Single Sign On with Kerberos
• ssii.ini

SuperSERVER audit logs now record a tabulation.query event for every query event. This can contain any or all of the following details of the query:

• Fact Tables
• Fields
• Value Sets
• Values

By default, the event will include details of the fact tables and fields involved in the query. You can use the server command in SuperADMIN to configure this.

• Configure
• Logged Events

The Tomcat startup parameters in the sw2.bat file that is provided with the full SuperWEB2 installation have been updated to more closely reflect our recommended RAM settings. This batch file now starts Tomcat/SuperWEB2 with 4GB RAM. In many cases you are recommended to increase the available RAM where possible, see Installation Requirements, Compatibility and Supported Platforms for more details.

In addition, some changes have been made to the batch file to remove some unnecessary error messages that were previously output to the SuperWEB2 log.

If you are using the supplied Tomcat installation and batch files to run SuperWEB2 then you may wish to consider applying the changes from the latest version supplied with this release.

It is now possible to configure different rate limits for the /schema and /table endpoints.

As a result of this change, the openDataApi.schema.numberOfRateLimitPerPeriod property in the configuration.properties file has been replaced with three new settings:

• openDataApi.rateLimit.global
• openDataApi.rateLimit.schema
• openDataApi.rateLimit.table

In addition, there are two new endpoints for checking the individual rate limits: /rate_limit_schema and /rate_limit_table.

• Rate Limit
• Configure API Performance Settings
• configuration.properties

The following new methods have been added to enable Data Control plugins to obtain additional weighting information: GetWeightedTableIdentifiersT(), GetDefaultMainWeightFieldIdentifierT(), and GetDefaultReplicateWeightFieldIdentifiersT().

• Weighting

For performance reasons, the displayoptions settings are now stored in the configuration server. Previously, these settings used an XML file (DisplayOptionsCatalog.xml).

Existing settings will be migrated automatically when SuperADMIN is upgraded, and no user action is required.

The command used to set and check these settings remains the same, although as part of this change, display options settings can now be set for record counts (previously they could only be set for measures).

• displayoptions
• Set Precision for Summation Options

As a result of changes to the user catalogue (see below), there has been a minor change to the behaviour of the account locking feature.

Previously, changing the global settings for maxattempts and locktime only applied to new accounts. Now, changes to the global defaults will apply to all accounts unless you have specifically applied custom settings to individual accounts.

In addition, it is now possible to configure individual groups or users whose accounts cannot be locked.

• Unlock a User Account

• Port Usage
• Configure User Registration

For performance reasons, SuperADMIN now uses a relational database to store all user data (previously, it used an XML file). When you upgrade to version 9.9 SuperADMIN will automatically convert the existing user data to an H2 database.

It is possible to switch to another RDBMS, although this change should only be made before going into production. Please contact STR for advice if you wish to change the RDBMS for a production system.

SuperADMIN also uses H2 to store the configuration server data and data related to user registrations. Switching to another RDBMS will move all of this data in addition to the user data.

• Use a Relational Database to store SuperADMIN data

Some additional details are now recorded in the audit logs:

• The SuperADMIN logs now record the client that generated the event.
• SuperWEB2 now records the user's IP address on login events.
• A unique job ID is now recorded for all events relating to a specific tabulation request. The same ID will be recorded in the SuperWEB2 and SuperSERVER log files.
• The structure of the SuperWEB2 logs has been modified slightly: event types have moved to an action element and the query.part event type has been removed.

• Logged Events

The outputscaling data control module now supports:

• Custom precision for individual measures.
• Negative values for precision (negative values round values to the left of the decimal point; for example -1 will cause all values to be expressed as multiples of 10).

As a result of this change, an additional column (precision) has been added to the measures scale file.

• Scaling Factor and Precision

The direct URLs feature has been extended. Previously if you wanted to link directly to a dataset you could construct a link that opens that dataset in Table View (/opendatabase?id=<dataset_id>). There is now an alternative option for linking to a dataset, which will take users to the dataset's info page (/openinfopage?id=<dataset_id>).

In addition, a new URL parameter (#db=<dataset_id>) has been added to allow you to link to other dataset info pages from within info pages.

• Use Direct URLs to Open a Specific Dataset or Table
• Configure the Text on the Data Catalogue Page

• Configure Graph View

A new configuration setting (global.enableGuestAccess) has been added to the configuration.properties file that allows you to disable guest access for a SuperWEB2 instance.

In addition, the username and password for guest access are now configured in the configuration.properties file, rather than web.xml.

• Configure Guest Access
• configuration.properties

The thresholds for large table mode are now configured in configuration.properties. Previously these settings were configured in CubeCatalog.xml.

In addition, the default total cell threshold for large table mode has been increased from 10,000 to 100,000.

• Configure Settings for Large Tables
• configuration.properties

Mandatory fields are now indicated in the field list by a small red * image that appears on top of the field icon and the field item icon.

A new configuration option has been added that allows you to specify whether mandatory fields will be automatically tabulated when opening a dataset.

• Mandatory Fields

• Setup a Metadata Database

• configuration.properties

• Getting Started

• Use SuperWEB2 to Build API Queries

SuperWEB2 now has an automated user registration system, designed to make it easy to allow users to sign up for an account on your SuperWEB2 deployment, set their own password, and change or reset their password.

As part of this change, users can now change their password and delete their account from the Account page (accessible from the top-right menu).

• User Registration
• configuration.properties

Since version 9.0 SuperCROSS has been using a particular formatting function from a third-party library when generating Excel files. This function adjusts the row height in the downloaded Excel file to fit the text in the row. This function has been found to cause performance issues when downloading very large tables.

SuperCROSS now runs the function only once per row when downloading very large tables, so in some cases it may be necessary to manually adjust the row height in the downloaded Excel file in order to see the full heading text.

A new setting has been added to the super.ini configuration file: AutoFitCellLimit=90

This setting controls the threshold for the new behaviour. Its value is the table cell count in millions.

For example, when this is set to 90:

• Any table under 90 million cells will use the auto formatting option on every header cell.
• Any table above this size will not have any custom row heights applied. You may need to manually adjust the row heights in the downloaded Excel file.

• super.ini

SuperWEB2 now allows users to create derivations within tables.

Derivations are automatically enabled by default, but you can disable them globally by changing the table.enableDerivations setting in the configuration.properties file.

As part of this change, a new property (externalLink.helpDerivationURL) has been added to the configuration.properties file to provide a link to context-sensitive help within the Define Derivation dialog.

• Derivations
• configuration.properties

• RulesEngine.xml

It is now possible to control the formatting of annotations in Excel downloads. This is particularly useful if you have long annotations as you can now control how the text wraps beneath the table.

In addition, the default Excel download templates have been updated to match the current default SuperWEB2 branding.

• Branding Excel and CSV Templates

A new setting (schemaTree.schemaFolderExpansionDepth) has been added that allows you to choose how many levels of folders to expand in the list of fields when opening a dataset.

In previous releases, the list of fields was always fully collapsed when opening the dataset (so only the names of the top level folders would be visible initially). Now you can choose how many levels to expand.

By default, all folders will now be expanded when you open a dataset.

• configuration.properties

The configuration setting for specifying the PDF template for downloads from Graph View has changed. Previously this used a single setting (externalLink.pdfTemplateURL) that specified the full path to the template PDF file.

This setting has been replaced by two individual settings in the configuration.properties file: graph.graphPdfTemplateLocation and graph.graphPdfTemplateName.

• Branding PDF Downloads
• configuration.properties

• sw2-chart-config-options.js

The location of the HTML info pages (which are displayed on the right of the database catalogue screen) has changed.

Previously these pages were located in <tomcat_home>\webapps\webapi\WEB-INF\resources\info\

Now, they are located in <tomcat_home>\webapps\webapi\info\

• Configure the Text on the Data Catalogue Page

• Setup a Metadata Database

• Customise the Interactive Tour

A number of usability changes have been made to the SuperWEB2 user interface:

• Some on screen elements have moved or been renamed.
• New icons have been added to help users distinguish between datasets, tables, folders and fields.
• By default, the logo in the top-left of SuperWEB2 is now displayed within the navigation bar, rather than above it. This change is fully backwards compatible, so you can choose to continue to display the logo above the navigation bar if you prefer. Your existing branding customisations can continue to be used with this version of SuperWEB2.

• User Guide
• Understanding the Field List
• Customise Header, Footer and Logo

• super.ini
• Row and Column Totals and Percentages

• Create a Quantile Field

It is now possible to configure the precision used when displaying quantile range boundaries and RSE values in the labels of quantile user defined fields.

The precision of these values is controlled by two new settings that you can add to the [Preferences] section of the cross.ini configuration file: Quantile Label Value Precision and Quantile Label Rse Precision.

• cross.ini

When using weighted databases:

• Relative Standard Error for user defined fields is now calculated correctly and added to tables in the same way as standard measures.
• Relative Standard Error values are now recalculated correctly if perturbation is turned off in SuperCROSS.

• statistics.xml

It is now possible to display both weighted and unweighted values in the same table when using SuperSERVER weightings in SuperCROSS.

In previous versions, changing between a weighted and unweighted option in the Weight By drop-down list in the Define Recode window would change all of the functions in the Recode Values list to be either weighted or unweighted. Now, you can add a combination of weighted and unweighted functions to the Recode Values list.

• Weighted Datasets
• Understanding SuperSERVER, SuperCROSS and SuperWEB2 Weighting Differences

Quantile perturbation requires three configuration files. In previous releases these had to be located in the same directory as the SXV4 file.

New properties have been added to the data control modules so that it is now possible to configure the location of these files.

• Quantiles and Ranges

Prior to version 9.7, it was necessary to configure a data control plugin to calculate RSE values. From 9.7 onwards this now runs automatically for all SuperSERVER weighted databases, and will use the jackknife formula for calculating the RSE. You can still configure the plugin if you wish to change some of its other settings, such as the formula, thresholds and unreliable table message, although if you just want to change the formula used then you can do this by editing a setting in the weightings file (if both are configured then the plugin settings take precedence).

As a result of this change, RSE values are now available for SuperSERVER weighted databases in SuperCROSS when confidentiality has been turned off using the padlock button.

• Relative Standard Error
• sa_weightings.xml

New configuration options have been added to enable consistent weighting configuration across SuperCROSS and SuperWEB2.

• Understanding SuperSERVER, SuperCROSS and SuperWEB2 Weighting Differences

Additional configuration options have been added to the confidentiality rule data control module:

• You can now configure the confidentiality rule so that it only applies when specific fields are in the table.
• You can now set different thresholds and frequency rules for totals and non totals.

• Advanced Confidentiality Rule Options

• Open Data API

The following settings have been added to the configuration.properties file:

• externalLink.helpCatalogueURL
• externalLink.helpGraphViewURL
• externalLink.helpMyTablesURL
• map.mapArcGISEndpointPattern
• openDataApi.schema.pageSize
• openDataApi.schema.numberOfRateLimitPerPeriod
• openDataApi.table.maximumResponseCellLimit
• openDataApi.table.maximumRequestItemLimit

• configuration.properties

• Setup Static Metadata Pages

A number of usability changes have been made to the SuperWEB2 user interface.

Some on screen elements have moved or been renamed. For example:

• The Catalogue option on the SuperWEB2 menu bar has moved from the top-right of the screen to the top left, and has been renamed to Datasets).
• The My Tables tab has been moved to the drop-down menu in the top-right of the screen.
• The Custom Data tab has been moved to a button at the bottom of the field list in Table View.

If you have added your own UI languages, then some translation changes may be required.

• SuperWEB2

• Perturbation

• Configure Field Level Security

This release adds the ability to load TXDs into the User Data Repository through SuperADMIN. This means you can save a TXD from SuperCROSS and then load it into the SuperADMIN catalogue so that SuperWEB2 users can access that table (in previous releases this required you to save the TXD file to a location within SuperWEB2 itself).

Once you have loaded a TXD into the catalogue, you can use the cat command in SuperADMIN to manage that table (for example you can move it to a folder, rename it, or delete it).

• Configure Folders and Shared Tables

• The server command

• How It Works

A new option has been added to the config.txt configuration file allowing you to set the encoding to use when channelling with the TDD driver.

You may need to change this setting if you are channelling data that contains characters that are not supported by the current system codepage.

• config.txt

Additional logging is now available when SuperCROSS is running in server mode. SuperCROSS will now log the following events to the Super.logs.txt file (located in C:\ProgramData\STR\SuperCROSS\logs):

• Errors loading a TXD:
  • Invalid dataset ID.
  • Missing dataset ID.
  • Invalid column name.
  • Invalid value sets.
• Saving output:
  • Invalid Output Module defined in SuperModule.ini.
  • Invalid Function in the Output Module.
  • Save failed.

• Set Up Batch Processing

• SuperModule.ini

• Performance Considerations - SuperSERVER

Some changes have been made to audit logging:

• SuperSERVER now logs additional events: tabulation.request, tabulation,started and tabulation.retrieved.
• All SuperSERVER audit events now log duration in milliseconds (this includes tabulation.complete, which previously logged duration in seconds.
• SuperSERVER audit logging is now enabled by default.

Some changes have been made to SuperSERVER and Metadata Server logging:

• The logs are now configured to roll to a new file every day, by default.
• The log settings are now configured in an XML file, rather than a .properties file.

• Logged Events
• Configure
• Logging - SuperSERVER
• Logging

The following settings have been added to the configuration.properties file:

• catalogue.infoPanel.displayDatabaseMetadataDescription
• schemaTree.selectAtLevelShowsValuesetName
• table.useSavedTableLanguage
• table.displayMetadataAsAnnotations
• metadata.scanForDescriptions
• udr.maxQueryParams

• configuration.properties

• Configure a Default Table for your Datasets

The information pages that are displayed on the dataset catalogue screen in SuperWEB2 now support more options:

• It is now possible to create static HTML pages for different languages, by including the language code in the filename of the HTML page.
• It is now possible to configure SuperWEB2 to retrieve the information page from the metadata server, rather than using static HTML pages.
• It is now possible to display metadata links (retrieved from the metadata database) on the dataset information pages.

• Configure the Text on the Data Catalogue Page
• Use the Metadata Server to Store the Data Catalogue Information Pages - SuperWEB2
• Display Link Metadata on the Data Catalogue Page - SuperWEB2

Some settings in the basemaps.properties file have changed. This file that is used to configure the base map images (the map imagery itself, such as the street map images or the satellite view).

• The urllist property is now two related properties: url and subdomains
• An additional option, BLANK, is now available for the type property.
• The order of the map types has changed, so that satellite imagery is now the default type of map (previously the street map was the default).
• The default base map image for street maps now uses imagery from ArcGIS online (previously the default street map imagery was provided by Map Data Services).

• Configure Base Map Properties
• Troubleshooting Mapping

• Map View

A number of new features have been added to metadata:

• There is now a configuration setting that instructs SuperWEB2 to automatically scan for metadata descriptions in the metadata database, and display metadata links in the field list against fields that have descriptions in the metadata database.
• There is now an option to automatically retrieve descriptions from the metadata database for any fields in the current table, and display them as annotations below the table.
• A new REST endpoint has been introduced to enable administrator users to flush SuperWEB2's local metadata cache.

• Display Metadata Icons in the Field List
• Display Metadata as Annotations Below the Table
• Clear Metadata Cache

In some cases, precision settings in TXDs now override the SuperWEB2 precision settings.

In previous releases, the summation options precision setting (defined using the displayoptions command in SuperADMIN), and SuperWEB2's global dataprecision setting took precedence. Now, custom decimal place settings in TXDs saved from SuperCROSS that are loaded into SuperWEB2 will take precedence.

• Set Precision for Summation Options

• Configure Field Drop-down Labels

A new configuration option has been added that allows you to prevent users from sorting a field alphabetically in SuperWEB2.

By default, when a field has been added to the table rows, SuperWEB2 allows users to sort it into ascending and descending alphabetical order. In some cases, this will not make sense (for example, a date field where it does not make sense to sort the months alphabetically).

• Prevent Fields from Sorting Alphabetically

A new property has been added to the mapping configuration: localFeatureUrlHostName.

This property sets the host name of the Tomcat instance that mapping will use when it needs to make a request to another of the REST endpoints.

• Mapping Property Reference

The following settings have been added to the configuration.properties file:

• download.formatNumbers
• login.requireTerms
• map.mapCacheLocation

• configuration.properties

• Caching Map Data

• Set Precision for Summation Options

• Add a Warning Message to Printed Tables

• Add a Terms and Conditions Screen

The following setting has been added to the configuration.properties file:

• table.enableDrillUpAndDrillDown

• configuration.properties

The following settings have been added to the configuration.properties file:

• catalogue.databaseFolderExpansionDepth
• table.totalsInitialState

• configuration.properties

• The record type hierarchy graph menu option

The following setting has been added to the configuration.properties file:

• graphView.maxCells

• configuration.properties

• Configuration Options for Graph View

• Setup a Metadata Database

• Target Attributes

• Add Defined Fields

• Unicode

• Record Count
• Perturbation

• Creating Data Control Plugins

• Installation Requirements, Compatibility and Supported Platforms

• The cfg command
• Port usage
• List of SuperWEB2 settings that have moved to Configuration Server

• Batch Insert Mode

• ColourMatrix

• Update a Dataset

• Bulk Selection

• Change the Sort Order of Tables and Datasets

A new SuperWEB2 configuration file, configuration.properties, has been added, and many settings have moved to this file. When upgrading from an earlier release you will need to manually reapply any required settings by editing the new configuration.properties file.

• List of SuperWEB2 settings that have moved to configuration.properties

• Configure a Default Table for your Datasets

• Getting Started

• Graph View

It is now possible to navigate up and down the levels of a hierarchical field from with the table.

In addition, the default settings have been changed so that it is now possible to mix items from different levels of a hierarchy within a table (for example adding both Victoria and Melbourne from an Area hierarchy to the table at the same time). In previous releases, this behaviour was disabled by default. You can revert to the previous behaviour by editing the rules in the RulesEngine.xml configuration file.

• Working With Hierarchies
• RulesEngine.xml

• Mandatory Fields

• How to configuring mapping

• Multilingual and Localisation Support

• Configure Quantiles and Ranges

• Zero Suppression