Field Exclusion Rules

You can use the field exclusion rules to limit the number of fields from a defined group that can be added to the table at any one time. This feature is typically used to restrict the number of sub-state geographies that can be added into a table at the one time.

There are two types of rule you can define:

Global rules apply to all datasets.
Dataset rules apply to a specific dataset, in addition to any global rules that have been defined.

The rules are defined as a list of fields and a maximum number. Users will not be able to add more than this maximum number of fields from this group to the table.

Field exclusion rules apply to fields (classifications) only. They cannot be applied to measures/summation options.

In addition, field exclusion rules do not apply when concatenating fields .

Step 1 - Confirm that the Field Exclusion Rule is Active

Open <tomcat_home>\webapps\webapi\WEB-INF\data\.repository\RulesEngine.xml in a text editor.

Locate the following section and confirm that the FieldExclusionRule is active and not commented out:

XML

<rules:RulesPipe name="CDataOnlineEditRules">
  <!-- <rules:rule-name name="OverrideDefaultSummationRule"/> -->
  <!-- <rules:rule-name name="MandatoryFieldsRule"/> -->
  <!-- <rules:rule-name name="DBSummationOptionsRule"/> -->
  <!-- <rules:rule-name name="GuestUserCellLimitRule"/> -->
  <rules:rule-name name="DemographicVariablesRule"/>
  <rules:rule-name name="TableCellsNumberRule"/>
  <rules:rule-name name="HierarchyLevelRule_All"/> 
  <rules:rule-name name="checkAddToDimensionRule"/>
  <rules:rule-name name="checkDoubleCountingRule"/>
  <rules:rule-name name="AutoRecodeItemRule"/>
  <rules:rule-name name="SummationOptionsRule"/>
  <rules:rule-name name="DutyOfCareRule"/>        
  <rules:rule-name name="FieldExclusionRule"/>   
</rules:RulesPipe>

Save your changes to the file (if any).

If you have made changes to the RulesEngine.xml file, then you will need to restart SuperWEB2 or Tomcat to apply the change.

Step 2 (Optional) - Configure Field Exclusion Rules to Apply to Saved Tables

By default, field exclusion rules are applied when creating new tables, but are not checked when loading a saved table. This means that any tables that were saved before the field exclusion rules were implemented will continue to open even if they breach the rules.

If you prefer to enforce the field exclusion rules for saved tables as well as new tables created by users, then you can configure this by making another change to the RulesEngine.xml file.

Locate the following section:

XML

    <rules:EditRule name="FieldExclusionRule"
        applicableOperations="add"
        implClassName="au.com.str.webapi.services.database.manager.rules.impl.FieldExclusionRule">
    </rules:EditRule>

Add ;new to the applicableOperations:

XML

    <rules:EditRule name="FieldExclusionRule"
        applicableOperations="add;new"
        implClassName="au.com.str.webapi.services.database.manager.rules.impl.FieldExclusionRule">
    </rules:EditRule>

Locate the following section:

XML

    <rules:RulesPipe name="NewTableRules">
        <!-- <rules:rule-name name="MandatoryFieldsRule"/> -->
    </rules:RulesPipe>

Add the FieldExclusionRule, as follows:
XML
```
<rules:RulesPipe name="NewTableRules">
    
    <rules:rule-name name="FieldExclusionRule"/>
</rules:RulesPipe>
```
Make sure you add the FieldExclusionRule as the last rule in the NewTableRules pipe, to ensure that the rules are checked after any other rules (such as mandatory fields or mandatory values) have run.
Save your changes, and restart SuperWEB2 or Tomcat to apply the change.

Step 3 - Define the Field Exclusion Rules in SuperADMIN

Before starting to create new rules, you should check whether there are any existing field exclusion rules defined. Login to SuperADMIN and use one of the following commands:

To check for existing global rules, use the following command:

CODE

cfg global superweb2.rules.fieldExclusion

To check for existing dataset rules, use the following command (replace <dataset_id> with the ID of the dataset):

CODE

cfg db <dataset_id> superweb2.rules.fieldExclusion

SuperWEB2 will either display the list of existing rules (if some rules are already defined), or display "not found".

Scenario A: No Existing Rules Defined

If there are no existing rules defined, SuperADMIN displays the message "not found":

Global Rules:

CODE

> cfg global superweb2.rules.fieldExclusion
cfg global superweb2.rules.fieldExclusion : not found

Dataset Rules:

CODE

> cfg db bank superweb2.rules.fieldExclusion
cfg db bank superweb2.rules.fieldExclusion : not found

If there are no existing rules defined, then you can create a new rule using the following commands:

Global rules:

CODE

cfg global superweb2.rules.fieldExclusion[<index>].limit set <limit>
cfg global superweb2.rules.fieldExclusion[<index>].fields set [ <list_of_fields> ]

Or:

CODE

cfg global superweb2.rules.fieldExclusion add { "fields": [ <list_of_fields> ], "limit": <limit> }

Dataset rules:

CODE

cfg db <dataset_id> superweb2.rules.fieldExclusion[<index>].limit set <limit>
cfg db <dataset_id> superweb2.rules.fieldExclusion[<index>].fields set [ <list_of_fields> ]

Or:

CODE

cfg db <dataset_id> superweb2.rules.fieldExclusion add { "fields": [ <list_of_fields> ], "limit": <limit> }

Where:

<index> is the rule number. This is only required when using the set version of the command. Make sure that each rule you add has a unique index number, starting from zero. If you use the add version of the command then your value will automatically be added as the next item in the array.
<dataset_id> is the ID of the dataset this rule applies to (you can obtain this using the cat command in SuperADMIN).
<limit> is the maximum number of fields from this group that can be included in a table at any one time.
<list_of_fields> is the list of fields. Specify each field in quotes, separated by commas and with square brackets at the start and end of the list.

You can either use the field display names (shown in SuperWEB2) or the field IDs (which you can obtain in SuperADMIN by typing cat <dataset_id> <field_name>). If you are using multilingual datasets, then you must use the field ID.

For example, the following commands specify that only one of these three fields (Age, Area and Marital Status) can be added to tables created from the Retail Banking dataset at any one time. As this is the first rule defined it has the index value of 0:

CODE

> cfg db bank superweb2.rules.fieldExclusion[0].limit set 1
superweb2.rules.fieldExclusion[0].limit : Created
> cfg db bank superweb2.rules.fieldExclusion[0].fields set ["Age", "Area", "Marital Status"]
superweb2.rules.fieldExclusion[0].fields : Created

If you want to define a second dataset rule, use the same command but increment the index value. For example (this example uses field IDs):

CODE

> cfg db bank superweb2.rules.fieldExclusion[1].limit set 1
superweb2.rules.fieldExclusion[1].limit : Created
> cfg db bank superweb2.rules.fieldExclusion[1].fields set ["SXV4__Retail_Banking__F_Customer__Occupation_FLD", "SXV4__Retail_Banking__F_Customer__Gender_FLD"]
superweb2.rules.fieldExclusion[1].fields : Created

Alternatively, you could add a new rule using the add command, as follows:

CODE

> cfg db bank superweb2.rules.fieldExclusion add { "fields": ["Age", "Area", "Occupation"], "limit": 2 }
superweb2.rules.fieldExclusion : Updated

Scenario B: Existing Rules Defined

If there are already existing rules defined, then SuperADMIN displays the details. For example:

CODE

> cfg db bank superweb2.rules.fieldExclusion
superweb2.rules.fieldExclusion :
[
    {
        "limit":1,
        "fields":[
            "Age",
            "Area",
            "Marital Status"
        ]
    },
    {
        "limit":1,
        "fields":[
            "SXV4__Retail_Banking__F_Customer__Occupation_FLD",
            "SXV4__Retail_Banking__F_Customer__Gender_FLD"
        ]
    }
]

In this example there are two rules defined for the Retail Banking dataset (each rule is enclosed in curly brackets).

You can either add a new rule or update one of these existing rules.

To add a new rule, either use the add command, or use the set command and make sure that you use the next index value up. In this example, there are already two rules, so to create a third rule you must use the index value of 2 (because the index numbering starts at 0). For example:
CODE
```
> cfg db bank superweb2.rules.fieldExclusion[2].limit set 1
superweb2.rules.fieldExclusion[2].limit : Created
> cfg db bank superweb2.rules.fieldExclusion[2].fields set ["Age Groups", "Gender"]
superweb2.rules.fieldExclusion[2].fields : Created
```
Alternatively, you can add the same rule without specifying the index value as follows:
CODE
```
cfg db bank superweb2.rules.fieldExclusion add { "fields": ["Age Groups", "Gender"], "limit": 1 }
```
To update an existing rule, simply use the index value of the existing rule. For example, to update the limit setting for the second rule (the Occupation/Gender rule), use the index value of 1:
CODE
```
> cfg db bank superweb2.rules.fieldExclusion[1].limit set 2
superweb2.rules.fieldExclusion[1].limit : Updated
```

Step 4 - Validate the Rules in SuperWEB2

When you have finished defining your rules, go to SuperWEB2 and create some tables to verify that the rules are defined correctly.

There is no need to restart SuperWEB2 to see the changes, but if Table View is already open, you must go back to the catalogue and reselect the dataset to access your updated configuration. This is also the case for any users who are already logged in: they will not see the impact of the new rules until they reselect the dataset from the catalogue.

For example:

Notes:

If a recode exists under a field that has the field exclusion rule applied, the same error will occur if a user tries to add the recode to a table and it exceeds the limit.
The rule also applies to hierarchical fields. If a user attempts to add a value from a field with multi-level hierarchies, an error will occur if the limit is exceeded.

Remove Field Exclusion Rules

If you want to remove a field exclusion rule, use the following commands:

Global Rules:

CODE

cfg global superweb2.rules.fieldExclusion[<index>] remove

Dataset Rules:

CODE

cfg db <dataset_id> superweb2.rules.fieldExclusion[<index>] remove

For example, to remove the second rule defined for the bank dataset, use the following command (the second rule has the index value of 1, because the rule numbering starts at 0):

CODE

> cfg db bank superweb2.rules.fieldExclusion[1] remove
superweb2.rules.fieldExclusion[1] : Removed

When you remove the rule, the index values of any subsequent rules will be updated accordingly to replace the removed rule. For example, if you have three rules defined, the index values of these rules will be 0, 1 and 2. When you remove the rule with index value 1, the index value of the third rule will be updated from 2 to 1.

If you want to remove all the existing field exclusion rules, use the following commands:

Global Rules:

CODE

cfg global superweb2.rules.fieldExclusion remove

Dataset Rules:

CODE

cfg db <dataset_id> superweb2.rules.fieldExclusion remove

For example, to remove all the rules defined for the bank dataset, use the following command:

CODE

> cfg db bank superweb2.rules.fieldExclusion remove
superweb2.rules.fieldExclusion : Removed