Update the Search Index for Single Language Datasets

If you are only using single language datasets, then you can update the search index by running the script BuildSXV4SearchIndex, which is located in the SuperADMIN program data directory. For example, on Windows if you install to the default location, it will be located at C:\ProgramData\STR\SuperADMIN\MetaData\MetaDataUtilities\BuildSXV4SearchIndex.bat

Before starting, check that you have plenty of available disk space (the exact amount required to store the final index will depend on the size of your dataset catalogue, but can be several gigabytes for a large catalogue). In addition, the indexing process will create some temporary files while it builds the index. These will be cleaned up automatically at the end of the process, but may cause the index to temporarily grow to around double its final size before these files are cleaned up.

If you encounter disk space errors during the indexing, you should either increase the size of the disk or use the DESTINATION_FOLDER  setting to change the location of the index to a partition that has sufficient disk space.

Step 1 - Update databases.txt

This is a text file that specifies which SuperSTAR datasets you want to include in the index. It is located in the same directory as the indexing script.

You need to update this file so that it contains a list of all the datasets on your deployment that you want to be indexed.

You should include all your datasets in the index. If you have datasets that not all users have access to, then you can still include these in your index. SuperADMIN will automatically take care of permissions when a user searches in SuperWEB2. The search results will automatically be filtered so that they only contain results from datasets and fields that the user has permission to access.

You can either update databases.txt manually or use the createdatabaselist command in SuperADMIN.

The file must use the following format (if you use the SuperADMIN command it will generate a list of all of your SuperSTAR datasets in this format):

<dataset_id>|<display_name>|<full_path_to_SXV4>

Where:

<dataset_id>

<display_name>

<full_path_to_SXV4>

For example, the shipped databases.txt file is as follows. This would instruct the batch process to index the sample People and Retail Banking datasets:

people|people|C:\ProgramData\STR\SuperSERVER SA\databases\People
bank|bank|C:\ProgramData\STR\SuperSERVER SA\databases\RetailBanking

When you have finished editing databases.txt, save the file.

You are recommended to save this file in the same location as the standard shipped file. If for any reason you want to save the file to a different location, you must update SET DB_FILE_LIST="databases.txt" in BuildSXV4SearchIndex  before running the script so that it contains the full path to your new location of the databases.txt file.

Step 2 (Linux Only) - Set the Index and Destination Directories

On Linux, you will need to edit some of the variables in the indexing script before you can run it:

• INDEX_DIR: set this to the full path of the MetaDataUtilities directory where the indexing script is located. For example: export INDEX_DIR=/home/str/superadmin/Metadata/MetaDataUtilities 
• DESTINATION_FOLDER: set this to the full path to the location where you want to generate the index files. This will need to match the configuration in SuperADMIN, which is set to use the meta_search_index directory in the SuperADMIN program data directory by default. See the next step for more details on this variable.

Step 3 - Check the Index Location in BuildSXV4SearchIndex

There is a setting in the script (DESTINATION_FOLDER) that instructs it where to generate the index files. There is also a setting in SuperADMIN that determines where it will look for the generated index when a user performs a search (you can check this using the command gc search indexDirectory).

By default, these are both set to the meta_search_index directory in the SuperADMIN program data directory (by default, C:\ProgramData\STR\SuperADMIN\server\meta_search_index on Windows). The predefined index that is supplied with SuperSTAR (which covers the sample Retail Banking and People datasets) is located in this directory.

When you run the script to update the index, the first thing it will do is to remove the existing index. This means that search will be unavailable to users until it is rebuilt.

As it may take some time to rebuild the index (particularly if you have a large number of datasets), then you may want to take the following steps:

1. Update the DESTINATION_FOLDER setting in the BuildSXV4SearchIndex script to point to a new location.
2. Run the script.
3. When the script finishes, update SuperADMIN to use the new location.

This will ensure that there is no downtime of the search index and that users can continue to search (using the old index files) while you are doing the update.

The following steps assume you are going to build the index to a new location to avoid downtime. If you are not concerned about search being unavailable, then you can simply execute the indexing script using the default output location. Once it has completed, log in to SuperWEB2 and check that search now includes all your datasets.

To update the index location and rebuild the index:

1. Open BuildSXV4SearchIndex in a text editor.

2. Check the DESTINATION_FOLDER setting. By default, this is set as follows:

   CODE

   set DESTINATION_FOLDER="%SA_PROGRAM_DATA%\server\meta_search_index"

   This indicates that the search index files will be saved to the meta_search_index directory in the SuperADMIN program data directory (by default, C:\ProgramData\STR\SuperADMIN\server\meta_search_index).

3. Update the location to a new directory. For example:

   CODE

   set DESTINATION_FOLDER="%SA_PROGRAM_DATA%\server\meta_search_index_july_2014"

   You do not need to create this directory; the index script will automatically create it when it runs. Please note that if you are running the script on Linux then the script does not define the variable %SA_PROGRAM_DATA% so you will need to specify the full path instead.

4. Save your changes to the file.
5. Run BuildSXV4SearchIndex, and wait for it to finish indexing your datasets.

6. Go to SuperADMIN and use the the following command to update the index location to your new location:

   CODE

   > gc search indexDirectory value "meta_search_index_july_2014"

7. Go to SuperWEB2 and check that search now includes all your datasets.