Job Expiry and Removal

If you wish to automatically remove jobs after a period of time you need to:

• Configure the job expiry time (this specifies that a job will expire a certain period of time after it has been downloaded at least once by the owning users).
  
• Activate the job removal scheduled task (this runs at your chosen interval and automatically removes any expired jobs from the database).

Job Expiry

To set the job expiry:

1. Go to the Tomcat instance that is running SuperWEB2 and open <tomcat_home>\webapps\webapi\WEB-INF\config\data-access-config-jqm.xml in a text editor.

   

   Make a backup copy of this file before making any changes.

2. Locate the following line:

   XML

   <!-- <property name="globalJobExpiryPeriod" value="7 days" /> -->

3. Remove the comments:

   XML

   <property name="globalJobExpiryPeriod" value="7 days" />

4. Change the value to set the expiry time. The expiry time must be set in days and must include the word day or days. The following are some examples of valid expiry periods: 5 days, 1 day, 0.2 day, 7.3 days
5. Save your changes to data-access-config-jqm.xml.
6. Restart the instance of Tomcat where you have made changes.
7. If you are running multiple SuperWEB2 instances, repeat the above steps to set the expiry time on each one.

Job Removal

Once you have configured an expiry period, you need to activate the scheduled task to automatically remove expired jobs from the database.

1. Go to the Tomcat instance that is running Job Queue Manager and open <tomcat_home>\webapps\queuedjobservice\WEB-INF\config\common\service-config.xml in a text editor.

   

   Make a backup copy of this file before making any changes.

2. Locate the following section:

   XML

   <!-- Un-comment the lines below to enable job expiry.
       <bean name="expiredJobRemover" class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
           <property name="jobClass" value="au.com.str.jobqueue.server.scheduling.ExpiredJobRemoval" />
           <property name="jobDataAsMap">
               <map>
                   <entry key="entityManagerFactory" value-ref="entityManagerFactory" />
               </map>
           </property>
       </bean>
       <bean id="scheduleTrigger" class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
           <property name="jobDetail" ref="expiredJobRemover"/>
           <!- run every minute: "0 0/1 * * * ?" ->
           <!- run every morning at 6 AM ->
           <property name="cronExpression" value="0 0 6 * * ?"/>
       </bean>
       <bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
           <property name="triggers">
               <list>
                   <ref bean="scheduleTrigger" />
               </list>
           </property>
       </bean>
    -->

3. Remove the comments from the start and end of this section:

   XML

       <bean name="expiredJobRemover" class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
           <property name="jobClass" value="au.com.str.jobqueue.server.scheduling.ExpiredJobRemoval" />
           <property name="jobDataAsMap">
               <map>
                   <entry key="entityManagerFactory" value-ref="entityManagerFactory" />
               </map>
           </property>
       </bean>
       <bean id="scheduleTrigger" class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
           <property name="jobDetail" ref="expiredJobRemover"/>
           <!- run every minute: "0 0/1 * * * ?" ->
           <!- run every morning at 6 AM ->
           <property name="cronExpression" value="0 0 6 * * ?"/>
       </bean>
       <bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
           <property name="triggers">
               <list>
                   <ref bean="scheduleTrigger" />
               </list>
           </property>
       </bean>
   

4. Edit the following line to specify when you want the scheduled task to run a cleanup of expired jobs.

   XML

           <property name="cronExpression" value="0 0 6 * * ?"/>

   You need to specify the scheduled time in the form of a cron expression. The values represent seconds, minutes, hours, day of month, month, day of week.

5. Remove the following lines (these are just examples; they need to be removed because once you have removed the comments that enclose the whole section, these lines will not be properly commented out):

   XML

           <!- run every minute: "0 0/1 * * * ?" ->
           <!- run every morning at 6 AM ->

   As an alternative to deleting these lines, if you want to leave them as comments for future reference, you can do this by adding an additional - symbol at the start and end of each line, to make them into valid comments. For example:

   XML

           <!-- run every minute: "0 0/1 * * * ?" -->
           <!-- run every morning at 6 AM --> 

6. Save your changes to service-config.xml.

7. Restart the instance of Tomcat where you have made changes.

By default, jobs remain in the database indefinitely and will continue to be available to download from the Manage Tables page.

Job Queue Manager offers two methods for periodically cleaning up the database to remove old jobs:

• Expired job removal: jobs will automatically be removed a (configurable) amount of time after they have been downloaded. 
• Dead job removal: jobs will be removed if they have been in the same state for longer than a (configurable) period of time. You can use this feature to clean up jobs in any state (whereas the expired job removal feature only applies to jobs that have been downloaded). 

Configure Expired Job Removal

To configure automatic removal of expired jobs, you first need to configure a global expiry time, and then activate the expiredJobRemover scheduled task.

Step 1 - Configure Job Expiry Time

This step sets the period of time after a job has been downloaded when it will be considered expired. The expiry time is shown to end users in the Expiry Time column on the Manage Tables page:

1. Go to the Tomcat instance that is running SuperWEB2 and open <tomcat_home>\webapps\webapi\WEB-INF\config\data-access-config-jqm.xml in a text editor.

   

   Make a backup copy of this file before making any changes.

2. Locate the following line:

   XML

   <!-- <property name="globalJobExpiryPeriod" value="7 days" /> -->

3. Remove the comments:

   XML

   <property name="globalJobExpiryPeriod" value="7 days" />

4. Change the value to set the expiry time. The expiry time must be set in days and must include the word day or days. The following are some examples of valid expiry periods: 5 days, 1 day, 0.2 day, 7.3 days.

   

   Jobs are not removed exactly on their expiry time: once a job has expired, it will be removed the next time the expiredJobRemover scheduled task runs. For example, if you set the expiry time to 7 days and configure the expiredJobRemover task to run once a day, then a job will be removed at some point between 7 and 8 days after it was downloaded. You will configure how frequently the scheduled task runs in the next step.

5. Save your changes to data-access-config-jqm.xml.
6. Restart the instance of Tomcat where you have made changes.
7. If you are running multiple SuperWEB2 instances, repeat the above steps to set the expiry time on each one.

Step 2 - Configure the Expired Job Removal Scheduled Task

Once you have configured an expiry period, you need to configure and activate the scheduled task to automatically remove expired jobs from the database.

1. Go to the Tomcat instance that is running Job Queue Manager and open <tomcat_home>\webapps\queuedjobservice\WEB-INF\config\common\service-config.xml in a text editor.

   

   Make a backup copy of this file before making any changes.

2. Locate the following section:

   XML

       <!-- Scheduled task to remove expired jobs in the COLLECTED state. The expiry time is saved in the database when
            queued. -->
       <!-- Example task to run every hour.
       <bean name="expiredJobRemover" class="au.com.str.jobqueue.server.scheduling.ExpiredJobRemoval.Factory">
           <property name="runFrequency" value="1h" />
           <property name="entityManagerFactory" ref="entityManagerFactory" />
           <property name="scheduler" ref="jobQueueManager" />
       </bean>
       -->

3. Remove the comments from around the <bean> ... </bean> element:

   XML

       <!-- Scheduled task to remove expired jobs in the COLLECTED state. The expiry time is saved in the database when
            queued. -->
   
       <bean name="expiredJobRemover" class="au.com.str.jobqueue.server.scheduling.ExpiredJobRemoval.Factory">
           <property name="runFrequency" value="1h" />
           <property name="entityManagerFactory" ref="entityManagerFactory" />
           <property name="scheduler" ref="jobQueueManager" />
       </bean>
   
   

4. Edit the following line to specify how frequently you want the job removal task to run. Specify the frequency as a number followed by the first letter of days, hours, minutes or seconds. For example 1h sets it to run every hour, and 1d sets it to run every day.

   XML

           <property name="runFrequency" value="1h" />

5. Save your changes to service-config.xml.

6. Restart the instance of Tomcat where you have made changes.

Configure Dead Job Removal

This feature allows you to configure Job Queue Manager to automatically remove jobs that have been in the same status for longer than a specific period of time.

1. Go to the Tomcat instance that is running Job Queue Manager and open <tomcat_home>\webapps\queuedjobservice\WEB-INF\config\common\service-config.xml in a text editor.

   

   Make a backup copy of this file before making any changes.

2. Locate the following section:

   XML

       <!-- Example task to delete any job that has been in the same status for over 30 days, running every hour.
       <bean name="deadJobRemover" class="au.com.str.jobqueue.server.scheduling.DeadJobRemoval.Factory">
           <property name="time" value="30d" />
           <property name="runFrequency" value="1h" />
           <property name="entityManagerFactory" ref="entityManagerFactory" />
           <property name="scheduler" ref="jobQueueManager" />
       </bean>
       -->

3. Remove the comments around the <bean> ... </bean> element:

   XML

       <bean name="deadJobRemover" class="au.com.str.jobqueue.server.scheduling.DeadJobRemoval.Factory">
           <property name="time" value="30d" />
           <property name="runFrequency" value="1h" />
           <property name="entityManagerFactory" ref="entityManagerFactory" />
           <property name="scheduler" ref="jobQueueManager" />
       </bean>
   

4. Configure the values for time and runFrequency (specify the values as a number followed by the first letter of days, hours, minutes or seconds; for example 30d means every 30 days, and 1d means every day):
   • time is the age of the job you want to clean up (for example in the above code block, 30d means any jobs that have been in the same status for 30 days).
   • runFrequency is how frequently you want to scheduled task to run (for example in the above code block, 1h means that the task will run every hour).  

5. The above example will apply to all jobs in any status. If you only want to clean up jobs with specific statuses, then add a statuses property with a comma separated list of status values that you want the rule to apply to. Possible status values are:

   • SUBMITTED
   • SCHEDULED
   • RUNNING
   • COMPLETE
   • COLLECTED
   • RECOVERABLE
   • ERROR

   For example, the following configuration will clean up jobs that have been in either the COLLECTED or ERROR statuses for 7 days, with the task running every hour:

   XML

       <bean name="deadJobRemover" class="au.com.str.jobqueue.server.scheduling.DeadJobRemoval.Factory">
           <property name="statuses" value="COLLECTED, ERROR" />
           <property name="time" value="7d" />
           <property name="runFrequency" value="1h" />
           <property name="entityManagerFactory" ref="entityManagerFactory" />
           <property name="scheduler" ref="jobQueueManager" />
       </bean>
   

6. (Optionally): if you want to configure different rules for different statuses, you can include multiple instances of the dead job remover scheduled task: simply copy the <bean> ... </bean> element and give each instance a unique name. For example, the following configuration ensures that jobs that have been collected will be removed after 7 days, with all other jobs being removed after 30 days (both scheduled tasks run every hour):

   XML

       <bean name="deadJobRemover" class="au.com.str.jobqueue.server.scheduling.DeadJobRemoval.Factory">
           <property name="time" value="30d" />
           <property name="runFrequency" value="1h" />
           <property name="entityManagerFactory" ref="entityManagerFactory" />
           <property name="scheduler" ref="jobQueueManager" />
       </bean>
   	
       <bean name="collectedJobRemover" class="au.com.str.jobqueue.server.scheduling.DeadJobRemoval.Factory">
           <property name="statuses" value="COLLECTED" />
           <property name="time" value="7d" />
           <property name="runFrequency" value="1h" />
           <property name="entityManagerFactory" ref="entityManagerFactory" />
           <property name="scheduler" ref="jobQueueManager" />
       </bean>

7. Save your changes to service-config.xml.

8. Restart the instance of Tomcat where you have made changes.