Batch Processor

Headless Client Plugin Guide / Batch Processor

Overview

The Batch Processor is a headless client that starts automatically when the server starts and runs without any direct user interaction. It is typically used for backend automation and maintenance tasks. Since this client is initiated as part of the server's lifecycle, it is ideal for tasks that need to execute consistently on startup or at regular intervals.

The Batch Processor can be configured with specific credentials or startup arguments, depending on your needs, and it can either run a one-time task (like server maintenance) or be scheduled to run recurrently using the Scheduler Plugin.

You can find more detailed information about Batch Processors here.

Features

• No User Interaction*: The Batch Processor runs automatically with no need for a logged-in user interface.

• Login Credentials: You can configure a user credential for authentication, managed through the Servoy Admin Page.

• Startup Arguments: Startup arguments can also be set, making it easier to customize the batch session's behavior during initialization.

• Scheduling: You can bind the processor to a startup method for execution upon server start, also schedule recurring tasks via the Scheduler Plugin.

Examples

Bind to a startup method

A simple method that runs when the server starts. It can be configured to perform any server-side tasks, like cleaning up logs or running diagnostics In this example, the Batch Processor will clean up old logs and session data in the performMaintenance() function.

/**
 * Callback method for when solution is opened.
 *
 * @param {String} arg startup argument part of the deeplink url with which the Client was started
 * @param {Object<Array<String>|String>} queryParams all query parameters of the deeplink url with which the Client was started, key>string if there was one value else key>Array<String>
 *
 * @properties={typeid:24,uuid:"E4E0B0D5-B12B-40B1-8E4B-CC5670C9C5D2"}
 */
function onSolutionOpen(arg, queryParams) {
    // Example: Initializing system-wide settings
    application.output('Batch Processor: Server has started. Initializing system...');
    
    // Perform any server-side maintenance or setup
    performMaintenance();
}

function performMaintenance() {
    // Maintenance logic
    // You could perform database cleanups, data synchronization, etc.
        application.output('Batch Processor: Starting maintenance tasks...');

    // 1. Clean up old server logs (keeping only the last 3 months)
    var cutoffDate = new Date();
    cutoffDate.setMonth(cutoffDate.getMonth() - 3);  // Logs older than 3 months

    var logQuery = databaseManager.createSelect('db:/example_db/log_table');
    logQuery.where.add(logQuery.columns.log_date.lt(cutoffDate));
    
    // Delete old log entries
    var deletedLogs = databaseManager.getFoundSet(logQuery).deleteAllRecords();
    application.output('Batch Processor: ' + deletedLogs + ' old log records deleted.');

    // 2. Remove outdated session data (keeping only the last 7 days)
    var sessionCutoffDate = new Date();
    sessionCutoffDate.setDate(sessionCutoffDate.getDate() - 7);  // Sessions older than 7 days

    var sessionQuery = databaseManager.createSelect('db:/example_db/session_table');
    sessionQuery.where.add(sessionQuery.columns.session_date.lt(sessionCutoffDate));

    // Delete old session entries
    var deletedSessions = databaseManager.getFoundSet(sessionQuery).deleteAllRecords();
    application.output('Batch Processor: ' + deletedSessions + ' old session records deleted.');

    application.output('Batch Processor: Maintenance tasks completed.');
}

Run once and shutdown

Sometimes, the processor only needs to perform a task once, such as server initialization or cleanup, and then stop.

/**
 * Callback method for when solution is opened.
 *
 * @param {String} arg startup argument part of the deeplink url with which the Client was started
 * @param {Object<Array<String>|String>} queryParams all query parameters of the deeplink url with which the Client was started, key>string if there was one value else key>Array<String>
 *
 * @properties={typeid:24,uuid:"E4E0B0D5-B12B-40B1-8E4B-CC5670C9C5D2"}
 */
function onSolutionOpen(arg, queryParams) {
    runOneTimeTask()
}

// Run a one-time task on server startup and then shut down
function runOneTimeTask() {
    application.output('Batch Processor: Running one-time task...');
    
    // Example: Processing data or initializing resources
    processData();

    // Shut down the batch processor after the task is done
    application.output('Batch Processor: Task completed. Shutting down.');
    application.exit();  // This will shut down the batch processor
}

// Example method for processing data
function processData() {
    application.output('Batch Processor: Starting data processing task...');
    // Insert data processing logic here 
    // 1. Use QB select to retrieve pending orders
    var orderQuery = databaseManager.createSelect('db:/example_db/orders');
    orderQuery.where.add(orderQuery.columns.status.eq('PENDING'));

    var orderFoundSet = databaseManager.getFoundSet(orderQuery);

    // 2. Loop through the foundset and process each order
    for (var i = 1; i <= orderFoundSet.getSize(); i++) {
        var record = orderFoundSet.getRecord(i);
        
        // Example processing logic: Update the status of the order
        if (record.status === 'PENDING') {
            record.status = 'PROCESSED';  // Mark order as processed
            if (databaseManager.saveData(record)) {
                application.output('Batch Processor: Order ID ' + record.id + ' marked as PROCESSED.');
            } else {
                application.output('Batch Processor: Failed to update Order ID ' + record.id + '.');
            }
        }
    }

    application.output('Batch Processor: Data processing task completed.');
}

In this scenario:

• The Batch Processor runs the runOneTimeTask() method.

• Once the task completes, the application.exit() method is called, which shuts down the batch processor.

This is ideal for one-time tasks that don't need to keep running after startup, such as initializing services or processing data at the server boot.

Schedule a recurring task

To run a task at recurring intervals, the Scheduler Plugin can be used. Here's how to schedule a task that runs every 10 minutes:

/**
 * Callback method for when solution is opened.
 *
 * @param {String} arg startup argument part of the deeplink url with which the Client was started
 * @param {Object<Array<String>|String>} queryParams all query parameters of the deeplink url with which the Client was started, key>string if there was one value else key>Array<String>
 *
 * @properties={typeid:24,uuid:"E4E0B0D5-B12B-40B1-8E4B-CC5670C9C5D2"}
 */
function onSolutionOpen(arg, queryParams) {
// Scheduling a recurring task using the Scheduler Plugin
    var jobName = 'recurringTaskJob';
    plugins.scheduler.addCronJob(jobName, '0 0/10 * * * ?', recurringTask); // Schedule a task to run every 10 minutes
}

// Define the recurring task
function recurringTask() {
    application.output('Running recurring task...');
    
    // Insert the task logic here (e.g., data sync, report generation)
    syncData();
}

// Example data synchronization task
function syncData() {
    // Logic to sync data goes here
    application.output('Batch Processor: Starting data synchronization task...');

    // 1. Use QB select to retrieve unsynced sales data
    var salesQuery = databaseManager.createSelect('db:/example_db/sales');
    salesQuery.where.add(salesQuery.columns.synced.eq(0));  // 0 for unsynced records

    var salesFoundSet = databaseManager.getFoundSet(salesQuery);

    // 2. Loop through the sales records and send data to the external system
    for (var i = 1; i <= salesFoundSet.getSize(); i++) {
        var saleRecord = salesFoundSet.getRecord(i);
        
        // Send data to the external system (assuming an API endpoint)
        var response = scopes.globals.sendDataToExternalSys('https://external-system.com/api/sales', {
            id: saleRecord.id,
            amount: saleRecord.sale_amount
        });

        // 3. If successful, update the record as synced
        if (response && response.statusCode === 200) {
            saleRecord.synced = 1;  // Mark the record as synced
            if (databaseManager.saveData(saleRecord)) {
                application.output('Batch Processor: Sale ID ' + saleRecord.id + ' successfully synced.');
            } else {
                application.output('Batch Processor: Failed to update Sale ID ' + saleRecord.id + ' after sync.');
            }
        } else {
            application.output('Batch Processor: Failed to sync Sale ID ' + saleRecord.id + '. Response: ' + response.statusText);
        }
    }

    application.output('Batch Processor: Data synchronization task completed.');
}

Key Points:

• Scheduler Plugin is used in onSolutionOpen to add a CRON job that runs every 10 minutes.

• The CRON expression '0 0/10 * * * ?' schedules the task to execute at the start of every 10-minute period.

• The recurringTask() method contains the logic that will be executed every time the job runs.

This setup is useful for tasks like recurring data synchronization, sending automated emails, or running periodic checks.