jolira

backgrounder

yet another library for creating background workers

This library is very similar to node-webworker, Q-Oper8 and several others.

Features of this library include:

Download

Github Repo: http://github.com/jolira/backgrounder

$ git clone git://github.com/jolira/backgrounder

Installation

npm install backgrounder

Usage

Master

backgrounder.spawn(filename[, config [, callback]])

Spawn a new background process. The filename must be the fully qualified name of the node.js file used to run the worker, as in:

var backgrounder = require("backgrounder");
var worker = backgrounder.spawn(__dirname + "/simple-worker.js");

The backgrounder automatically set up a two way communciation channel with the worker process using stdin and stdout. Any console messages produced by the child process are automatically forwarded to the parent process and printed there.

If the config parameter is passed, the process sends the data (as long as it is valid JSON) to the worker. The worker must call process.on('config', nofifier) to receive the message. The data will be passed as the first parameter to the notifier. The second parameter of the notifier is a callback function that must be called when the when the configuration process is completed. If a callback parameter was passed to the spawn call, this callback will be invoked when the configuration completes.

var backgrounder = require("backgrounder");
backgrounder.spawn(__dirname + "/worker.js", {
    "primaryDirective": "don't interfere",
    "overdrive": true
  }, function(worker){
    console.log("Master: worker finished configuring!");
    worker.terminate();
});

To worker much register to receive the "config" event and call the callback function passed to the event as the second parameter exactly once:

process.on('config', function(config, callback) {
    console.log('Worker: received configuration ', config);
    callback();
});
Pools of Workers

Passing "children-count" as part of the config parameter creats a work-pool instead of a single worker. The backgrounder automatically chooses an idle worker when sending messages.

var backgrounder = require("backgrounder");
var worker = backgrounder.spawn(__dirname + "/multi-worker.js", {
    "children-count" : 5
});

worker.on(event, callback)

Register to receive messages form the client. Messages include 'message' indicating that the worker process sent an arbitrary user-defined message using the process.send(..) call, 'idle' to indicate that the worker has completed processing all messages in its queue (there may be asynchronous tasks pending tough), and 'completed' for ever message that was processed by the worker.

worker.on("idle", function(message) {
    console.log("Master: worker became idle");
});

worker.send(message[, callback])

Send a user-defined message to the worker The message must be valid JSON. This call creates 'message' event for the worker. The worker must call process.on('message', nofifier) to receive the message. The message will be passed as the first parameter to the notifier. If a callback is provided, the notifier will have a function-object a the second parameter. Invoking this function-object in the worker process will in turn call the callback function in the parent process and pass the same paramter as passed to the function-object in the worker (as long as all parameters are valid JSON).

var backgrounder = require("backgrounder");
var worker = backgrounder.spawn(__dirname + "/simple-worker.js");

worker.send({
    "company": "jolira"
  }, function(arg1, arg2, arg3) {
    console.log("Master: client called the callback with %s arguments:", arguments.length, arg1, arg2, arg3);
    worker.terminate();
});

Worker

No special require statement is necessary on the client as the client is launched by the backgrounder library, which defines all necessary call and symbols before invoking the client code.

It is very important to note that worker code must not directly interact with stdin, stdout, and sterr as the backgrounder uses these facilities to communicate with the parent process. Instead, use the console object to print error and other information. The console has been modifed for the worker to forward all logs and error to the parent process so they can be printed there.

process.on(event, nofifier)

For workers the backgrounder adds new events to the process object. The new events are 'message' (for user-defined messages sent by the parent using worker.send(message[, callback])) and 'config' (for configuration messages sent by the parent using worker.config(message[, callback])). Both these events have a first prameter, which contains the JSON data sent by the parent and an optional second parameter, which is the callback function to invoked to send information back to the parent.

process.on('message', function(message, callback) {
    console.log('Worker: "processing" ', message);

    callback("received", "this", message);
});

process.send(message)

Send a user-defined message back to the parent. The parent must register worker.on("message", callback) to receive the message.

process.send({
    "answer": 42
});

Examples

master.js is a every comprehensive example for all the different features available for creating background workers and communicating with them. The associated worker code can be found at worker.js.

Using the Callback API

A very simple example for using this library can be found at simple-master.js. The code for the matching worker can be found at simple-worker.js.

When running the simple-master.js, the console output should look like this:

jfk@graz:~/workspace/1/backgrounder$ node examples/simple-master.js
Worker: Started!
Worker: "processing"  { company: 'jolira' }
Master: client called the callback with 3 arguments: received this { company: 'jolira' }

Pools of Workers

An example for using a pool of workers can be found at multi-master.js. The code for the matching worker can be found at multi-worker.js.

When running the multi-master.js, the console output should look something like this:

jfk@graz:~/workspace/1/backgrounder$ node examples/multi-master.js
Worker: Started 6478!
Worker: Started 6480!
Worker: Started 6482!
Worker: Started 6479!
Worker: Started 6481!
Master: client called the callback with 3 arguments: 6478 status-code: 200
Master: client called the callback with 3 arguments: 6478 status-code: 200
Master: client called the callback with 3 arguments: 6482 status-code: 200
Master: client called the callback with 3 arguments: 6478 status-code: 200
Master: client called the callback with 3 arguments: 6478 status-code: 200
Master: client called the callback with 3 arguments: 6478 status-code: 200
Master: client called the callback with 3 arguments: 6480 status-code: 200
Master: client called the callback with 3 arguments: 6478 status-code: 200
Master: client called the callback with 3 arguments: 6478 status-code: 200
Master: client called the callback with 3 arguments: 6478 status-code: 200

Your output may vary as the clients are identified by the process id and the OS scheduler assigns work to the processes in a manner that differs frorm run to run.

License

Copyright (c) 2011 jolira

Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.

Authors

Jolira Development <dev@jolira.com>