HPCx homepage
Services User support Projects Research About us Sitemap Search  
  line          
Helpdesk User Guide Notices Bulletin Documentation
Training Porting Filestore Batch Interactive FAQ
               
home > support > FAQ > taskfarm
spacer
hr
spacer

Task Farming on HPCx

spacer
I have written a very straightforward harness called taskfarm, currently in /usr/local/packages/bin, that tries to help people do simple task farming of serial codes. It's basically nothing more than a wrapper MPI program that calls system(...) to run the user's code, but hopefully the interface is flexible enough to be used in most cases.

Basically, in your LoadLeveler script you do something like:

taskfarm "echo hello from %d"

(quotes aren't necessary here). Each process executes the command with %d replaced by its own process id, so if you specify 3 CPUs in the your LoadLeveler script and run the above you should get output like:

hello from 0
hello from 1
hello from 2
In practice you want each executable to read different input files and write to different output, so you would do something like:
  taskfarm "cd rundir%d; serialjob < input > output.log"
or
  taskfarm "serialjob< input.%d > output.%d.log"
(here the quotes ARE necessary).

The only option is "-v" for verbose in case you want to debug, eg on 3 processes:

  taskfarm -v "echo hello from %d"

  taskfarm: called with 4 arguments: echo hello from %d
  taskfarm: process 0 executing "echo hello from 0"
  taskfarm: process 1 executing "echo hello from 1"
  taskfarm: process 2 executing "echo hello from 2"
  hello from 0
  hello from 1
  hello from 2
  taskfarm: return code on process 0 is 0
  taskfarm: return code on process 1 is 0
  taskfarm: return code on process 2 is 0
I hope that this is useful!

David Henty

spacer
hr
spacer
http://www.hpcx.ac.uk/support/FAQ/taskfarm.html contact email - www@hpcx.ac.uk © UoE HPCX Ltd