> home > support > FAQ > omptaskfarm

Task Farming OpenMP Programs on HPCx

To recap, the taskfarm program, located in /usr/local/packages/bin/, was written to run multiple copies of a serial program. For an explanation see the short task farming guide.

However, it turns out that the program that is task-farmed doesn't actually have to be a serial program: all that is required is that it is not an MPI program. The three challenges with running multiple OpenMP jobs in this manner are:

ensuring that each OpenMP program runs on a separate SMP node;
ensuring that the correct number of programs is run.

To set the number of threads you just use the OMP_NUM_THREADS environment variable as usual, ie you should set it to 16 (the number of processors in an HPCx SMP node). To ensure that you only run one OpenMP program per node you must set tasks_per_node = 1. With this value set to 1, the standard cpus variable now specifies the number of nodes to use (ie the number of independent programs to be run) rather than the total number of processors to use.

In short, with tasks_per_node = 1, the number of processors you use is 16 times the value of cpus.

As an example, imagine a simple OpenMP "Hello World" program which simply prints "Hello" from each thread. Run in the following way:

#@ cpus=4
#@ tasks_per_node=1

export PATH=$PATH:/usr/local/packages/bin

export OMP_NUM_THREADS=16

taskfarm "./omphello > omphello.%d.log"

will use 64 processors to run four independent jobs each comprising 16 OpenMP threads. You should see four output files called omphello.0.log, omphello.1.log, etc., corresponding to each of the four jobs. Each log file should contain 16 "hello world" lines, one from each thread.

Before running a real OpenMP program in this manner I would recommend running a simple Hello World program as above to check that everything is working correctly.

I hope that this is useful!

David Henty

http://www.hpcx.ac.uk/support/FAQ/omptaskfarm.html

contact email - www@hpcx.ac.uk