----------------------------------------------------------------------

H   H  PPP    CCC                           UK  NATIONAL  HPC  SERVICE
H   H  P  P  C   C                          --------------------------
HHHHH  PPP   C      x  x                    provided by
H   H  P     C   C   xx                     EPCC and
H   H  P      CCC   x  x                    CCLRC Daresbury Laboratory

----------------------------------------------------------------------
HPCx User Mailing 071                                      27 May 2005
----------------------------------------------------------------------

Contents

** RDMA: new recommended MPI job script

** "Capability Computing": HPCx newsletter


----------------------------------------------------------------------

Greetings--

RDMA: NEW RECOMMENDED MPI JOB SCRIPT

RDMA (Remote Direct Memory Access) is an upgrade to the microcode of
the HPS switch which connects the frames in the HPCx system.  It was
installed on HPCx as part of the recent SP12 software upgrade.  When
it's enabled, it results in a significant increase to the maximum
bandwidth available via the switch network.

However, RDMA is not enabled by default.  To enable it, you have to
add the following line to the header of your batch script:

  #@ bulkxfer = yes

We recommend this for every MPI job.  Accordingly, the new recommended
settings for MPI jobs are as follows:

  #@ shell = /bin/ksh
  #
  #@ job_name = myrun
  #
  #@ job_type = parallel
  #@ cpus = 256
  #@ node_usage = not_shared
  #
  #@ network.MPI = csss,shared,US
  #@ bulkxfer = yes
  #
  #@ wall_clock_limit = 00:20:00
  #@ account_no = z001
  #
  #@ output = $(job_name).$(schedd_host).$(jobid).out
  #@ error  = $(job_name).$(schedd_host).$(jobid).err
  #@ notification = never
  #
  #@ queue

  # suggested environment settings:
  export MP_EAGER_LIMIT=65536
  export MP_SHARED_MEMORY=yes
  export MEMORY_AFFINITY=MCM
  export MP_TASK_AFFINITY=MCM

  poe ./my_executable

For more information, please see the User Guide:

  http://www.hpcx.ac.uk/support/documentation/UserGuide/HPCxuser/HPCxuser.html

We are preparing a technical report on RDMA on HPCx;  this should be
available shortly.


----------------------------------------------------------------------

"CAPABILITY COMPUTING": HPCx NEWSLETTER

Issue 5 of "Capability Computing" has been published, and has been
sent out by post.  If you haven't received your copy by, say, the end
of next week please let me know.

You can find all the issues of "Capability Computing" on the website:

  http://www.hpcx.ac.uk/about/newsletter/index.html


----------------------------------------------------------------------

Regards

--John

----------------------------------------------------------------------
Earlier mailings: http://www.hpcx.ac.uk/support/notices/index.html
To be removed from the mailing list: log into your website account,
go to the "Update" page, and click the "Opt out of user emails"
field;  then click "Commit update".

--
John Fisher                               j.fisher@epcc.ed.ac.uk
HPCx User Administration and Helpdesk
HPCx:  http://www.hpcx.ac.uk              Helpdesk: support@hpcx.ac.uk
Phone: +44 131 650 5029                   Fax: +44 131 650 6555