User Resource Management

What resources do I have - time?

Users can always get a brief summary of their projects and the amount of resource remaining in those projects with the budgets command:

bash-2.05a$ budgets
z001:     999999 AU      416666:15:0

What resources do I have - storage?

Diskspace

Disk space on the HPCx service is available in several varieties: homespace, workspace and temporary scratch space.

• Homespace is intended for permanent files: source and binary codes, permanent small datasets, etc. This is the area that is backed up.

• Workspace is not backed up. We expect this to be used for input and output files for current work. Workspace is considerably larger than homespace.

  Workspace for each user is located at:

  /hpcx/work/project_code/group_code/username

  where project_code is the code for your project (eg, x01); group_code is the code for your project group, if your project has groups, or the same as the project code, if not (eg, x01-a) username is your login name.

  For example:

  /hpcx/work/z001/z001/johnf

• Temporary scratch space is available for the duration of your job. To access this, you have to set the variable

  	JTMPDIR=`lljtmp`
  

  in your LoadLeveler script. The variable JTMPDIR contains the name of the temporary directory and can be used with the standard UNIX commands, e.g.

  	cd $JTMPDIR
  

  The temporary scratch space (4.5TB at the time of writing) is shared between all jobs running on the machine on a first come first served basis. Unfortunately no guarantees -- another job(s) might have taken it all.

  Important note: The scratch space is erased once your job finishes. You have to copy the data from inside your LoadLeveler script into your home or work space.

User and group quotas can be interrogated with the mmlsquota command:

bash-2.05a$ mmlsquota -g z001
Block Limits | File Limits
Filesystem type KB quota limit in_doubt grace | files quota limit in_doubt grace

Disk quotas for group z001 (gid 900):
hsm GRP 3900800 0 10485760 146624 none | 14910 0 0 115 none
work GRP no limits
tmpchkpt GRP no limits
home GRP no limits

Note, this command is in /usr/lpp/mmfs/bin/ which should be added to the path before use.

Tapestorage

The tape storage comprises a 4 frame 3584 tape library (1363 tape capacity) with 48 LTO tape drives. There is a separate user guide for the tape archiver http://www.hpcx.ac.uk/research/hpc/technical_reports/HPCxTR0405.pdf

The HPCx Administration Web Site

All users have a login and password on the HPCx Administration Web Site (aka the SAF page):

https://www.hpcx.ac.uk/login.jsp

Once logged into this web page, users can find out much about their usage of the HPCx system, including:

• machine status

• project details

• detailed usage information including report generation

These features are largely self-documenting and are not described further here.

See here and here for more details of the web-site.

Charging

The standard charging policy on HPCx is to multiply the elapsed wall-clock time for each job by the number of nodes used. Since parallel production jobs are granted exclusive access to their nodes, each node is charged with the cost of all its 16 CPUs, even if your job uses only a fraction of it. For example, a job lasting 1 hour on 50 CPUs from four nodes will be charged as 64 CPU hours. For most jobs it will be most economic to ask for multiples of 16 CPUs, that is 16, 32, 48, 64, ... CPUs.

However, there are two exceptions to this rule: serial and interactive jobs. In both these cases users are only charged for the CPU time actually used.

For serial jobs we would expect this to be almost identical to the wall-clock time as we don't over subscribe CPUs (ie: there are never more serial processes than the number of CPUs dedicated to serial jobs). For interactive jobs we allow only one process to run on each CPU, so users will be charged for the number of processes multiplied by the elapsed wall-clock time.