---------------------------------------------------------------------- H H PPP CCC UK NATIONAL HPC SERVICE H H P P C C -------------------------- HHHHH PPP C x x provided by H H P C C xx EPCC and H H P CCC x x CCLRC Daresbury Laboratory ---------------------------------------------------------------------- HPCx User Mailing 050 29 July 2004 ---------------------------------------------------------------------- Contents ** Switch microcode upgrade and memory changes ** IBM Project Workshop on planning HPC software development ** Tape archive ---------------------------------------------------------------------- Greetings-- SWITCH MICROCODE UPGRADE AND MEMORY CHANGES As planned (see mailing 049) the microcode of the HPS switch was upgraded during the maintenance session yesterday (28 July). There were also some associated changes in the rest of the software. Joachim Hein has been running some benchmarking jobs, and already he has found: * Improved zero message latency (Pallas). Latency went down from 10.25 usec to 5.85 usec * Best observed bandwidth between frames has increased from 4.4 GB/s to 8.02 GB/s * Improved handling of simulataneous messages originating from different tasks on the same frame. * For messages between 8kB and 64kB, bandwidth improved by a factors between 2.1 and 4.4. These are dramatic improvements in the performance of the switch. Joachim's benchmarks were run as ordinary jobs, while the service was open to other users. We are very pleased with these results. There is a downside, however: the new microcode requires us to reserve more memory on each frame for the switch adapter drivers. As a result we have had to reduce the Resident Set Size (RSS), which is the total amount of memory available to a task: it is now: 26.96 Gb / tasks_per_node You can find a discussion of memory management here: http://www.hpcx.ac.uk/support/FAQ/stack.txt A few users may find that jobs which ran before will now fail. You may need to reduce tasks_per_node (which is 32 by default) so as to increase the size of your RSS. I appreciate that this will be a disadvantage for those users, but in our judgment the improvements in the performance of the switch make it worthwhile. If anyone needs help with this, please contact the helpdesk at support@hpcx.ac.uk. ---------------------------------------------------------------------- IBM PROJECT WORKSHOP ON PLANNING HPC SOFTWARE DEVELOPMENT [The following message is from IBM] IBM will hold a one day HPCx Project Workshop to investigate, discuss and understand users' requirements for developments to the IBM software which runs on HPCx. The Workshop will take place in Conference Room 5, IBM Manchester, Imperial Court, Building No. 2, Exchange Quay, Manchester, M5 3EB on Tuesday August 31st, starting at 10am. IBM Manchester is adjacent to the Exchange Quay stop on the Manchester Metrolink. The sorts of things you might want to raise might include: * Functional enhancements to the implementation of MPI provided by IBM's Parallel Environment software. * Additions to the customised and optimised subroutine libraries provided by ESSL and Parallel ESSL. but does not need to be restricted to these areas. In addition, we would like to be able to quantify the benefits any proposed changes would bring to you, the users of the system. This is the first step in the process in which IBM UK will facilitate a Project Workshop: * to investigate and collect a list of user requirements in much greater detail * to arrange their prioritisation * to match them against the existing known IBM Development road map * to identify those currently not in the IBM Development plans * to negotiate with IBM Development their possible inclusion into the plans taking into account the costs, effort and timescales involved. * to document the results of the Workshop and the subsequent agreed actions The return for your attendance will be to ensure that any requirements you have on IBM software on HPCx are formally documented and fed to IBM Development for inclusion in IBM's development plans. The meeting will be run by Jonathan Follows. Also present from IBM will be: * Luigi Brochard (IBM Distinguished Engineer and HPC Architect who designed the HPCx system) * Angelo Rossi (IBM Advanced Computing Technology Center, http://www.research.ibm.com/actc/People.html, who recently visited Edinburgh to work with HPCx support staff). Members of HPCx staff will also attend. For further information, and to register, please contact: Jonathan Follows Deep Computing, Certified I/T Specialist Jonathan_Follows@uk.ibm.com ---------------------------------------------------------------------- TAPE ARCHIVE If you delete a file from a tape on the tape archive, the space it occuped can be reused. However, for this to happen, the remaining files must be moved, to concatenate the free areas. This can only be done if there are at least two tapes within the group - in our terms, allocated to the same project group. For this reason, we are planning to change the administrative website so that it will not allow a PI or manager to allocate less than two tapes to a project group. You may find it useful in advance of this change to allocate an extra tape to any project group which has only one. If this means that a project needs more tapes, please send in a query to support@hpcx.ac.uk, and they will be provided. ---------------------------------------------------------------------- Regards --John ---------------------------------------------------------------------- Earlier mailings: http://www.hpcx.ac.uk/support/notices/index.html To be removed from the mailing list: log into your website account, go to the "Update" page, and click the "Opt out of user emails" field; then click "Commit update". -- John Fisher j.fisher@epcc.ed.ac.uk HPCx User Administration and Helpdesk HPCx: http://www.hpcx.ac.uk Helpdesk: support@hpcx.ac.uk Phone: +44 131 650 5029 Fax: +44 131 650 6555