Technical Specifications of NIWA’s HPCF

The specifications for our current machine, and those of the system after it is upgraded in 2011/2012.

Phase 1 (2010)

IBM p575/p6 supercomputer with 56 POWER6, 32 way 4.7 GHz nodes for a total of 1792 processors and 5.4 terabytes of memory;
Capable of performing up to 34 trillion (million million) calculations per second
The bandwidth within each 32 way node is as high as 80 GB/s. The nodes can communicate with each other at 16 GB/s per connection in each direction with a latency of around 4μs (MPI) for short messages.
0.5 petabytes of useable disk storage on 740 disks in an IBM DCS9900 storage array;
Two IBM TS3500 Automatic Tape Libraries, with 12 LTO-5 drives and each capable of storing 2.5 Petabytes of data. One tape library will be for disaster recovery, and will contain copies of all data on the primary tape library;
Eight p520/p6 servers for HPC Management, GPFS (General Parallel File System), and TSM (Tivoli Storage Manager) functions; and
One BladeCenter with 96 Xeon 2.53 GHz processors and 384 Gigabytes of memory for pre- and post- processing tasks.

AIX operating system on all p520 servers and p575 nodes;
General Parallel File System (GPFS) – accessible from all HPCF processors – both POWER 6 and Bladecenter X-Series;
Tivoli Storage Manager (TSM) to provide backup and transparent movement of data between storage media (Hierarchical Storage Management);
Tivoli Workload Scheduler software – to manage and schedule work on the HPCF;
XL Fortran, C and C++ compilers, and
IBM High Performance Computing Toolkit.

370 kW to run the system;
Another 120 kW for cooling, using chilled water (which removes around 70% of the heat) and air (which removes the rest of the heat);
Weight: 12.6 tonnes, including water.

The new computer hall has been built to support a planned upgrade to

An additional 52 POWER6, 32 way 4.7 GHz nodes, for a total of 3456 processors and 9.0 terabytes of memory;
Be capable of performing 65 TeraFLOPS;
An additional 1.4 Petabytes of disk storage for a total 2 Petabytes (i.e. two million gigabytes)

The existing computer hall has already been designed to