slideshow 3


V100 GPU nodes available in Minerva
5 new GPU nodes with a total of 20 V100 GPU cards are available in Minera. Currently, We have Centos7.6/Centos7.5 installed on all the GPU nodes, with nvidia driver version 410.79 which supports CUDA 10.0.

To access GPU node through LSF, please use following the command :
$ bsub -n 1 -P acc_xxx -q gpu -R "v100" R "rusage[ngpus_excl_p=1]" -W 0:05 echo "Hello World"
$ bsub -n 1 -P acc_xxx -q gpu -R "p100" R "rusage[ngpus_excl_p=1]" -W 0:05 echo "Hello World"

Chimera status update
Lenovo completed reliability test and HPL performance test. Chimera is able to achieve a peak performance of 833.3TF with 69.8% efficiency. HPC team is building OS image and setting up environment. Currently, 5 GPU nodes are made available in the Minerva partition.

Minerva user survey results
The results for user survey is available at
The response to the comments is available at

Slides for Minerva Town Hall meeting
The slides for Minerva Town Hall meeting are available on our website: (MUG Meeting Nov 29, 2018).

Highlights from the meeting:

  • Minerva usage status and our accomplishments in 2018
  • Main upgrade plans for 2019:
    • New compute partition Chimera: specs and installation plan
    • Storage upgrade: GPFS upgrade to 5.x and new filesystem /sc/hydra
    • OS upgrade, package rebuild and new module system
    • Queue structure changes
    • Container support

New compute partition: Chimera
We are happy to announce that we purchased a new compute partition Chimera. It will be available to users in early 2019. Specs of the new compute partition:

4x login nodes - Intel 8168 24C, 2.7GHz - 384 GB memory
280+ compute nodes - Intel 8168 24C, 2.7GHz - 192 GB memory
4x high memory nodes - Intel 8168 24C, 2.7GHz - 1.5 TB memory
48 V100 GPUs in 12 nodes - Intel 6142 16C, 2.6GHz - 384 GB memory - 4x V100-16 GB GPU
10x gateway nodes

New NFS storage (for users home directories) - 192 TB raw / 160 TB usable RAID6

Mellanox EDR Infiniband fat tree fabric (100Gb/s)

Total system memory (computes + GPU + high mem) = 65.7 TB
Total number of cores (computes + GPU + high mem) = 14,304 cores
Peak performance (computes + GPU + high mem, CPU only) = 1.2 PFlops/s

We will send out necessary announcement during the integration and make this transition seamless for all users.

We do not backup user data. It is the users' responsibility to ensure data is backed up in accordance with their funding agency or private organization requirements. Refer to "Preserving Data: Who’s Got Your Back(up)?" for helpful hints in developing your individual backup strategy.

All Announcements/Reminders.

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer