Добавил:
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз: Предмет: Файл:
Скачиваний:
23
Добавлен:
28.06.2014
Размер:
3.27 Mб
Скачать

IBM Research

Achieving Strong Scaling On Blue Gene/L: Case Study with NAMD

Sameer Kumar,

Blue Gene Software Group,

IBM T J Watson Research Center, Yorktown Heights, NY sameerk@us.ibm.com

© 2005 IBM Corporation

IBM Research

Outline

Motivation

NAMD and Charm++

BGL Techniques

Problem mapping

Overlap of communication with computation

Grain size

Load-balancing

Communication optimizations

Summary

2

© 2005 IBM Corporation

 

IBM Research

Blue Gene/L

© 2005 IBM Corporation

Blue Gene/L

System

64 Racks, 64x32x32

Rack

32 Node Cards

 

Node Card

180/360 TF/s

 

(32 chips 4x4x2)

32 TB

 

16 compute, 0-2 IO cards

 

 

2.8/5.6 TF/s

Compute

Card

512 GB

 

2 chips, 1x2x1

 

Chip

90/180 GF/s

16 GB

2 processors

5.6/11.2 GF/s

1.0 GB

2.8/5.6 GF/s

4 MB

IBM Research

Application Scaling

Weak

Problem size increases with processors

Strong

Constant problem size

Linear to sub-linear decrease in computation time with processors

Cache performance

Communication overhead

Communication to computation ratio

5

© 2005 IBM Corporation

 

IBM Research

Scaling on Blue Gene/L

Several applications have demonstrated weak scaling

Strong scaling on a large number of benchmarks still needs to be achieved

6

© 2005 IBM Corporation

 

IBM Research

NAMD and Charm++

© 2005 IBM Corporation

IBM Research

NAMD: A Production MD program

NAMD

Fully featured programNIH-funded development

Distributed free of charge

(thousands downloads so far)Binaries and source code

Installed at NSF centersUser training and support

Large published simulations

(e.g., aquaporin simulation featured in keynote)

8

© 2005 IBM Corporation

 

IBM Research Simulation

NAMD, CHARMM27, PME NpT ensemble at 310 or 298 K

1ns equilibration, 4ns production

Protein: ~ 15,000 atoms Lipids (POPE): ~ 40,000 atoms

Water: ~ 51,000 atoms

Total: ~ 106,000 atoms

3.5 days / ns - 128 O2000 CPUs

11 days / ns - 32 Linux CPUs

.35 days/ns–512 LeMieux CPUs

F. Zhu, E.T., K. Schulten, FEBS Lett. 504, 212 (2001)

M. Jensen, E.T., K. Schulten, Structure 9, 1083 (2001)

9

IBM Research

Molecular Dynamics in NAMD

Collection of [charged] atoms, with bonds

Newtonian mechanics

Thousands of atoms (10,000 - 500,000)

At each time-step

Calculate forces on each atom

Bonds:

Non-bonded: electrostatic and van der Waal’s

Short-distance: every timestep

Long-distance: using PME (3D FFT)

Multiple Time Stepping : PME every 4 timesteps

Calculate velocities and advance positions

Challenge: femtosecond time-step, millions needed!

10

© 2005 IBM Corporation