I. Building Ubuntu Linux Libraries

To ensure similar environment, some common software libraries will be installed on each computing node.

$ : Ubuntu shell response

Scripts for Preparing Dependences (on Ubuntu 16.04
1. Prepare dependences
 $ sudo apt-get update && sudo apt-get dist-upgrade
 $ sudo apt-get install -y build-essential
 $ sudo apt-get install autotools-dev autoconf automake cmake cmake-curses-gui
 $ sudo apt-get install gcc-multilib gfortran
 $ sudo apt-get install rpm openjdk-7-jre-headless csh menu
 $ sudo apt-get install x11proto-print-dev motif-clients libx11-dev
 $ sudo apt-get install freeglut3-dev libxmu-dev libxi-dev libgl1-mesa-glx libgl1-mesa-dri libglu1-mesa libglu1-mesa-dev
 $ sudo apt-get install libboost-dev libxml2-dev
 $ sudo apt-get install libcurl-ocaml-dev libcurl4-gnutls-dev gnuplot grace
 $ sudo apt-get install doxygen swig

2. Remove any obsolete libraries
 $ sudo apt-get autoremove && sudo apt-get autoclean

This will produce a common environment to support the installment of simulation packages on each computing node.

II. Building OpenMPI and nVidia GPU Driver

Currently the state-of-the art CPU has an architecture of several cores. Furthermore, most personal computers for multimedia applications possess GPU cards, which are also suitable for parallel computing. So why not use it effectively to establish a hybrid cpu-gpu parallel computing facility.

OpenMPI is one of the leading MPI-2 implementations. The OpenMPI project has the stated aim of building the best Message Passing Interface (MPI) library available.

(A) Prepare dependences and configure gnu compilers

1. Install and upgrade the necessary files
 $ sudo add-apt-repository ppa:ubuntu-toolchain-r/test
 $ sudo apt-get update && sudo apt-get dist-upgrade
 $ sudo apt-get install --upgrade build-essential autotools-dev autoconf automake

2. Install gcc and gfortran compilers
 $ sudo apt-get install g++-5 gfortran-5
 $ sudo apt-get install g++-6 gfortran-6
 $ dpkg -l | grep gcc | awk '{print $2}'
 --> gcc, gcc-5, gcc-5-base:amd64; gcc-6, gcc-6-base:amd64; gir1.2-packagekitglib-1.0, libcaca0:amd64, libgcc-5-dev:amd64, libgcc-6-dev:amd64, libgcc1:amd64, libpackagekit-glib2-16:amd64, libunity-action-qt1:amd64, libwebrtc-audio-processing-0:amd64, qtchooser qtdeclarative5-unity-action-plugin:amd64, qtdeclarative5-unity-action-plugin:amd64

 $ dpkg -l | grep g++ | awk '{print $2}'
 --> g++, g++-5, g++-6

 $ dpkg -l | grep gfortran | awk '{print $2}'
 --> gfortran, gfortran-5, gfortran-6, libgfortran-5-dev:amd64, libgfortran-6-dev:amd64, libgfortran3:amd64

3. Configure gcc and gfortran compilers
 $ sudo update-alternatives --remove-all gcc
 $ sudo update-alternatives --remove-all g++
 $ sudo update-alternatives --remove-all cc
 $ sudo update-alternatives --remove-all c++
 $ sudo update-alternatives --remove-all gfortran

 $ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 10
 $ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-6 20

 $ sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-5 30
 $ sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-6 40

 $ sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc 50
 $ sudo update-alternatives --set cc /usr/bin/gcc

 $ sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++ 60
 $ sudo update-alternatives --set c++ /usr/bin/g++

 $ sudo update-alternatives --config gcc
 $ sudo update-alternatives --config g++

 $ sudo update-alternatives --install /usr/bin/gfortran gfortran /usr/bin/gfortran-5 70
 $ sudo update-alternatives --install /usr/bin/gfortran gfortran /usr/bin/gfortran-6 80

 $ sudo update-alternatives --config gfortran

 $ sudo apt-get install csh menu x11proto-print-dev motif-clients
 $ sudo apt-get install freeglut3-dev libx11-dev libxmu-dev libxi-dev libgl1-mesa-glx libglu1-mesa libglu1-mesa-dev libgl1-mesa-dri libcurl-ocaml-dev libcurl4-gnutls-dev gnuplot grace
 $ sudo apt-get autoremove

(B) Install OpenMPI

1. Grab the necessary file from the web
 $ wget http://www.open-mpi.org/software/ompi/v1.10/downloads/openmpi-1.10.2.tar.gz
 $ tar xzvf openmpi-1.10.2.tar.gz
 $ cd openmpi-1.10.2

2. Building OpenMPI-1.10.2 :with Cuda-aware support:
 $ ./configure --prefix="/usr/local/openmpi" --enable-orterun-prefix-by-default CC=gcc CXX=g++ F90=gfortran FC=gfortran --enable-static --with-cuda

 Build using maximum number of physical cores
 $ n=`cat /proc/cpuinfo | grep "cpu cores" | uniq | awk '{print $NF}'`
 $ make -j $n

 $ sudo make install
 $ make distclean

3. Setup path in Environment Variable in /etc/environment:
 export PATH="/usr/local/openmpi/bin:$PATH"
 export LD_LIBRARY_PATH="/usr/local/openmpi/lib:$LD_LIBRARY_PATH"
 Quit and save.

4. Test OpenMPI
 (a) Prepare a c-program like this , which is a parallel version of c-program for calculating the summation of inverse factorials.

 (b) Compile the program with:
   $ mpicc testOMPI.c -o testOMPI

 (c) Test the code on the dell-m4800 node:
   $ mpirun -H dell-m4800 -np 4 testOMPI 10000000 -->
   Result: 2.7182818285
   Time elapsed: 16.366959ms

 (d) Test the code on the remote dell-t7500 node:
   $ mpirun -H dell-t7500 -np 4 testOMPI 10000000 -->
   Result: 2.7182818285
   Time elapsed: 22.429943ms

 (e) Test the code on the two nodes:
   $ mpirun --mca btl_tcp_if_include "10.42.0.0/16" -H dell-m4800,dell-t7500 -np 8 testOMPI 10000000 -->
   Result: 2.7182818285
   Time elapsed: 13.143063ms

(C) Install nVidia GPU Driver and Cuda-7.5 SDK

1. Check the VGA hardware information
 $ lspci  |  grep VGA -->
   01:00.0 VGA compatible controller: NVIDIA Corporation GK106GLM [Quadro K2100M] (rev a1)

2. Install nVidia GPU driver and Cuda-7.5:
 (a) Clean up all pre-installed nVidia-drivers
   $ sudo apt-get purge "nvidia-*" "libcuda1-*"
   $ sudo apt-get autoremove | sudo apt-get update
   $ sudo apt-get install dkms build-essential linux-headers-generic
   $ sudo add-apt-repository --remove ppa:xorg-edgers/ppa

 (b) Successful installment of nvidia-driver depends on the linux kernel. Thus, as you update linux kernel, you may need to re-install nvidia-driver.
   First, check your current linux kernel
   $ uname -r --> 4.4.0-24-generic

 (c) Install and Check Nvidia Graphic Driver.
   Install Nvidia-361 driver on Ubuntu 16.04
   I). Launch Web Browser and go to http://www.ubuntuupdates.org/package/core/xenial/restricted/proposed/nvidia-352
   Hit "APT INSTALL" red button and follow the instructions

   II). Check the installed nvidia graph card driver
   $ cd /usr/lib
   $ ls -lh | grep nvidia
   -rw-r--r-- 1 root root 1.7M Thursday 5 22:35 libnvidia-gtk2.so.361.42
   -rw-r--r-- 1 root root 1.7M Thursday 5 22:35 libnvidia-gtk3.so.361.42
   lrwxrwxrwx 1 root root 53 Saturday 2 10:37 libvdpau_nvidia.so -> /etc/alternatives/x86_64-linux-gnu_libvdpau_nvidia.so
   drwxr-xr-x 2 root root 4.0K Saturday 1 15:58 nvidia
   drwxr-xr-x 6 root root 4.0K Saturday 2 10:37 nvidia-361
   drwxr-xr-x 2 root root 4.0K Saturday 2 10:37 nvidia-361-prime

   III). Reboot
   $ sudo shutdown -r now

   IV). Check the nvidia setting.
   Use CLI to ask which card driver in use:
   $ prime-select query
   nvidia

   Select nvidia driver:
   $ sudo prime-select nvidia
   Info: the current GL alternatives in use are: ['nvidia-361', 'nvidia-361']
   Info: the current EGL alternatives in use are: ['nvidia-361', 'nvidia-361']
   Info: the nvidia profile is already in use.

 (d) Once your NVIDIA driver is set, install the CUDA 7.5 Debian package:
   I).Prepare clean environment for installing Cuda    $ sudo apt-get update
   $ sudo apt-get upgrade
   $ sudo shutdown -r now

   II).After reboot, install relevant dependences:
   $ sudo apt-get install ca-certificates-java default-jre default-jre-headless fonts-dejavu-extra freeglut3 freeglut3-dev java-common libatk-wrapper-java libatk-wrapper-java-jni libdrm-dev libgl1-mesa-dev libglu1-mesa-dev libgnomevfs2-0 libgnomevfs2-common libice-dev libpthread-stubs0-dev libsctp1 libsm-dev libx11-dev libx11-doc libx11-xcb-dev libxau-dev libxcb-dri2-0-dev libxcb-dri3-dev libxcb-glx0-dev libxcb-present-dev libxcb-randr0-dev libxcb-render0-dev libxcb-shape0-dev libxcb-sync-dev libxcb-xfixes0-dev libxcb1-dev libxdamage-dev libxdmcp-dev libxext-dev libxfixes-dev libxi-dev libxmu-dev libxmu-headers libxshmfence-dev libxt-dev libxxf86vm-dev lksctp-tools mesa-common-dev x11proto-damage-dev x11proto-dri2-dev x11proto-fixes-dev x11proto-gl-dev x11proto-input-dev x11proto-kb-dev x11proto-xext-dev x11proto-xf86vidmode-dev xorg-sgml-doctools xtrans-dev libgles2-mesa-dev nvidia-modprobe build-essential

   III). Download "cuda_7.5.18_linux.run" from https://developer.nvidia.com/cuda-downloads
   On ubuntu 16.04. it is necessary to install the CUDA manually from the ".run" file. Dependencies that would normally get pulled in from doing a CUDA install will have to be added.
   $ cd ~/Downloads
   $ sudo bash cuda_7.5.18_linux.run --override
   The "--override" is needed so you don't get the error, "Toolkit: Installation Failed. Using unsupported Compiler."

   --> Do you accept the previously read EULA? (accept/decline/quit): accept
   You are attempting to install on an unsupported configuration. Do you wish to continue? ((y)es/(n)o) [ default is no ]: y
   Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 352.39? ((y)es/(n)o/(q)uit): n
   Install the CUDA 7.5 Toolkit? ((y)es/(n)o/(q)uit): y
   Enter Toolkit Location [ default is /usr/local/cuda-7.5 ]:
   Do you want to install a symbolic link at /usr/local/cuda? ((y)es/(n)o/(q)uit): y
   Install the CUDA 7.5 Samples? ((y)es/(n)o/(q)uit): y
   Enter CUDA Samples Location [ default is /home/kinghorn ]: /usr/local/cuda-7.5
   Installing the CUDA Toolkit in /usr/local/cuda-7.5 ...
   Finished copying samples.

   = Summary =
   Driver: Not Selected
   Toolkit: Installed in /usr/local/cuda-7.5
   Samples: Installed in /usr/local/cuda-7.5

   Add a couple of system files.
   $ sudo nano /etc/profile.d/cuda.sh
   Add 'export PATH=$PATH:/usr/local/cuda/bin' into the file and save it. This allows cuda.sh to be executed when you login.
   $ sudo nano /etc/ld.so.conf.d/cuda.conf
   Add '/usr/local/cuda/lib64' into the file and save it. Run:
   $ sudo ldconfig

   Note: gcc-6 will not work with nvcc. Force cuda to work with gcc 5.
   $ sudo nano /usr/local/cuda/include/host_config.h
   Go to line: 115 and comment out error
   // #error -- unsupported GNU version! gcc versions later than 4.9 are not supported!
   $ sudo nano /usr/local/cuda/samples/common/findgllib.mk
   Go to line: 61, changed nvidia-352 to nvidia-361:
   UBUNTU_PKG_NAME = "nvidia-361"
   $ sudo nano /usr/lib/gcc/x86_64-linux-gnu/5/include/x86intrin.h
   comment out: // #include

3. Test the Cuda-7.5 SDK
 (a) Copy cuda programs to the user directory and compile the programs:
   $ cp -a /usr/local/cuda/samples   ~/cuda_samples
   $ cd ~/cuda_samples/
   Specify GLPATH=/usr/lib on the make line: GLPATH=/usr/lib make
   $ GLPATH=/usr/lib make -j $n

 (b) After compilation, test the programs with
   $ cd ~/cuda_samples/bin/x86_64/linux/release/
   $ ./deviceQuery

   $ ./nbody -benchmark -numbodies=128000
   --> Compute 3.0 CUDA device: [Quadro K2100M]
   number of bodies = 128000
   128000 bodies, total time for 10 iterations: 9678.305 ms
   = 16.929 billion interactions per second
   = 338.572 single-precision GFLOP/s at 20 flops per interaction

   $ ./particles -benchmark
   --> grid: 64 x 64 x 64 = 262144 cells
   particles: 16384
   Run 16384 particles simulation for 300 iterations...
   particles, Throughput = 8682.0585 KParticles/s, Time = 0.00189 s, Size = 16384 particles, NumDevsUsed = 1, Workgroup = 0

III. Building Mathematical Libraries

Open source simulation packages such as Quantum Espresso (QE), ab initio molecular dynamics package CP2K, classical molecular dynamics simulators (OpenMM, NAMD and Gromacs) need linear algebra libraries (OpenBLAS, ScaLAPACK) and FFTW3 libraries built on each computer note. We will show here how to compile the source codes with gcc/gfortran compilers and install those libraries with automake/cmake. OpenMPI must be installed first. Some environment variables need to be set to use OpenMPI by using LD_LIBRARY_PATH.

Building OpenBLAS, ScaLAPACK, ELPA and FFTW3
OpenBLAS (which also includes LAPACK) is an optimized version of BLAS based on GotoBLAS2. ScaLAPACK is a library of high-performance linear algebra routines for parallel distributed memory. ScaLAPACK solves dense and banded linear systems, least squares problems, eigenvalue problems, and singular value problems.

(1) Build OpenBLAS
 Download OpenBLAS-0.2.14.tar.gz from git hub:
 $ git clone git://github.com/xianyi/OpenBLAS.git

 $ cd OpenBLAS
 $ export OMP_NUM_THREADS=$n
 $ make CC=gcc FC=gfortran HOSTCC=gcc USE_OPENMP=1
 $ sudo make install PREFIX=/usr/local/lib/OpenBLAS
 $ cd /usr/lib
 $ ln -s /usr/local/lib/OpenBLAS/lib/libopenblas.a   libopenblas.a
 $ ln -s /usr/local/lib/OpenBLAS/lib/libopenblas.so   libopenblas.so
 $ ln -s /usr/local/lib/OpenBLAS/lib/libopenblas.so.0   libopenblas.so.0

(2) Build scaLAPACK lib with OpenMPI compiler
 $ wget http://www.netlib.org/scalapack/scalapack-2.0.2.tgz
 $ tar zxf scalapack.tgz
 $ cd scalapack-2.0.2
 $ cp SLmake.inc.example   SLmake.inc
 $ nano SLmake.inc --> Edit the following lines
  mpicc = /usr/local/openmpi/bin/mpicc
  mpif90 = /usr/local/openmpi/bin/mpif90
  BLASLIB = -lopenblas   -L/usr/local/lib/OpenBLAS/lib
  LAPACKLIB = -lopenblas   -L/usr/local/lib/OpenBLAS/lib
  LIBS = $(LAPACKLIB) $(BLASLIB)
 $ make
 $ chmod 555 libscalapack.a
 $ sudo cp ./libscalapack.a   /usr/local/lib/
 $ make clean

(3) Build ELPA library
  ELPA is a Fortran-based high-performance computational library for the massively parallel solution of symmetric or Hermitian, generalized eigenvalue problems. ELPA is designed to be compiled (Fortran) on its own.
  Download elpa-2016.5.003.tar.gz and untar the file
  $ tar zxvf elpa-2016.5.003.tar.gz
  $ cd elpa-2016.5.003

  Check the available options with "./configure --help".
  $ export omp="/usr/local/openmpi/bin"
  $ ./configure --prefix="/usr/local/lib/elpa" FC=$omp/mpifort CC=$omp/mpicc CXX=$omp/mpicxx CFLAGS="-O2 -march=native" CXXFLAGS="-O2 -march=native" FCFLAGS="-O2"
  $ make
  $ make check
  $ sudo make install

  This will build libelpa.a and libelpa.so in the folder "/usr/local/lib/elpa/lib". For the linking of your program to use ELPA, remember to add the linking flags. The most easy way would be to add the library specification '-L -lelpa'

(4) Set up FFTW3 with OpenMPI
  Download fftw-3.3.5.tg.gz and untar the file:
  $ tar zxf fftw-3.3.5.tar.gz
  $ cd fftw-3.3.5
  $ make distclean
  $ ./configure --prefix=/usr/local/fftw3 CC="/usr/bin/gcc" CXX="/usr/bin/g++" CFLAGS=-O3 FFLAGS=-O3 --enable-shared --enable-sse2 --enable-float --enable-openmp --enable-threads
  $ make -j $n
  $ make check
  $ sudo make install

(5) Set up LibINT---to be used by CP2K
  Download libint-1.1.4.tar.gz and untar the file:
  $ tar zxvf libint-1.1.4.tar.gz
  $ cd libint-1.1.4
  $ ./configure --prefix=/usr/local/lib/libint
  $ make
  $ sudo make install

(6) Set up Libxc---an exchange-correlation density functional library to be used by CP2K
  Download libxc-2.2.2.tg.gz and untar the file:
  $ tar zxf libxc-2.2.2.tg.gz
  $ cd libxc-2.2.2
  $ ./configure --prefix=/usr/local/lib/libxc
  $ make
  $ sudo make install

IV. Building Analytical Tools for Simulation Data

In this section, several open source software tools useful for analyzing simulation data were described. The tools include python packages, MDTraj, PyMol, VMD, R, bio3D, Xcrysden and Pwgui, etc.

(A) Install Python package manager (pip) and python packages :
 Check python version on the machine --> $ python -V
 $ sudo apt-get update
 $ sudo apt-get install --upgrade libpng-dev libfreetype6-dev
 $ sudo apt-get install --upgrade libffi-dev libxslt1-dev python-dev libssl-dev libxml2-dev
 -->
 libffi-dev is already the newest version (3.2.1-4).
 libxslt1-dev is already the newest version (1.1.28-2.1).
 python-dev is already the newest version (2.7.11-1).
 libssl-dev is already the newest version (1.0.2g-1ubuntu4.1).
 libxml2-dev is already the newest version (2.9.3+dfsg1-1ubuntu0.1).

 $ sudo apt-get install --upgrade python-pip
 $ sudo pip install -U pip
 $ sudo pip install -U pyopenssl ndg-httpsclient pyasn1

 Create virtual environment for Python
 $ pip show virtualenv

 $ sudo pip install numpy scipy
 $ sudo pip install matplotlib
 $ sudo pip install ipython
 $ sudo pip install pyzmq jinja2 tornado jsonschema
 $ sudo pip install scikit-learn

(B) Install R
(1) Prepare a clean environment for R installment
 $ sudo apt-get remove r-base-core
 $ sudo nano /etc/apt/sources.list --> Add following entry
 deb http://cran.rstudio.com/bin/linux/ubuntu trusty/
 save the file and quit.

 $ gpg --keyserver keyserver.ubuntu.com --recv-key E084DAB9
 $ gpg -a --export E084DAB9 | sudo apt-key add -
 $ sudo apt-get update && sudo apt-get upgrade

(2) Install R-base core
 $ sudo apt-get install r-base

(3) Launch R-base
 $ sudo R
 > install.packages(
   lib = lib <- .libPaths()[1],
   pkgs = as.data.frame(installed.packages(lib), stringsAsFactors=FALSE)$Package,
   type = 'source'
   )
 > install.packages('ggplot2', dependencies=TRUE)

(4) Install bio3D for analyzing MD trajectories
 The installment requires some packages, which can be installed directly through the official package manager system with
 $ sudo apt-get install netcdf-bin libnetcdf-dev libxml2-dev libcurl3-dev seaview muscle< br />
 $ sudo R
 > install.packages("bio3d", dependencies=TRUE)
 > q(  )

(C) Install MDTraj for Ananlyzing MD Trajectories
 $ sudo su
 $ pip install nose tables cython -U
 $ pip install pandas ipywidgets scripttest
 $ pip install mdtraj -U

 Testing the Installation with
 $ nosetests mdtraj -v

(D) Install the Xcrysden
 $ sudo apt-get update
 $ sudo apt-get install tcl8.5
 $ sudo make install tk8.5

 Download xcrysden-1.5.60.tar.gz and untar the file
 $ tar zxvf xcrysden-1.5.60.tar.gz
 $ cd xcrysden-1.5.60
 $ cp system/Make.sys-semishared   Make.sys
 $ make all
 $ sudo make install

 Xcrysden will be installed in "/usr/local/bin"
 $ xcrysden --help
 $ xcrysden -v

(E) Install PyMol
 $ sudo apt-get install pymol

(F) Install Pwgui
 1. Install itk
 $ sudo apt-get install insighttoolkit4-python
 $ sudo apt-get install libinsighttoolkit4-dev
 $ sudo apt-get install libinsighttoolkit4-dbg

 2. Check which version of tcl and iwidgets on ubuntu
 $ sudo apt-get install iwidgets4
 --> The following NEW packages will be installed:
 itcl3 itk3 iwidgets4

 $ sudo dpkg --get-selections | grep iwidgets

 3. Download PWgui-5.4.0.tar.gz and untar the file
 $ tar zxvf PWgui-5.4.0.tar.gz

 4. Copy "pwgui" launcher to some common place. Move the generated pwgui to /usr/local/bin/ by typing:
 $ sudo mv ./PWgui-5.4.0 /usr/local/bin/
 $ cd /usr/local/bin/
 $ sudo ln -s /usr/local/bin/PWgui-5.4.0/pwgui pwgui

 To launch the program, type:
 $ pwgui

(G) Install VMD on Ubuntu 14.04 (64-bit)
 $ sudo apt-get update

 Download vmd-1.9.2.bin.LINUXAMD64.opengl.tar.gz and untar the file
 $ tar -zxvf vmd-1.9.2.bin.LINUXAMD64.opengl.tar.gz

 $ cd vmd-1.9.2
 $ ./configure
 --> using configure.options: LINUXAMD64 OPENGL OPENGLPBUFFER FLTK TK ACTC CUDA IMD LIBSBALL XINERAMA XINPUT LIBTACHYON VRPN NETCDF COLVARS TCL PYTHON PTHREADS NUMPY SILENT ICC
 $ cd src
 $ sudo make install

(H) Install the online pdbfixer tool
 $ cd ~/Downloads
 $ git clone https://github.com/pandegroup/pdbfixer
 $ cd pdbfixer
 $ sudo python setup.py install --record inst.txt