Build Hyak Stack from Scratch with Spack

This page describes the steps needed on Hyak to build the latest version of R with a up-to-date GCC compiler.

1. Work on the build node.

For Mox, run:

srun -p build --time=6:00:00 --mem=40G --pty /bin/bash

TODO, add Klone:

TODO

For RSPH HPC, run:

srun -N 1 -n 1 --cpus-per-task=32 -p day-long-cpu --time=4:00:00 --mem=180G --pty /bin/bash

These are good functions to set up aliases for. (TODO: Link to alias page)

2. Clone Spack (https://spack.readthedocs.io/en/latest/#)

Move to the folder you'd like to store everything (this can be your personal home directory), then clone spack.

git clone https://github.com/spack/spack.git

After cloning, load Spack functions and variables into the environment:

. spack/share/spack/setup-env.sh

3. Tell Spack about Your Existing Compilers

Spack needs to know what to compile new packages with. This will find them and store them in a configuration file.

spack compiler find

If you are having any challenges with compiling GCC, below, you have have a previous version of this compiler configuration file. In this case, just clear that all out and restart with the following. Note, replace sjenness with your Hyak user name.

rm -rf ~/.spack
spack compiler find

4. Install GCC

Install the spack gcc package. This takes a while, around 2.5 hours, so just let it run.

spack install gcc 

If you need/want to run this on one core, use:

spack install -j1 gcc 

5. Reload Spack Environment

Reload the Spack environment to identify the new GCC module that you have built, then load that module with spack load.

. spack/share/spack/setup-env.sh
spack load gcc@X.X.X

where X.X.X is the version of GCC that you just installed.

Tell Spack to add this new compiler to the list with:

spack compiler find

6. Build R

This will build the latest version of R with the new GCC.

spack install r %gcc@X.X.X

where X.X.X is the version of GCC that you just installed.

If you want to build an older version of R, specify it like this:

spack install r@Y.Y.Y %gcc@X.X.X

where Y.Y.Y is the version of R that you want.

You can look up all available versions of R for download with:

spack info r

When complete, you can review the set of available spack packages with:

spack find