Installing bcl2fastq without rebuilding the container

[arisp99]

Currently, we advise users that if they would like to demultiplex files they must first download bcl2fastq, move the .zip file into the programs directory, and then re-build the container. It would be convenient if users were able to add custom software into the container without needing to re-build it, perhaps by binding to a certain location within the container.

This issue was discussed previously by @JeffAndBailey in #32 (comment) and by @aydemiro in #32 (comment):

Let's see if we can download and build externally bcl2fastq and install it as a working version with any needed libraries or accessory files. If that is possible then really our fixed builds san bcl2fastq will be fine for reproducibility.

Originally posted by @JeffAndBailey in #32 (comment)

As for the bcl2fastq issue, I agree that we should explore building the software outside and providing the binary to the container as a binding. However, this is a compiled c++ program and how to create a portable binary is beyond my capabilities at the moment. Nick is probably the best person to consult on this.

Originally posted by @aydemiro in #32 (comment)

[arisp99]

There may be a potential solution for this in the use of the fakeroot feature. Administrators will need to ensure that user namespace UID/GID mapping is properly configured, which is outlined in the admin guide. It seems that admins can use singularity config fakeroot to set up the correct configuration for a user.

Users may need to configure their distribution so that unprivileged user namespace creation is enabled. In the case of REHL (which is what our server runs on):

sudo sh -c 'echo user.max_user_namespaces=15000 \
    >/etc/sysctl.d/90-max_net_namespaces.conf'
sudo sysctl -p /etc/sysctl.d /etc/sysctl.d/90-max_net_namespaces.conf

Then we could let users use fakeroot by doing:

sudo singularity config fakeroot --add <username>

This command will edit /etc/subuid and /etc/subgid with the correct mapping for a user.