Installation#
The VirtualLab platform has been designed so that only a small number of prerequisites are needed for its use. Containers are used to house various codes and software, meaning that a containerisation tool is required, with Apptainer the chosen tool. The VirtualLab python package only requires gitpython (and therefore git), and can be used with either native python (including pip) or conda.
VirtualLab Supports the following operating systems:
Operating System |
Version |
Notes |
---|---|---|
Linux |
Mint 19/Ubuntu 20.04+ |
Any reasonably modern distro should work. VirtualLab has been tested on various desktops and laptops running Ubuntu 20.04 LTS and a supercomputer running Redhat Linux enterprise 9. However, as with all things Linux results may vary on other distros. |
As Apptainer is only available on Linux this is currently the only officially supported OS.
Note
Apptainer’s website does contain instructions for using it Windows and MacOs. However, this through a virtual machine which prevents the use of GPUs for modules that support them. It also has a negative impact on performance as such we don’t recommend using Apptainer on non Linux systems.
Quick Installation#
You may run the following ‘one line’ command on a fresh installation of a supported Linux distro to install VirtualLab and the containerisation tool. The only requirement for this installation is that you have either python installed (including pip), or conda.
Warning
If you are not using a fresh OS it is highly recommended that you read the rest of this page before trying this method, to better understand how the installation is carried out in case you have pre-installed dependencies which might brake.
This ‘one line’ commant has only been tested on Ubuntu 20.04 LTS.
Terminal:
wget https://gitlab.com/ibsim/virtuallab/-/raw/master/Scripts/Install/Host/Install_main.sh && \
chmod 755 Install_main.sh && \
./Install_main.sh -y && \
rm Install_main.sh && \
source ~/.VLprofile
Running the above will install both git and apptainer along with the VirtualLab python package using standard python.
Note
Install_main.sh contains a couple of sudo commands, therefore you will be promted to enter your password.
The -y
in the third line of the above signals to skip the installation dialogue and install VirtualLab in the default location /home/$USER/VirtualLab
. This flag can be removed if youd require a non-standard install.
Note
If youd like to install VirtualLab using conda or to get the latest development version see here.
To test out the install follow the steps outlined here.
Standard install#
To use the install/update script you will need to install git. This can be easily done by either following the instructions on git’s website or, on Ubuntu based distros, you can run the following in a terminal.
sudo apt install git
Next you will need to install Apptainer. This can either be installed using the following:
wget https://gitlab.com/ibsim/virtuallab/-/raw/master/Scripts/Install/Host/Install_Apptainer-bin.sh && \
chmod 755 Install_Apptainer-bin.sh && \
sudo ./Install_Apptainer-bin.sh -y && \
rm Install_Apptainer-bin.sh
or by following the most up-to-date instructions from their website:
VirtualLab is primarily command line only so you will need to run the following commands in a terminal to install the VirtualLab python package
BRANCH=master
wget https://gitlab.com/ibsim/virtuallab/-/raw/${BRANCH}/Scripts/Install/Host/Install_VirtualLab.sh && \
chmod 755 Install_VirtualLab.sh && \
./Install_VirtualLab.sh -B $BRANCH && \
rm Install_VirtualLab.sh && \
source ~/.VLprofile
The installer will then take you through a series of menus and download the latest version of the code.
Note
If you’d prefer to use conda instead of native python you will need to add -I conda
to the third line, e.g.
./Install_VirtualLab.sh -I conda
This will create an environment named VirtualLab.
Note
The above will install the most recent version of VirtualLab from the master branch. For the most recent development version change BRANCH=dev in the above.
At this point the VirtualLab package will have been installed, however none of the containers it requires have yet been downloaded. These will be installed as and when they are needed for the analysis in question.
The size of these containers can be quite large. As standard, these containers will be saved to a directory named ‘Containers’ in the VirtualLab directory. If you’d prefer these containers be saved elsewhere, this can be changed in VLconfig.py
file in the VirtualLab directory, see code configuration for more details.
To test out the install follow the steps outlined here.
Instalaltion on HPC system#
Due to the limited number of prerequisits, installation of VirtualLab on HPC clusters is straight forward. The steps outlined here are for the sunbird cluster, however a similar procedure should work for other HPC systems.
Firstly, a setup file is required which contains all of the packages required for VirtualLab to work correctly. For sunbird, this file looks like this
module load anaconda/2023.03 git/2.19.2 apptainer/1.0.3 mpi/mpich/3.2.1
# sunbird specific bug fix to ensure conda works correctly
source /apps/local/languages/anaconda/2023.03/etc/profile.d/conda.sh # must match loaded anaconda version
set -a
. /apps/local/languages/anaconda/2023.03/etc/profile.d/conda.sh
set +a
# sources .VLprofile (if its available) to ensure paths such as the VirtualLab directory are discovarable
if [ -f "$HOME/.VLprofile" ] ; then
source $HOME/.VLprofile
fi
This loads anaconda, git, apptainer and mpich (which is needed for multi-node use, see MPI for details). This file also fixes a small bug with conda specific to sunbird. It then sources .VLprofile
, which is created as part of the VirtualLab install.
This file can be easily downloaded and sourced with the following
cd # more convenient if this file is in home directory, but doesn't have to be
wget "https://gitlab.com/ibsim/virtuallab_bin/-/raw/dev/VL_setup.sh"
source VL_setup.sh
On other HPC systems the versions for the packages would need to be changed to reflect the versions available.
Next VirtualLab can be installed with the following
BRANCH=master # branch to install from
wget "https://gitlab.com/ibsim/virtuallab/-/raw/"$BRANCH"/Scripts/Install/Host/Install_VirtualLab.sh"
chmod 755 Install_VirtualLab.sh
./Install_VirtualLab.sh -B $BRANCH -I conda
rm Install_VirtualLab.sh
This installation is exactly the same as that outlined here, where more details can be found.
Test out that the installation has worked correctly with the following
VirtualLab --test
see here for more details on this.
Note
On sunbird the computing nodes do not have access to the internet, therefore any containers required for an analysis will need to be pulled and built using the login node. Therefore the above test will need to be performed on the login node.
Performing analysis using VirtualLab on sunbird can then performed using the a SLURM script such as this
#!/bin/bash --login
#SBATCH --job-name=VirtualLab
#SBATCH --output=VL%J
#SBATCH --time=0-00:10
#SBATCH --ntasks=1
#SBATCH --mem-per-cpu=6000
#SBATCH --account=scwXXXX
module purge
source ~/VL_setup.sh # gets all the necessary packages
VirtualLab -f ~/VirtualLab/Run.py # runs analysis
The first few lines specify the resources required to run the analysis, followed by sourcing VL_setup.sh to ensure the required packages are loaded. Following this the analysis outlined Run.py
in the VirtualLab
directory is performed, which is a simple tensile test.
Note
This analysis requires salome meca to create meshes and perform FEA, therefore this container will need to be built on the login node before hand.
A convenient method of building container is by using the -C
option followed by the name of the containers. To build the salome meca container use the following command
VirtualLab -C SalomeMeca
The above SLURM script can then be submitted with the following
sbatch #PATH/#TO/#SLURM_FILE
Testing#
To test out that the installation has worked as expected run the following command
VirtualLab --test
This will download VirtualLab’s manager container along with a small test container to make sure things are set up correctly. It also spits out a randomly selected programming joke as a nice whimsical bonus.
For more on how to use VirtualLab we recommend working through the Tutorials section.
MPI#
VirtualLab is able to perform analysis on multi-node systems as well as personal computers. For this MPI is required, and needs to be compatible with the MPI installed within VirtualLab’s manager container, which is MPICH. To install MPICH run the following command
sudo apt install mpich
To test out that VirtualLab is compatible with MPI run the following
VirtualLab -f RunFiles/MPI_test.py
You should see an output similar to this (order will differ)
Hello! I'm rank 0 from 5 running in total...
Hello! I'm rank 2 from 5 running in total...
Hello! I'm rank 1 from 5 running in total...
Hello! I'm rank 4 from 5 running in total...
Hello! I'm rank 3 from 5 running in total...
Warning
GlibC issues with Ubuntu 22.04+
We note, at this stage, that there is a known bug with Salome-Meca Running in VirtualLab with Ubuntu 22.04, along with some newer versions of Fedora.
If you are using these you may find you get an error containing something similar to the following:
version `GLIBC_2.34' not found (required by /.singularity.d/libs/libGLX.so.0)
The issue is a bug in the way that the --nv
flag loads nvidia libraries. The short version is that the --nv
flag isn’t very sophisticated when it comes to libraries. It looks for a list of library files on the host which is defined in nvliblist.conf
.
The issue is that the latest version(s) of Ubuntu are compiled against a newer version of libGLX than is included within the Salome container. This causes problems in Apptainer.
To fix this you have two options. Firstly, you can use the -N
option to turn off the nvidia libraries. The drawback to this is that you will be running in ‘software rendering mode’ and thus you will not benefit from any GPU acceleration.
The second option is to use the following workaround.
Search for a file named
nvliblist.conf
in your installation. It should be under your Apptainer installation directory. By default this is under/etc/apptainer
.Make a back-up of this file
mv nvliblist.conf nvliblist.conf.bak
.Open the file
nvliblist.conf
using a text editor.Delete all of the following lines that appear
libGLX.so
,libGLX.so.0
,libglx.so
,libglx.so.0
andlibGLdispatch.so
. Note, depending on you exact system, the file may not contain all of them.
Try running the Salome container again, it should work this time.
Reference: https: //github.com/apptainer/apptainer/issues/598
One caveat with this workaround, however, is that involves messing with configs that apply system wide. As such, it may have unintended side-effects with other software/containers that use Apptainer. Our team have not yet reported any issues. However, this does not mean they do not exist. Therefore, we cannot 100% guarantee you won’t have any issues. This is also the reason we recommend backing up your original config in step 2, just in case. Also, for future reference, these fixes where applied to ubuntu 22.04 with Apptainer version 1.0.5. Your millage may vary with future updates.