This is an old revision of the document!
How to use Berzelius
Getting Started
- below information is from memory and should be updated by the next person going through the process
- Get a login to SNIC: https://supr.snic.se.
- You will be walked throught the process.
- Note that you will have to accept the SNIC user agreements.
- After every step you will receive an email from supr@supr.snic.se. Read the emails carefully, they will tell you what to do next.
- At some point you will have to write a small project proposal: https://supr.snic.se/proposal/
- - Go to Rounds page
- - In the sub-menue, select AI/ML and then select LiU Berzelius
- - Create the proposal for getting computation time on Berzelius
- You will get a confirmation email from snic.se and NSC Berzelius and later a confirmation email that your project was accepted.
- Follow the guidelines in the emails. At some point you will have to accept another user agreement for using Berzelius. Once accepted your account is created.
- before you can log in you need a 2-Factor Authorization (2FA): https://www.nsc.liu.se/support/2fa/migration/
- - go through the section “How to enable 2FA for your cluster login account - detailed version”
- Finally, you can run
ssh berzelius.nsc.liu.se
. You get asked for the password and then the 6-digit number from the authenticator account. - … and you should be in.
Once you are logged in
- To not have to put in your password on every login, use an SSH key: https://www.nsc.liu.se/support/security/
- Do not work on the shell you arrive at, it is a shared resource. Use the commant
interactive -n 1
go get your own CPU. Work from there. Useexit
to leave the interactive session and free that CPU for others and to return to the login shell. From there, you can log off or request another interactive session.
Working with Conda
If you want to use conda:
- - On your private workstation, use
conda env export > environment.yml
to get a description of your conda envionment. Use scp or rsync to copyenvironment.yml
to berzelius. - - run
module load Anaconda/2021.05-nsc1
to load the conda module. It is a good idea to add this to your .bashrc. - -
ln -s ~/.conda /proj/<your_project_dir>/users/$(id -un)
, don't forget to replace <your_project_dir> with your project id. In my case, it is “berzelius-2022-58”. - -
conda env create -f environment.yml
will replicate your home conda environment with the same name, etc. Then, runconda activate…
Note that libraries you have installed with pip are not installed. You have to do that manually. - -
conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
- - If you need ai-gym and atari-games, use
conda install -c conda-forge gym
. - - If you need the ATARI-ROMs, then the steps become a bit myserious: See https://github.com/mgbellemare/Arcade-Learning-Environment and perhaps the outdated https://github.com/openai/atari-py
- I have tested PyTorch so far, it seems that it can only see a single GPU and apparently not fully supports the NVIDIA A100-SXM4-40GB GPU. under investigation
Working with Singularity
- under investigation…