documentation:ml:berzelius

This is an old revision of the document!


How to use Berzelius

  1. below in formation is from memory and should be updated by the next person going through the process
  2. Get a login to SNIC: https://supr.snic.se.
  3. You will be walked throught the process.
  4. Note that you will have to accept the SNIC user agreements.
  5. After every step you will receive an email from supr@supr.snic.se. Read the emails carefully, they will tell you what to do next.
  6. At some point you will have to write a small project proposal: https://supr.snic.se/proposal/
  7. - Go to Rounds page
  8. - In the sub-menue, select AI/ML and then select LiU Berzelius
  9. - Create the proposal for getting computation time on Berzelius
  10. You will get a confirmation email from snic.se and NSC Berzelius and later a confirmation email that your project was accepted.
  11. Follow the guidelines in the emails. At some point you will have to accept another user agreement for using Berzelius. Once accepted your account is created.
  12. before you can log in you need a 2-Factor Authorization (2FA): https://www.nsc.liu.se/support/2fa/migration/
  13. - go through the section “How to enable 2FA for your cluster login account - detailed version”
  14. Finally, you can run ssh berzelius.nsc.liu.se. You get asked for the password and then the 6-digit number from the authenticator account.
  15. … and you should be in.

If you want to use conda:

  • - On your private workstation, use conda env export > environment.yml to get a description of your conda envionment. Use scp or rsync to copy environment.yml to berzelius.
  • - run module load Anaconda/2021.05-nsc1 to load the conda module. It is a good idea to add this to your .bashrc.
  • - ln -s ~/.conda /proj/<your_project_dir>/users/$(id -un), don't forget to replace <your_project_dir> with your project id. In my case, it is “berzelius-2022-58”.
  • - conda env create -f environment.yml will replicate your home conda environment with the same name, etc. Then, run conda activate… Note that libraries you have installed with pip are not installed. You have to do that manually.
  • - conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
  • - If you need ai-gym and atari-games, use conda install -c conda-forge gym.
  • - If you need the ATARI-ROMs, then the steps become a bit myserious: See https://github.com/mgbellemare/Arcade-Learning-Environment and perhaps the outdated https://github.com/openai/atari-py
  • I have tested PyTorch so far, it seems that it can only see a single GPU and apparently not fully supports the NVIDIA A100-SXM4-40GB GPU.
  • under investigation…
  • documentation/ml/berzelius.1648033266.txt.gz
  • Last modified: 2022/09/02 14:04
  • (external edit)