documentation:ml:berzelius

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
documentation:ml:berzelius [2022-03-28 15:03] – Robin Göranssondocumentation:ml:berzelius [2022-09-02 16:04] (current) – external edit 127.0.0.1
Line 46: Line 46:
  
 ==== Working with Singularity ==== ==== Working with Singularity ====
 +
 +Install Singularity by following this guide: https://sylabs.io/guides/3.0/user-guide/quick_start.html
 +
 +Creating a simple Singularity image using a recipe file: https://sylabs.io/guides/3.0/user-guide/definition_files.html
 +  - Create and open a recipe file using ''vim Singularity.recipe''
 +  - Choose an bootstrap agent that will create the base OS you want to use and add the corresponding lines to the recipe file:
 +      * Bootstrap: docker
 +      * From: ubuntu:20.04
 +  - Create the %post section which will execute commands within the singularity container:
 +      * %post
 +      * apt -y update
 +      * apt -y install python3
 +      * apt -y install pip
 +      * pip3 install --upgrade pip
 +      * pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113
 +  - Add a %files section if any files are to be used by the singularity container:
 +      * %files
 +      * main.py /
 +      * eval.py /
 +      * logs/ /
 +  - Specify in the %runscript section the standard script to be run:
 +      * %runscript
 +      * python3 main.py
 +  - Create a .sif file from the recipe by running ''sudo singularity build image.sif Singularity.recipe''
 +  - The script in the %runscript section can then be run with ''singularity run image.sif''
 +  - Other scripts can be run using ''singularity exec image.sif python3 eval.py''
 +
 +Using .sif file on Berzelius:
 +  - Upload .sif file to Berzelius using ''scp image.sif <username>@berzelius1.nsc.liu.se:/proj/<project name>/users/<username>''
 +  - Also upload all needed files to the same folder on Berzelius.
 +  - Log into Berzelius, request computing resources and change directory to ''/proj/<project name>/users/<username>''
 +  - Run script using, for example ''singularity exec --nv image.sif python3 main.py''
  
   * under investigation...   * under investigation...
  • documentation/ml/berzelius.1648472637.txt.gz
  • Last modified: 2022-09-02 16:04
  • (external edit)