documentation:ml:berzelius

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
documentation:ml:berzelius [2022/03/28 13:19] – [Working with Singularity] robingdocumentation:ml:berzelius [2022/09/02 14:04] (current) – external edit 127.0.0.1
Line 54: Line 54:
       * Bootstrap: docker       * Bootstrap: docker
       * From: ubuntu:20.04       * From: ubuntu:20.04
-  - Next, create the %post section which will execute commands within the singularity container:+  - Create the %post section which will execute commands within the singularity container:
       * %post       * %post
       * apt -y update       * apt -y update
       * apt -y install python3       * apt -y install python3
       * apt -y install pip       * apt -y install pip
-      * pip3 install torch +      * pip3 install --upgrade pip 
 +      * pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113 
 +  - Add a %files section if any files are to be used by the singularity container: 
 +      * %files 
 +      * main.py / 
 +      * eval.py / 
 +      * logs/ / 
 +  - Specify in the %runscript section the standard script to be run: 
 +      * %runscript 
 +      * python3 main.py 
 +  - Create a .sif file from the recipe by running ''sudo singularity build image.sif Singularity.recipe'' 
 +  - The script in the %runscript section can then be run with ''singularity run image.sif'' 
 +  - Other scripts can be run using ''singularity exec image.sif python3 eval.py''
  
 +Using .sif file on Berzelius:
 +  - Upload .sif file to Berzelius using ''scp image.sif <username>@berzelius1.nsc.liu.se:/proj/<project name>/users/<username>''
 +  - Also upload all needed files to the same folder on Berzelius.
 +  - Log into Berzelius, request computing resources and change directory to ''/proj/<project name>/users/<username>''
 +  - Run script using, for example ''singularity exec --nv image.sif python3 main.py''
  
   * under investigation...   * under investigation...
  • documentation/ml/berzelius.1648473560.txt.gz
  • Last modified: 2022/09/02 14:04
  • (external edit)