In this document, we will outline how to create and modify conda environments, as well as how to install python packages on Andromeda.
1. Loading Miniconda
Andromeda provides a distribution of Miniconda 3. You can see it with ‘module avail miniconda’ and load it with ‘module load miniconda3/miniconda’
[1001]$ module avail miniconda
------------------ /cm/shared/modulefiles ------------------
miniconda3/miniconda miniconda3/test
[1001]$ module load miniconda3/miniconda
[1001]$ module list
Currently Loaded Modulefiles:
1) slurm/current 2) miniconda3/miniconda
[1001]$
2. Creating an environment
Once your shell is initialized, you can create a conda environment with conda create -n env_name python=3.9This will create a conda environment called env_name, using python version 3.9. You can change the environment name and python version as needed. Activate the new environment with conda activate env_nameOnce your environment is activated, you can install packages and run scripts using the new environment.
(env_name) $ which python
/path/to/conda/.envs/env_name/bin/python
(env_name) $ which pip
/path/to/conda/.envs/env_name/bin/pip
3. Installing packages
You can install python packages in your conda environment with pip, as you would normally. For example, to install numpy, just run pip install numpy with your environment active. The package will be installed to your conda environment. You can also install other conda packages with conda install (package name)For example, conda install jupyter will install Jupyter to your conda environment.
4. Using conda environment in Slurm job
To use your conda environment in your slurm batch job, you will need to activate it in your script. For example,
#!/bin/bash
#SBATCH --job-name=job_test
#SBATCH --partition=shared
#SBATCH --output=slurm_%x_%j.out
#SBATCH --error=slurm_%x_%j.err
#SBATCH --time=00:10:00
#SBATCH --ntasks=1
#SBATCH --mem-per-cpu=1gb
module load miniconda3/miniconda
conda activate env_name
cd (path to working directory)
python my_script.py
5. Miscellaneous conda usage
You can deactivate your conda environment with conda deactivateDelete an environment with conda remove -n env_name --alland view a list of all available conda environments with conda info --envs