ETH RDM Workshop Series - Spring 2024

Workshop 4 - Reproducible Scientific Computing and Data Analysis

Henry Lütcke - henry.lutcke@id.ethz.ch
Scientific IT Services, ETH Zurich

13 March 2024

1 Introduction to Conda

1.1 Reference

1.2 Starting conda

Command Line Interface

$ conda deactivate
Conda on Windows
Conda on Windows
Conda on macOS
Conda on macOS

1.3 Version, Info and Help

conda Environment

A directory that contains specific installed packages (system libraries, python/R modules, …).

base Environment

The default environment that you have when installing conda.

$ conda --version

$ conda --help

$ conda info

1.3.1 Exercise

  1. Which version of conda are you running?

  2. Inspect the output of conda info especially: platform, Python version, active environment, base environment and channel URLs.

1.4 Create / Clone and List all Envs

$ conda create --name py38 python=3.8

$ conda create --name py38-clone --clone py38

$ conda env list

1.5 Activate and Deactivate Env

Check Python version on:

  1. the system,
  2. the base env, and
  3. the new py38 env.
      $ python --version

      $ conda activate 
(base)$ python --version

(base)$ conda activate py38
(py38)$ python --version

(py38)$ conda deactivate --help
(py38)$ conda deactivate

1.6 List Installed Packages

      $ conda activate py38
(py38)$ conda list
(py38)$ conda list --name base

(py38)$ conda list python
(py38)$ conda list --name base python

1.7 Search and Install Packages

Conda channels

Locations where packages are stored, e.g. conda-forge, bioconda, r;

(py38)$ conda deactivate && conda deactivate
      $ conda search numpy=1.21
      $ conda search --channel conda-forge numpy=1.21

      $ conda install --channel conda-forge numpy=1.21.6 --name py38
      $ conda activate py38
(py38)$ conda install pandas="<1.5"

1.8 Update Packages and Revisions

(py38)$ conda update --channel conda-forge pandas
(py38)$ conda update --all

(py38)$ conda list --revisions
(py38)$ conda install --revision 2 --channel conda-forge
(py38)$ conda list --revisions

Note: In restoring to a revision the additional channels are needed explicitly.

1.9 Remove Package / Env and Clean

(py38)$ conda env list
(py38)$ conda remove --name py38 pandas
(py38)$ conda remove --name py38-clone --all
(py38)$ conda env list

(py38)$ conda clean --all

Note: It is not possible to remove the active environment.

1.9.1 Exercise

  • List all revisions in py38 environment. Do you recognize the changes in the last revision?

1.10 Environment File - YAML Format

Environment File

A YAML file that contains all information to create an environment.

YAML file: seaborn.yml

name: seaborn
channels:
  - defaults
dependencies:
  - seaborn=0.12.2

Note: seaborn is is a Python data visualization library based on matplotlib.

1.11 Environment File to Environment and back

 (py38)$ conda env create --file seaborn.yml
 (py38)$ conda activate seaborn

(seaborn)$ conda env export
(seaborn)$ conda env export --no-builds
(seaborn)$ conda env export --from-history
(seaborn)$ conda env export --no-builds > seaborn_export.yaml
(seaborn)$ conda env create --file seaborn_export.yaml --name seaborn_export

Note: The build specification is generally platform dependent.

Note: The packages might not be available on all platforms.

1.12 Identical Environments - Spec File

Spec File

A text file that contains all information to create an identical environment.

(seaborn)$ conda list --explicit 

(seaborn)$ conda list --explicit > seaborn_spec_file.txt
(seaborn)$ conda create --name seaborn_explicit --file seaborn_spec_file.txt

Warning: Use the spec file on a similar platform.

1.12.1 Exercise

  1. List all environments.
  2. Read the help for conda remove and pay attention at the flag -y, --yes .
  3. Remove seaborn_export and seaborn_explicit environments.

1.13 Conda Optional Arguments

There are long -- and potentially equivalent short - optional arguments.

long: -- short: - Remark
--name -n
--channel -c
--help -h
--file -f not always
--all -a not always
--revisions -r
--yes -y

Note: If an argument is not working as expected use --help.

1.14 What Did We Learn?