Skip to content

INSTALLATION.md

Latest commit

 

History

History
248 lines (180 loc) · 7.37 KB

INSTALLATION.md

File metadata and controls

248 lines (180 loc) · 7.37 KB
Raw

Installation

PyTorch Connectomics targets Python 3.11 in a conda environment named pytc (both are configurable). Pick the path that matches your level of control.

Quickest path

curl -fsSL https://raw.githubusercontent.com/zudi-lin/pytorch_connectomics/master/quickstart.sh | bash
cd pytorch_connectomics
conda activate pytc
python scripts/main.py --demo

The quickstart.sh script installs Miniconda if missing, clones the repo (when not already inside one), and runs install.py. The cd step is needed because curl ... | bash runs the script in a child process — your interactive shell stays where it started.

Install with options

For control over the env name, Python version, CUDA wheel, or extras:

git clone https://github.com/zudi-lin/pytorch_connectomics.git
cd pytorch_connectomics
python install.py --install-type basic --python 3.11 --env-name pytc

install.py flags:

Flag Default Purpose
--env-name NAME pytc Conda env name
--python VER 3.11 Python version (3.8–3.12)
--cuda X.Y auto Force a specific CUDA version (e.g. 12.1)
--cpu-only off CPU-only PyTorch wheel
--install-type basic basic, dev, or full
--force-recreate off Wipe and recreate an existing env (default: reuse)
--interactive off Prompt before each step
--pip-options STR "" Extra args passed to pip install -e .
--no-color off Disable ANSI colors

If the target env already exists, install.py reuses it by default and continues installing on top. Pass --force-recreate to wipe and recreate it.

install.py auto-detects CUDA from nvidia-smi, nvcc, the module system, and /usr/local/cuda-*. Passing --cuda or --cpu-only overrides detection.

On Apple silicon, the standard macOS PyTorch wheel includes the MPS backend. The installer selects a pthreads OpenBLAS build to avoid loading a second copy of libomp, and runtime config resolves system.accelerator: auto to mps. Apple MPS is exposed as one device, so use num_gpus: 1 (or -1 for automatic device count):

system:
  accelerator: auto  # auto | cpu | cuda | mps
  num_gpus: 1

PyTC uses full precision on MPS when a tutorial requests mixed precision, because PyTorch Lightning's MPS mixed-precision path is not consistently supported across macOS/PyTorch releases.

Manual install

Use this when you want full control or are on a host where install.py does not work cleanly:

conda create -n pytc python=3.11 -y
conda activate pytc

# Pre-built wheels for the few packages that compile against numpy/h5py.
conda install -c conda-forge numpy h5py cython connected-components-3d -y

# Pick the PyTorch wheel that matches your CUDA. See
# https://pytorch.org/get-started/locally/ for the index URL.
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121

# Editable install of pytorch_connectomics.
git clone https://github.com/zudi-lin/pytorch_connectomics.git
cd pytorch_connectomics
pip install -e .

# Optional: just (command runner used by README tutorial commands).
conda install -c conda-forge just -y

Install via Claude Code or Codex CLI

If you already have an authenticated Claude Code or Codex CLI on this machine, you can delegate the install to the agent. The agent runs install.py and diagnoses failures using this doc — it does not invent new install steps.

Prerequisites: claude or codex CLI installed and authenticated; just (used by the recipes — direct invocations without just are shown below).

Default: interactive (recommended)

The agent reads prompts/INSTALL.md, runs install.py, and asks for your approval on each shell command before executing it. Use this when you want a guided install without typing the commands yourself.

just install-claude     # opens claude with prompts/INSTALL.md
# or, without just:
claude "$(cat prompts/INSTALL.md)"
just install-codex      # opens codex with prompts/INSTALL.md
# or, without just:
codex "$(cat prompts/INSTALL.md)"

You will see prompts for conda create -n pytc python=3.11, the PyTorch pip install, pip install -e ., and the demo command. Approve each. The agent exits when install.py reports success.

Unattended (advanced — trusted machines only)

Skip per-command approval. These flags grant the agent full shell access; only run them on trusted, isolated machines (throwaway VMs, a fresh user account, or a sandboxed container). Do not use on shared workstations or development boxes that hold private data.

claude -p --permission-mode bypassPermissions --allowedTools "Bash(*)" \
       "$(cat prompts/INSTALL.md)"

codex exec --dangerously-bypass-approvals-and-sandbox - < prompts/INSTALL.md

Optional extras

pip install -e .[full]   # tifffile, wandb, optuna, gputil, neuroglancer
pip install -e .[dev]    # pytest, pytest-benchmark, ruff, etc.
pip install git+https://github.com/PytorchConnectomics/MedNeXt.git

wandb is bundled inside the [full] extra — there is no separate [wandb] extra in setup.py / pyproject.toml.

Common install issues

"No module named 'connectomics'"

Reactivate the env and reinstall in editable mode:

conda activate pytc
pip install -e . --no-build-isolation

"NumPy requires GCC >= 9.3"

Common on HPC clusters with old toolchains. Use conda for the packages that build native extensions:

conda activate pytc
conda install -c conda-forge numpy h5py cython connected-components-3d -y
pip install -e . --no-build-isolation

"AttributeError: module 'numpy' has no attribute 'float'"

Mahotas < 1.4.18 is incompatible with NumPy 2.x:

pip install --upgrade mahotas numpy

"CUDA not available" but you have a GPU

Either PyTorch was installed CPU-only, or CUDA is not loaded:

# Reinstall with CUDA wheel (adjust to your CUDA version).
pip uninstall -y torch torchvision
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121

# Or load CUDA module on HPC.
module load cuda/12.1
python -c "import torch; print(torch.cuda.is_available())"

macOS aborts with "libomp.dylib already initialized"

Current install.py prevents this by selecting pthreads OpenBLAS. For an environment created by an older installer, repair it with:

conda install -n pytc -c conda-forge 'libopenblas=*=*pthreads*'

"ImportError: libcudnn.so.8: cannot open shared object file"

module load cudnn/8.9.0
# or:
conda install -c conda-forge cudnn -y

"ABI mismatch on import cc3d"

install.py probes for this and repairs automatically. If you see it after a manual install:

pip uninstall -y connected-components-3d
pip install --no-cache-dir connected-components-3d

Cluster-specific helpers

If you are on the BC SLURM cluster, scripts/setup_slurm.sh (also exposed as just setup-slurm) detects the cluster's CUDA/cuDNN modules and patches ~/.bashrc to load them automatically. This is not part of the canonical install path; use it only if you know your cluster's module conventions.

For Docker, see docker/README.md. The canonical image is built from the repository root, pins an official CUDA-enabled PyTorch base, and runs as a non-root user whose UID/GID can be matched to the host.