Skip to content

Commit 2643987

Browse files
ClaptarClaptar
authored
Merge pull request #1 from cellgeni/dev
Subset command and tests
2 parents bb945d7 + 6b2a250 commit 2643987

20 files changed

Lines changed: 2550 additions & 218 deletions

‎.github/workflows/quay-on-tag.yml‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
name: Build & Push Docker image (tag)
2+
3+
on:
4+
push:
5+
tags:
6+
- "*"
7+
8+
concurrency:
9+
group: docker-${{ github.ref }}
10+
cancel-in-progress: true
11+
12+
jobs:
13+
docker:
14+
runs-on: ubuntu-latest
15+
permissions:
16+
contents: read
17+
18+
steps:
19+
- uses: actions/checkout@v4
20+
21+
- uses: docker/setup-buildx-action@v3
22+
23+
- name: Login to Quay
24+
uses: docker/login-action@v3
25+
with:
26+
registry: quay.io
27+
username: ${{ secrets.QUAY_USERNAME }}
28+
password: ${{ secrets.QUAY_TOKEN }}
29+
30+
- name: Docker metadata
31+
id: meta
32+
uses: docker/metadata-action@v5
33+
with:
34+
images: quay.io/cellgeni/h5ad-cli
35+
tags: |
36+
type=ref,event=tag
37+
# push "latest" only for tags WITHOUT hyphens (excludes pre-releases like 1.0.0-beta):
38+
type=raw,value=latest,enable=${{ !contains(github.ref_name, '-') }}
39+
40+
- name: Build and push
41+
uses: docker/build-push-action@v6
42+
with:
43+
context: .
44+
file: Dockerfile
45+
push: true
46+
tags: ${{ steps.meta.outputs.tags }}
47+
labels: ${{ steps.meta.outputs.labels }}
48+
cache-from: type=gha
49+
cache-to: type=gha,mode=max

‎.github/workflows/tests.yml‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
name: Tests
2+
3+
on:
4+
push:
5+
branches: [main, dev]
6+
pull_request:
7+
branches: [main, dev]
8+
9+
concurrency:
10+
group: tests-${{ github.workflow }}-${{ github.ref }}
11+
cancel-in-progress: true
12+
13+
jobs:
14+
test:
15+
runs-on: ubuntu-latest
16+
timeout-minutes: 20
17+
18+
permissions:
19+
contents: read
20+
checks: write # needed for EnricoMi/publish-unit-test-result-action on PRs
21+
22+
strategy:
23+
fail-fast: false
24+
matrix:
25+
python-version: ["3.12"] # add "3.13" if you want
26+
27+
steps:
28+
- uses: actions/checkout@v4
29+
30+
- uses: actions/setup-python@v5
31+
with:
32+
python-version: ${{ matrix.python-version }}
33+
34+
# Prefer an action over curl|sh
35+
- name: Set up uv
36+
uses: astral-sh/setup-uv@v3
37+
with:
38+
enable-cache: true
39+
40+
- name: Install dependencies (frozen)
41+
run: uv sync --extra dev --frozen
42+
43+
- name: Run tests with coverage
44+
run: |
45+
uv run pytest -v \
46+
--cov=h5ad \
47+
--cov-report=term-missing \
48+
--cov-report=xml \
49+
--cov-report=html \
50+
--junitxml=pytest-results.xml
51+
52+
- name: Publish test results summary
53+
uses: EnricoMi/publish-unit-test-result-action@v2
54+
if: always()
55+
with:
56+
files: pytest-results.xml
57+
check_name: Test Results
58+
59+
- name: Upload coverage artifacts
60+
uses: actions/upload-artifact@v4
61+
if: always()
62+
with:
63+
name: coverage
64+
path: |
65+
coverage.xml
66+
htmlcov/
67+
pytest-results.xml
68+
retention-days: 30
69+
70+
- name: Upload coverage to Codecov
71+
uses: codecov/codecov-action@v4
72+
if: always()
73+
with:
74+
files: coverage.xml
75+
fail_ci_if_error: false

‎Dockerfile‎

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,10 @@
11
# Base image: Python 3.12 + uv preinstalled (Debian slim)
22
FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim
33

4+
# Environment variables
5+
ENV UV_NO_DEV=1
6+
ENV VENV=/env
7+
48
# Work directory inside the container
59
WORKDIR /cli
610

@@ -9,19 +13,22 @@ RUN apt-get update \
913
&& apt-get install -y --no-install-recommends git \
1014
&& rm -rf /var/lib/apt/lists/*
1115

12-
# Use only production deps (no dev extras if you add them later)
13-
ENV UV_NO_DEV=1
1416

15-
# Clone the repo into /app
16-
RUN git clone --branch 0.1.0 https://github.com/cellgeni/h5ad-cli.git .
17+
# Copy the project files (from GitHub Actions checkout context)
18+
COPY pyproject.toml uv.lock ./
19+
COPY src/ ./src/
1720

1821

1922
# Install the project according to pyproject.toml + uv.lock
2023
# --locked asserts that uv.lock is in sync with pyproject.toml
2124
RUN uv sync --locked
2225

26+
# Create separate venv for csvkit to avoid dependency conflicts
27+
RUN uv venv $VENV --python 3.12 && \
28+
uv pip install --python $VENV/bin/python csvkit
29+
2330
# Put the project venv on PATH so `h5ad` is directly runnable
24-
ENV PATH="/cli/.venv/bin:${PATH}"
31+
ENV PATH="/cli/.venv/bin:${VENV}/bin:${PATH}"
2532

2633
# Default entrypoint: run the CLI
2734
ENTRYPOINT ["h5ad"]

‎README.md‎

Lines changed: 34 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,32 @@
1-
## h5ad CLI
1+
# h5ad CLI
22

3-
h5ad is a Typer-powered command-line tool for exploring huge `.h5ad` (AnnData) files without loading them fully into memory. It streams data directly from disk so you can inspect structure, metadata, and matrices on demand.
3+
A command-line tool for exploring huge `.h5ad` (AnnData) files without loading them fully into memory. Streams data directly from disk for efficient inspection of structure, metadata, and matrices.
44

5-
### Features
6-
- **`info`** – Reports `n_obs × n_var` dimensions and highlights top-level groups with their child keys.
7-
- **`table`** – Exports obs or var metadata tables to CSV in memory-efficient chunks.
8-
- Colored Rich output for clearer, more informative summaries.
5+
## Features
96

10-
### Usage
11-
Install dependencies with `uv sync`, then invoke any subcommand via `uv run h5ad ...`:
7+
- **`info`** – Show file structure and dimensions (`n_obs × n_var`)
8+
- **`table`** – Export obs/var metadata to CSV with chunked streaming
9+
- **`subset`** – Filter h5ad files by cell/gene names (supports dense and sparse CSR/CSC matrices)
10+
- Memory-efficient chunked processing for large files
11+
- Rich terminal output with colors and progress bars
12+
13+
## Installation
1214

1315
```bash
1416
uv sync
17+
```
18+
19+
For development and testing:
20+
```bash
21+
uv sync --extra dev
22+
```
23+
24+
See [docs/TESTING.md](docs/TESTING.md) for testing documentation.
25+
26+
## Usage
27+
Invoke any subcommand via `uv run h5ad ...`:
28+
29+
```bash
1530
uv run h5ad --help
1631
```
1732

@@ -37,4 +52,14 @@ uv run h5ad table data.h5ad --axis obs --cols cell_type,donor
3752
uv run h5ad table data.h5ad --axis var --chunk-rows 5000 --out var_metadata.csv
3853
```
3954

40-
Each command streams from disk, so even multi-GB `.h5ad` files remain responsive.
55+
**Subset by cell names:**
56+
```bash
57+
uv run h5ad subset input.h5ad output.h5ad --obs cells.txt
58+
```
59+
60+
**Subset by both cells and genes:**
61+
```bash
62+
uv run h5ad subset input.h5ad output.h5ad --obs cells.txt --var genes.txt
63+
```
64+
65+
All commands stream from disk, so even multi-GB `.h5ad` files remain responsive.

‎docs/TESTING.md‎

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
# H5AD Testing Documentation
2+
3+
## Quick Start
4+
5+
Install development dependencies:
6+
```bash
7+
uv sync --extra dev
8+
```
9+
10+
Run all tests:
11+
```bash
12+
uv run pytest
13+
```
14+
15+
Run specific test file:
16+
```bash
17+
uv run pytest tests/test_subset.py
18+
```
19+
20+
Run specific test class:
21+
```bash
22+
uv run pytest tests/test_subset.py::TestSubsetH5ad
23+
```
24+
25+
Run specific test function:
26+
```bash
27+
uv run pytest tests/test_subset.py::TestSubsetH5ad::test_subset_h5ad_obs_only
28+
```
29+
30+
Run with coverage:
31+
```bash
32+
uv run pytest --cov=h5ad --cov-report=html
33+
```
34+
35+
Run verbose:
36+
```bash
37+
uv run pytest -v
38+
```
39+
40+
Run and show print statements:
41+
```bash
42+
uv run pytest -s
43+
```
44+
45+
Run with short tracebacks:
46+
```bash
47+
uv run pytest --tb=short
48+
```
49+
50+
## Test Structure
51+
52+
- **`conftest.py`** - Pytest fixtures for test h5ad files (dense, CSR sparse, CSC sparse, categorical)
53+
- **`test_info_read.py`** - Tests for info.py and read.py utility functions (58 tests total)
54+
- Axis length calculation
55+
- String decoding (bytes/unicode/object arrays)
56+
- Categorical column reading with caching
57+
- Column chunk reading
58+
- **`test_subset.py`** - Tests for subsetting operations
59+
- Name file reading and parsing
60+
- Index matching with chunked processing
61+
- Axis group subsetting (including categorical columns)
62+
- Dense matrix subsetting (chunked)
63+
- Sparse matrix subsetting (CSR and CSC formats)
64+
- Integration tests for complete subset workflows
65+
- **`test_cli.py`** - CLI command integration tests
66+
- `info` command (file structure inspection)
67+
- `table` command (CSV export with filters)
68+
- `subset` command (file subsetting)
69+
- Error handling and validation
70+
71+
## Test Coverage
72+
73+
The test suite provides comprehensive coverage:
74+
75+
- ✅ **CLI Commands**: All commands (info, table, subset) with various options
76+
- ✅ **Dense Matrices**: Chunked reading and subsetting
77+
- ✅ **Sparse Matrices**: CSR and CSC format support
78+
- ✅ **Categorical Data**: Proper handling of categorical columns
79+
- ✅ **Chunked Processing**: Memory-efficient operations on large datasets
80+
- ✅ **Error Handling**: Invalid inputs, missing files, malformed data
81+
- ✅ **Edge Cases**: Empty results, missing names, single row/column
82+
83+
**Current Status**: 58 tests, all passing ✅
84+
85+
## Test Fixtures
86+
87+
### Available Fixtures
88+
89+
- `temp_dir`: Temporary directory for test files
90+
- `sample_h5ad_file`: Basic h5ad file with dense matrix
91+
- `sample_sparse_csr_h5ad`: H5ad file with CSR sparse matrix
92+
- `sample_sparse_csc_h5ad`: H5ad file with CSC sparse matrix
93+
- `sample_categorical_h5ad`: H5ad file with categorical columns
94+
95+
## Coverage Reports
96+
97+
Generate terminal coverage report:
98+
```bash
99+
uv run pytest --cov=h5ad --cov-report=term-missing
100+
```
101+
102+
Generate HTML coverage report:
103+
```bash
104+
uv run pytest --cov=h5ad --cov-report=html
105+
```
106+
107+
View HTML report:
108+
```bash
109+
open htmlcov/index.html # macOS
110+
xdg-open htmlcov/index.html # Linux
111+
```
112+
113+
Coverage configuration is in `pyproject.toml` under `[tool.coverage.*]`.

‎pyproject.toml‎

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,37 @@
11
[project]
22
name = "h5ad"
3-
version = "0.1.0"
3+
version = "0.2.0"
44
description = "Streaming CLI utilities for exploring large AnnData .h5ad files"
55
readme = "README.md"
66
requires-python = ">=3.12"
77
dependencies = [
88
"h5py>=3.15.1",
9+
"numpy>=2.3.5",
10+
"rich>=14.2.0",
911
"typer>=0.20.0",
1012
]
1113

14+
[project.optional-dependencies]
15+
dev = [
16+
"pytest>=8.3.4",
17+
"pytest-cov>=6.0.0",
18+
]
19+
1220
[build-system]
1321
requires = ["uv_build>=0.8.0,<0.9.0"]
1422
build-backend = "uv_build"
1523

1624
[project.scripts]
17-
h5ad = "h5ad.cli:main"
25+
h5ad = "h5ad.cli:main"
26+
27+
[tool.coverage.run]
28+
source = ["src/h5ad"]
29+
omit = [
30+
"*/tests/*",
31+
"*/__pycache__/*",
32+
]
33+
34+
[tool.coverage.report]
35+
precision = 2
36+
show_missing = true
37+
skip_covered = false

‎pytest.ini‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
[pytest]
2+
testpaths = tests
3+
python_files = test_*.py
4+
python_classes = Test*
5+
python_functions = test_*
6+
addopts = -v --strict-markers --tb=short
7+
markers =
8+
slow: marks tests as slow (deselect with '-m "not slow"')
9+
integration: marks tests as integration tests

0 commit comments

Comments
 (0)