Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 8 additions & 13 deletions .github/workflows/backend.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,23 +21,18 @@ jobs:
- name: Checkout
uses: actions/checkout@v5

- name: Setup Python
uses: actions/setup-python@v5
- name: Setup uv
uses: astral-sh/setup-uv@v7
with:
python-version: "3.13"
version: "0.12.19"

- name: Install system dependencies
run: apt-get update && apt-get install -y --no-install-recommends ca-certificates python3

- name: Install dependencies
run: |
apt-get update && apt-get install -y curl python3.13-dev libpq-dev gcc
python -m pip install --upgrade pip
curl -sSL https://install.python-poetry.org | python3 -
export PATH="/github/home/.local/bin:$PATH"
echo "$HOME/.local/bin" >> $GITHUB_PATH # allow poetry to be used in next step
poetry env use python3.13
poetry install --no-interaction --no-root
run: uv sync --locked --no-build --no-python-downloads --python /usr/bin/python3
working-directory: ./src/backend

- name: Run tests
run: |
poetry run pytest ./tests
run: uv run --no-sync pytest ./tests
working-directory: ./src/backend
9 changes: 7 additions & 2 deletions .github/workflows/frontend.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,18 @@ jobs:
- name: Checkout
uses: actions/checkout@v5

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '26.10.0'

- name: Setup pnpm
uses: pnpm/action-setup@v4
with:
version: 8
version: 12.8.1

- name: Install dependencies
run: pnpm install
run: pnpm install --frozen-lockfile
working-directory: ./src/frontend/acc

- name: Lint
Expand Down
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ __pycache__
.idea
/.mypy_cache/
**/venv
**/.venv
.vscode
.devcontainer
build/
build/
4 changes: 2 additions & 2 deletions docs/backend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
- [services](./services/)

# Environment variables
- `ACC_DB_URL` - PostgreSQL database connection string.
- `ACC_DB_URL` - PostgreSQL database connection string using Psycopg 3: `postgresql+psycopg://user:password@host:5432/database`. Custom environment overrides must use this driver scheme too.
- `ACC_DATA_DIR` - Directory for storing uploaded files and calculation results.
- `ACC_EXAMPLES_DIR` - Directory for storing precalculated examples.
- `ACC_LOG_DIR` - Directory for storing logs.
Expand All @@ -23,4 +23,4 @@
- `OIDC_REDIRECT_URL` - Redirect URL after successful Life Science auth.
- `OIDC_DISCOVERY_URL` - URL for fetching OIDC Life Science infomation (auth endpoint, ...).
- `OIDC_CLIENT_ID` - Client ID of the registered Life Science Application.
- `OIDC_CLIENT_SECRET` - Client secret of the registered Life Science Application.
- `OIDC_CLIENT_SECRET` - Client secret of the registered Life Science Application.
4 changes: 2 additions & 2 deletions docs/backend/db/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@ Migrations are handled by `alembic`, with related code being located [here](../.
### Creating a new migration
```bash
$ cd src/backend/app
$ poetry run alembic revision --autogenerate -m "<migration name>"
$ uv run --no-sync alembic revision --autogenerate -m "<migration name>"
```

### Running migrations
```bash
$ cd src/backend/app
$ poetry run alembic upgrade head
$ uv run --no-sync alembic upgrade head
```

## Updating the database schema
Expand Down
1 change: 0 additions & 1 deletion src/backend/.python-version

This file was deleted.

22 changes: 8 additions & 14 deletions src/backend/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,27 +2,21 @@ FROM chargefw2-base:acc

USER root

ENV POETRY_HOME=/opt/poetry
ENV POETRY_VIRTUALENVS_CREATE=false
ENV PATH="/opt/poetry/bin:$PATH"
COPY --from=ghcr.io/astral-sh/uv:0.12.19 /uv /usr/local/bin/uv

ENV UV_PYTHON_DOWNLOADS=never
ENV UV_PROJECT_ENVIRONMENT=/opt/venv
ENV PATH="/opt/venv/bin:$PATH"

RUN apt-get update && \
apt-get install -y --no-install-recommends \
ca-certificates \
curl \
gcc \
python3-dev \
libpq-dev && \
python3 && \
apt-get clean && rm -rf /var/lib/apt/lists/*

# Install Poetry
RUN curl -sSL https://install.python-poetry.org | python3 - && \
poetry config virtualenvs.create false


WORKDIR /acc
COPY poetry.lock pyproject.toml ./
RUN poetry install --no-interaction --no-ansi --no-root
COPY uv.lock pyproject.toml ./
RUN uv sync --locked --no-cache --no-build --python /usr/bin/python3

COPY ./app /acc/app
COPY ./entrypoint.sh /acc/
Expand Down
25 changes: 17 additions & 8 deletions src/backend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,20 +26,28 @@ $ cmake .. -DCMAKE_INSTALL_PREFIX=<WHERE-TO-INSTALL> -DPYTHON_MODULE=ON
All required environment variables are located in the [.env file](./app/.env) except `OIDC_CLIENT_ID` and `OIDC_CLIENT_SECRET` (required for Life Science auth, can be ignored for local setup). All used environment variables are described in [docs](../../docs/backend/README.md). How to obtain the abovementioned environment variables is also mentioned [here](../../docs/backend/life-science/README.md).

### Installing dependencies
ACC III uses [Poetry](https://python-poetry.org/) for depencency management.
ACC III uses [uv](https://docs.astral.sh/uv/) for dependency management. Docker and CI use uv 0.12.19.

#### Install Poetry
#### Install uv
```bash
$ curl -sSL https://install.python-poetry.org | python3 -
$ curl -LsSf https://astral.sh/uv/0.12.19/install.sh | sh
```

#### Install project dependencies
*Note:* Poetry will automatically create a virtual environment inside the project before the installation. Configured [here](./poetry.toml).
uv creates a `.venv` in the backend directory and installs the versions recorded in `uv.lock`. Use Python 3.13 or newer, matching the Python version used to build your ChargeFW2 bindings. Docker uses Ubuntu's Python and does not download another interpreter.

```bash
$ poetry install
$ uv sync --locked --no-python-downloads --python python3
```

For API tests using Starlette's `TestClient`, enable the optional `api-test` group:

```bash
$ uv sync --locked --group api-test --no-python-downloads --python python3
```

This supplies `httpx2` for `TestClient` without changing the application's `httpx` client. The group is not installed in production images. Use `uv run --no-sync` after syncing to retain the optional test dependencies.

### Startup
We firstly need to start the database. Easiest way is by using an official postgresql docker image. Connection string is located in the `.env` file.
```bash
Expand All @@ -53,13 +61,14 @@ $ cd app

After the database is ready, we need to run migrations:
```bash
$ poetry run alembic upgrade head
$ uv run --no-sync alembic upgrade head
```

API can now be started just by running the main file:
Prepare the shared example files once, then start the API workers. Run these commands in order:

```bash
$ poetry run gunicorn --workers 4 --worker-class uvicorn.workers.UvicornWorker main:web_app
$ uv run --no-sync python main.py
$ uv run --no-sync gunicorn --workers 4 --worker-class uvicorn_worker.UvicornWorker main:web_app
```

API runs by default on `--bind 127.0.0.1:8000`. Documentation (Swagger) is available on `/docs`. Alternatively you can use Redoc available on `/redoc`.
Expand Down
2 changes: 1 addition & 1 deletion src/backend/app/.env
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
PYTHONPATH="/opt/chargefw2/lib"

ACC_DB_URL="postgresql://postgres:postgres@localhost:5432/postgres"
ACC_DB_URL="postgresql+psycopg://postgres:postgres@localhost:5432/postgres"
ACC_DATA_DIR="/home/acc/data"
ACC_EXAMPLES_DIR="/home/acc/examples"
ACC_LOG_DIR="/var/log/acc"
Expand Down
8 changes: 4 additions & 4 deletions src/backend/app/api/v1/routes/charges.py
Original file line number Diff line number Diff line change
Expand Up @@ -386,7 +386,7 @@ async def calculate_charges(
if available_b <= 0:
quota_mb = quota_b / 1024 / 1024
raise BadRequestError(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail="Unable to calculate charges. Quota exceeded. "
+ f"Maximum storage space is {quota_mb} MB.",
)
Expand Down Expand Up @@ -440,7 +440,7 @@ async def calculate_charges(
if total_size > io_service.max_file_size:
max_file_size_mb = io_service.max_file_size / 1024 / 1024
raise BadRequestError(
status_code=status.HTTP_413_REQUEST_ENTITY_TOO_LARGE,
status_code=status.HTTP_413_CONTENT_TOO_LARGE,
detail="Unable to calculate charges. Calculation is too large. "
+ f"Maximum allowed size is {max_file_size_mb} MB.",
)
Expand Down Expand Up @@ -576,7 +576,7 @@ async def setup(
if available_b <= 0:
quota_mb = quota_b / 1024 / 1024
raise BadRequestError(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail="Unable to set up calculation. Quota would be exceeded. "
+ f"Maximum storage space is {quota_mb} MB.",
)
Expand All @@ -589,7 +589,7 @@ async def setup(
if total_size > io_service.max_file_size:
max_file_size_mb = io_service.max_file_size / 1024 / 1024
raise BadRequestError(
status_code=status.HTTP_413_REQUEST_ENTITY_TOO_LARGE,
status_code=status.HTTP_413_CONTENT_TOO_LARGE,
detail="Unable to set up calculation. Calculation is too large. "
+ f"Maximum allowed size is {max_file_size_mb} MB.",
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ def store(self, session: Session, calculation_set: CalculationSet) -> None:
def _paginate(
self, session: Session, statement: Select, page: int, page_size: int
) -> PagedList[CalculationSet]:
total_statement = select(func.count()).select_from(statement)
total_statement = select(func.count()).select_from(statement.subquery())
items_statement = statement.limit(page_size).offset((page - 1) * page_size)

total_count = (session.execute(total_statement)).unique().scalar()
Expand Down
10 changes: 8 additions & 2 deletions src/backend/app/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@ def move_example_files() -> None:
def create_app() -> FastAPI:
"""Creates FastAPI apps with routers and middleware."""

move_example_files()
if not load_dotenv():
raise EnvironmentError("Could not load environment variables.")

# Create DI container
container = Container()
Expand All @@ -49,6 +50,8 @@ def create_app() -> FastAPI:
title="Atomic Charge Calculator III API",
root_path="/api",
swagger_ui_parameters={"syntaxHighlight": False},
# Preserve the existing logging setup without automatic telemetry exporters.
telemetry={"tracing": False, "metrics": False, "logs": False, "auto_configure": False},
)

container.wire()
Expand Down Expand Up @@ -78,4 +81,7 @@ def create_app() -> FastAPI:
return app


web_app = create_app()
if __name__ == "__main__":
move_example_files()
else:
web_app = create_app()
31 changes: 0 additions & 31 deletions src/backend/app/services/calculation_storage.py
Original file line number Diff line number Diff line change
Expand Up @@ -271,37 +271,6 @@ def filter_existing_calculations(
self.logger.error(f"Error filtering existing calculations: {traceback.format_exc()}")
raise e

def get_calculation_results(self, computation_id: str) -> list[CalculationResultDto]:
"""Get calculation results from database."""

try:
self.logger.info(f"Getting calculation results for computation {computation_id}.")
with self.session_manager.session() as session:
calculation_set = self.set_repository.get(session, computation_id)

if not calculation_set:
return []

result = [
CalculationResultDto(
config=CalculationConfigDto.model_validate(config),
calculations=[
CalculationDto.model_validate(calculation)
for calculation in calculation_set.calculations
if calculation.config == config
],
)
for config in calculation_set.configs
]

return result

except Exception as e:
self.logger.error(
f"Error getting calculation results for computation {computation_id}: {traceback.format_exc()}"
)
raise e

def delete_calculation_set(self, computation_id: str) -> None:
"""Delete calculation set from database."""

Expand Down
6 changes: 3 additions & 3 deletions src/backend/app/services/io.py
Original file line number Diff line number Diff line change
Expand Up @@ -602,7 +602,7 @@ def ensure_upload_files_sizes_valid(self, files: list[UploadFile]) -> None:
if any((file.size or 0) > self.max_file_size for file in files):
max_file_size_mb = self.max_file_size / 1024 / 1024
raise BadRequestError(
status_code=status.HTTP_413_REQUEST_ENTITY_TOO_LARGE,
status_code=status.HTTP_413_CONTENT_TOO_LARGE,
detail="Unable to upload files. One or more files are too large. "
+ f"Maximum allowed size is {max_file_size_mb} MB.",
)
Expand All @@ -612,7 +612,7 @@ def ensure_upload_files_sizes_valid(self, files: list[UploadFile]) -> None:
if upload_size_b > self.max_upload_size:
max_upload_size_mb = self.max_upload_size / 1024 / 1024
raise BadRequestError(
status_code=status.HTTP_413_REQUEST_ENTITY_TOO_LARGE,
status_code=status.HTTP_413_CONTENT_TOO_LARGE,
detail="Unable to upload files. Maximum upload size exceeded. "
+ f"Maximum upload size is {max_upload_size_mb} MB.",
)
Expand All @@ -633,7 +633,7 @@ def ensure_quota_not_exceeded(
if upload_size_b > available_b:
quota_mb = quota_b / 1024 / 1024
raise BadRequestError(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail="Unable to upload files. Quota exceeded. "
+ f"Maximum storage space is {quota_mb} MB.",
)
Expand Down
10 changes: 4 additions & 6 deletions src/backend/app/services/oidc.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,10 @@
import os

import httpx

import jwt
from cachetools import TTLCache
from dotenv import load_dotenv

from jose import JWTError, jwt

from jwt.exceptions import PyJWTError
from services.logging.base import LoggerBase


Expand Down Expand Up @@ -102,13 +100,13 @@ async def verify_token(self, token: str) -> dict | None:

payload = jwt.decode(
token,
rsa_key,
jwt.PyJWK.from_dict(rsa_key, algorithm="RS256").key,
algorithms=["RS256"],
audience=self.client_id,
issuer=config["issuer"],
)
return payload
except JWTError as e:
except PyJWTError as e:
self.logger.warn(f"Invalid token: {str(e)}")
return None
except Exception as e:
Expand Down
7 changes: 5 additions & 2 deletions src/backend/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,11 @@ cd /acc/app
# run database migrations
alembic upgrade head

# Prepare shared examples once, before workers import the application.
python main.py || exit $?

# use gunicorn with 4 workers
# uvicorn.workers.UvicornWorker is used for async processing
# uvicorn_worker.UvicornWorker is used for async processing
# setting timeout to 1 hour for long lasting computations
# running on port 8000
exec gunicorn --workers 4 --worker-class uvicorn.workers.UvicornWorker --timeout 6000 --bind 0.0.0.0:8000 main:web_app
exec gunicorn --workers 4 --worker-class uvicorn_worker.UvicornWorker --timeout 6000 --bind 0.0.0.0:8000 main:web_app
Loading
Loading