Skip to content

Commit da870b1

Browse files
Amir-GolmoradiAmir-Golmoradi
authored
Replace infrastructure scripts with Vault Agent, Ansible, and Flyway (#40)
* Replace infrastructure scripts with Vault Agent and native tools * Keep Flyway test migrations in the payguard schema
1 parent c904d71 commit da870b1

110 files changed

Lines changed: 5416 additions & 2496 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.dockerignore‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,17 @@
99
.local
1010
**/target
1111
**/__pycache__
12+
13+
# Only the four packaged service JARs enter application build contexts.
14+
!payguard-identity-service/target/
15+
payguard-identity-service/target/*
16+
!payguard-identity-service/target/payguard-identity-service-1.0.0-SNAPSHOT.jar
17+
!payguard-wallet-service/target/
18+
payguard-wallet-service/target/*
19+
!payguard-wallet-service/target/payguard-wallet-service-1.0.0-SNAPSHOT.jar
20+
!payguard-loan-service/target/
21+
payguard-loan-service/target/*
22+
!payguard-loan-service/target/payguard-loan-service-1.0.0-SNAPSHOT.jar
23+
!payguard-collateral-service/target/
24+
payguard-collateral-service/target/*
25+
!payguard-collateral-service/target/payguard-collateral-service-1.0.0-SNAPSHOT.jar

‎.env.example‎

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,19 @@
1-
# Secrets are generated into ignored protected files by scripts/local/prepare.py.
2-
# No runtime database passwords or HMAC keys belong here.
1+
# Non-secret deployment identity. Never put passwords or tokens here.
32
COMPOSE_PROJECT_NAME=payguard
43
VAULT_PORT=8200
5-
KAFKA_PORT=9092
64
PAYGUARD_LOCAL_UID=1000
75
PAYGUARD_LOCAL_GID=1000
6+
PAYGUARD_DOCKER_GID=1000
7+
# Set to the SAME absolute path on the host and operations container.
8+
PAYGUARD_REPO_ROOT=/workspace/payguard
9+
PAYGUARD_RUNTIME_ROOT=/dev/shm/payguard
10+
# PostgreSQL 18 image default. Inspect existing PG_VERSION locations before adoption.
11+
PAYGUARD_PGDATA=/var/lib/postgresql/18/docker
12+
# Existing volume names are explicit and external; they are never auto-created on startup.
13+
PAYGUARD_POSTGRES_IDENTITY_VOLUME=payguard_postgres_identity_data
14+
PAYGUARD_POSTGRES_WALLET_VOLUME=payguard_postgres_wallet_data
15+
PAYGUARD_POSTGRES_LOAN_VOLUME=payguard_postgres_loan_data
16+
PAYGUARD_POSTGRES_COLLATERAL_VOLUME=payguard_postgres_collateral_data
17+
PAYGUARD_VAULT_DATA_VOLUME=payguard_vault_data
18+
PAYGUARD_VAULT_AUDIT_VOLUME=payguard_vault_audit
19+
PAYGUARD_KAFKA_DATA_VOLUME=payguard_kafka_data

‎.github/workflows/_database-security.yml‎

Lines changed: 60 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,55 +1,80 @@
1-
name: PostgreSQL and Vault verification
2-
1+
name: Native infrastructure verification
32
on:
43
workflow_call:
5-
64
permissions:
75
contents: read
8-
96
jobs:
10-
real-dependencies:
7+
isolated-dependencies:
118
runs-on: ubuntu-latest
12-
timeout-minutes: 25
9+
timeout-minutes: 35
10+
env:
11+
COMPOSE_PROJECT_NAME: payguard-it-ci
12+
PAYGUARD_RUNTIME_ROOT: /dev/shm/payguard-it-ci
13+
PAYGUARD_POSTGRES_IDENTITY_VOLUME: payguard-it-ci_identity
14+
PAYGUARD_POSTGRES_WALLET_VOLUME: payguard-it-ci_wallet
15+
PAYGUARD_POSTGRES_LOAN_VOLUME: payguard-it-ci_loan
16+
PAYGUARD_POSTGRES_COLLATERAL_VOLUME: payguard-it-ci_collateral
17+
PAYGUARD_VAULT_DATA_VOLUME: payguard-it-ci_vault
18+
PAYGUARD_VAULT_AUDIT_VOLUME: payguard-it-ci_audit
19+
PAYGUARD_KAFKA_DATA_VOLUME: payguard-it-ci_kafka
1320
steps:
1421
- uses: actions/checkout@v7
1522
- uses: actions/setup-java@v6
1623
with:
1724
distribution: temurin
1825
java-version: '25'
1926
cache: maven
20-
- name: PostgreSQL migrations and competing transactions
21-
run: >-
22-
./mvnw -B -ntp -pl payguard-identity-service,payguard-loan-service,payguard-collateral-service -am
23-
-Dtest=PostgreSqlTlsTest,PostgresqlMigrationRehearsalTest,SanitizedDependencyErrorTest,OtpInfrastructureIntegrationTest,OtpApplicationContractsTest,LoanServiceApplicationTests,CollateralServiceApplicationTests
24-
-Dsurefire.failIfNoSpecifiedTests=false clean test
25-
- name: Build protected migration and integration runners
26-
run: >-
27-
./mvnw -B -ntp -pl payguard-identity-service,payguard-wallet-service,payguard-loan-service,payguard-collateral-service -am -DskipTests package dependency:copy-dependencies -DincludeScope=test
28-
- name: Verify synthetic export and approved identifier mapping
29-
run: python3 -m unittest discover -s scripts/migration -p 'test_*.py'
30-
- name: Prepare isolated TLS infrastructure
27+
- name: Install native Ansible dependencies and offline test dependencies
3128
run: |
32-
python3 scripts/local/prepare.py
33-
docker compose --env-file .env.example up -d --wait postgres-wallet postgres-loan postgres-collateral postgres-identity vault
34-
python3 scripts/local/vault_bootstrap.py
35-
- name: Apply separately privileged Flyway migrations
29+
python3 -m venv /tmp/payguard-ansible
30+
/tmp/payguard-ansible/bin/pip install -r infrastructure/operations/requirements.txt -r infrastructure/tests-requirements.txt
31+
/tmp/payguard-ansible/bin/ansible-galaxy collection install -r infrastructure/operations/requirements.yml
32+
echo /tmp/payguard-ansible/bin >> "$GITHUB_PATH"
33+
echo "PAYGUARD_REPO_ROOT=$GITHUB_WORKSPACE" >> "$GITHUB_ENV"
34+
echo "PAYGUARD_LOCAL_UID=$(id -u)" >> "$GITHUB_ENV"
35+
echo "PAYGUARD_LOCAL_GID=$(id -g)" >> "$GITHUB_ENV"
36+
echo "PAYGUARD_DOCKER_GID=$(stat -c %g /var/run/docker.sock)" >> "$GITHUB_ENV"
37+
mkdir -m 700 "$PAYGUARD_RUNTIME_ROOT"
38+
- name: Verify source contracts and build applications
3639
run: |
37-
python3 scripts/local/migrate.py identity
38-
python3 scripts/local/migrate.py loan
39-
python3 scripts/local/migrate.py collateral
40-
- name: Verify actual pools, scoped secrets and Vault audit
41-
run: python3 scripts/local/verify_vault.py
42-
- name: Boot actual packaged services with scoped credentials
43-
run: python3 scripts/local/verify_applications.py
44-
- name: Preserve test reports
40+
python3 -m unittest discover -s infrastructure/tests -v
41+
./mvnw -B -ntp -pl payguard-identity-service,payguard-wallet-service,payguard-loan-service,payguard-collateral-service -am test package
42+
- name: Prepare isolated fixture and allocate only new explicit volumes
43+
run: |
44+
ansible-playbook -i localhost, -c local infrastructure/operations/prepare.yml -e fresh_target=true
45+
ansible-playbook -i localhost, -c local infrastructure/operations/allocate.yml -e fresh_target=true
46+
docker compose -f docker-compose.yaml -f infrastructure/compose.operations.yaml --profile operations build operations
47+
docker compose up -d --wait postgres-identity postgres-wallet postgres-loan postgres-collateral
48+
docker compose up -d vault
49+
- name: Initialize and provision the new fixture through Ansible
50+
run: |
51+
docker compose -f docker-compose.yaml -f infrastructure/compose.operations.yaml run --rm --no-deps operations infrastructure/operations/vault-lifecycle.yml -e operation=initialize -e fresh_target=true -e local_rehearsal=true
52+
docker compose -f docker-compose.yaml -f infrastructure/compose.operations.yaml run --rm --no-deps operations infrastructure/operations/site.yml -e fresh_target=true
53+
docker compose -f docker-compose.yaml -f infrastructure/compose.operations.yaml up -d agent-operator
54+
- name: Run all four CLI migrations twice
55+
run: |
56+
for pass in 1 2; do
57+
for service in identity wallet loan collateral; do
58+
docker compose -f docker-compose.yaml -f infrastructure/compose.operations.yaml run --rm --no-deps operations infrastructure/operations/migrate.yml -e service="$service"
59+
done
60+
done
61+
- name: Verify privilege boundaries and native Agent security
62+
run: |
63+
docker compose -f docker-compose.yaml -f infrastructure/compose.operations.yaml run --rm --no-deps operations infrastructure/operations/tests/integration.yml
64+
docker compose -f docker-compose.yaml -f infrastructure/compose.operations.yaml run --rm --no-deps operations infrastructure/operations/tests/agent.yml -e isolated_target=true
65+
- name: Boot and verify actual packaged applications
66+
run: |
67+
docker compose --profile applications up -d --build
68+
docker compose -f docker-compose.yaml -f infrastructure/compose.operations.yaml run --rm --no-deps operations infrastructure/operations/tests/applications.yml
69+
- name: Rehearse an isolated Vault outage and recovery
70+
run: docker compose -f docker-compose.yaml -f infrastructure/compose.operations.yaml run --rm --no-deps operations infrastructure/operations/tests/recovery.yml
71+
- name: Preserve Java test reports
4572
if: always()
4673
uses: actions/upload-artifact@v7
4774
with:
48-
name: database-security-reports
49-
path: |
50-
**/target/surefire-reports/TEST-*.xml
51-
payguard-shared/target/vault-test-reports/*.xml
75+
name: native-infrastructure-java-reports
76+
path: '**/target/surefire-reports/TEST-*.xml'
5277
if-no-files-found: warn
53-
- name: Stop isolated infrastructure
78+
- name: Stop only this isolated fixture
5479
if: always()
55-
run: docker compose --env-file .env.example stop
80+
run: docker compose -f docker-compose.yaml -f infrastructure/compose.operations.yaml --profile applications --profile operations stop

‎README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -103,15 +103,15 @@ Domain rules remain in the domain, use-case coordination in application code, an
103103

104104
## Build and local rehearsal
105105

106-
Use JDK **25**, Docker/Compose, Python 3 and OpenSSL. The Maven wrapper is included. Database integration tests start real PostgreSQL containers; they do not require an unset opt-in flag.
106+
Use JDK **25**, Docker/Compose, Ansible and OpenSSL. The Maven wrapper is included. Database integration tests start real PostgreSQL containers; they do not require an unset opt-in flag.
107107

108108
```bash
109109
./mvnw -B -ntp clean verify
110110
```
111111

112-
The committed Identity integration passed full Maven verification with 91 tests and zero failures, errors or skips, including the existing domain regressions and real PostgreSQL tests. Formatting and static-analysis checks also passed. See [the publication repair log](docs/tasks/identity-publication-repair.md) for the exact source revision and evidence.
112+
Historical Identity verification is recorded in [the publication repair log](docs/tasks/identity-publication-repair.md). It does not validate the new Vault Agent lifecycle; see [current infrastructure validation](docs/infrastructure-validation.md).
113113

114-
For the isolated, persistent TLS PostgreSQL/Vault rehearsal, follow [the implementation guide](docs/guides/postgresql-vault-migration.md). It gives the exact preparation, initialization, migration and test commands. Generated credentials, certificates and unseal material live under ignored `.local/payguard` with protected permissions. `.env.example` contains only public settings. Never copy secrets into an image or publish that directory.
114+
For the Compose, Vault Agent, Ansible and Flyway CLI deployment, follow [the infrastructure runbook](docs/infrastructure.md). It gives the exact preparation, initialization, migration and test commands. Generated credentials, certificates and unseal material live under ignored `.local/payguard` with protected permissions. `.env.example` contains only public settings. Never copy secrets into an image or publish that directory.
115115

116116
Runtime Flyway is disabled: apply migrations using the migration identity before starting applications. The `applications` Compose profile is separate from the database/Vault infrastructure. Provider delivery, customer enrollment, external authentication, production certificate issuance, HA/auto-unseal and durable event delivery are not implemented by this migration.
117117

0 commit comments

Comments
 (0)