How do you make Doco-CD read an .env properly without having to copy it?

[AnonymousWP]

This may sound like a dumb question, but hold on.

I have a Docker VM running in Proxmox, and I use Doco-CD, which I love to use by the way. It's just that with some applications, I notice they sometimes require .env files. So what I do is reference them with Doco-CD, no problem there. However, I obviously don't commit secrets to my (private) repository, and that's .gitignore'd as well, so what I do is create a .env.example. When I deploy a stack, .env.example is deployed, but then obviously the service fails because it needs an .env, not an .env.example. How do you do this properly? Yeah, I could use a secret manager, and I saw documentation about it, but is there another way? What I have to do now is:

1. Deploy stack with .env.example
2. Do a git pull on my server
3. Edit the .env.example with VI (in my case)
4. Copy the values to /var/lib/docker/volumes/doco-cd_data/<path-to-env-that-dococd-copied/deployed>
5. Restart the containers so it's being picked up

This all sounds too complicated, and I believe I'm doing it wrong. Doco-CD should be much easier than this, can't imagine others also do it this 'complicated'.

[qianlongzt]

I use SOPS to encrypt secrets and share the SOPS Age key with doco-cd and myself. It's not the best practice, but it's sufficient for a homelab.

Encrypt every secret file with SOPS and commit the encrypted files to the Git repository. doco-cd should start with access to the SOPS Age key so it can decrypt the secrets at runtime.

Alternatively, you can keep your .env file outside the repository and configure doco-cd to mount it. You may need to specify the environment file explicitly. In this setup, make sure to keep .env and .env.example in sync.

[kimdre]

If you use Docker swarm, you can create Docker secrets and reference them in your stacks. Otherwise I also do it like @qianlongzt

[AnonymousWP]

Thanks for the replies. It's not just for secrets, but for .env files in general. How it works now is just a hassle (not blaming Doco-CD here, because I don't know how you'd solve this): let's say you deploy Immich. That application requires a .env file. However, I don't want to commit stuff like that to the repo, so I create a .env.example and deploy that with Doco-CD. But then I still need to log onto the server to modify it (but Immich of course will fail because it only found a .env.example, not a .env. I don't use Docker Swarm.

What do you mean with keeping .env and .env.example 'in sync'?

[kimdre]

In general I just encrypt all files (not only .env files) with SOPS when they contain something I don't want to have in my repo as clear text (Secrets, personal/private data, certificares or any other sensitive data). It's very easy to also edit and view the encrypted file contents later again with SOPS too.

The only thing I have there in clear text is my AGE public key that I use to encrypt the data and I use some shell scripts in my repo to encrypt and edit the files. I also have my private AGE key there locally for later file edits, but gitignore'd so I don't commit it by accident.

encrypt.sh

#!/usr/bin/env bash

set -euo pipefail

export EDITOR=vim

PUBLIC_KEY="ageXXX"

_encrypt_completion() {
	local cur
	cur="${COMP_WORDS[COMP_CWORD]}"
	COMPREPLY=($(compgen -f -- "$cur"))
}

complete -o filenames -F _encrypt_completion encrypt.sh

if [ "$#" -ne 1 ]; then
		echo "Usage: $0 <file-to-encrypt>"
		exit 1
fi

FILE_TO_ENCRYPT="$(realpath "$1")"
TMP_ENCRYPTED_FILE="$(mktemp "${FILE_TO_ENCRYPT}.enc.XXXXXX")"

cleanup() {
	rm -f "$TMP_ENCRYPTED_FILE"
}
trap cleanup EXIT

is_top_level_yaml_sequence() {
	local first_content_line
	first_content_line="$(
		awk '
			/^[[:space:]]*($|#)/ { next }
			/^[[:space:]]*---[[:space:]]*$/ { next }
			{ print; exit }
		' "$1"
	)"

	[[ -n "$first_content_line" ]] && [[ "$first_content_line" =~ ^[[:space:]]*-[[:space:]] ]]
}

if [[ "$FILE_TO_ENCRYPT" =~ \.ya?ml$ ]] && is_top_level_yaml_sequence "$FILE_TO_ENCRYPT"; then
	sops --encrypt --input-type binary --output-type binary --age "$PUBLIC_KEY" "$FILE_TO_ENCRYPT" > "$TMP_ENCRYPTED_FILE"
else
	sops --encrypt --age "$PUBLIC_KEY" "$FILE_TO_ENCRYPT" > "$TMP_ENCRYPTED_FILE"
fi

mv "$TMP_ENCRYPTED_FILE" "$FILE_TO_ENCRYPT"

edit.sh

#!/usr/bin/env bash

set -euo pipefail

export SOPS_AGE_KEY_FILE=sops_age_key.txt
export EDITOR=vim

PUBLIC_KEY="ageXXX"

_edit_completion() {
	local cur
	cur="${COMP_WORDS[COMP_CWORD]}"
	COMPREPLY=($(compgen -f -- "$cur"))
}

complete -o filenames -F _edit_completion edit.sh

if [ "$#" -ne 1 ]; then
	echo "Usage: $0 <file-to-edit>"
	exit 1
fi

FILE_TO_EDIT="$1"
# Get the absolute path, handling non-existent files on macOS
if [ -e "$FILE_TO_EDIT" ]; then
	FILE_TO_EDIT="$(realpath "$FILE_TO_EDIT")"
else
	FILE_TO_EDIT="$(cd "$(dirname "$FILE_TO_EDIT")" && pwd)/$(basename "$FILE_TO_EDIT")"
fi

TMP_DECRYPTED_FILE="$(mktemp "${FILE_TO_EDIT}.dec.XXXXXX")"
TMP_ENCRYPTED_FILE="$(mktemp "${FILE_TO_EDIT}.enc.XXXXXX")"

cleanup() {
	rm -f "$TMP_DECRYPTED_FILE" "$TMP_ENCRYPTED_FILE"
}
trap cleanup EXIT

is_binary_sops_payload() {
	grep -Eq '"data"[[:space:]]*:[[:space:]]*"ENC\[' "$1" && grep -Eq '"sops"[[:space:]]*:' "$1"
}

if [ ! -f "$FILE_TO_EDIT" ]; then
	# File doesn't exist - open editor with the path and let user decide to save
	"$EDITOR" "$FILE_TO_EDIT"
	# Only proceed if file was actually created
	if [ -f "$FILE_TO_EDIT" ]; then
		sops --encrypt --age "$PUBLIC_KEY" "$FILE_TO_EDIT" > "$TMP_ENCRYPTED_FILE"
		mv "$TMP_ENCRYPTED_FILE" "$FILE_TO_EDIT"
	fi
elif is_binary_sops_payload "$FILE_TO_EDIT"; then
	sops --decrypt --input-type binary --output-type binary "$FILE_TO_EDIT" > "$TMP_DECRYPTED_FILE"
	"$EDITOR" "$TMP_DECRYPTED_FILE"
	sops --encrypt --input-type binary --output-type binary --age "$PUBLIC_KEY" "$TMP_DECRYPTED_FILE" > "$TMP_ENCRYPTED_FILE"
	mv "$TMP_ENCRYPTED_FILE" "$FILE_TO_EDIT"
else
	sops edit "$FILE_TO_EDIT"
fi

[AnonymousWP]

I looked into it again, and I think there's no other way than using SOPS. Seems reasonably easy to maintain/deploy/use, so I think I'll go with that. Thanks for the replies!