Add documentation for managing encrypted secrets with sops and age
This commit is contained in:
parent
908c90ff49
commit
f7d8bec945
GPG key ID:
087F0A95E5766D72
1 changed files with 212 additions and 0 deletions
212
secrets.md
Normal file
212
secrets.md
Normal file
|
|
@ -0,0 +1,212 @@
|
||||||
|
# Secrets — managing encrypted secrets with sops (age)
|
||||||
|
|
||||||
|
This document explains how we manage secrets for the PRG server configuration using sops and the age encryption protocol. It keeps the original workflow and file layout but presents it in a more structured, readable format with examples and useful tips.
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> This file documents repository-side secrets management only. Do not paste private keys or unencrypted secrets into the repository. Keep private key material on the host machines or in a secure vault.
|
||||||
|
|
||||||
|
|
||||||
|
## Quick overview
|
||||||
|
|
||||||
|
We encrypt secrets with sops using the age tool. The repository contains a `.sops.yaml` that documents which public recipients can decrypt which files (via creation_rules). Encrypted files are safe to commit to git; only machines with the corresponding private age keys can decrypt them.
|
||||||
|
|
||||||
|
|
||||||
|
## Files and conventions (what to look for)
|
||||||
|
|
||||||
|
- `.sops.yaml` — repository configuration that lists public age recipients and creation_rules that map file paths to key groups.
|
||||||
|
- `nix-system-configs/secrets/...` — the repository path where secrets are stored, organized by subsystem (for example `database`, `mail`, `songsheet`, `build_machine`, `traefik`, etc.).
|
||||||
|
- Nix module files that use sops (examples in `nix-system-configs/modules/secrets-config/`) — those show how secrets are exported into NixOS as file or string secrets.
|
||||||
|
|
||||||
|
Inspect the repository's `.sops.yaml` (example shown later) and the secrets folders before creating or editing secrets.
|
||||||
|
|
||||||
|
|
||||||
|
## Creating and registering age keys (on the machine that will hold the private key)
|
||||||
|
|
||||||
|
> [!TIP]
|
||||||
|
> Prefer creating the age key on the machine that needs to decrypt the secret (for example the server). Keep the private key on that machine only.
|
||||||
|
|
||||||
|
Example using nix-shell (macOS or Nix-enabled machine):
|
||||||
|
|
||||||
|
```zsh
|
||||||
|
# open a temporary shell with `age` available, create a dir for sops-nix keys and generate a key
|
||||||
|
nix-shell -p age --run 'sudo mkdir -p /var/lib/sops-nix && sudo chown $USER /var/lib/sops-nix && age-keygen -o /var/lib/sops-nix/key.txt'
|
||||||
|
|
||||||
|
# The generated file contains a public and private key pair. Copy the PUBLIC key (the line starting with "age1...") into the repository's .sops.yaml.
|
||||||
|
# DO NOT add the private key to the repo. Keep it under /var/lib/sops-nix on the machine.
|
||||||
|
```
|
||||||
|
|
||||||
|
What the generated key file contains (example):
|
||||||
|
|
||||||
|
- private key: starts with `AGE-SECRET-KEY-` (keep private)
|
||||||
|
- public key (recipient): looks like `age1...` — this is what you add to `.sops.yaml`
|
||||||
|
|
||||||
|
|
||||||
|
## Inspecting / editing `.sops.yaml`
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> `.sops.yaml` controls which public keys can decrypt files and where new secret files should live. When adding a public key, add the recipient under the `keys:` list and update `creation_rules:` if needed.
|
||||||
|
|
||||||
|
Example `.sops.yaml` entries (repository already contains similar entries):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
keys:
|
||||||
|
- &admin_one age1x...
|
||||||
|
- &server_two age1kzsr...
|
||||||
|
- &server_mail age1p...
|
||||||
|
creation_rules:
|
||||||
|
- path_regex: nix-system-configs/secrets/example_project/[^/]+\.(yaml|json|env|ini)$
|
||||||
|
key_groups:
|
||||||
|
- age:
|
||||||
|
- *admin_one
|
||||||
|
- *server_two
|
||||||
|
```
|
||||||
|
|
||||||
|
Notes:
|
||||||
|
- The repository's actual `.sops.yaml` uses YAML anchors (for example `&admin_christine`) and path-specific creation_rules for `songsheet`, `traefik`, `database`, `christine`, `wireguard`, `build_machine`, and `mail`.
|
||||||
|
- `path_regex` is matched relative to the repository root. Keep the files under `nix-system-configs/secrets/...` for consistency with existing rules.
|
||||||
|
|
||||||
|
|
||||||
|
## Creating a new encrypted secret file
|
||||||
|
|
||||||
|
1. Ensure the public recipient(s) for the intended end systems exist in `.sops.yaml`.
|
||||||
|
2. Use sops to create or edit an encrypted file. If sops cannot find an age key in the usual locations, pass `SOPS_AGE_KEY_FILE` pointing to the generated private key (only when editing locally with your private key present).
|
||||||
|
|
||||||
|
Example (editor micro) — note we use a temporary environment so sops can locate the key you generated locally:
|
||||||
|
|
||||||
|
```zsh
|
||||||
|
# from the repository root, create/edit a new secret for the build machine
|
||||||
|
EDITOR=micro nix-shell -p sops --run 'SOPS_AGE_KEY_FILE=$HOME/.config/sops/age/keys.txt sops nix-system-configs/secrets/example_project/secrets.yaml'
|
||||||
|
```
|
||||||
|
|
||||||
|
What you will see inside the new file while editing:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# plain YAML content (sops will encrypt on save)
|
||||||
|
runner_token: "example-runner-token"
|
||||||
|
# any other keys as required by your service
|
||||||
|
```
|
||||||
|
|
||||||
|
After saving, sops will replace plaintext with encrypted blocks and add a `sops:` metadata section — commit this encrypted file to git.
|
||||||
|
|
||||||
|
|
||||||
|
#### Example secret file
|
||||||
|
|
||||||
|
The repository includes `nix-system-configs/secrets/build_machine/secrets.yaml` which contains an encrypted `runner_token` plus the sops metadata. That file shows the encrypted enc blocks and recipients under `sops:`, thus you can use it as a reference for structure.
|
||||||
|
|
||||||
|
|
||||||
|
## How to add a new secrets set and wire it into Nix (step‑by‑step)
|
||||||
|
|
||||||
|
This is a practical walkthrough that follows the pattern used in `nix-system-configs/modules/secrets-config/sops-database.nix` and `sops-mail.nix`.
|
||||||
|
|
||||||
|
1) Create the secrets file in the repo
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> Do this only as a template: paste the examples below into the appropriate repo paths when you're ready. This file intentionally keeps everything inline so you don't need separate helper files to understand the pattern.
|
||||||
|
|
||||||
|
2) Create a small `sops`-based Nix module for this secret set
|
||||||
|
|
||||||
|
- Copy one of the existing modules (for example `sops-database.nix`) and adapt it. Example module: create `nix-system-configs/modules/secrets-config/sops-project_example.nix` with the following content:
|
||||||
|
|
||||||
|
```nix
|
||||||
|
{
|
||||||
|
config,
|
||||||
|
pkgs,
|
||||||
|
lib,
|
||||||
|
...
|
||||||
|
}: {...
|
||||||
|
|
||||||
|
# This will add secrets.yaml to the Nix store; you can instead reference a path string
|
||||||
|
sops.defaultSopsFile = ../../secrets/project_example/secrets.yaml;
|
||||||
|
|
||||||
|
# NOTE: sops.age.keyFile is where the private age key is expected on the target system
|
||||||
|
sops.age.keyFile = "/var/lib/sops-nix/key.txt"; # SEE NOTE ABOUT THIS
|
||||||
|
sops.age.generateKey = true; # SEE NOTE ABOUT THIS
|
||||||
|
|
||||||
|
# declare secrets that sops-nix will expose
|
||||||
|
sops.secrets."project_example/hello" = {
|
||||||
|
format = "yaml";
|
||||||
|
sopsFile = ../../secrets/project_example/secrets.yaml;
|
||||||
|
owner = "root";
|
||||||
|
mode = "0400";
|
||||||
|
};
|
||||||
|
|
||||||
|
sops.secrets."project_example/example_key" = {
|
||||||
|
format = "yaml";
|
||||||
|
sopsFile = ../../secrets/project_example/secrets.yaml;
|
||||||
|
owner = "root";
|
||||||
|
mode = "0400";
|
||||||
|
};
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Minimal example `secrets.yaml` (paste to `nix-system-configs/secrets/project_example/secrets.yaml` or edit in-place with `sops`):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# project_example secrets - plaintext template (encrypt with sops before committing)
|
||||||
|
hello: "replace-with-actual-hello-value"
|
||||||
|
example_key: "replace-with-actual-example-key"
|
||||||
|
```
|
||||||
|
|
||||||
|
3) Import the new module into your system
|
||||||
|
|
||||||
|
- Add the new Nix module to the `imports = [ ... ]` block of the system configuration that should receive the secrets. For example, in your system file (e.g. `nix-system-configs/modules/system/your-system.nix`) add:
|
||||||
|
|
||||||
|
```nix
|
||||||
|
imports = [
|
||||||
|
# ...existing imports...
|
||||||
|
../../modules/secrets-config/sops-project_example.nix
|
||||||
|
];
|
||||||
|
```
|
||||||
|
|
||||||
|
- You can also import it from a central place where you manage secrets modules; the important part is that the module is available in the `imports` set used for the machine(s) that need it.
|
||||||
|
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> This section is only required if you did not already generate and register age keys for the target end system. If you pregenerated the key pair and added the public recipient to `.sops.yaml`, you can skip these steps.
|
||||||
|
>
|
||||||
|
> If `sops.age.generateKey = true` and no key exists, the module will create `/var/lib/sops-nix/key.txt` on the target end system when the module activates. After the module generates a key, extract the public recipient (the `age1...` line) and add it to the repository `.sops.yaml` under `keys:` and to the appropriate `creation_rules` so future `sops` edits include the new recipient.
|
||||||
|
>
|
||||||
|
> Example commands to extract the public recipient (run on the target machine; use sudo if needed):
|
||||||
|
>
|
||||||
|
> ```zsh
|
||||||
|
> # print the whole key file (DO NOT commit the private key)
|
||||||
|
> sudo cat /var/lib/sops-nix/key.txt
|
||||||
|
> ```
|
||||||
|
>
|
||||||
|
> Important: never commit the private key (the `AGE-SECRET-KEY-...` line) into the repository — only the `age1...` public recipient belongs in `.sops.yaml`.
|
||||||
|
|
||||||
|
|
||||||
|
4) Reference the secret from other Nix code and services
|
||||||
|
|
||||||
|
- After the module is imported and active, `sops-nix` exposes each declared secret in `config.sops.secrets` under the quoted secret name. The repo uses the flat, quoted key names (for example `"project_example/hello"`) so you can reference them in Nix like this:
|
||||||
|
|
||||||
|
```nix
|
||||||
|
# use the path to the secret file in another module
|
||||||
|
someService.extraConfig = {
|
||||||
|
passwordPath = config.sops.secrets."project_example/example_key".path;
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
- If you declared a secret that should be used by a Docker Compose conversion (compose2nix), annotate the service in `docker-compose.yml` with the label used by compose2nix and pass `--sops_file` when running `compose2nix`:
|
||||||
|
|
||||||
|
```text
|
||||||
|
config.sops.secrets."project_example/hello".path
|
||||||
|
config.sops.secrets."project_example/example_key".path
|
||||||
|
```
|
||||||
|
|
||||||
|
Then run compose2nix with the SOPS file:
|
||||||
|
|
||||||
|
```zsh
|
||||||
|
compose2nix --sops_file nix-system-configs/secrets/project_example/secrets.yaml ...other args...
|
||||||
|
```
|
||||||
|
|
||||||
|
This instructs the generated Nix expression to reference the secret name (which compose2nix and/or your Nix code can resolve via `config.sops.secrets."project_example/hello".path`).
|
||||||
|
|
||||||
|
|
||||||
|
## Common pitfalls & troubleshooting
|
||||||
|
|
||||||
|
- Never commit private keys. Only public age recipients belong in `.sops.yaml`.
|
||||||
|
- If sops fails to encrypt/decrypt locally, check that you have the corresponding private key available and `SOPS_AGE_KEY_FILE` points to it when running from a shell that cannot otherwise find it.
|
||||||
|
- `path_regex` in `.sops.yaml` is matched relative to the repo root — use it carefully so creation rules apply where expected.
|
||||||
|
- If `sops.age.generateKey` is enabled and you rely on the module to generate keys, remember you must register the generated public recipient in the repo `.sops.yaml` after first activation. Extract the `age1...` line with one of the commands above and add it to `keys:` and the appropriate `creation_rules`.
|
||||||
|
- If the Nix module expects the private key at `/var/lib/sops-nix/key.txt`, make sure that is where the key is placed on the target end system and that ownership/mode are correct for the system service that reads it.
|
||||||
Loading…
Add table
Add a link
Reference in a new issue