Welcome to the Polyteknisk Radiogruppe's Server Documentation

This documentation outlines how the server is built and maintained, covering both short- and long-term management practices.

Note

Important: This document should be actively maintained whenever functional changes occur - no matter how small or large. This includes compilation flag changes, password key updates, new standard packages, and any other modifications.

Documentation File Structure

As this document is in alpha release, the following package has been made with the standard Rust project template in the case of having to create custom binaries for the system of the sysadmin or any related responsibilites/positions. Thusly, the following document focus should be in the nix-system-configs folder, which is the main folder for the system configuration files and scripts.

[!TIP] For documentation maintainers: Update the directory tree using lsd --tree from the project root. Be careful not to expose secrets and other things when documenting from your local system.

 .
├──  backup-key.private
├──  Cargo.lock
├──  Cargo.toml
├──  compose2nix-readme.md
├──  docker-compose.wellknown.override.yml
├──  documentation_titlepage.md
├──  dummy.txt
├──  LICENSE
├──  nginx
│   └──  matrix.conf
├──  nix-system-configs
│   ├──  build
│   │   └──  configuration.nix
│   ├──  christine-server_do_not_use.nix
│   ├──  database-deprecated
│   │   └──  configuration.nix
│   ├──  dns
│   │   ├──  configuration-knot.nix
│   │   ├──  configuration-pihole-green.nix
│   │   ├──  configuration-pihole-inside-system.nix
│   │   └──  configuration-pihole-orange.nix
│   ├──  example-composed.nix
│   ├──  forgejo-deprecated
│   │   ├──  forgejo-localconfig.nix
│   │   ├──  pull.zsh
│   │   └──  push.zsh
│   ├──  gateway
│   │   └──  gateway.nix
│   ├──  modules
│   │   ├──  bootloader
│   │   ├──  desktop-manager
│   │   ├──  lix-default.nix
│   │   ├──  local
│   │   ├──  scripts
│   │   ├──  secrets-config
│   │   ├──  songsheet
│   │   ├──  system
│   │   ├──  system_scripts
│   │   └──  toolsets
│   ├──  old-server-notes.md
│   ├──  prg-blank-setup
│   │   ├──  blank-proxmox-partition.nix
│   │   └──  example.nix
│   ├──  secrets
│   │   ├──  database
│   │   ├──  songsheet
│   │   └──  traefik
│   └──  traefik-deprecated
│       ├──  pull.zsh
│       ├──  push.zsh
│       └──  traefik-config.nix
├──  README.md
├──  sops-nix-readme.md
├── 󱧼 src
│   └──  main.rs
└──  target
    ├──  CACHEDIR.TAG
    └──  debug
        ├──  build
        ├──  deps
        ├──  examples
        └──  incremental

Module Architecture

The system is organized into reusable modules across all NixOS machines in the PRG network. The system module serves as the core configuration for Linux systems running on Proxmox, while supporting diverse deployments through modular composition.

Configuration Pattern

Use blank_system_USE_THIS_AS_COPY.nix as a template for new system configurations. This file demonstrates the standard pattern of declaring system variables using Nix syntax to create custom local options that are callable across imported modules.

Core Modules

The imports = [ ... ] block specifies which modules are attached to your system configuration. These core modules ensure consistency and functionality:

Bootloader [bootloader] (required): Handles system boot initialization. Configuration varies by installation type (ISO-assigned or Proxmox-assigned).
Local [local] (required): Provides internet connectivity, hostname resolution, DNS configuration, and user settings.
Standard Toolset [lix-default.nix] (strongly recommended): Ensures consistent tools across the PRG network—file transfers (rsync), SSH with key-based authentication, security features (U2F and SSH agent passwordless sudo), package management, and automated daily system upgrades.
Desktop/Window Manager [desktop-manager] (optional): Simplifies VM interaction within Proxmox and supports systems requiring graphical interfaces.
Toolsets [toolsets] (optional): Contains reusable configurations for specific tools or services that may be needed across multiple systems, such as remote building configurations.
Secrets Config [secrets-config] (required when needing password management): Manages sensitive information like database credentials and Traefik secrets using SOPS-encrypted Nix files (see this and that). This module is essential for systems that require secure handling of secrets.
Docker Containers [songsheet] (optional): This special module handles Docker Compose files that are converted to Nix. Currently it is used for a single VM but can be expanded to contain all compose2nix generated configurations.
System Scripts [system_scripts] (variable): system_scripts contains scripts (for example, database backup scripts) that are often easier to write and maintain in shell than in Nix.

Module Organization

 .
├──  bootloader
│   ├──  seabios-assigned-iso-at-birth.nix
│   └──  seabios-assigned-proxmox-at-birth.nix
├──  desktop-manager
│   ├──  gnome.nix
│   └──  sway_greetd_homemanager.nix
├──  lix-default.nix
├──  local
│   ├──  hostname_username.nix
│   └──  networking_local.nix
├──  scripts
│   ├──  compose_two_nix.zsh
│   ├──  pull.zsh
│   └──  push.zsh
├──  secrets-config
│   ├──  sops-composesongsheet.nix
│   └──  sops-database.nix
├──  songsheet
│   └──  wavelog
├──  system
│   ├──  blank_system_USE_THIS_AS_COPY.nix
│   ├──  compose-songsheet.nix
│   ├──  database.nix
│   ├──  forgejo.nix
│   ├──  teamspeak.nix
│   └──  traefik.nix
├──  system_scripts
│   ├──  backup_strategem
│   └──  gcloud_backup.nix
└──  toolsets
└──  remote_building.nix

Module Scripts

[!INFO] As you saw the odd one in the file structure, the scripts folder contains helper scripts to simplify maintenance and to make updating system configuration files on remote machines easier. These helper scripts live under nix-system-configs/modules/scripts.

`pull.zsh`

Deploys configuration changes from the repository to the local system. This script:

Prompts you to select a system configuration (or accepts one as a command-line argument)
Pulls the latest changes from the git repository
Syncs the modules directory to /etc/nixos/modules
Syncs the secrets directory to /etc/nixos/secrets (if present)
Syncs the system_scripts directory to /etc/nixos/system_scripts (if present)
Copies the selected system configuration to /etc/nixos/configuration.nix
Executes nixos-rebuild switch --upgrade-all to apply changes

`push.zsh`

Synchronizes local system changes back to the repository. This script:

Prompts you to select a system configuration (or accepts one as a command-line argument)
Rebuilds the local NixOS system with nixos-rebuild switch --upgrade-all
Pulls the latest changes from the git repository
Copies the active configuration from /etc/nixos/configuration.nix back to the repository
Syncs changes from /etc/nixos/modules back to the repository
Stages, commits, and pushes changes to the remote git repository with a timestamped commit message

How to maintain the local machines

[!NOTE] The following maintenance scripts should be made less unwieldy to use, so they will be changed in the next possible project restructuring.

From the home machine:

The remote access IP address should be reachable via the VPN service you use (Tailscale, OpenVPN, etc.). To connect to the remote machine (or other machines on the same network) with established keys, use:

ssh -A REMOTE_MACHINE_USERNAME@ACCESS_IP_ADDRESS

To connect to other machines:

ssh -A OTHER_REMOTE_MACHINE_USERNAME@LOCAL_IP_ADDRESS

After having had connected to the remote machine, run the following commands, although this assumes you have git cloned this repository into the home directory of the remote machine. Now, running through the selection of the different systems as per script and upon selection of the script, it will then try to upgrade the system.

cd the-prg-server-structure
git pull
zsh nix-system-configs/modules/scripts/pull.zsh

9.2 KiB Raw Blame History