Storage Architecture & Data Safety

Two-folder strategy

h5h/
├── code/          ← this git repo (clone on any new machine)
│   ├── apps/
│   │   └── web/           h5h.me — portfolio (Vercel)
│   ├── docker/
│   │   ├── docker-compose.yml
│   │   ├── .env.example
│   │   ├── traefik/       static + dynamic config
│   │   ├── homepage/      Homepage dashboard config (sh.h5h.me)
│   │   ├── prometheus/    metrics + alert rules
│   │   ├── grafana/       dashboards + provisioning
│   │   ├── loki/          log aggregation config
│   │   ├── promtail/      log shipping config
│   │   ├── cloudflared/
│   │   └── README.md
│   ├── scripts/           setup.sh, generate-secrets.sh, protect-data.sh, setup-plex.sh, setup-qbittorrent.sh
│   ├── Dockerfile         portfolio image (Docker Hub safe)
│   ├── Makefile
│   ├── package.json
│   └── vercel.json
│
└── data/          ← all media, databases, caches (gitignored)
    ├── immich/
    ├── nextcloud/
    ├── plex/
    ├── torrent/       qBittorrent config + downloads
    ├── postgres/
    ├── grafana/
    ├── prometheus/
    ├── traefik/       acme certs + access logs
    ├── tailscale/     VPN state
    ├── crowdsec/      intrusion detection data
    ├── loki/          log storage
    └── backups/

Migrating to a mini PC? git clone for code, copy data/ to the HDD, cd code && make init && make up.

Secrets & data safety

This is a public repo. Configuration files (Traefik rules, Prometheus scrape targets) are committed. Secrets are never committed.

What lives where

Layer	Contents	Committed?
code/docker/.env.example	Template with placeholder values	Yes
code/docker/.env	Real secrets (tokens, passwords)	Never — .gitignore'd
data/	Databases, uploads, certs, logs	Never — outside code/

data/ protection

data/ is protected against accidental deletion using macOS chflags uchg on the data/ directory only (not subdirectories). This single flag gives full protection while keeping apps functional:

Action	Result
rm -rf data/	Blocked — data/ is immutable
rm -rf data/immich	Blocked — can't modify entries in immutable data/
git clean -fdx	Blocked — can't remove data/ or any top-level subdirectory
mv data/ somewhere/	Blocked — can't rename immutable directory
Apps creating runtime files/dirs	Allowed — subdirectories are fully writable
Docker writing to data/loki/, data/grafana/, etc.	Allowed

Why only data/ and not subdirectories? The uchg flag prevents adding/removing entries inside the flagged directory. If data/loki/ had uchg, Loki couldn't create runtime directories like tsdb-shipper-cache/. By protecting only data/, all subdirectories (immich/, loki/, postgres/, etc.) are shielded from deletion but remain writable for apps.

Additional protection	Covers	How
.gitignore + data/.gitignore	git clean -fd	Respects .gitignore — data/ is safe
make clean	Only runs docker system prune	Explicitly never touches data/
make protect / make unprotect	Manual lock/unlock	Toggle immutable flag
make init	Auto-locks on first setup	Runs protect-data.sh lock

make protect      # Lock data/ (done automatically by make init)
make unprotect    # Unlock for maintenance
make data-status  # Check protection status

Never commit checklist

Before every git push, verify none of these appear in your commit:

• [ ] CF_TUNNEL_TOKEN — Cloudflare Tunnel token
• [ ] CF_DNS_API_TOKEN — Cloudflare DNS API token
• [ ] AUTHENTIK_SECRET_KEY, AUTHENTIK_BOOTSTRAP_PASSWORD, AUTHENTIK_DB_PASSWORD
• [ ] GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET — Google OAuth credentials
• [ ] AUTHENTIK_GRAFANA_CLIENT_SECRET — Grafana OIDC secret
• [ ] IMMICH_DB_PASSWORD, NEXTCLOUD_DB_PASSWORD, NEXTCLOUD_ADMIN_PASSWORD
• [ ] PLEX_CLAIM — Plex claim token
• [ ] GF_SECURITY_ADMIN_PASSWORD — Grafana admin password
• [ ] TS_AUTHKEY — Tailscale auth key
• [ ] CROWDSEC_BOUNCER_KEY — CrowdSec bouncer API key
• [ ] HOMEPAGE_VAR_* — Homepage widget API tokens
• [ ] data/traefik/acme/ — Let's Encrypt private keys

Future improvement: Migrate from .env to Docker secrets for database passwords and API tokens. On macOS this adds complexity (secrets require Swarm mode or Compose v2.23+), so .env is fine for v1.

Backups

Two mechanisms exist; only one should run in production.

offen/docker-volume-backup (primary, nightly)

Enabled by make backup. Runs at 0 3 * * * from inside the stack and tars ${DATA_DIR} into ${DATA_DIR}/backups/h5h-backup-<timestamp>.tar.gz with 7-day pruning.

Critically, it respects the h5h-backup-stop Docker label. Services that carry this label (all Postgres instances, all Redis instances, Obsidian) are gracefully stopped for the duration of the archive and restarted afterwards. This avoids capturing a half-written database page. Services without the label (Immich server, Nextcloud, Plex, Grafana, etc.) keep serving during the backup — their state files are crash-consistent.

To add another service to the stop-list, add h5h-backup-stop: "true" to its Docker labels.

scripts/backup.sh (manual fallback)

A hand-callable tar wrapper that writes to the same data/backups/ directory and keeps the last 5 tarballs. Useful before a risky upgrade when the nightly job hasn't run yet. Interleaves filenames with the offen output — tracked in known_issues.md #10. Prefer make backup-now (which triggers the offen container immediately) over calling backup.sh directly.