db as source of truth: wiki→seeds, archive old artifacts, knowledge ingestion

- Migrations 010 (content_hash) + 011 (search tsvector column) - new: internal/knowledge/seed.go — knowledge seed ingest engine - new: internal/httpapi/knowledge.go — SearchKnowledge + GetEntityKnowledge - wire knowledge ingest into oikos seed pipeline - convert all 36 wiki docs + 6 investigations + 12 runbooks → seeds/knowledge.yaml - archive: knowledge/wiki/→archive/, oikos/cards/→archive/, .hermes/plans/→archive/ - delete: 9 superseded Python kernel files, ledger/, mcp/build_host_files.py - remove empty knowledge/ directory tree
2026-07-07 20:22:30 +02:00
parent b2bfa26f64
commit 6b75f7302d
125 changed files with 7557 additions and 1938 deletions
--- a/archive/knowledge/infrastructure/ssh-access.md
+++ b/archive/knowledge/infrastructure/ssh-access.md
@@ -0,0 +1,206 @@
+# SSH access
+
+How to reach every host in the fleet from any workstation, with LAN as
+the primary path and Netbird as the automatic backup.
+
+## Architecture
+
+SSH access relies on three layers:
+
+1. **Homelab inventory (`inventory.yaml`)** — the single source of truth
+   for every host's LAN IP, Netbird addresses, SSH user, and port.
+2. **Key distribution (`ssh/deploy-keys.sh`)** — deploys workstation SSH
+   public keys to hubris and every running LXC, so any key-authorized
+   workstation can log in anywhere.
+3. **Config generation (`homelab ssh-config --install`)** — generates
+   `~/.ssh/config.d/homelab` with short hostname aliases for every host,
+   using LAN IPs (routed via Netbird's `192.168.8.0/24` subnet route when
+   off-LAN) with Netbird FQDN fallbacks (`<name>-mesh`) for roaming
+   workstations.
+
+### How it works
+
+- **From on-LAN:** `ssh gitea` resolves to `192.168.8.121` directly.
+- **From off-LAN (Netbird):** The same `192.168.8.121` works because
+  hubris routes the `192.168.8.0/24` subnet through Netbird.
+- **Roaming workstations:** `ssh mac-mini-mesh` or `ssh republic-laptop-mesh`
+  uses the Netbird FQDN as a fallback when the workstation is off its
+  home subnet.
+
+The `homelab ssh <host>` CLI command also has built-in LAN probing:
+it tries a 1.5s TCP connect to the LAN IP, and if that fails, falls
+back to the Netbird FQDN.
+
+## Key distribution
+
+Each workstation's SSH public key lives in the repo at:
+`ssh/authorized_keys/<hostname>.pub`
+
+To deploy or re-deploy all workstation keys to hubris + every running LXC:
+
+```bash
+# From hubris (or via homelab pct):
+sudo bash /opt/homelab-context/ssh/deploy-keys.sh
+
+# Or from any workstation:
+ssh root@192.168.8.77 "bash /opt/homelab-context/ssh/deploy-keys.sh"
+```
+
+This script:
+- Reads all `.pub` files from `ssh/authorized_keys/`
+- Adds any missing keys to `/etc/pve/priv/authorized_keys` on hubris
+- For each running LXC, appends keys to `/root/.ssh/authorized_keys`
+- Is idempotent — skips keys already present
+
+## Config generation
+
+To generate the SSH config on any workstation:
+
+```bash
+homelab ssh-config --install
+```
+
+This writes to `~/.ssh/config.d/homelab` and ensures
+`Include ~/.ssh/config.d/homelab` is present in `~/.ssh/config`.
+
+The config is regenerated automatically on every `homelab sync` (which
+kicks the 5-minute context sync timer).
+
+## Adding a new workstation
+
+When onboarding a new machine:
+
+1. Hostname must match an entry in `inventory.yaml`.
+2. If the workstation will be on the LAN, add its `lan_ip` to
+   `inventory.yaml` and push. This gives it a primary LAN entry in the
+   generated SSH config.
+3. Enable SSH Remote Login:
+   - **macOS:** `sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist`
+   - **Linux:** `sudo systemctl enable --now sshd`
+4. Generate an SSH keypair if one doesn't exist:
+   ```bash
+   ssh-keygen -t ed25519 -a 100
+   ```
+5. Publish the public key to the repo:
+   ```bash
+   cp ~/.ssh/id_ed25519.pub /opt/homelab-context/ssh/authorized_keys/<hostname>.pub
+   cd /opt/homelab-context && git add ssh/authorized_keys/ && git commit -m 'ssh: add <hostname> pubkey' && git push
+   ```
+6. Deploy the key to all hosts:
+   ```bash
+   ssh root@192.168.8.77 "cd /opt/homelab-context && git pull --ff-only && bash ssh/deploy-keys.sh"
+   ```
+7. Generate the local SSH config:
+   ```bash
+   homelab ssh-config --install
+   ```
+
+## Hosts
+
+### Hubris + strong (PVE cluster: `Homelab`)
+
+Both nodes share `/etc/pve/priv/authorized_keys` — it's Proxmox
+cluster-synced, so a key added on either node is authorized on both.
+
+| Detail | hubris | strong |
+|--------|--------|-----------|
+| LAN IP | `192.168.8.77` | `192.168.178.181` |
+| Cluster node name | `hubris` | `strong` (OS hostname kept as-is from install) |
+| Netbird | `100.122.38.109` (`proxmox-server.netbird.selfhosted`) | not enrolled yet |
+| Netbird SSH port | `22022` (mesh-only, OIDC auth) | n/a |
+| SSH user | `root` | `root` |
+
+Authorized root keys currently deployed (cluster-wide):
+- `root@hubris` (self, RSA)
+- `d.toro.v@pm.me` (ed25519) — mac-mini
+- `root@strong` (RSA) — strong's own key, added 2026-07-01 for the cluster join
+
+### LXCs
+
+Every LXC at `192.168.8.x` accepts root SSH via authorized_keys. Keys
+are managed by `ssh/deploy-keys.sh`. SSH user is `root`.
+
+| LXC | Name | LAN IP | Role |
+|-----|------|--------|------|
+| 101 | jellyfin | `192.168.8.206` | media-server |
+| 102 | nfs-export | `192.168.8.200` | storage-export |
+| 103 | paperless | `192.168.8.130` | document-archive |
+| 104 | gitea | `192.168.8.121` | git-server |
+| 105 | apps | `192.168.8.205` | docker-apps |
+| 106 | auth-outpost | `192.168.8.184` | authentik-outpost |
+| 107 | dns | `192.168.8.185` | dns-helper |
+| 114 | nextcloud | `192.168.8.224` | file-sync |
+| 118 | elementsynapse | `192.168.8.239` | matrix-server |
+| 119 | sophia | `192.168.8.157` | workshop |
+| 120 | mule-images | `192.168.8.136` | photo-management |
+| 121 | caddy | `192.168.8.175` | reverse-proxy |
+| 122 | arriman | `192.168.8.132` | arr-stack |
+
+### Workstations
+
+| Name | OS | LAN IP | Netbird FQDN | SSH user |
+|------|----|--------|--------------|----------|
+| mac-mini | macOS | `192.168.8.174` | `mac-mini-234-17.netbird.selfhosted` | `dtoro` |
+| republic-laptop | Linux | TBD | `republic-laptop.netbird.selfhosted` | `dtoro` |
+
+strong moved out of this table 2026-07-01 — it's a Proxmox host now, see the cluster table above.
+
+### VPS (external)
+
+| Detail | Value |
+|--------|-------|
+| Public IP | `82.165.190.79` |
+| Netbird | `100.122.165.149` (FQDN: `netbird-ionos.netbird.selfhosted`) |
+| SSH user | `root` |
+| Access | Mesh-only — public port 22 is blocked by nftables. Key-only auth. |
+
+## VPS
+
+Access is mesh-only. From a mesh-connected peer:
+
+```bash
+ssh root@100.122.165.149
+ssh root@netbird-ionos.netbird.selfhosted
+# or via homelab:
+homelab ssh netbird-vps
+```
+
+## Verification
+
+```bash
+# From any workstation after running homelab ssh-config --install:
+for name in hubris gitea apps sophia paperless caddy jellyfin nextcloud; do
+  ssh -o BatchMode=yes "$name" "hostname" && echo "$name OK"
+done
+```
+
+## Related
+
+- [Mesh migration](mesh.md)
+- [VPS hardening](vps-hardening.md)
+- [Agent enrollment](../../../.agents/operations/agent-enrollment.md)
+- [Homelab CLI](../../../bin/homelab)
+
+## Changelog
+
+### 2026-07-01 — strong reformatted to Proxmox, joined cluster; table corrected
+strong moved from the Workstations table to the PVE-cluster table (was showing a stale `192.168.8.133`, never actually reachable — the real LAN IP has always been `192.168.178.181`, matching hosts/strong.yaml). Root key access bootstrapped via one-time console password, then key-only going forward. See [hosts/hubris.md#cluster](../hosts/hubris.md#cluster) and [hosts/strong.md](../hosts/strong.md).
+
+### 2026-06-02 — universal SSH reachability
+
+Replaced ad-hoc per-workstation SSH configs with inventory-generated
+configs (`ssh/gen-config.py`, `homelab ssh-config`). Added centralized
+key distribution (`ssh/deploy-keys.sh`, `ssh/authorized_keys/`). All
+LXCs now accept root SSH from any workstation whose pubkey is in the
+repo. mac-mini Remote Login enabled. Netbird subnet route
+(192.168.8.0/24 via hubris) provides off-LAN reachability for all LAN
+IPs.
+
+### 2026-04-28 — wiki entry created
+Initial documentation.
+
+### 2026-04-23 — VPS SSH hardened to mesh-only
+Public `:22` blocked at nftables. Key-only sshd.
+
+### 2026-04-22 — iMac key authorized on hubris
+`d.toro.v@pm.me` added to `/etc/pve/priv/authorized_keys`.