dtoro
b5c1247093
Problem: after the wiki-hq reorg, agent-instruction and human-doc domains were still scattered across the repo root, with three now-redundant stub files cluttering it. The organizing principle wasn't visible in the layout. Change — enforce three clear buckets: - .agents/ = how agents operate: OIKOS.md, HERMES.md (moved from root), shared/ conventions, domains/ schemas, skills/, and operations/ (operator cheatsheet + enrollment + hermes-agent, moved from root). - knowledge/ = what exists + evidence: wiki/, GLOSSARY.md, and sources/ now including investigations/ (incident records are evidence/sources). - root = substrate + two entry points (AGENTS.md, README.md), plus plans/ as its own design-intent domain. Moves: - investigations/ -> knowledge/sources/investigations/ (incl. archive/, index). - operations/ -> .agents/operations/. - HERMES.md -> .agents/HERMES.md. - Deleted unreferenced root stubs CAVEMAN.md, CONTRIBUTING.md, and OIKOS.md (its 7 remaining linkers repointed to .agents/OIKOS.md). Consumers updated: - inventory.yaml doc_page (agent-enrollment) + regenerated hosts/*.yaml + cards. - tools/setup-hermes-soul.sh and bootstrap.sh (x2) -> .agents/HERMES.md. - bin/homelab help string -> .agents/operations/hermes-agent.md. - knowledge/operations schemas, llm-wiki, page-templates, incident-investigation skill, AGENTS.md/README nav -> new investigations/operations paths. - All markdown links rewritten via the path-resolving mapper. Left in place (substrate/executable/separate-domain): hosts/, ledger/, tools/, plans/, oikos/, mcp/, secrets/, bin/, inventory.yaml. Verification: docs-lint at baseline (2 intentional cross-repo refs, no new breakage); gen-topology.py --check exit 0; build_host_files.py idempotent; all doc_page targets resolve; Hermes provisioning scripts point at the new path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
35 lines
1.6 KiB
Markdown
35 lines
1.6 KiB
Markdown
---
|
|
name: incident-investigation
|
|
risk_class: read_only
|
|
inputs: [symptom, affected_entity]
|
|
verification: "n/a — investigation produces a written record, not a state change"
|
|
docs_update_checklist: [investigations_entry]
|
|
---
|
|
|
|
# Incident investigation
|
|
|
|
Goal: understand what broke and why, before touching anything.
|
|
|
|
1. `homelab service <name> explain` (or `homelab node <name> relations`
|
|
if the affected entity is a host) — get the blast radius and doc
|
|
pointer first. Don't start pulling logs blind.
|
|
2. `homelab service <name> health` + `homelab service <name> log` (or
|
|
MCP `get_service_status` / `tail_log`) for the affected service.
|
|
3. Walk the blast radius: is a shared dependency down (`caddy`, `dns`,
|
|
`authentik`, or the backend host itself)? `homelab node <name>
|
|
relations` shows "affected by" — check those first.
|
|
4. `homelab apt-audit` if the symptom looks like a dpkg/upgrade
|
|
interaction.
|
|
5. Check the change ledger for recent mutations to the affected entity
|
|
or anything upstream of it: `homelab service <name> history` (once
|
|
populated) or grep `ledger/*.jsonl`.
|
|
6. Write findings to a new `knowledge/sources/investigations/<date>-<slug>.md` — symptom,
|
|
timeline, root cause, fix applied, prevention. This is the durable
|
|
record; don't rely on chat history.
|
|
|
|
Docs-update checklist: always create the investigation entry. If the
|
|
root cause was stale/wrong inventory data (a `doc_page`, `config_repo`,
|
|
or `backend` that didn't match reality — this happened during Week 1
|
|
kernel work, see the `authentik` backend fix), correct `inventory.yaml`
|
|
in the same session.
|