docs(R5): rewrite knowledge schema + llm-wiki for DB-native model; deprecate root inventory.yaml

Rewrote .agents/domains/knowledge/schema.md and .agents/shared/llm-wiki.md
which described the deleted Python substrate (bin/homelab, oikos/cards/,
oikos/ledger.py, root inventory.yaml, knowledge/sources/, get_page/
search_docs MCP tools). Now reflect ADR 0003: Postgres DB is the single
source of truth for structured data and narrative knowledge; seeds/*.yaml
are bootstrap+DR manifests (content-hashed via seed_versions); archive/
knowledge/ is the frozen legacy wiki; MCP search_knowledge/get_entity_
knowledge replace get_page/search_docs.

Swept substrate refs in .agents/shared/{writing-style,page-templates}.md
and .agents/domains/operations/schema.md: bare inventory.yaml ->
seeds/inventory.yaml; knowledge/sources/ -> archive/knowledge/sources/
(historical); get_changelog/oikos/ledger.py -> DB audit trail / structured
document changelog field; HERMES -> Nomos.

Root inventory.yaml (618-line Python-era file superseded 2026-07-07 by
seeds/inventory.yaml) replaced with a deprecation stub pointing to the seed
and DB. Kept as a stub rather than deleted because AGENTS.md §1/§2 still
point clients at /opt/homelab-context/inventory.yaml; full on-client path
reconciliation deferred to R13.

Flagged export gap: oikos export regenerates seeds/{ontology,inventory,
policy}.yaml but NOT seeds/knowledge.yaml — API-added knowledge lives only
in the DB until hand-edited into the seed.

VERSION 0.7.7 -> 0.7.8. Plan R5 marked done.
This commit is contained in:
2026-07-17 22:36:41 +02:00
parent d2950dd09d
commit a2410cf9c2
8 changed files with 175 additions and 702 deletions

View File

@@ -1,48 +1,93 @@
# Knowledge domain — schema
The knowledge domain is the durable, authoritative current-state documentation of the homelab: one
page per node and per cross-cutting system, synthesized from live state and evidence. It answers
"what exists and how does it work right now."
The knowledge domain is the durable, authoritative current-state documentation of the homelab:
narrative for every node and cross-cutting system, synthesized from live state and evidence. It
answers "what exists and how does it work right now."
It follows the [LLM Wiki layer model](../../shared/llm-wiki.md) and the
[writing-style](../../shared/writing-style.md) and [page-templates](../../shared/page-templates.md)
rules.
## The narrative / substrate split
## Source of truth — the database
The knowledge wiki is **narrative**. It sits alongside a **machine-readable substrate** that it
describes but never contains. The split is load-bearing: several programs read the substrate at
fixed paths, so the wiki reorganization never moves it.
Per ADR 0003, the Postgres database is the single source of truth for all structured data **and**
narrative knowledge. The narrative/substrate split of the Python era is gone: the DB holds both the
structured graph (entities, relationships, status, metrics) and the narrative layer (documents,
investigations, runbooks) in the `knowledge_entities` table.
| Layer | Location | Consumed by |
|-------|----------|-------------|
| Substrate — source of truth | `inventory.yaml` (root) | MCP server, `homelab` CLI, `oikos/` scheduler/drift/relations/gen-topology |
| Substrate — generated host records | `inventory.yaml` (root) | Go `internal/mcp/` server, `bin/homelab`; the single source of truth |
| Substrate — kernel + context cards | `oikos/` (code, `oikos/cards/`, `oikos/state.json`) | MCP `explain`, scheduler |
| Narrative — synthesized wiki | `archive/knowledge/{hosts,containers,vms,infrastructure}/` | humans, agents via MCP `get_page` / `search_docs` |
| Evidence — immutable sources | `knowledge/sources/` (references + investigations) | synthesis into wiki pages |
| Concern | Where it lives | How it gets there |
|---------|----------------|-------------------|
| Knowledge content — documents, investigations, runbooks | `knowledge_entities` table (rows linked to `entities` via `documents` / `about` edges) | Seeded from `seeds/knowledge.yaml` at deploy; mutated at runtime via the API |
| Seed manifest (bootstrap + DR) | `seeds/knowledge.yaml` | Hand-edited or regenerated; ingested idempotently (content-hashed via `seed_versions`) |
| Structured graph — hosts, services, entity types, relationships | `entities`, `relationships`, `entity_types` tables | Seeded from `seeds/{ontology,inventory}.yaml`; mutated via API/MCP |
| Archived narrative wiki (read-only history) | `archive/knowledge/` | Frozen 2026-07-07 when the DB became source of truth |
## Wiki pages
### Seed ingest
- **Node pages** (`archive/knowledge/containers/<id>-<name>.md`, `.../vms/<id>-<name>.md`,
`.../hosts/<name>.md`) follow the container/host template in
[page-templates.md](../../shared/page-templates.md): opening definition, `## At a glance`,
`## Role`, service/port map, storage, auto-deploy, `## Related`, `## Changelog`.
- **Cross-cutting pages** (`archive/knowledge/infrastructure/<topic>.md`) follow the cross-cutting
template: `## Why`, `## Components`, `## How to apply`, `## Gotchas`, `## Related`, `## Changelog`.
- Each `inventory.yaml` host entry carries a `doc_page:` field pointing at its narrative page.
Changing where a page lives means updating that field (read by `bin/homelab`).
`seeds/knowledge.yaml` has three top-level lists — `documents`, `investigations`, `runbooks` — each
entry carrying `slug`, `title`, `content` (markdown), and tags. `internal/knowledge/seed.go`
ingests each entry by:
1. `getOrCreateEntity` — ensures the slug exists in `entities` (type `document` / `investigation` /
`runbook`).
2. `upsertKnowledgeEntity` — writes the markdown body into `knowledge_entities`, keyed by
`content_hash` so re-ingest is a no-op when nothing changed.
3. `createEdge` — links the knowledge entity to its subject(s) via `documents` (for `document`) or
`about` (for `investigation`) edges. Runbooks bind to an `entity_type` via `applies_to_type`
rather than to a single entity.
### Runtime mutation
Agents register or update knowledge through the API, not by editing the seed:
- `POST /api/v1/knowledge/{entity_slug}` — upsert a document/investigation on an entity
(`upsert_knowledge` MCP tool).
- `update_entity_attributes` — merge a discovered fact (IP, version, port) into an entity.
- `create_relationship` — record a discovered edge (`depends-on`, `hosts`, `routes-to`).
> **Export gap.** `oikos export` regenerates `seeds/{ontology,inventory,policy}.yaml` from the DB
> for version control, but **not** `seeds/knowledge.yaml`. Knowledge added via the API today lives
> only in the DB until someone hand-edits the seed. Tracked as a follow-up.
## Knowledge kinds
- **Documents** (`document` entities, linked via `documents` edges) — node and cross-cutting
narrative pages. Carry `at_glance` (structured attributes) and a parsed `changelog`. Follow the
container / cross-cutting templates in [page-templates.md](../../shared/page-templates.md).
- **Investigations** (`investigation` entities, linked via `about` edges) — incident evidence,
written once at incident time. Sections: `## Summary`, `## Timeline`, `## Root cause`,
`## Mitigations applied`, `## Open questions`.
- **Runbooks** (`runbook` entities, bound by `applies_to_type`) — repeatable procedures. Carry
`risk_class` and a JSON-schema-validated `procedure`. **Runbooks also live as `SKILL.md` files
under `.agents/skills/<name>/`** — the DB row is the policy/lifecycle framing, the SKILL.md is
the executable procedure the agent loads. See
[the operations schema](../operations/schema.md).
## The two logs
- The per-page **`## Changelog`** records infrastructure changes and is machine-parsed
(`get_changelog`, the Oikos ledger). Keep the `### YYYY-MM-DD — title` shape.
- **`knowledge/log.md`** is append-only and records *documentation-maintenance* operations only
(restructures, source ingests, lint sweeps): `## [YYYY-MM-DD] <op> | <summary>`. It never
duplicates the Oikos change ledger (`oikos/ledger.py`).
- The per-document **`## Changelog`** records infrastructure changes to that node. Keep the
`### YYYY-MM-DD — title` shape so the parsed `changelog` field stays structured.
- **`archive/knowledge/log.md`** is the append-only record of *documentation-maintenance*
operations on the legacy wiki (restructures, source ingests, lint sweeps):
`## [YYYY-MM-DD] <op> | <summary>`. It is frozen with the rest of `archive/knowledge/`; new
doc-maintenance operations are recorded in the DB audit trail instead.
## Querying knowledge
Use MCP, not grep:
- `search_knowledge(query)` — ILIKE search over documents, investigations, and runbooks in
`knowledge_entities`.
- `get_entity_knowledge(entity_slug)` — every document, investigation, and runbook linked to one
entity, in one call.
- `get_entity(slug)` / `get_relations(entity)` — the structured graph around an entity.
Grep the clone only when MCP is unreachable, and prefer `archive/knowledge/` for historical
narrative (it is not updated when the DB changes).
## Same-session update rule
A change to a node updates every page that references it in the same session — the node page, the
section `README.md` table, the root `README.md`, the Caddy/DNS/ingress pages, the host page, and
`inventory.yaml`. See [page-templates.md](../../shared/page-templates.md#same-session-update-rule).
A change to a node updates the DB in the same session — the entity's attributes, the relationships
that reference it, and any document whose `at_glance` or changelog should reflect the new state. See
[page-templates.md](../../shared/page-templates.md#same-session-update-rule) for the legacy wiki
equivalent (now scoped to `archive/knowledge/` history).

View File

@@ -6,9 +6,10 @@ follows [writing-style](../../shared/writing-style.md); runbooks and plans use t
exception.
Where each kind lives: runbooks are skills under [`.agents/skills/`](../../skills/); operator
reference (command cheatsheet, enrollment, Hermes agent) lives in
[`.agents/operations/`](../../operations/); investigations are sources under
`knowledge/sources/investigations/`; plans stay in the repo-root `plans/` folder (below).
reference (command cheatsheet, enrollment, Nomos agent) lives in
[`.agents/operations/`](../../operations/); investigations are `investigation` entities in the DB
(historically `archive/knowledge/sources/investigations/`); plans stay in the repo-root `plans/`
folder (below).
## Plans always live in `plans/`
@@ -21,7 +22,7 @@ message.** An agent drafting a plan:
3. On completion, moves it to `plans/done/` and updates the index status.
This is the single source for homelab design intent; keeping it in-repo means the plan is
versioned, reviewable, and reachable by MCP `get_page`/`search_docs` like any other doc.
versioned, reviewable, and reachable by MCP `search_knowledge` like any other doc.
## Runbooks
@@ -44,12 +45,14 @@ transition: "<from> -> <to>" # only for lifecycle runbooks
## Investigations
Incident records live in `knowledge/sources/investigations/YYYY-MM-DD-slug.md` and are **evidence sources** — written
once at incident time, then linked from the changelogs of the nodes they implicate. Sections:
`## Summary`, `## Timeline`, `## Root cause`, `## Mitigations applied`, `## Open questions`. Resolved
incidents move to `knowledge/sources/investigations/archive/`.
Incident records are `investigation` entities in the DB, linked to the entities they implicate via
`about` edges. They are **evidence sources** — written once at incident time, then back-linked from
the changelogs of the nodes they implicate. Sections: `## Summary`, `## Timeline`, `## Root cause`,
`## Mitigations applied`, `## Open questions`. The legacy file-based investigations live at
`archive/knowledge/sources/investigations/` (frozen 2026-07-07); new investigations go in the DB.
## The operations log
`plans/log.md` and `knowledge/log.md` are append-only records of documentation operations on
those areas (`## [YYYY-MM-DD] <op> | <summary>`), distinct from the Oikos change ledger.
`plans/log.md` is the append-only record of documentation operations on plans
(`## [YYYY-MM-DD] <op> | <summary>`), distinct from the DB audit trail. The legacy
`archive/knowledge/log.md` is frozen with the rest of the archived wiki.

View File

@@ -1,40 +1,49 @@
# LLM Wiki — the documentation contract
How the narrative documentation in this repo is organized. The pattern is borrowed from the
`sources / wiki / index / log` model: a durable synthesized layer (`archive/knowledge/`) built on top
of immutable evidence (`knowledge/sources/`, incident records), with pure-listing indexes and an
append-only operations log.
How documentation in this repo is organized. The pattern is the `sources / wiki / index / log`
model: a durable synthesized layer built on top of immutable evidence, with pure-listing indexes and
an append-only operations log.
This contract governs the **narrative layer only**. The machine-readable substrate — `inventory.yaml`,
`secrets/`, `scripts/`, `bin/` — is not part of the wiki and never
moves under it. See [the knowledge schema](../domains/knowledge/schema.md) for the split.
This contract governs the **narrative layer only**. The machine-readable source of truth — the
Postgres database, bootstrapped from `seeds/` — is not part of the wiki and never moves under it.
See [the knowledge schema](../domains/knowledge/schema.md) for the split, and ADR 0003 for the
DB-native model.
## Layers
- **Sources** are immutable raw material: incident records (`knowledge/sources/investigations/`), external reference
docs (`knowledge/sources/references/`), and the live system itself (`pct config`, `docker inspect`).
Read them; do not rewrite them into other sources.
- **Wiki** (`archive/knowledge/`) is the synthesized, authoritative current-state layer: one page per
node (`containers/`, `vms/`, host narratives) and per cross-cutting system (`infrastructure/`). A
reader understands the topic from the wiki page without reading the sources.
- **Source of truth** is the Postgres database. Structured data (entities, relationships, status,
metrics) and narrative knowledge (documents, investigations, runbooks) both live there, in the
`entities` / `relationships` / `knowledge_entities` tables. It is bootstrapped at deploy time from
`seeds/{ontology,inventory,policy,knowledge}.yaml` (idempotent, content-hashed via
`seed_versions`) and mutated at runtime via the API/MCP. `oikos export` regenerates
`seeds/{ontology,inventory,policy}.yaml` for version control.
- **Sources** are immutable raw material: incident records (now `investigation` entities in the DB,
historically `archive/knowledge/sources/investigations/`), external reference docs, and the live
system itself (`pct config`, `docker inspect`). Read them; do not rewrite them into other sources.
- **Wiki** — the synthesized, authoritative current-state layer. Today this is the set of
`document` entities in the DB (one per node and per cross-cutting system), queried via MCP
`search_knowledge` / `get_entity_knowledge`. The legacy file-based wiki is frozen at
`archive/knowledge/{hosts,containers,vms,infrastructure}/` for historical reference only.
- **Index** (`index.md` / folder `README.md`) is a pure listing — every page in scope with a
one-line summary, and nothing else. Anything the section wants to say up front goes into a page
the index lists, not into the index.
- **Log** (`log.md`) is append-only, recording *doc-maintenance operations* (restructures, source
ingests, lint sweeps) in single-line format: `## [YYYY-MM-DD] <op> | <summary>`.
- **Log** is append-only, recording *doc-maintenance operations* (restructures, source ingests,
lint sweeps) in single-line format: `## [YYYY-MM-DD] <op> | <summary>`. The active log is the DB
audit trail; `archive/knowledge/log.md` is the frozen legacy equivalent.
## Two logs, kept distinct
- **`## Changelog`** on each node/topic page records *infrastructure* changes to that node. It is
machine-parsed (`get_changelog`, the Oikos ledger) — keep the `### YYYY-MM-DD — title` shape.
- **`log.md`** per area records *documentation* operations only. It never duplicates the Oikos
change ledger (`oikos/ledger.py`), which stays authoritative for infra changes with
who/what/risk/approval/verification.
- **`## Changelog`** on each node/topic document records *infrastructure* changes to that node. It
is stored as a structured field on the `document` entity — keep the `### YYYY-MM-DD — title`
shape so it parses cleanly.
- **Doc-maintenance logs** record *documentation* operations only. They never duplicate the
infrastructure changelog, which stays authoritative for infra changes with
who/what/risk/approval/verification (now the DB audit trail, formerly `oikos/ledger.py`).
## Rules
- Wiki pages stay short and focused. A page past ~300 lines splits.
- Pages stay flat under `wiki/<section>/` until there are enough to warrant a sub-group.
- Pages stay flat under their section until there are enough to warrant a sub-group.
- Every page follows [writing-style.md](writing-style.md).
- Plans and design docs always live in the repo `plans/` folder (`plans/YYYY-MM-DD-slug.md`),
listed in `plans/index.md`, moved to `plans/done/` on completion — never a scratch path or a chat

View File

@@ -14,7 +14,7 @@ in [writing-style.md](writing-style.md); the layer model (sources / wiki / index
**Content / narrative pages:** lowercase-with-dashes, date-prefixed as needed
- **Container pages:** `<id>-<name>.md` (e.g. `101-jellyfin.md`, `132-rclone.md`). The `<id>` is the LXC/VM ordinal from `inventory.yaml`.
- **Container pages:** `<id>-<name>.md` (e.g. `101-jellyfin.md`, `132-rclone.md`). The `<id>` is the LXC/VM ordinal from the entity's attributes in the DB (seeded via `seeds/inventory.yaml`).
- **Infrastructure / cross-cutting pages:** `<topic>.md` (e.g. `dns.md`, `auto-deploy.md`, `mesh.md`). Describes a system, not a specific node.
- **Plans / investigations:** `YYYY-MM-DD-<slug>.md` (e.g. `2026-07-05-oikos-prometheus-lxc.md`). Date-sorted; slug is lowercase.
- **Section indices:** `README.md` (lowercase, conventional). Prefer in folders; `index.md` only if both intro prose and listing coexist.
@@ -118,7 +118,7 @@ What it looks like after.
Changelog entries to write, index status to update.
```
### Investigation (`knowledge/sources/investigations/YYYY-MM-DD-slug.md`)
### Investigation (`investigation` entity in the DB; historically `archive/knowledge/sources/investigations/YYYY-MM-DD-slug.md`)
```markdown
# YYYY-MM-DD — <title>
@@ -152,16 +152,18 @@ Changelog entries to write, index status to update.
## Same-session update rule
When you make a change to a node — migrate an LXC, update an IP, change a
mount, deploy a new service — **update every relevant doc page in the same
session.** A change that touches a container page must also update:
mount, deploy a new service — **update the DB and every relevant doc page in
the same session.** A change that touches a container must also update:
- The `containers/index.md` table (IPs, host, mounts, status)
- The `entities` / `relationships` rows for the node (via the API/MCP) —
this is the source of truth
- The `document` entity's `at_glance` and `## Changelog` for the container
- The `containers/index.md` table in the archived wiki (IPs, host, mounts,
status) — historical reference, update for consistency where still consulted
- The `README.md` table (if the change affects listed columns)
- The Caddy page site list (if the change affects `*.hubris.network` routing)
- The DNS / ingress infrastructure pages (if the change affects routing)
- The `hosts/{hubris,strong}.md` host page (if container count changes)
- The `inventory.yaml` host entry (single source of truth)
- The `infrastructure/topology.md` (generated from inventory, but regen if needed)
The pattern of updating only one page and leaving stale references on others
is a bug. If you're doing a multi-step migration, document the intermediate

View File

@@ -36,7 +36,7 @@ Every doc-level page follows the same shape so a reader scans it in one pass.
1. **One H1 = the page title.** Node pages use `# <id> — \`<name>\``; topic pages use `# <Topic>`.
2. **Opening definition.** First paragraph, 13 sentences, says what the thing is. No motivation, no marketing, no setup.
3. **Body sections** in the natural order for the topic. Reuse the section templates in [page-templates.md](page-templates.md).
4. **`## Changelog`** at the bottom of every node/topic page — reverse-chronological, append-only. This section is machine-parsed (Go MCP `get_changelog` in `internal/mcp/server.go`); keep the `### YYYY-MM-DD — title` shape.
4. **`## Changelog`** at the bottom of every node/topic page — reverse-chronological, append-only. This section is stored as a structured field on the `document` entity in the DB; keep the `### YYYY-MM-DD — title` shape so it parses cleanly.
5. **Related links** only at the bottom, only when a reference cannot be woven inline.
## Section indexes (folder READMEs)
@@ -53,12 +53,12 @@ duplicated prose, no narrative between the intro and the table.
- Prefer **tables** for enumerable items with internal structure (service/port maps, field lists, status grids). Reserve bullets for short non-structured lists.
- Use the **bold-leading-phrase pattern** for structured points: `**Read-only by construction.** The MCP server never mutates state.` — a bold noun phrase, a period, then the explanation.
- When enumerating across services or nodes, give each its own `###` sub-section or a table row, not one run-on paragraph.
- Use backticks for code, paths, hostnames, and file names (`inventory.yaml`, `192.168.8.77`, `pct config`); italics for first-mention terminology.
- Use backticks for code, paths, hostnames, and file names (`seeds/inventory.yaml`, `192.168.8.77`, `pct config`); italics for first-mention terminology.
- Use `>` blockquotes for caveats and gaps that interrupt the main flow: `> **Outstanding gap.** DNS-vs-inventory drift check not yet wired.` One thought per blockquote.
## Diagrams
- Mermaid is the default for topology and flow diagrams. `infrastructure/topology.md` is generated by `oikos/gen-topology.py` — do not hand-edit it. (Go DB-native topology generation planned.)
- Mermaid is the default for topology and flow diagrams. `infrastructure/topology.md` in the archived wiki was generated by the retired `oikos/gen-topology.py`; the DB-native equivalent is a future task — do not hand-edit the archived file expecting it to regenerate.
- ASCII box diagrams are fine for small shape diagrams; keep them to one screen.
## Sourcing and cross-references