Compare commits
28 Commits
750f0e088d
...
claude/cha
| Author | SHA1 | Date | |
|---|---|---|---|
| 614c38ea7c | |||
| 22412d2fa3 | |||
| 5686b9de40 | |||
| aa6017e0ca | |||
| cbfd09c5df | |||
| 851b5dce67 | |||
| 279549c8c9 | |||
| a39e67b6e9 | |||
| 551497e0b3 | |||
| 4a5e68bafe | |||
| ef00e5b8e4 | |||
| a512d40669 | |||
| b8bb29464b | |||
| 81acadec1d | |||
| 291b45565b | |||
| d45f2326b6 | |||
| 4bf811a383 | |||
| 35feada286 | |||
| cca2ae4621 | |||
| d13f6991b1 | |||
| 2de3602ebc | |||
| 2908b0a377 | |||
| cff05c0768 | |||
| 5d02126e16 | |||
| e8e230b4a5 | |||
| 2b3aa248b1 | |||
| 9d9cbb63c4 | |||
| ae3b150dcc |
@@ -1,4 +1,4 @@
|
||||
# HERMES.md — Agent persona for homelab clients
|
||||
# NOMOS.md — Agent persona for homelab clients
|
||||
|
||||
This file is the canonical agent persona for **all** AI agents running on
|
||||
machines in the **hubris** homelab. It prescribes behaviour, token-efficiency
|
||||
@@ -32,8 +32,8 @@ approval flow, ontology).
|
||||
|
||||
| Agent | Loading mechanism |
|
||||
|-------|------------------|
|
||||
| **Hermes** | `tools/setup-hermes-soul.sh` (auto-setup) → provisions `~/.hermes/SOUL.md` from this file |
|
||||
| **Goose** | `.goosehints` symlink at `~/.config/goose/.goosehints` → `/opt/homelab-context/HERMES.md` |
|
||||
| **Nomos** | `tools/setup-nomos-soul.sh` (auto-setup) → provisions `~/.nomos/SOUL.md` from this file |
|
||||
| **Goose** | `.goosehints` symlink at `~/.config/goose/.goosehints` → `/opt/homelab-context/NOMOS.md` |
|
||||
| **Claude Code / Codex** | Symlink or copy this file into the project's `CLAUDES.md` / `.claude` instructions |
|
||||
|
||||
**Do not edit SOUL.md or .goosehints directly.** Edit this file in the
|
||||
@@ -87,10 +87,10 @@ Caveman templates live at `~/templates/`:
|
||||
ls ~/bin/caveman_wrapper.sh && echo "caveman ready"
|
||||
```
|
||||
|
||||
## Important note for Hermes agents
|
||||
## Important note for Nomos agents
|
||||
|
||||
If you are reading this as a Hermes agent, your SOUL.md was auto-provisioned
|
||||
by `tools/setup-hermes-soul.sh`. This file is the canonical original — you
|
||||
If you are reading this as a Nomos agent, your SOUL.md was auto-provisioned
|
||||
by `tools/setup-nomos-soul.sh`. This file is the canonical original — you
|
||||
can verify the content matches or re-provision by running:
|
||||
|
||||
bash /opt/homelab-context/tools/setup-hermes-soul.sh
|
||||
bash /opt/homelab-context/tools/setup-nomos-soul.sh
|
||||
@@ -137,13 +137,13 @@ in the Go binary.
|
||||
- Go packages: `internal/scheduler/`, `internal/actuator/`,
|
||||
`internal/learning/`, `internal/notifier/`, `internal/policy/`.
|
||||
|
||||
**Phase 4 — Agent / Hermes (DONE):**
|
||||
- Standalone Hermes MCP client binary (`cmd/hermes`) with gateway mode
|
||||
**Phase 4 — Agent / Nomos (DONE):**
|
||||
- Standalone Nomos MCP client binary (`cmd/nomos`) with gateway mode
|
||||
(:8092). Structured queries + natural-language routing to 15 MCP tools.
|
||||
Agent activity logging on every tool call. No SSH keys.
|
||||
- `hermes/` directory with config, SOUL.md, homelab-ops skill.
|
||||
- Hermes Docker service in `docker-compose.yml` (profile: full).
|
||||
- Go packages: `cmd/hermes/`, `compose/hermes/`.
|
||||
- `nomos/` directory with config, SOUL.md, homelab-ops skill.
|
||||
- Nomos Docker service in `docker-compose.yml` (profile: full).
|
||||
- Go packages: `cmd/nomos/`, `compose/nomos/`.
|
||||
|
||||
**Phase 5 — Secrets / Infisical (DONE):**
|
||||
- `internal/secrets/`: backend abstraction (Manager) with primary
|
||||
@@ -159,7 +159,7 @@ in the Go binary.
|
||||
lint, test, docker build).
|
||||
- Deploy: `scripts/deploy.sh` (git pull → docker build → compose up →
|
||||
health check), SHA-tagged images, rolling restart.
|
||||
- Caddy config: `compose/caddy/Caddyfile.oikos` (oikos/mcp/hermes →
|
||||
- Caddy config: `compose/caddy/Caddyfile.oikos` (oikos/mcp/nomos →
|
||||
mac-mini mesh :8090/:8092).
|
||||
- Watchdog: `scripts/watchdog.sh` (2min cron, Matrix alert on failure).
|
||||
- Verification: `scripts/verify-phase6.sh` (14/14 checks pass).
|
||||
@@ -168,7 +168,7 @@ in the Go binary.
|
||||
|
||||
**Current deployment:**
|
||||
- **Production**: Docker stack on mac-mini (`--profile full`: postgres, api,
|
||||
scheduler, notifier, hermes). Deployed 2026-07-07 with full knowledge seed.
|
||||
scheduler, notifier, nomos). Deployed 2026-07-07 with full knowledge seed.
|
||||
The Python MCP server and secrets-issuance on apps/105 have been stopped
|
||||
(see `scripts/cutover-checklist.md`).
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ see [CONTRIBUTING.md](../../CONTRIBUTING.md) for a human-friendly version.
|
||||
```
|
||||
cmd/oikos/main.go Entry point. Subcommands: api, scheduler, notifier, migrate,
|
||||
seed, export, secret, all
|
||||
cmd/hermes/main.go Hermes MCP client gateway (standalone binary)
|
||||
cmd/nomos/main.go Nomos MCP client gateway (standalone binary, formerly Hermes)
|
||||
internal/httpapi/ REST + MCP server. Chi router. OpenAPI-generated types from
|
||||
internal/httpapi/gen/api.gen.go. Strict server in impl.go.
|
||||
internal/mcp/ MCP tool implementations (get_entity, search_knowledge, etc.)
|
||||
@@ -31,10 +31,10 @@ api/codegen.yaml oapi-codegen config → generates internal/httpapi/g
|
||||
migrations/ Forward-only SQL. Format: NNN_name.up.sql. No down migrations.
|
||||
seeds/ Bootstrap YAML. ontology.yaml, inventory.yaml, policy.yaml,
|
||||
knowledge.yaml. Regenerated from DB via oikos export.
|
||||
compose/ Dockerfiles. oikos/ (multi-stage), hermes/ (distroless).
|
||||
compose/ Dockerfiles. oikos/ (multi-stage), nomos/ (distroless).
|
||||
Caddy config at compose/caddy/Caddyfile.oikos.
|
||||
scripts/ Deploy, rollback, watchdog, verification, cutover checklist.
|
||||
hermes/ Hermes config.yaml, SOUL.md, skills.
|
||||
nomos/ Nomos config.yaml, SOUL.md, skills.
|
||||
.agents/ Agent instruction files, domains, shared conventions, skills.
|
||||
plans/ Design documents. active/ + done/.
|
||||
docs/adr/ Architecture decision records. Numbered, prefix-sorted.
|
||||
|
||||
@@ -6,8 +6,8 @@ this repo that auto-syncs every 5 min, a per-client age key for SOPS
|
||||
decryption, the `homelab` CLI, and an MCP endpoint in Claude Code's config.
|
||||
|
||||
> Onboarding a Nous-Hermes-powered Goose agent on top of standard enrollment?
|
||||
> See [hermes-agent.md](hermes-agent.md). It uses the same `bootstrap.sh`
|
||||
> with an additional `--with-hermes` flag.
|
||||
> See [nomos-agent.md](nomos-agent.md). It uses the same `bootstrap.sh`
|
||||
> with an additional `--with-nomos` flag.
|
||||
|
||||
Architecture in [project_homelab_context_plan](https://… memory link); the
|
||||
operational reference is here.
|
||||
@@ -354,9 +354,9 @@ Added a new "Post-bootstrap: SSH reachability" section covering SSH key
|
||||
generation, pubkey publication, deployment to hosts, SSH config generation,
|
||||
and LAN IP registration. New workstations enrolled via this doc will
|
||||
automatically join the universal SSH mesh.
|
||||
### 2026-05-31 — cross-link to nomos-agent.md
|
||||
|
||||
### 2026-05-31 — cross-link to hermes-agent.md
|
||||
Added a sibling page covering Nous-Hermes-on-Goose enrollment ([hermes-agent.md](hermes-agent.md)) and noted it at the top of this page. The Hermes flow extends `bootstrap.sh` with `--with-hermes` and `homelab client add` with the same flag; it does not change the underlying enrollment steps documented here.
|
||||
Added a sibling page covering Nous-Hermes-on-Goose enrollment ([nomos-agent.md](nomos-agent.md)) and noted it at the top of this page. The Nomos flow extends `bootstrap.sh` with `--with-nomos` and `homelab client add` with the same flag; it does not change the underlying enrollment steps documented here.
|
||||
|
||||
### 2026-05-21 — netbird-ssh JWT issuer + username + LAN-fallback troubleshooting rows
|
||||
Added three rows to the troubleshooting table covering issues surfaced during the netbird vanilla migration: (1) post-migration SSH JWT validator cache stuck on old Dex issuer (full `systemctl stop/start` required, not `restart`), (2) `user not found` from netbird-ssh's local-username default (use explicit `root@`), and (3) homelab CLI's LAN→netbird-FQDN fallback for off-LAN operators. Companion code change: per-host `ssh.user` field in `inventory.yaml` + `homelab` CLI's `ssh_target()` helper.
|
||||
|
||||
@@ -14,7 +14,7 @@ Run from the [hubris host](../../archive/knowledge/hosts/hubris.md) as root. Whe
|
||||
| `pvesm status` | Storage pools status |
|
||||
| `pvesh get /nodes --output-format json` | Node summary as JSON |
|
||||
| `pvesh get /nodes/hubris/lxc/<id>/status/current` | Live container status |
|
||||
| `pvesh get /cluster/resources --type vm --output-format json` | Bulk per-LXC CPU/mem/disk (used by the `homelab-health-watchdog` Hermes cron — see [monitoring](../../archive/knowledge/infrastructure/monitoring.md); the old `claudio-monitor` this once fed is deprecated) |
|
||||
| `pvesh get /cluster/resources --type vm --output-format json` | Bulk per-LXC CPU/mem/disk (used by the `homelab-health-watchdog` Nomos cron — see [monitoring](../../archive/knowledge/infrastructure/monitoring.md); the old `claudio-monitor` this once fed is deprecated) |
|
||||
| `pveversion` | PVE version |
|
||||
| `journalctl -u pve-cluster -n 100` | PVE service logs |
|
||||
|
||||
@@ -77,7 +77,7 @@ See [OIKOS.md](../OIKOS.md) for the operating model. Quick reference:
|
||||
| `homelab change preflight <service>` | Dry-run report before mutating: risk class, current health, config repo, verification command |
|
||||
| `homelab decide <action> <entity>` | Decision classifier: risk × blast radius × confidence → auto-act or escalate |
|
||||
| `homelab signal list\|raise\|ack\|resolve\|mute` | The attention layer — pending updates, thresholds, drift, anything needing attention |
|
||||
| `homelab approval request\|list\|reply\|check` | Escalate-route grants (Matrix-delivered via Hermes, or the Oikos Console's `/approvals` page) |
|
||||
| `homelab approval request\|list\|reply\|check` | Escalate-route grants (Matrix-delivered via Nomos, or the Oikos Console's `/approvals` page) |
|
||||
| `homelab restart <service> [--approval-id <id>]` | `--approval-id` is required whenever the service's risk class needs approval (e.g. `caddy`, `dns`) — refuses mechanically without a valid grant |
|
||||
|
||||
Oikos Console (read-mostly dashboard): `oikos.hubris.network` once deployed — see [oikos/console/deploy/README.md](../../archive/oikos-cards/).
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Hermes agent — Nous-Hermes-powered Goose sessions on a homelab client
|
||||
# Nomos agent — LLM-powered terminal sessions on a homelab client
|
||||
|
||||
Onboards [Nous Research's Hermes](https://nousresearch.com/) (a fine-tuned
|
||||
Llama variant) as a working terminal agent on a homelab client. Builds on top
|
||||
@@ -8,13 +8,13 @@ of standard client enrollment (see [agent-enrollment.md](agent-enrollment.md))
|
||||
The agent runs as a [Goose](https://goose-docs.ai/) session. Goose provides:
|
||||
|
||||
- The chat loop, multi-turn history, and streaming
|
||||
- The OpenRouter provider that routes to Nous Hermes
|
||||
- The OpenRouter provider that routes to the configured LLM
|
||||
- The built-in `developer` extension (shell + file editor — same surface Claude
|
||||
Code has)
|
||||
- A remote MCP extension pointed at `mcp.hubris.network` for read-only
|
||||
homelab context (`list_lxcs`, `tail_log`, `search_docs`, etc.)
|
||||
|
||||
The persona is `/opt/homelab-context/HERMES.md`, symlinked as Goose's global
|
||||
The persona is `/opt/homelab-context/NOMOS.md`, symlinked as Goose's global
|
||||
`.goosehints` so it's injected into the system prompt on every session.
|
||||
|
||||
## Prerequisites
|
||||
@@ -23,7 +23,7 @@ The persona is `/opt/homelab-context/HERMES.md`, symlinked as Goose's global
|
||||
| --- | --- |
|
||||
| Standard enrollment complete (`homelab whoami` works) | [agent-enrollment.md](agent-enrollment.md) |
|
||||
| `secrets/openrouter-api-key.yaml` exists with a real `sk-or-...` value | See "Seeding the OpenRouter key" below |
|
||||
| The host's `age_pubkey` is on the openrouter-api-key.yaml sops rule | `homelab client add <host> --finalize-pubkey <age1...> --with-hermes` |
|
||||
| The host's `age_pubkey` is on the openrouter-api-key.yaml sops rule | `homelab client add <host> --finalize-pubkey <age1...> --with-nomos` |
|
||||
|
||||
## Onboarding flow
|
||||
|
||||
@@ -33,37 +33,37 @@ homelab client add new-machine
|
||||
|
||||
# 2. Join new-machine to Netbird (setup-key or OIDC).
|
||||
|
||||
# 3. On new-machine: bootstrap with --with-hermes.
|
||||
# 3. On new-machine: bootstrap with --with-nomos.
|
||||
TOKEN=... # gitea PAT, read:repository
|
||||
curl -fsSL -u "dtoro:$TOKEN" \
|
||||
https://git.hubris.network/dtoro/oikos/raw/branch/main/bootstrap.sh \
|
||||
-o /tmp/bootstrap.sh
|
||||
sudo HOMELAB_GITEA_TOKEN=$TOKEN bash /tmp/bootstrap.sh --with-mcp --with-hermes
|
||||
sudo HOMELAB_GITEA_TOKEN=$TOKEN bash /tmp/bootstrap.sh --with-mcp --with-nomos
|
||||
|
||||
# 4. Back on hubris: finalize the age pubkey AND grant the Hermes secret.
|
||||
# 4. Back on hubris: finalize the age pubkey AND grant the Nomos secret.
|
||||
homelab client add new-machine \
|
||||
--finalize-pubkey age1... \
|
||||
--with-hermes
|
||||
--with-nomos
|
||||
|
||||
# 5. Wait ≤5 min for sync, then on new-machine:
|
||||
hermes "what LXCs are running?"
|
||||
nomos "what LXCs are running?"
|
||||
```
|
||||
|
||||
The bootstrap `--with-hermes` flag does five things, all idempotent:
|
||||
The bootstrap `--with-nomos` flag does five things, all idempotent:
|
||||
|
||||
1. Downloads the latest Goose binary into the operator's `~/.local/bin/goose`
|
||||
(upstream installer) and symlinks `/usr/local/bin/goose` to it.
|
||||
2. Symlinks `/opt/homelab-context/bin/hermes` → `/usr/local/bin/hermes`.
|
||||
3. Symlinks `/opt/homelab-context/HERMES.md` → `/root/HERMES.md` (Linux) or
|
||||
`/etc/HERMES.md` (macOS) for `cat`-as-operator convenience.
|
||||
2. Symlinks `/opt/homelab-context/bin/nomos` → `/usr/local/bin/nomos`.
|
||||
3. Symlinks `/opt/homelab-context/NOMOS.md` → `/root/NOMOS.md` (Linux) or
|
||||
`/etc/NOMOS.md` (macOS) for `cat`-as-operator convenience.
|
||||
4. Drops `~/.config/goose/config.yaml` pinning the provider, model, and
|
||||
extensions (preserves any keys the operator added by hand).
|
||||
5. Symlinks `~/.config/goose/.goosehints` → HERMES.md, so the persona is
|
||||
5. Symlinks `~/.config/goose/.goosehints` → NOMOS.md, so the persona is
|
||||
injected as the system prompt on every session.
|
||||
|
||||
## Seeding the OpenRouter key
|
||||
|
||||
The first time anyone enrolls with `--with-hermes`, the encrypted file
|
||||
The first time anyone enrolls with `--with-nomos`, the encrypted file
|
||||
`secrets/openrouter-api-key.yaml` contains a placeholder. On hubris (or any
|
||||
existing recipient):
|
||||
|
||||
@@ -75,19 +75,19 @@ git -C /opt/homelab-context commit -m 'openrouter-api-key: seed real key'
|
||||
git -C /opt/homelab-context push
|
||||
```
|
||||
|
||||
Until this step happens, `hermes …` exits with `openrouter-api-key.yaml still
|
||||
Until this step happens, `nomos …` exits with `openrouter-api-key.yaml still
|
||||
contains the placeholder`. Subsequent enrollees get the real key automatically
|
||||
via `--with-hermes` (which adds them as a sops recipient on
|
||||
via `--with-nomos` (which adds them as a sops recipient on
|
||||
`secrets/openrouter-api-key.yaml`).
|
||||
|
||||
## Granting the OpenRouter key to an already-enrolled host
|
||||
|
||||
If a host was enrolled without `--with-hermes` and you want to add it later:
|
||||
If a host was enrolled without `--with-nomos` and you want to add it later:
|
||||
|
||||
```bash
|
||||
# On hubris:
|
||||
PUBKEY=$(homelab whoami --hostname <host> | grep age_pubkey | awk '{print $2}')
|
||||
homelab client add <host> --finalize-pubkey "$PUBKEY" --with-hermes
|
||||
homelab client add <host> --finalize-pubkey "$PUBKEY" --with-nomos
|
||||
```
|
||||
|
||||
`--finalize-pubkey` is required by the existing flow even when the pubkey is
|
||||
@@ -101,12 +101,12 @@ re-run; only the secret recipient list changed.
|
||||
```bash
|
||||
homelab whoami # standard enrollment OK
|
||||
homelab secret openrouter-api-key | head -c 8 # decrypts (prints `api_key:`)
|
||||
which goose && which hermes # binaries present
|
||||
which goose && which nomos # binaries present
|
||||
goose info -v # provider/model wiring sane
|
||||
hermes "what LXCs are running?" # interactive Goose session
|
||||
nomos "what LXCs are running?" # interactive Goose session
|
||||
|
||||
# Non-interactive smoke test:
|
||||
echo "List the homelab MCP tools you have available" | hermes
|
||||
echo "List the homelab MCP tools you have available" | nomos
|
||||
```
|
||||
|
||||
## Configuration
|
||||
@@ -135,9 +135,9 @@ extensions:
|
||||
Override via env on a single bootstrap run:
|
||||
|
||||
```bash
|
||||
HOMELAB_HERMES_MODEL=nousresearch/hermes-3-llama-3.1-405b \
|
||||
HOMELAB_HERMES_MCP_URI=https://mcp.hubris.network/mcp \
|
||||
sudo bash /tmp/bootstrap.sh --with-hermes
|
||||
HOMELAB_NOMOS_MODEL=nousresearch/hermes-3-llama-3.1-405b \
|
||||
HOMELAB_NOMOS_MCP_URI=https://mcp.hubris.network/mcp \
|
||||
sudo bash /tmp/bootstrap.sh --with-nomos
|
||||
```
|
||||
|
||||
Any keys you add by hand (e.g. `GOOSE_TEMPERATURE`, extra `extensions.*`) are
|
||||
@@ -156,22 +156,22 @@ every tool call, use `approve`. See
|
||||
|
||||
| Symptom | Cause | Fix |
|
||||
| --- | --- | --- |
|
||||
| `hermes: could not decrypt secrets/openrouter-api-key.yaml` | Host isn't a recipient on the sops rule | `homelab client add <host> --finalize-pubkey <age1...> --with-hermes` from hubris |
|
||||
| `hermes: openrouter-api-key.yaml still contains the placeholder` | No real key has been seeded yet | See "Seeding the OpenRouter key" above |
|
||||
| Goose hangs on first `hermes` invocation with no output | Goose's interactive `configure` ran on first launch and is awaiting input | Re-run; the installer is supposed to skip it (CONFIGURE=false). If it persists, run `goose configure` once manually in a real terminal to commit the config. |
|
||||
| `nomos: could not decrypt secrets/openrouter-api-key.yaml` | Host isn't a recipient on the sops rule | `homelab client add <host> --finalize-pubkey <age1...> --with-nomos` from hubris |
|
||||
| `nomos: openrouter-api-key.yaml still contains the placeholder` | No real key has been seeded yet | See "Seeding the OpenRouter key" above |
|
||||
| Goose hangs on first `nomos` invocation with no output | Goose's interactive `configure` ran on first launch and is awaiting input | Re-run; the installer is supposed to skip it (CONFIGURE=false). If it persists, run `goose configure` once manually in a real terminal to commit the config. |
|
||||
| `homelab` extension fails to connect / no MCP tools listed | MCP server upgraded in Go rewrite (`internal/mcp/server.go`, Streamable HTTP via official MCP SDK). Old FastMCP SSE transport is deprecated. | Run `docker compose --profile full up` on mac-mini, or wait for the production cutover from apps/105. |
|
||||
| `goose: command not found` after bootstrap | Upstream installer dropped binary in `~/.local/bin/` but `/usr/local/bin/goose` symlink didn't land | Re-run bootstrap with `--with-hermes`; the symlink step is at the end of the install block. If still missing, `ln -sfn ~/.local/bin/goose /usr/local/bin/goose` manually. |
|
||||
| `goose: command not found` after bootstrap | Upstream installer dropped binary in `~/.local/bin/` but `/usr/local/bin/goose` symlink didn't land | Re-run bootstrap with `--with-nomos`; the symlink step is at the end of the install block. If still missing, `ln -sfn ~/.local/bin/goose /usr/local/bin/goose` manually. |
|
||||
| Tool calls hit OpenRouter rate limits | One shared key across many hosts | Future: per-host keys; for now, see the rate-limits guide referenced in `goose info -v`. |
|
||||
|
||||
## Cross-references
|
||||
|
||||
- [agent-enrollment.md](agent-enrollment.md) — base client onboarding the
|
||||
Hermes flow assumes is done.
|
||||
- [`HERMES.md`](../HERMES.md) — the persona the Hermes agent reads on every
|
||||
Nomos flow assumes is done.
|
||||
- [`NOMOS.md`](../NOMOS.md) — the persona the Nomos agent reads on every
|
||||
session start (via `~/.config/goose/.goosehints`).
|
||||
- [`bin/hermes`](../../bin/hermes) — the wrapper that decrypts the OpenRouter key
|
||||
- [`bin/nomos`](../../bin/nomos) — the wrapper that decrypts the OpenRouter key
|
||||
and execs `goose session`.
|
||||
- [`bootstrap.sh`](../../bootstrap.sh) — the `--with-hermes` flag's install block.
|
||||
- [`bootstrap.sh`](../../bootstrap.sh) — the `--with-nomos` flag's install block.
|
||||
|
||||
## Follow-ups
|
||||
|
||||
@@ -182,7 +182,7 @@ every tool call, use `approve`. See
|
||||
that's changed, the `homelab` MCP extension in Goose will fail to connect.
|
||||
The developer extension (shell + edit) covers most ops without it; this is
|
||||
a polish item, not a blocker.
|
||||
2. **Per-host OpenRouter keys** for billing attribution. Today all Hermes
|
||||
2. **Per-host OpenRouter keys** for billing attribution. Today all Nomos
|
||||
hosts share one key.
|
||||
3. **Pin the model version** rather than tracking `nousresearch/hermes-4-405b`
|
||||
directly — OpenRouter periodically rotates the underlying weights.
|
||||
@@ -204,7 +204,7 @@ templating + `~/bin/caveman_wrapper.sh` + `~/templates/*.txt` for token-
|
||||
efficient CLI output. Replaces raw `git pull` in launchd/systemd timers.
|
||||
Also created `tools/caveman/` with the wrapper script, JS renderer, and
|
||||
templates — the canonical source for all agent hosts.
|
||||
Captures the Hermes-on-Goose onboarding flow added in the same commit as
|
||||
`bootstrap.sh --with-hermes`, `bin/hermes`, the sops rule for
|
||||
`secrets/openrouter-api-key.yaml`, and the `homelab client add --with-hermes`
|
||||
Captures the Nomos-on-Goose onboarding flow added in the same commit as
|
||||
`bootstrap.sh --with-nomos`, `bin/nomos`, the sops rule for
|
||||
`secrets/openrouter-api-key.yaml`, and the `homelab client add --with-nomos`
|
||||
extension. MCP streamable_http migration is queued as follow-up #1.
|
||||
@@ -30,4 +30,4 @@ Code/commits/PRs: write normal. "stop caveman" or "normal mode": revert. Level p
|
||||
---
|
||||
|
||||
Source: https://github.com/JuliusBrussee/caveman
|
||||
Copy to `~/.hermes/skills/` for Hermes Agent, or `~/.claude/projects/<name>/SKILL.md` for Claude Code.
|
||||
Copy to `~/.nomos/skills/` for Nomos agent, or `~/.claude/projects/<name>/SKILL.md` for Claude Code.
|
||||
|
||||
@@ -26,7 +26,7 @@ the full walkthrough; this runbook is the risk/lifecycle framing.
|
||||
routed `192.168.8.0/24` Netbird network resource. Skip this step for
|
||||
LAN-only nodes; do it (out-of-band, console or setup key) only for
|
||||
hosts that need independent off-LAN reachability.
|
||||
3. On the new host: run `bootstrap.sh` (add `--with-hermes` to also
|
||||
3. On the new host: run `bootstrap.sh` (add `--with-nomos` to also
|
||||
enroll the Hermes agent). This provisions `/etc/age/key.txt`, the
|
||||
sync timer, and prints an age pubkey.
|
||||
4. Back on an enrolled client: `homelab client add <hostname>
|
||||
|
||||
11
.claude/launch.json
Normal file
11
.claude/launch.json
Normal file
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"version": "0.0.1",
|
||||
"configurations": [
|
||||
{
|
||||
"name": "web",
|
||||
"runtimeExecutable": "npm",
|
||||
"runtimeArgs": ["--prefix", "web", "run", "dev"],
|
||||
"port": 5173
|
||||
}
|
||||
]
|
||||
}
|
||||
11
.gitignore
vendored
11
.gitignore
vendored
@@ -5,9 +5,9 @@ __pycache__/
|
||||
# Regenerated every scheduler run; ephemeral health-probe cache.
|
||||
oikos/state.json
|
||||
|
||||
# Compiled binaries (Go rewrite — bin/oikos, bin/hermes)
|
||||
# Compiled binaries (Go rewrite — bin/oikos, bin/nomos)
|
||||
bin/oikos
|
||||
bin/hermes
|
||||
bin/nomos
|
||||
oikos/oikos
|
||||
|
||||
# Legacy Python oikos (superseded by cmd/oikos Go binary — Phase 1-6 rewrite).
|
||||
@@ -17,3 +17,10 @@ oikos/oikos
|
||||
backups/
|
||||
.env
|
||||
.infisical-credentials
|
||||
|
||||
# Web UI (Svelte 5) — build artifacts. Ignore built output but keep the
|
||||
# .gitkeep placeholder so `//go:embed all:dist` (web/embed.go) compiles on a
|
||||
# fresh checkout before the UI is built.
|
||||
web/dist/*
|
||||
!web/dist/.gitkeep
|
||||
web/node_modules/
|
||||
|
||||
@@ -86,11 +86,11 @@ creation_rules:
|
||||
age1z62ff2ak9zj5ctcvaxwyyhedwjvlwgm2dkn9nk3wrwk8fkavcpmsqwc2vs
|
||||
|
||||
- path_regex: archive/secrets-sops-backupopenrouter-api-key\.yaml$
|
||||
# OpenRouter API key consumed by the `hermes` wrapper (bin/hermes) when
|
||||
# OpenRouter API key consumed by the `nomos` wrapper (bin/nomos) when
|
||||
# spawning a Goose session. Recipients are any host that should run a
|
||||
# Nous-Hermes agent. Add a host's age_pubkey here, then
|
||||
# Nomos agent. Add a host's age_pubkey here, then
|
||||
# `sops updatekeys -y secrets/openrouter-api-key.yaml`.
|
||||
# See operations/hermes-agent.md.
|
||||
# See operations/nomos-agent.md.
|
||||
age: >-
|
||||
age1xkklkvnk5z0fsnh6cfgv70hy9ksfy8rdprwerzw4yk3p4p7cxcqs2yvpz6,
|
||||
age1vf8h7s8mqsn2q5eadgpdupsj4mwn8zguc77d85ws3xj40sl9rgksx2rxw6,
|
||||
|
||||
12
AGENTS.md
12
AGENTS.md
@@ -124,9 +124,9 @@ per the DB-as-source-of-truth plan.
|
||||
|
||||
## 5. Acting on the homelab
|
||||
|
||||
- **Read state**: use MCP tools. Hermes (the AI agent) is the primary
|
||||
- **Read state**: use MCP tools. Nomos (the AI agent) is the primary
|
||||
operator interface — it has 21 MCP tools for observe/orient/decide/act.
|
||||
- **Actions** (restart, logs, apt, pct exec): Hermes calls `request_execution`
|
||||
- **Actions** (restart, logs, apt, pct exec): Nomos calls `request_execution`
|
||||
via MCP. `reversible_low` actions execute immediately; `config_mutation`
|
||||
and `destructive` actions are queued for operator approval via Matrix.
|
||||
- **Secrets**: managed by Infisical (`oikos secret` subcommand for migration).
|
||||
@@ -152,10 +152,10 @@ Currently auto-setup:
|
||||
- **Caveman + templates** (`tools/setup-caveman.sh`): Installs Caveman npm
|
||||
package, wrapper scripts, and compact output templates for token-efficient
|
||||
CLI output. Wrapper at `~/bin/caveman_wrapper.sh`.
|
||||
- **Hermes agent persona** (`tools/setup-hermes-soul.sh`): Provisions
|
||||
`~/.hermes/SOUL.md` from `HERMES.md` on Hermes agents. This ensures every
|
||||
Hermes agent follows the canonical homelab persona (token efficiency, source
|
||||
of truth hierarchy). No-op on non-Hermes agents.
|
||||
- **Nomos agent persona** (`tools/setup-nomos-soul.sh`): Provisions
|
||||
`~/.nomos/SOUL.md` from `NOMOS.md` on Nomos agents. This ensures every
|
||||
Nomos agent follows the canonical homelab persona (token efficiency, source
|
||||
of truth hierarchy). No-op on non-Nomos agents.
|
||||
|
||||
To add a new auto-setup, create `tools/<name>.setup.sh` in the repo,
|
||||
commit and push. All enrolled clients pick it up within 5 minutes.
|
||||
|
||||
@@ -41,7 +41,7 @@ curl -fsSL https://git.hubris.network/dtoro/oikos/raw/main/bootstrap.sh | sudo b
|
||||
|
||||
# Or with optional tooling:
|
||||
curl ... | sudo bash -s -- --with-mcp # wire Claude's MCP config
|
||||
curl ... | sudo bash -s -- --with-hermes # install Goose + Hermes
|
||||
curl ... | sudo bash -s -- --with-nomos # install Goose + Nomos
|
||||
```
|
||||
|
||||
This calls `POST /api/v1/clients/enroll` on the Oikos API, which:
|
||||
@@ -56,7 +56,7 @@ This calls `POST /api/v1/clients/enroll` on the Oikos API, which:
|
||||
### What changes on your machine
|
||||
|
||||
- `/opt/homelab/` — agent orientation files (CLIENTS.md, AGENTS.md, OIKOS.md)
|
||||
- `/opt/homelab/tools/` — tooling scripts (caveman, hermes-soul)
|
||||
- `/opt/homelab/tools/` — tooling scripts (caveman, nomos-soul)
|
||||
- `/etc/age/key.txt` — age private key for SOPS decryption (fallback)
|
||||
- `/etc/infisical/identity` — Infisical machine identity (primary secrets)
|
||||
- Context poller — launchd/systemd timer hits `GET /api/v1/clients/{slug}/context` every 5 minutes for agent file updates
|
||||
|
||||
@@ -28,7 +28,7 @@ make build
|
||||
|
||||
```
|
||||
cmd/oikos/ Single-binary entry point
|
||||
cmd/hermes/ Hermes MCP client gateway
|
||||
cmd/nomos/ Nomos MCP client gateway
|
||||
internal/ All Go packages
|
||||
httpapi/ REST + MCP server (OpenAPI-generated)
|
||||
mcp/ MCP tool implementations
|
||||
@@ -47,7 +47,7 @@ migrations/ Forward-only SQL migrations (TimescaleDB)
|
||||
seeds/ Bootstrap YAML: ontology, inventory, policy, knowledge
|
||||
compose/ Dockerfiles + Caddy config
|
||||
scripts/ Deploy, watchdog, rollback
|
||||
hermes/ Hermes config, persona, skills
|
||||
nomos/ Nomos config, persona, skills
|
||||
.agents/ Agent instruction files + skills
|
||||
plans/ Design documents
|
||||
docs/adr/ Architecture decision records
|
||||
|
||||
18
README.md
18
README.md
@@ -1,8 +1,8 @@
|
||||
# Oikos
|
||||
|
||||
Agentic homelab operating system written in Go. Single binary (`cmd/oikos`),
|
||||
Docker-deployed on mac-mini, with a standalone Hermes MCP agent gateway
|
||||
(`cmd/hermes`). Manages the **hubris** Proxmox homelab autonomously — observes
|
||||
Docker-deployed on mac-mini, with a standalone Nomos MCP agent gateway
|
||||
(`cmd/nomos`). Manages the **hubris** Proxmox homelab autonomously — observes
|
||||
state, classifies actions against policy, executes approved procedures over SSH,
|
||||
learns from outcomes, and escalates when uncertain.
|
||||
|
||||
@@ -16,7 +16,7 @@ learns from outcomes, and escalates when uncertain.
|
||||
# Dev stack (postgres + api + scheduler + notifier)
|
||||
docker compose --profile dev up -d
|
||||
|
||||
# Full stack (adds Hermes agent gateway)
|
||||
# Full stack (adds Nomos agent gateway)
|
||||
docker compose --profile full up -d
|
||||
|
||||
# Build standalone binary
|
||||
@@ -33,7 +33,7 @@ OIKOS_DATABASE_URL="postgres://oikos:oikos_dev@localhost:5432/oikos?sslmode=disa
|
||||
┌──────────────────────────────────┐
|
||||
│ mac-mini (Docker) │
|
||||
│ │
|
||||
Workstation ─── │ hermes (8092) ──MCP── api (8090) │
|
||||
Workstation ─── │ nomos (8092) ──MCP── api (8090) │
|
||||
(mesh) │ MCP gateway REST + MCP │
|
||||
│ │
|
||||
│ scheduler ── notifier ── postgres │
|
||||
@@ -46,7 +46,7 @@ OIKOS_DATABASE_URL="postgres://oikos:oikos_dev@localhost:5432/oikos?sslmode=disa
|
||||
| `oikos api` | 8090 | REST API + MCP server (15 tools) |
|
||||
| `oikos scheduler` | — | Probe runner, signal lifecycle, metrics |
|
||||
| `oikos notifier` | — | Approval tokens, Matrix alerts |
|
||||
| `hermes serve` | 8092 | MCP client gateway, query routing |
|
||||
| `nomos serve` | 8092 | MCP client gateway, query routing |
|
||||
|
||||
## Phases
|
||||
|
||||
@@ -55,7 +55,7 @@ OIKOS_DATABASE_URL="postgres://oikos:oikos_dev@localhost:5432/oikos?sslmode=disa
|
||||
| 1 — Ontology + DB | ✅ | TimescaleDB, migrations, seeds, blast_radius |
|
||||
| 2 — API | ✅ | OpenAPI-first REST + MCP, auth, SSE, audit |
|
||||
| 3 — Control loop | ✅ | Scheduler, actuator, learning, classifier, notifier |
|
||||
| 4 — Hermes agent | ✅ | Standalone MCP client gateway, agent activity |
|
||||
| 4 — Nomos agent | ✅ | Standalone MCP client gateway, agent activity |
|
||||
| 5 — Secrets | ✅ | Infisical backend + SOPS fallback, rotation runbooks |
|
||||
| 6 — Deploy | ✅ | CI pipeline, cutover checklist, watchdog, rollback |
|
||||
|
||||
@@ -71,7 +71,7 @@ curl http://localhost:8090/api/v1/health # fleet health
|
||||
curl http://localhost:8090/api/v1/agent-activity # agent log
|
||||
```
|
||||
|
||||
### Hermes queries
|
||||
### Nomos queries
|
||||
|
||||
```bash
|
||||
# Structured tool call
|
||||
@@ -101,7 +101,7 @@ oikos secret migrate # SOPS → Infisical
|
||||
|
||||
```
|
||||
cmd/oikos/ Go entry point — single binary
|
||||
cmd/hermes/ Hermes MCP client gateway
|
||||
cmd/nomos/ Nomos MCP client gateway
|
||||
internal/ Go packages (httpapi, mcp, scheduler, actuator, learning,
|
||||
notifier, policy, secrets, db, config, ontology, domain,
|
||||
knowledge)
|
||||
@@ -110,7 +110,7 @@ migrations/ Forward-only SQL migrations (TimescaleDB)
|
||||
seeds/ Bootstrap YAML (ontology, inventory, policy, knowledge)
|
||||
compose/ Dockerfiles + Caddy config
|
||||
scripts/ Deploy, watchdog, verification, rollback
|
||||
hermes/ Hermes config, persona, skills
|
||||
nomos/ Nomos config, persona, skills
|
||||
.agents/ Agent instruction files, shared conventions, skills
|
||||
archive/ Historical reference (legacy wiki, plans, SOPS backups)
|
||||
plans/ Design documents (active + done)
|
||||
|
||||
125
api/openapi.yaml
125
api/openapi.yaml
@@ -312,6 +312,17 @@ paths:
|
||||
type: string
|
||||
style: form
|
||||
explode: true
|
||||
- name: include
|
||||
in: query
|
||||
schema:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
enum:
|
||||
- status
|
||||
style: form
|
||||
explode: true
|
||||
description: include=status joins entity_status and populates GraphView.health
|
||||
responses:
|
||||
'200':
|
||||
description: Graph view
|
||||
@@ -1696,6 +1707,22 @@ paths:
|
||||
$ref: '#/components/schemas/HealthSummary'
|
||||
default:
|
||||
$ref: '#/components/responses/Problem'
|
||||
/dashboard/summary:
|
||||
get:
|
||||
tags:
|
||||
- observability
|
||||
operationId: getDashboardSummary
|
||||
summary: One-round-trip overview for the control room home page
|
||||
x-required-scope: viewer
|
||||
responses:
|
||||
'200':
|
||||
description: Dashboard summary
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/DashboardSummary'
|
||||
default:
|
||||
$ref: '#/components/responses/Problem'
|
||||
/export:
|
||||
get:
|
||||
tags:
|
||||
@@ -1887,6 +1914,21 @@ components:
|
||||
updated_at:
|
||||
type: string
|
||||
format: date-time
|
||||
health:
|
||||
type: string
|
||||
description: last observed health, when the entity is monitored
|
||||
nullable: true
|
||||
enum:
|
||||
- healthy
|
||||
- degraded
|
||||
- down
|
||||
- unknown
|
||||
- stale
|
||||
last_check_at:
|
||||
type: string
|
||||
format: date-time
|
||||
nullable: true
|
||||
description: when health was last observed
|
||||
EntityCreate:
|
||||
type: object
|
||||
required:
|
||||
@@ -1983,6 +2025,17 @@ components:
|
||||
truncated:
|
||||
type: boolean
|
||||
description: True if node cap was hit
|
||||
health:
|
||||
type: object
|
||||
description: entity id -> health, present when include=status was requested
|
||||
additionalProperties:
|
||||
type: string
|
||||
enum:
|
||||
- healthy
|
||||
- degraded
|
||||
- down
|
||||
- unknown
|
||||
- stale
|
||||
EntityType:
|
||||
type: object
|
||||
required:
|
||||
@@ -2207,6 +2260,7 @@ components:
|
||||
- disk
|
||||
- cert-expiry
|
||||
- drift
|
||||
- ping
|
||||
- ssh-script
|
||||
target:
|
||||
type: string
|
||||
@@ -2246,6 +2300,7 @@ components:
|
||||
- disk
|
||||
- cert-expiry
|
||||
- drift
|
||||
- ping
|
||||
- ssh-script
|
||||
target:
|
||||
type: string
|
||||
@@ -2992,6 +3047,9 @@ components:
|
||||
type: integer
|
||||
unknown:
|
||||
type: integer
|
||||
stale:
|
||||
type: integer
|
||||
description: last observation older than the check's expected cadence
|
||||
entities:
|
||||
type: array
|
||||
items:
|
||||
@@ -3012,6 +3070,7 @@ components:
|
||||
- degraded
|
||||
- down
|
||||
- unknown
|
||||
- stale
|
||||
trend:
|
||||
type: string
|
||||
enum:
|
||||
@@ -3024,6 +3083,72 @@ components:
|
||||
type: string
|
||||
format: date-time
|
||||
nullable: true
|
||||
DashboardSummary:
|
||||
type: object
|
||||
required:
|
||||
- entities_by_type
|
||||
- entities_by_state
|
||||
- health
|
||||
- signals_by_severity
|
||||
- approvals_pending
|
||||
- executions_by_state
|
||||
- event_rate
|
||||
properties:
|
||||
entities_by_type:
|
||||
type: object
|
||||
description: entity counts keyed by type
|
||||
additionalProperties:
|
||||
type: integer
|
||||
entities_by_state:
|
||||
type: object
|
||||
description: entity counts keyed by state
|
||||
additionalProperties:
|
||||
type: integer
|
||||
health:
|
||||
type: object
|
||||
required:
|
||||
- healthy
|
||||
- degraded
|
||||
- down
|
||||
- unknown
|
||||
properties:
|
||||
healthy:
|
||||
type: integer
|
||||
degraded:
|
||||
type: integer
|
||||
down:
|
||||
type: integer
|
||||
unknown:
|
||||
type: integer
|
||||
stale:
|
||||
type: integer
|
||||
description: last observation older than the check's expected cadence
|
||||
signals_by_severity:
|
||||
type: object
|
||||
description: open (non-resolved) signal counts keyed by severity
|
||||
additionalProperties:
|
||||
type: integer
|
||||
approvals_pending:
|
||||
type: integer
|
||||
executions_by_state:
|
||||
type: object
|
||||
description: execution counts keyed by state, last 24h
|
||||
additionalProperties:
|
||||
type: integer
|
||||
event_rate:
|
||||
type: array
|
||||
description: event counts bucketed by 5-minute interval, most recent last
|
||||
items:
|
||||
type: object
|
||||
required:
|
||||
- bucket
|
||||
- count
|
||||
properties:
|
||||
bucket:
|
||||
type: string
|
||||
format: date-time
|
||||
count:
|
||||
type: integer
|
||||
EnrollRequest:
|
||||
type: object
|
||||
required:
|
||||
|
||||
26
bootstrap.sh
26
bootstrap.sh
@@ -3,7 +3,7 @@
|
||||
#
|
||||
# Thin client model (rev 2): no git clone, no sync timer. Fetches only the
|
||||
# agent orientation files (CLIENTS.md, AGENTS.md, OIKOS.md) and tooling
|
||||
# (caveman, hermes-soul) from the raw Gitea URL. Enrolls via the Oikos API
|
||||
# (caveman, nomos-soul) from the raw Gitea URL. Enrolls via the Oikos API
|
||||
# to receive an age keypair and Infisical machine identity. A lightweight
|
||||
# context poller replaces the old 5-minute git pull.
|
||||
#
|
||||
@@ -11,7 +11,7 @@
|
||||
# curl -fsSL https://git.hubris.network/dtoro/oikos/raw/main/bootstrap.sh \
|
||||
# | sudo bash
|
||||
# curl ... | sudo bash -s -- --with-mcp # wire Claude's .mcp.json
|
||||
# curl ... | sudo bash -s -- --with-hermes # install Goose + Hermes
|
||||
# curl ... | sudo bash -s -- --with-nomos # install Goose + Nomos
|
||||
# curl ... | sudo bash -s -- --dry-run # show what would happen
|
||||
#
|
||||
# Prerequisites:
|
||||
@@ -28,11 +28,11 @@ REPO_RAW_URL="${HOMELAB_RAW_URL:-https://git.hubris.network/dtoro/oikos/raw/main
|
||||
OIKOS_API_URL="${HOMELAB_OIKOS_URL:-https://oikos.hubris.network/api/v1}"
|
||||
CLONE_DIR="${HOMELAB_CONTEXT_DIR:-/opt/homelab}"
|
||||
MCP_URL="${HOMELAB_MCP_URL:-https://mcp.hubris.network/mcp}"
|
||||
HERMES_MCP_URI="${HOMELAB_HERMES_MCP_URI:-https://mcp.hubris.network/mcp}"
|
||||
HERMES_MODEL="${HOMELAB_HERMES_MODEL:-nousresearch/hermes-4-405b}"
|
||||
NOMOS_MCP_URI="${HOMELAB_NOMOS_MCP_URI:-https://mcp.hubris.network/mcp}"
|
||||
NOMOS_MODEL="${HOMELAB_NOMOS_MODEL:-nousresearch/hermes-4-405b}"
|
||||
|
||||
WITH_MCP=0
|
||||
WITH_HERMES=0
|
||||
WITH_NOMOS=0
|
||||
DRY_RUN=0
|
||||
|
||||
GITEA_TOKEN="${HOMELAB_GITEA_TOKEN:-}"
|
||||
@@ -79,7 +79,7 @@ detect_mesh_ip() {
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--with-mcp) WITH_MCP=1 ;;
|
||||
--with-hermes) WITH_HERMES=1 ;;
|
||||
--with-nomos) WITH_NOMOS=1 ;;
|
||||
--dry-run) DRY_RUN=1 ;;
|
||||
--gitea-token) GITEA_TOKEN="$2"; shift ;;
|
||||
--gitea-user) GITEA_USER="$2"; shift ;;
|
||||
@@ -148,7 +148,7 @@ done
|
||||
|
||||
# ── fetch tools ──────────────────────────────────────────────────────
|
||||
log "fetching tools..."
|
||||
for tool in setup-caveman.sh setup-hermes-soul.sh caveman.js caveman_wrapper.sh post-pull.sh; do
|
||||
for tool in setup-caveman.sh setup-nomos-soul.sh caveman.js caveman_wrapper.sh post-pull.sh; do
|
||||
url="$REPO_RAW_URL/tools/${tool}"
|
||||
dest="$CLONE_DIR/tools/${tool}"
|
||||
dry mkdir -p "$(dirname "$dest")"
|
||||
@@ -319,15 +319,15 @@ if [ "$WITH_MCP" -eq 1 ]; then
|
||||
log " + MCP wired to $MCP_URL"
|
||||
fi
|
||||
|
||||
# ── --with-hermes: install Goose + Hermes wrapper ────────────────────
|
||||
if [ "$WITH_HERMES" -eq 1 ]; then
|
||||
log "installing Hermes agent..."
|
||||
# ── --with-nomos: install Goose + Nomos wrapper ────────────────────
|
||||
if [ "$WITH_NOMOS" -eq 1 ]; then
|
||||
log "installing Nomos agent..."
|
||||
GOOSE_URL="https://github.com/block/goose/releases/latest/download/goose-${OS}-${ARCH:-amd64}"
|
||||
if [ "$OS" = Darwin ]; then GOOSE_URL="https://github.com/block/goose/releases/latest/download/goose-darwin-${ARCH:-arm64}"; fi
|
||||
dry curl -fsSL "$GOOSE_URL" -o /usr/local/bin/goose 2>/dev/null && chmod +x /usr/local/bin/goose || warn "goose not installed"
|
||||
# Drop Hermes persona
|
||||
cp "$CLONE_DIR/HERMES.md" "$CLONE_DIR/.agents/HERMES.md" 2>/dev/null || true
|
||||
log " + Hermes agent installed"
|
||||
# Drop Nomos persona
|
||||
cp "$CLONE_DIR/NOMOS.md" "$CLONE_DIR/.agents/NOMOS.md" 2>/dev/null || true
|
||||
log " + Nomos agent installed"
|
||||
fi
|
||||
|
||||
# ── netbird SSH JWT cache ────────────────────────────────────────────
|
||||
|
||||
23
checks/backup_freshness.sh
Normal file
23
checks/backup_freshness.sh
Normal file
@@ -0,0 +1,23 @@
|
||||
#!/usr/bin/env bash
|
||||
# backup_freshness.sh — check that backups exist and are recent.
|
||||
set -euo pipefail
|
||||
|
||||
BACKUP_DIRS="${OIKOS_BACKUP_DIRS:-/var/backups /opt/backups /mnt/backups}"
|
||||
GRACE_HOURS="${OIKOS_BACKUP_GRACE:-48}"
|
||||
STALE=""
|
||||
|
||||
for dir in $BACKUP_DIRS; do
|
||||
[ -d "$dir" ] || continue
|
||||
NEWEST=$(find "$dir" -type f -mmin -$((GRACE_HOURS * 60)) 2>/dev/null | head -1 || true)
|
||||
if [ -z "$NEWEST" ]; then
|
||||
LATEST_TS=$(find "$dir" -type f -printf '%T@ %p\n' 2>/dev/null | sort -rn | head -1 | awk '{print $1}' || echo "0")
|
||||
LATEST_HOURS=$(awk "BEGIN {printf \"%.0f\", ($(date +%s) - ${LATEST_TS:-0})/3600}")
|
||||
STALE="$STALE $dir(${LATEST_HOURS}h)"
|
||||
fi
|
||||
done
|
||||
|
||||
if [ -n "$STALE" ]; then
|
||||
echo "{\"health\":\"degraded\",\"signalKind\":\"backup-stale\",\"evidence\":\"stale backups:$(echo "$STALE" | sed 's/ /, /g')\"}"
|
||||
else
|
||||
echo '{"health":"healthy"}'
|
||||
fi
|
||||
46
checks/caddy_error_rate.sh
Normal file
46
checks/caddy_error_rate.sh
Normal file
@@ -0,0 +1,46 @@
|
||||
#!/usr/bin/env bash
|
||||
# caddy_error_rate.sh — 5xx error rate from Caddy JSON access logs.
|
||||
set -euo pipefail
|
||||
|
||||
LOG_DIR=""
|
||||
if [ -d /var/log/caddy ]; then
|
||||
LOG_DIR="/var/log/caddy"
|
||||
elif [ -d /var/lib/caddy/logs ]; then
|
||||
LOG_DIR="/var/lib/caddy/logs"
|
||||
elif [ -d /opt/caddy/logs ]; then
|
||||
LOG_DIR="/opt/caddy/logs"
|
||||
fi
|
||||
|
||||
if [ -z "$LOG_DIR" ]; then
|
||||
echo '{"health":"healthy"}'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
CUTOFF=$(date -u -d "5 minutes ago" +%Y-%m-%dT%H:%M 2>/dev/null || date -u -v-5M +%Y-%m-%dT%H:%M 2>/dev/null || echo "")
|
||||
|
||||
TOTAL=0
|
||||
ERR_5XX=0
|
||||
|
||||
for LOG in "$LOG_DIR"/access*.log "$LOG_DIR"/access*.json "$LOG_DIR"/*.log 2>/dev/null; do
|
||||
[ -f "$LOG" ] || continue
|
||||
[ -r "$LOG" ] || continue
|
||||
|
||||
if [ -n "$CUTOFF" ]; then
|
||||
NEW=$(awk -v cutoff="$CUTOFF" '$0 >= cutoff {print}' "$LOG" 2>/dev/null | wc -l | tr -d ' ' || echo 0)
|
||||
if [ "$NEW" -gt 0 ]; then
|
||||
TOTAL=$((TOTAL + NEW))
|
||||
ERR_5XX=$((ERR_5XX + $(awk -v cutoff="$CUTOFF" '$0 >= cutoff && /"status":5[0-9][0-9]/ {print}' "$LOG" 2>/dev/null | wc -l | tr -d ' ' || echo 0)))
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
if [ "$TOTAL" -gt 100 ]; then
|
||||
RATE=$(awk "BEGIN {printf \"%.1f\", $ERR_5XX*100/$TOTAL}")
|
||||
if [ "$(echo "$RATE > 5" | bc 2>/dev/null || echo 0)" = "1" ]; then
|
||||
echo "{\"health\":\"degraded\",\"signalKind\":\"caddy-errors\",\"evidence\":\"${RATE}% 5xx rate ($ERR_5XX/$TOTAL)\",\"metrics\":{\"caddy_5xx_rate\":$RATE,\"caddy_requests\":$TOTAL,\"caddy_5xx\":$ERR_5XX}}"
|
||||
exit 0
|
||||
fi
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"caddy_5xx_rate\":$RATE,\"caddy_requests\":$TOTAL,\"caddy_5xx\":$ERR_5XX}}"
|
||||
else
|
||||
echo '{"health":"healthy"}'
|
||||
fi
|
||||
20
checks/cpu_check.sh
Normal file
20
checks/cpu_check.sh
Normal file
@@ -0,0 +1,20 @@
|
||||
#!/usr/bin/env bash
|
||||
# cpu_check.sh — CPU usage % and thermal temperature.
|
||||
set -euo pipefail
|
||||
|
||||
USAGE=$(top -bn1 2>/dev/null | awk '/^%Cpu/ {print 100 - $8}' || true)
|
||||
if [ -z "$USAGE" ]; then
|
||||
CORES=$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 1)
|
||||
USAGE=$(awk -v cores="$CORES" '{print ($1+$2+$3)*100/cores}' /proc/loadavg 2>/dev/null || echo "0")
|
||||
fi
|
||||
|
||||
TEMP=""
|
||||
if [ -f /sys/class/thermal/thermal_zone0/temp ]; then
|
||||
TEMP=$(awk '{printf "%.1f", $1/1000}' /sys/class/thermal/thermal_zone0/temp 2>/dev/null || true)
|
||||
fi
|
||||
|
||||
if [ -n "$TEMP" ]; then
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"cpu_pct\":$USAGE,\"cpu_temp\":$TEMP}}"
|
||||
else
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"cpu_pct\":$USAGE}}"
|
||||
fi
|
||||
29
checks/disk_smart_check.sh
Normal file
29
checks/disk_smart_check.sh
Normal file
@@ -0,0 +1,29 @@
|
||||
#!/usr/bin/env bash
|
||||
# disk_smart_check.sh — SMART pre-failure indicators for physical disks.
|
||||
set -euo pipefail
|
||||
|
||||
if ! command -v smartctl >/dev/null 2>&1; then
|
||||
echo '{"health":"healthy"}'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
DISKS=$(lsblk -ndo NAME,TYPE 2>/dev/null | awk '$2=="disk"{print "/dev/"$1}' || true)
|
||||
if [ -z "$DISKS" ]; then
|
||||
echo '{"health":"healthy"}'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
FAILED=""
|
||||
for dev in $DISKS; do
|
||||
INFO=$(smartctl -H "$dev" 2>/dev/null || true)
|
||||
if ! echo "$INFO" | grep -q "PASSED\|OK"; then
|
||||
MODEL=$(smartctl -i "$dev" 2>/dev/null | awk -F': ' '/Device Model|Product/{print $2; exit}' || echo "$dev")
|
||||
FAILED="$FAILED $MODEL"
|
||||
fi
|
||||
done
|
||||
|
||||
if [ -n "$FAILED" ]; then
|
||||
echo "{\"health\":\"degraded\",\"signalKind\":\"disk-smart-fail\",\"evidence\":\"SMART check failed for:$(echo "$FAILED" | sed 's/ /, /g')\"}"
|
||||
else
|
||||
echo '{"health":"healthy"}'
|
||||
fi
|
||||
26
checks/disk_usage_check.sh
Normal file
26
checks/disk_usage_check.sh
Normal file
@@ -0,0 +1,26 @@
|
||||
#!/usr/bin/env bash
|
||||
# disk_usage_check.sh — disk usage and inode usage per mountpoint.
|
||||
set -euo pipefail
|
||||
|
||||
MOUNTS=$(df -k 2>/dev/null | awk 'NR>1 && $1 ~ /^\// && $NF !~ /^\/(snap|dev|proc|sys|run|private)/ {print $NF}' || true)
|
||||
FIRST=1
|
||||
|
||||
echo -n '{"health":"healthy","metrics":{'
|
||||
for m in $MOUNTS; do
|
||||
LINE=$(df -k "$m" 2>/dev/null | awk 'NR==2 {print $3, $4, $5, $7}' | tr -d '%' || true)
|
||||
if [ -z "$LINE" ]; then continue; fi
|
||||
USED=$(echo "$LINE" | awk '{print $1}')
|
||||
FREE=$(echo "$LINE" | awk '{print $2}')
|
||||
PCT=$(echo "$LINE" | awk '{print $3}')
|
||||
|
||||
INODE_LINE=$(df -i "$m" 2>/dev/null | awk 'NR==2 {print $5}' | tr -d '%' || echo "0")
|
||||
INODE_PCT=$(echo "${INODE_LINE:-0}" | sed 's/-/0/')
|
||||
|
||||
KEY=$(echo "$m" | sed 's|/|_|g' | sed 's|^_||')
|
||||
[ -z "$KEY" ] && KEY="root"
|
||||
|
||||
if [ $FIRST -eq 0 ]; then echo -n ','; fi
|
||||
FIRST=0
|
||||
echo -n "\"disk_${KEY}_pct\":$PCT,\"inode_${KEY}_pct\":$INODE_PCT"
|
||||
done
|
||||
echo '}}'
|
||||
19
checks/docker_health_check.sh
Normal file
19
checks/docker_health_check.sh
Normal file
@@ -0,0 +1,19 @@
|
||||
#!/usr/bin/env bash
|
||||
# docker_health_check.sh — detect unhealthy Docker containers.
|
||||
set -euo pipefail
|
||||
|
||||
if ! command -v docker >/dev/null 2>&1; then
|
||||
echo '{"health":"healthy"}'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
UNHEALTHY=$(docker ps --filter "health=unhealthy" --format "{{.Names}}" 2>/dev/null || true)
|
||||
|
||||
if [ -n "$UNHEALTHY" ]; then
|
||||
COUNT=$(echo "$UNHEALTHY" | wc -l | tr -d ' ')
|
||||
NAMES=$(echo "$UNHEALTHY" | tr '\n' ',' | sed 's/,$//')
|
||||
echo "{\"health\":\"degraded\",\"signalKind\":\"docker-unhealthy\",\"evidence\":\"$COUNT unhealthy container(s): $NAMES\",\"metrics\":{\"docker_unhealthy\":$COUNT}}"
|
||||
else
|
||||
TOTAL=$(docker ps -q 2>/dev/null | wc -l | tr -d ' ' || echo 0)
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"docker_unhealthy\":0,\"docker_total\":$TOTAL}}"
|
||||
fi
|
||||
13
checks/fd_check.sh
Normal file
13
checks/fd_check.sh
Normal file
@@ -0,0 +1,13 @@
|
||||
#!/usr/bin/env bash
|
||||
# fd_check.sh — open file descriptor usage ratio.
|
||||
set -euo pipefail
|
||||
|
||||
FD_PCT=0
|
||||
if [ -r /proc/sys/fs/file-nr ]; then
|
||||
read -r ALLOC _ LIMIT < /proc/sys/fs/file-nr
|
||||
if [ "$LIMIT" -gt 0 ]; then
|
||||
FD_PCT=$(awk "BEGIN {printf \"%.1f\", $ALLOC*100/$LIMIT}")
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"fd_pct\":$FD_PCT}}"
|
||||
29
checks/install.sh
Normal file
29
checks/install.sh
Normal file
@@ -0,0 +1,29 @@
|
||||
#!/usr/bin/env bash
|
||||
# install all check scripts into the canonical directory.
|
||||
# Auto-setup hook called by tools/post-pull.sh.
|
||||
set -euo pipefail
|
||||
|
||||
CLONE_DIR="${HOMELAB_CONTEXT_DIR:-/opt/homelab}"
|
||||
CHECK_SRC="$CLONE_DIR/checks"
|
||||
CHECK_DST="${OIKOS_CHECK_DIR:-/opt/oikos/checks}"
|
||||
|
||||
if [ ! -d "$CHECK_SRC" ]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
mkdir -p "$CHECK_DST"
|
||||
|
||||
for script in "$CHECK_SRC"/*.sh; do
|
||||
name=$(basename "$script")
|
||||
if [ "$name" = "install.sh" ]; then continue; fi
|
||||
if [ -f "$CHECK_DST/$name" ]; then
|
||||
if cmp -s "$script" "$CHECK_DST/$name"; then
|
||||
continue
|
||||
fi
|
||||
fi
|
||||
cp "$script" "$CHECK_DST/$name"
|
||||
chmod 755 "$CHECK_DST/$name"
|
||||
echo "[setup-checks] installed $name"
|
||||
done
|
||||
|
||||
echo "[setup-checks] done"
|
||||
15
checks/journal_check.sh
Normal file
15
checks/journal_check.sh
Normal file
@@ -0,0 +1,15 @@
|
||||
#!/usr/bin/env bash
|
||||
# journal_check.sh — count journal errors in the last check interval.
|
||||
set -euo pipefail
|
||||
|
||||
ERRORS=0
|
||||
|
||||
if command -v journalctl >/dev/null 2>&1; then
|
||||
ERRORS=$(journalctl -p err --since "-5min" --no-pager 2>/dev/null | wc -l | tr -d ' ' || echo 0)
|
||||
fi
|
||||
|
||||
if [ "$ERRORS" -gt 0 ]; then
|
||||
echo "{\"health\":\"degraded\",\"signalKind\":\"journal-errors\",\"evidence\":\"$ERRORS error entries in last 5min\",\"metrics\":{\"journal_errors\":$ERRORS}}"
|
||||
else
|
||||
echo '{"health":"healthy","metrics":{"journal_errors":0}}'
|
||||
fi
|
||||
8
checks/load_check.sh
Normal file
8
checks/load_check.sh
Normal file
@@ -0,0 +1,8 @@
|
||||
#!/usr/bin/env bash
|
||||
# load_check.sh — system load average scaled by CPU count.
|
||||
set -euo pipefail
|
||||
|
||||
LOAD=$(awk '{print $1}' /proc/loadavg 2>/dev/null || sysctl -n vm.loadavg 2>/dev/null | awk '{print $2}' || echo "0")
|
||||
CORES=$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 1)
|
||||
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"load1\":$LOAD,\"cores\":$CORES}}"
|
||||
27
checks/memory_check.sh
Normal file
27
checks/memory_check.sh
Normal file
@@ -0,0 +1,27 @@
|
||||
#!/usr/bin/env bash
|
||||
# memory_check.sh — RAM usage percentage.
|
||||
set -euo pipefail
|
||||
|
||||
TOTAL=1
|
||||
AVAIL=1
|
||||
|
||||
if [ -r /proc/meminfo ]; then
|
||||
TOTAL=$(awk '/^MemTotal:/ {printf "%d", $2}' /proc/meminfo)
|
||||
AVAIL=$(awk '/^MemAvailable:/ {printf "%d", $2}' /proc/meminfo)
|
||||
elif [ "$(uname)" = "Darwin" ]; then
|
||||
MEM=$(vm_stat 2>/dev/null | awk '
|
||||
/page size/ {ps=$8}
|
||||
/Pages free/ {free+=$NF}
|
||||
/Pages active/ {active+=$NF}
|
||||
/Pages wired/ {wired+=$NF}
|
||||
END {printf "%.2f %.2f", ps*(free+active+wired)/1048576, ps*wired/1048576}')
|
||||
TOTAL=$(echo "$MEM" | awk '{printf "%.0f", $1}')
|
||||
AVAIL=$(echo "$MEM" | awk '{printf "%.0f", $1 - $2}')
|
||||
fi
|
||||
|
||||
if [ "$TOTAL" -eq 0 ]; then TOTAL=1; fi
|
||||
if [ "$AVAIL" -lt 0 ]; then AVAIL=0; fi
|
||||
|
||||
USED_PCT=$(awk "BEGIN {printf \"%.1f\", (1 - $AVAIL/$TOTAL)*100}")
|
||||
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"mem_pct\":$USED_PCT}}"
|
||||
17
checks/oom_check.sh
Normal file
17
checks/oom_check.sh
Normal file
@@ -0,0 +1,17 @@
|
||||
#!/usr/bin/env bash
|
||||
# oom_check.sh — detect OOM kills since last boot.
|
||||
set -euo pipefail
|
||||
|
||||
OOM_COUNT=0
|
||||
|
||||
if command -v dmesg >/dev/null 2>&1; then
|
||||
OOM_COUNT=$(dmesg 2>/dev/null | grep -ci 'out of memory\|oom-killer\|Killed process' || echo 0)
|
||||
elif command -v journalctl >/dev/null 2>&1; then
|
||||
OOM_COUNT=$(journalctl -k --no-pager 2>/dev/null | grep -ci 'out of memory\|oom-killer\|Killed process' || echo 0)
|
||||
fi
|
||||
|
||||
if [ "$OOM_COUNT" -gt 0 ]; then
|
||||
echo "{\"health\":\"degraded\",\"signalKind\":\"oom-kills\",\"evidence\":\"$OOM_COUNT OOM events detected since boot\",\"metrics\":{\"oom_count\":$OOM_COUNT}}"
|
||||
else
|
||||
echo '{"health":"healthy","metrics":{"oom_count":0}}'
|
||||
fi
|
||||
22
checks/process_check.sh
Normal file
22
checks/process_check.sh
Normal file
@@ -0,0 +1,22 @@
|
||||
#!/usr/bin/env bash
|
||||
# process_check.sh — systemd service liveness.
|
||||
set -euo pipefail
|
||||
|
||||
SERVICE="${1:-}"
|
||||
if [ -z "$SERVICE" ]; then
|
||||
echo '{"health":"unknown","signalKind":"process-check","evidence":"no service name provided"}'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if ! command -v systemctl >/dev/null 2>&1; then
|
||||
echo '{"health":"unknown","signalKind":"process-check","evidence":"systemctl not found"}'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
STATE=$(systemctl is-active "$SERVICE" 2>/dev/null || echo "unknown")
|
||||
|
||||
if [ "$STATE" = "active" ]; then
|
||||
echo "{\"health\":\"healthy\"}"
|
||||
else
|
||||
echo "{\"health\":\"degraded\",\"signalKind\":\"$SERVICE\",\"evidence\":\"$SERVICE is $STATE\"}"
|
||||
fi
|
||||
22
checks/swap_check.sh
Normal file
22
checks/swap_check.sh
Normal file
@@ -0,0 +1,22 @@
|
||||
#!/usr/bin/env bash
|
||||
# swap_check.sh — swap usage percentage.
|
||||
set -euo pipefail
|
||||
|
||||
if [ -r /proc/meminfo ]; then
|
||||
SWAP_TOTAL=$(awk '/^SwapTotal:/ {printf "%d", $2}' /proc/meminfo)
|
||||
SWAP_FREE=$(awk '/^SwapFree:/ {printf "%d", $2}' /proc/meminfo)
|
||||
elif [ "$(uname)" = "Darwin" ]; then
|
||||
SP=$(sysctl vm.swapusage 2>/dev/null | awk '{print $4, $9}' | tr -d 'M' || echo "0 0")
|
||||
SWAP_TOTAL=$(echo "$SP" | awk '{printf "%.0f", $1*1024}')
|
||||
SWAP_FREE=$(echo "$SP" | awk '{printf "%.0f", ($1-$2)*1024}')
|
||||
else
|
||||
SWAP_TOTAL=0
|
||||
SWAP_FREE=0
|
||||
fi
|
||||
|
||||
if [ "$SWAP_TOTAL" -eq 0 ]; then
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"swap_pct\":0}}"
|
||||
else
|
||||
USED_PCT=$(awk "BEGIN {printf \"%.1f\", (1 - $SWAP_FREE/$SWAP_TOTAL)*100}")
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"swap_pct\":$USED_PCT}}"
|
||||
fi
|
||||
26
checks/time_check.sh
Normal file
26
checks/time_check.sh
Normal file
@@ -0,0 +1,26 @@
|
||||
#!/usr/bin/env bash
|
||||
# time_check.sh — NTP synchronization status and clock drift.
|
||||
set -euo pipefail
|
||||
|
||||
DRIFT_S=0
|
||||
SYNCED=true
|
||||
|
||||
if command -v chronyc >/dev/null 2>&1; then
|
||||
TRACKING=$(chronyc tracking 2>/dev/null || true)
|
||||
DRIFT_NS=$(echo "$TRACKING" | awk '/System time/ {print $4}' | sed 's/-//' || echo "0")
|
||||
DRIFT_S=$(awk "BEGIN {printf \"%.6f\", $DRIFT_NS/1e9}")
|
||||
# chronyc returns a very small value when synced (nanoseconds)
|
||||
elif command -v timedatectl >/dev/null 2>&1; then
|
||||
STATUS=$(timedatectl show 2>/dev/null || true)
|
||||
if echo "$STATUS" | grep -q "NTPSynchronized=no"; then
|
||||
SYNCED=false
|
||||
fi
|
||||
fi
|
||||
|
||||
if ! $SYNCED; then
|
||||
echo '{"health":"degraded","signalKind":"time-drift","evidence":"NTP not synchronized"}'
|
||||
elif [ "$(echo "$DRIFT_S > 1" | bc 2>/dev/null || echo 0)" = "1" ]; then
|
||||
echo "{\"health\":\"degraded\",\"signalKind\":\"time-drift\",\"evidence\":\"clock drift ${DRIFT_S}s exceeds 1s threshold\",\"metrics\":{\"clock_drift_s\":$DRIFT_S}}"
|
||||
else
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"clock_drift_s\":$DRIFT_S}}"
|
||||
fi
|
||||
17
checks/updates_check.sh
Normal file
17
checks/updates_check.sh
Normal file
@@ -0,0 +1,17 @@
|
||||
#!/usr/bin/env bash
|
||||
# updates_check.sh — pending apt security updates and reboot-required flag.
|
||||
set -euo pipefail
|
||||
|
||||
SECURITY=0
|
||||
REBOOT=0
|
||||
|
||||
if command -v apt >/dev/null 2>&1; then
|
||||
apt update -qq >/dev/null 2>&1 || true
|
||||
SECURITY=$(apt list --upgradable 2>/dev/null | grep -c '\-security' || true)
|
||||
fi
|
||||
|
||||
if [ -f /var/run/reboot-required ]; then
|
||||
REBOOT=1
|
||||
fi
|
||||
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"security_updates\":$SECURITY,\"reboot_required\":$REBOOT}}"
|
||||
7
checks/uptime_check.sh
Normal file
7
checks/uptime_check.sh
Normal file
@@ -0,0 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
# uptime_check.sh — detect unexpected reboots by monitoring uptime.
|
||||
set -euo pipefail
|
||||
|
||||
UPTIME=$(awk '{printf "%.0f", $1}' /proc/uptime 2>/dev/null || sysctl -n kern.boottime 2>/dev/null | awk '{print $4}' | tr -d ',' || echo "0")
|
||||
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"uptime_seconds\":$UPTIME}}"
|
||||
29
checks/zfs_check.sh
Normal file
29
checks/zfs_check.sh
Normal file
@@ -0,0 +1,29 @@
|
||||
#!/usr/bin/env bash
|
||||
# zfs_check.sh — ZFS pool health.
|
||||
set -euo pipefail
|
||||
|
||||
if ! command -v zpool >/dev/null 2>&1; then
|
||||
echo '{"health":"healthy"}'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
STATUS=$(zpool status -x 2>&1 || true)
|
||||
SCRUB_OVERDUE=""
|
||||
|
||||
if echo "$STATUS" | grep -q "all pools are healthy"; then
|
||||
for pool in $(zpool list -Ho name 2>/dev/null || true); do
|
||||
LAST=$(zpool status "$pool" 2>/dev/null | awk '/scan:/{print $0}' || true)
|
||||
if [ -z "$LAST" ] || echo "$LAST" | grep -q "scrub repaired"; then
|
||||
SCRUB_OVERDUE="$pool"
|
||||
break
|
||||
fi
|
||||
done
|
||||
if [ -n "$SCRUB_OVERDUE" ]; then
|
||||
echo "{\"health\":\"degraded\",\"signalKind\":\"zfs-scrub-overdue\",\"evidence\":\"pool $SCRUB_OVERDUE scrub has errors or is overdue\"}"
|
||||
else
|
||||
echo '{"health":"healthy"}'
|
||||
fi
|
||||
else
|
||||
HEALTH=$(echo "$STATUS" | grep "state:" | awk '{print $2}' | head -1)
|
||||
echo "{\"health\":\"degraded\",\"signalKind\":\"zfs-degraded\",\"evidence\":\"pool state: $HEALTH\"}"
|
||||
fi
|
||||
@@ -1,341 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
func main() {
|
||||
if len(os.Args) < 2 {
|
||||
fmt.Fprintln(os.Stderr, "usage: hermes serve")
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
mcpURL := os.Getenv("HERMES_MCP_URL")
|
||||
if mcpURL == "" {
|
||||
mcpURL = "http://localhost:8090/mcp"
|
||||
}
|
||||
|
||||
agentSlug := os.Getenv("HERMES_AGENT_SLUG")
|
||||
if agentSlug == "" {
|
||||
agentSlug = "agent:hermes"
|
||||
}
|
||||
|
||||
switch os.Args[1] {
|
||||
case "serve":
|
||||
ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT)
|
||||
defer cancel()
|
||||
|
||||
client, err := newMCPClient(mcpURL)
|
||||
if err != nil {
|
||||
slog.Error("hermes: mcp connect", "url", mcpURL, "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(200)
|
||||
w.Write([]byte("ok"))
|
||||
})
|
||||
mux.HandleFunc("/query", func(w http.ResponseWriter, r *http.Request) {
|
||||
handleQuery(w, r, client, agentSlug, mcpURL)
|
||||
})
|
||||
|
||||
addr := os.Getenv("HERMES_LISTEN")
|
||||
if addr == "" {
|
||||
addr = ":8092"
|
||||
}
|
||||
|
||||
srv := &http.Server{Addr: addr, Handler: mux}
|
||||
go func() {
|
||||
slog.Info("hermes: gateway listening", "addr", addr, "mcp", mcpURL)
|
||||
if err := srv.ListenAndServe(); err != http.ErrServerClosed {
|
||||
slog.Error("hermes: serve", "error", err)
|
||||
}
|
||||
}()
|
||||
|
||||
<-ctx.Done()
|
||||
slog.Info("hermes: shutting down")
|
||||
srv.Shutdown(context.Background())
|
||||
client.close()
|
||||
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "unknown command: %s\n", os.Args[1])
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
// handleQuery maps structured queries to MCP tool calls.
|
||||
func handleQuery(w http.ResponseWriter, r *http.Request, client *mcpClient, agentSlug, mcpURL string) {
|
||||
if r.Method != http.MethodPost {
|
||||
http.Error(w, "method not allowed", 405)
|
||||
return
|
||||
}
|
||||
|
||||
var req struct {
|
||||
Query string `json:"query"`
|
||||
Tool string `json:"tool"`
|
||||
Args map[string]any `json:"args"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
http.Error(w, "bad request: "+err.Error(), 400)
|
||||
return
|
||||
}
|
||||
|
||||
start := time.Now()
|
||||
var result any
|
||||
var err error
|
||||
|
||||
// Direct tool call (structured)
|
||||
if req.Tool != "" {
|
||||
result, err = client.callTool(req.Tool, req.Args)
|
||||
} else {
|
||||
// Natural-language-ish query routing
|
||||
q := strings.ToLower(req.Query)
|
||||
result, err = routeQuery(client, q, agentSlug)
|
||||
}
|
||||
|
||||
duration := time.Since(start).Milliseconds()
|
||||
|
||||
if err != nil {
|
||||
slog.Error("hermes: query failed", "query", req.Query, "error", err)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{
|
||||
"error": err.Error(),
|
||||
"elapsed_ms": duration,
|
||||
"agent_slug": agentSlug,
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{
|
||||
"result": result,
|
||||
"elapsed_ms": duration,
|
||||
"agent_slug": agentSlug,
|
||||
"mcp_url": mcpURL,
|
||||
})
|
||||
}
|
||||
|
||||
// routeQuery maps natural-language-style queries to MCP tool calls.
|
||||
func routeQuery(client *mcpClient, query, agentSlug string) (any, error) {
|
||||
switch {
|
||||
case strings.Contains(query, "depends on") || strings.Contains(query, "depend on"):
|
||||
entity := extractEntity(query)
|
||||
if entity == "" {
|
||||
return nil, fmt.Errorf("no entity found in query: %s", query)
|
||||
}
|
||||
return client.callTool("get_blast_radius", map[string]any{
|
||||
"entity_id": entity,
|
||||
})
|
||||
|
||||
case strings.Contains(query, "what is") || strings.Contains(query, "describe"):
|
||||
entity := extractEntity(query)
|
||||
if entity == "" {
|
||||
entity = query
|
||||
}
|
||||
return client.callTool("get_entity", map[string]any{
|
||||
"slug_or_id": entity,
|
||||
})
|
||||
|
||||
case strings.Contains(query, "health") || strings.Contains(query, "status"):
|
||||
return client.callTool("get_health_summary", map[string]any{})
|
||||
|
||||
case strings.Contains(query, "restart") || strings.Contains(query, "reload"):
|
||||
entity := extractEntity(query)
|
||||
if entity == "" {
|
||||
return nil, fmt.Errorf("no entity found in query: %s", query)
|
||||
}
|
||||
return client.callTool("request_execution", map[string]any{
|
||||
"target": entity,
|
||||
"action": "restart",
|
||||
})
|
||||
|
||||
case strings.Contains(query, "what can you do") || strings.Contains(query, "help"):
|
||||
return client.callTool("tools/list", nil)
|
||||
|
||||
default:
|
||||
return client.callTool("get_health_summary", map[string]any{})
|
||||
}
|
||||
}
|
||||
|
||||
// extractEntity guesses an entity slug from a query.
|
||||
func extractEntity(query string) string {
|
||||
for _, slug := range []string{"authentik", "caddy", "vaultwarden", "gitea", "immich"} {
|
||||
if strings.Contains(query, slug) {
|
||||
return "service:" + slug
|
||||
}
|
||||
}
|
||||
if strings.Contains(query, "mac-mini") {
|
||||
return "host:mac-mini"
|
||||
}
|
||||
if strings.Contains(query, "hubris") {
|
||||
return "host:hubris"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// ─── MCP Streamable HTTP client ────────────────────────────────────────
|
||||
|
||||
type mcpClient struct {
|
||||
baseURL string
|
||||
sessionID string
|
||||
http *http.Client
|
||||
nextID int
|
||||
}
|
||||
|
||||
func newMCPClient(baseURL string) (*mcpClient, error) {
|
||||
c := &mcpClient{
|
||||
baseURL: baseURL,
|
||||
http: &http.Client{Timeout: 30 * time.Second},
|
||||
}
|
||||
|
||||
// Initialize session
|
||||
resp, err := c.doRequest("initialize", map[string]any{
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": map[string]any{},
|
||||
"clientInfo": map[string]any{"name": "hermes", "version": "2.0"},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("initialize: %w", err)
|
||||
}
|
||||
if resp.sessionID == "" {
|
||||
return nil, fmt.Errorf("no session ID in initialize response")
|
||||
}
|
||||
c.sessionID = resp.sessionID
|
||||
|
||||
// Send initialized notification
|
||||
c.doRequest("notifications/initialized", map[string]any{})
|
||||
|
||||
slog.Info("hermes: mcp connected", "session", c.sessionID[:16]+"...")
|
||||
return c, nil
|
||||
}
|
||||
|
||||
type mcpJSONRPCResponse struct {
|
||||
sessionID string
|
||||
Result json.RawMessage `json:"result"`
|
||||
Error json.RawMessage `json:"error"`
|
||||
}
|
||||
|
||||
func (c *mcpClient) doRequest(method string, params map[string]any) (*mcpJSONRPCResponse, error) {
|
||||
c.nextID++
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"jsonrpc": "2.0",
|
||||
"method": method,
|
||||
"params": params,
|
||||
"id": c.nextID,
|
||||
})
|
||||
|
||||
req, err := http.NewRequest(http.MethodPost, c.baseURL, bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Accept", "application/json, text/event-stream")
|
||||
if c.sessionID != "" {
|
||||
req.Header.Set("Mcp-Session-Id", c.sessionID)
|
||||
}
|
||||
|
||||
resp, err := c.http.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
result := &mcpJSONRPCResponse{}
|
||||
result.sessionID = resp.Header.Get("Mcp-Session-Id")
|
||||
|
||||
// Parse SSE stream: "event: message\ndata: <json>\n\n"
|
||||
scanner := bufio.NewScanner(resp.Body)
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
if strings.HasPrefix(line, "data: ") {
|
||||
data := line[6:]
|
||||
if err := json.Unmarshal([]byte(data), result); err != nil {
|
||||
return nil, fmt.Errorf("parse response: %w", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if result.Error != nil {
|
||||
return nil, fmt.Errorf("rpc error: %s", string(result.Error))
|
||||
}
|
||||
|
||||
if result.sessionID != "" {
|
||||
c.sessionID = result.sessionID
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) callTool(name string, args map[string]any) (any, error) {
|
||||
resp, err := c.doRequest("tools/call", map[string]any{
|
||||
"name": name,
|
||||
"arguments": args,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Parse MCP content: { "content": [{ "type": "text", "text": "..." }] }
|
||||
var toolResult struct {
|
||||
Content []struct {
|
||||
Type string `json:"type"`
|
||||
Text string `json:"text"`
|
||||
} `json:"content"`
|
||||
}
|
||||
if err := json.Unmarshal(resp.Result, &toolResult); err != nil {
|
||||
return string(resp.Result), nil
|
||||
}
|
||||
|
||||
var texts []string
|
||||
for _, c := range toolResult.Content {
|
||||
if c.Type == "text" {
|
||||
// Try to parse as JSON for structured display
|
||||
var parsed any
|
||||
if json.Unmarshal([]byte(c.Text), &parsed) == nil {
|
||||
return parsed, nil
|
||||
}
|
||||
texts = append(texts, c.Text)
|
||||
}
|
||||
}
|
||||
if len(texts) == 1 {
|
||||
return texts[0], nil
|
||||
}
|
||||
return texts, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) listTools() ([]string, error) {
|
||||
resp, err := c.doRequest("tools/list", map[string]any{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var tr struct {
|
||||
Tools []struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
} `json:"tools"`
|
||||
}
|
||||
if err := json.Unmarshal(resp.Result, &tr); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var names []string
|
||||
for _, t := range tr.Tools {
|
||||
names = append(names, t.Name)
|
||||
}
|
||||
return names, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) close() {
|
||||
// MCP sessions are ephemeral; no explicit close needed
|
||||
}
|
||||
440
cmd/nomos/agent.go
Normal file
440
cmd/nomos/agent.go
Normal file
@@ -0,0 +1,440 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/openai/openai-go"
|
||||
"github.com/openai/openai-go/option"
|
||||
"github.com/openai/openai-go/shared"
|
||||
)
|
||||
|
||||
const maxIterations = 15
|
||||
|
||||
type agent struct {
|
||||
client *mcpClient
|
||||
system string
|
||||
provider *openai.Client
|
||||
model string
|
||||
store *store
|
||||
agentID uuid.UUID
|
||||
reqOpts []option.RequestOption
|
||||
}
|
||||
|
||||
func newAgent(ctx context.Context, mcpClient *mcpClient, st *store, agentSlug string) (*agent, error) {
|
||||
system := loadSoul()
|
||||
apiKey := os.Getenv("OPENROUTER_API_KEY")
|
||||
model := os.Getenv("NOMOS_MODEL")
|
||||
if model == "" {
|
||||
model = "deepseek/deepseek-v4-flash"
|
||||
}
|
||||
|
||||
provider := openai.NewClient(
|
||||
option.WithBaseURL("https://openrouter.ai/api/v1"),
|
||||
option.WithAPIKey(apiKey),
|
||||
)
|
||||
|
||||
agentID := st.resolveAgentID(ctx, agentSlug)
|
||||
if agentID == uuid.Nil {
|
||||
slog.Warn("nomos: agent entity not found; tool-call activity will not be logged", "slug", agentSlug)
|
||||
}
|
||||
|
||||
// OpenRouter provider routing. data_collection=deny pins to zero-data-
|
||||
// retention providers (privacy: conversations + tool results transit
|
||||
// OpenRouter); require_parameters ensures the routed provider actually
|
||||
// supports tool calling. NOMOS_PROVIDER_SORT (price|throughput|latency)
|
||||
// and Exacto tool-accuracy routing are opt-in — the latter via a model
|
||||
// suffix in NOMOS_MODEL (e.g. "deepseek/deepseek-v4-flash:exacto"), so an
|
||||
// unsupported value never silently breaks the confirmed routing below.
|
||||
providerRouting := map[string]any{
|
||||
"data_collection": "deny",
|
||||
"require_parameters": true,
|
||||
}
|
||||
if sort := os.Getenv("NOMOS_PROVIDER_SORT"); sort != "" {
|
||||
providerRouting["sort"] = sort
|
||||
}
|
||||
reqOpts := []option.RequestOption{option.WithJSONSet("provider", providerRouting)}
|
||||
|
||||
return &agent{
|
||||
client: mcpClient,
|
||||
system: system,
|
||||
provider: &provider,
|
||||
model: model,
|
||||
store: st,
|
||||
agentID: agentID,
|
||||
reqOpts: reqOpts,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func loadSoul() string {
|
||||
paths := []string{"/app/nomos/SOUL.md", "nomos/SOUL.md"}
|
||||
for _, p := range paths {
|
||||
if data, err := os.ReadFile(p); err == nil {
|
||||
return string(data)
|
||||
}
|
||||
}
|
||||
return `You are Nomos, the steward of the oikos — the AI agent for the hubris homelab.
|
||||
You have access to MCP tools to query topology, health, knowledge, and request
|
||||
gated mutations through request_execution. Be concise. Prefer tools over guessing.`
|
||||
}
|
||||
|
||||
type toolDef struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
InputSchema map[string]any `json:"inputSchema"`
|
||||
}
|
||||
|
||||
type agentEvent struct {
|
||||
Type string `json:"type"`
|
||||
Data any `json:"data,omitempty"`
|
||||
SessionID string `json:"session_id,omitempty"`
|
||||
Iteration int `json:"iteration,omitempty"`
|
||||
}
|
||||
|
||||
func (a *agent) chat(ctx context.Context, sessionID, message string, emit func(agentEvent)) {
|
||||
correlationID := uuid.New().String()
|
||||
|
||||
tools, err := a.buildTools()
|
||||
if err != nil {
|
||||
emit(agentEvent{Type: "error", Data: fmt.Sprintf("build tools: %v", err), SessionID: sessionID})
|
||||
return
|
||||
}
|
||||
|
||||
// Rebuild conversation context from persisted history so sessions are
|
||||
// multi-turn. The current user turn is saved by the HTTP handler before
|
||||
// this runs, so it is already included in the history for real sessions.
|
||||
// Prior tool_use/tool_result pairs are replayed as a tool-calling
|
||||
// assistant message followed by matching tool-role results, so the agent
|
||||
// starts each turn already knowing what it already checked instead of
|
||||
// re-querying the same tools from scratch. Ephemeral sessions (no store)
|
||||
// fall back to the single incoming message.
|
||||
system := a.system
|
||||
if snapshot := a.fleetSnapshot(); snapshot != "" {
|
||||
system += "\n\n" + snapshot
|
||||
}
|
||||
messages := []openai.ChatCompletionMessageParamUnion{openai.SystemMessage(system)}
|
||||
history, _ := a.store.getMessages(ctx, sessionID)
|
||||
for _, m := range history {
|
||||
text := extractText(m.Content)
|
||||
switch m.Role {
|
||||
case "user":
|
||||
messages = append(messages, openai.UserMessage(text))
|
||||
case "assistant":
|
||||
if calls := extractToolCalls(m.Content); len(calls) > 0 {
|
||||
messages = append(messages, assistantToolCallMessage(calls))
|
||||
for _, c := range calls {
|
||||
messages = append(messages, openai.ToolMessage(c.resultText(), c.id))
|
||||
}
|
||||
}
|
||||
if text != "" {
|
||||
messages = append(messages, openai.AssistantMessage(text))
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(history) == 0 {
|
||||
messages = append(messages, openai.UserMessage(message))
|
||||
}
|
||||
|
||||
for i := 0; i < maxIterations; i++ {
|
||||
params := openai.ChatCompletionNewParams{
|
||||
Model: openai.ChatModel(a.model),
|
||||
Messages: messages,
|
||||
Tools: tools,
|
||||
}
|
||||
|
||||
// Stream the completion, emitting token deltas as they arrive. The
|
||||
// accumulator reassembles the full message (content + tool calls) for
|
||||
// the loop's control flow.
|
||||
stream := a.provider.Chat.Completions.NewStreaming(ctx, params, a.reqOpts...)
|
||||
acc := openai.ChatCompletionAccumulator{}
|
||||
for stream.Next() {
|
||||
chunk := stream.Current()
|
||||
acc.AddChunk(chunk)
|
||||
if len(chunk.Choices) > 0 {
|
||||
if delta := chunk.Choices[0].Delta.Content; delta != "" {
|
||||
emit(agentEvent{Type: "text_delta", Data: delta, SessionID: sessionID, Iteration: i + 1})
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := stream.Err(); err != nil {
|
||||
emit(agentEvent{Type: "error", Data: fmt.Sprintf("llm: %v", err), SessionID: sessionID})
|
||||
return
|
||||
}
|
||||
if len(acc.Choices) == 0 {
|
||||
emit(agentEvent{Type: "error", Data: "no choices in response", SessionID: sessionID})
|
||||
return
|
||||
}
|
||||
|
||||
msg := acc.Choices[0].Message
|
||||
|
||||
if len(msg.ToolCalls) == 0 {
|
||||
emit(agentEvent{Type: "text", Data: msg.Content, SessionID: sessionID})
|
||||
emit(agentEvent{Type: "done", Data: map[string]any{
|
||||
"session_id": sessionID,
|
||||
"usage": acc.Usage,
|
||||
"correlation_id": correlationID,
|
||||
"iterations": i + 1,
|
||||
}, SessionID: sessionID})
|
||||
return
|
||||
}
|
||||
|
||||
slog.Info("nomos: tool calls", "count", len(msg.ToolCalls), "iter", i+1, "correlation", correlationID)
|
||||
|
||||
messages = append(messages, msg.ToParam())
|
||||
|
||||
for _, tc := range msg.ToolCalls {
|
||||
var args map[string]any
|
||||
if err := json.Unmarshal([]byte(tc.Function.Arguments), &args); err != nil {
|
||||
args = map[string]any{}
|
||||
}
|
||||
|
||||
emit(agentEvent{
|
||||
Type: "tool_use",
|
||||
Data: map[string]any{"name": tc.Function.Name, "args": args, "id": tc.ID},
|
||||
SessionID: sessionID,
|
||||
Iteration: i + 1,
|
||||
})
|
||||
|
||||
start := time.Now()
|
||||
result, callErr := a.client.callTool(tc.Function.Name, args)
|
||||
elapsed := int(time.Since(start).Milliseconds())
|
||||
|
||||
inputJSON, _ := json.Marshal(args)
|
||||
inputStr := string(inputJSON)
|
||||
|
||||
if callErr != nil {
|
||||
a.store.logActivity(ctx, a.agentID, sessionID, tc.Function.Name, inputStr, callErr.Error(), elapsed, false, correlationID)
|
||||
|
||||
emit(agentEvent{
|
||||
Type: "tool_result",
|
||||
Data: map[string]any{"name": tc.Function.Name, "error": callErr.Error(), "id": tc.ID},
|
||||
SessionID: sessionID,
|
||||
Iteration: i + 1,
|
||||
})
|
||||
messages = append(messages, openai.ToolMessage(callErr.Error(), tc.ID))
|
||||
slog.Error("nomos: tool error", "tool", tc.Function.Name, "error", callErr, "ms", elapsed)
|
||||
continue
|
||||
}
|
||||
|
||||
resultJSON, _ := json.Marshal(result)
|
||||
a.store.logActivity(ctx, a.agentID, sessionID, tc.Function.Name, inputStr, string(resultJSON), elapsed, true, correlationID)
|
||||
|
||||
emit(agentEvent{
|
||||
Type: "tool_result",
|
||||
Data: map[string]any{"name": tc.Function.Name, "result": result, "id": tc.ID},
|
||||
SessionID: sessionID,
|
||||
Iteration: i + 1,
|
||||
})
|
||||
messages = append(messages, openai.ToolMessage(string(resultJSON), tc.ID))
|
||||
slog.Info("nomos: tool success", "tool", tc.Function.Name, "ms", elapsed)
|
||||
}
|
||||
}
|
||||
|
||||
emit(agentEvent{Type: "text", Data: "Agent loop: max iterations reached without final answer.", SessionID: sessionID})
|
||||
emit(agentEvent{Type: "done", Data: map[string]any{
|
||||
"session_id": sessionID,
|
||||
"correlation_id": correlationID,
|
||||
"iterations": maxIterations,
|
||||
}, SessionID: sessionID})
|
||||
}
|
||||
|
||||
// extractText pulls the "text" field from a persisted message's JSONB content.
|
||||
func extractText(content json.RawMessage) string {
|
||||
var m struct {
|
||||
Text string `json:"text"`
|
||||
}
|
||||
if err := json.Unmarshal(content, &m); err != nil {
|
||||
return ""
|
||||
}
|
||||
return m.Text
|
||||
}
|
||||
|
||||
// persistedCall is one merged tool_use+tool_result pair from a persisted
|
||||
// assistant message's tool_calls array. The store keeps them as two entries
|
||||
// sharing the same id (mirroring the SSE event pair); replay needs one
|
||||
// entry per id to build a valid tool-calling assistant message.
|
||||
type persistedCall struct {
|
||||
id string
|
||||
name string
|
||||
args json.RawMessage
|
||||
result json.RawMessage
|
||||
errMsg string
|
||||
}
|
||||
|
||||
func (c persistedCall) resultText() string {
|
||||
if c.errMsg != "" {
|
||||
return c.errMsg
|
||||
}
|
||||
if len(c.result) > 0 {
|
||||
return string(c.result)
|
||||
}
|
||||
return "null"
|
||||
}
|
||||
|
||||
// extractToolCalls parses and merges a persisted message's tool_calls array,
|
||||
// preserving first-seen order across ids.
|
||||
func extractToolCalls(content json.RawMessage) []persistedCall {
|
||||
var m struct {
|
||||
ToolCalls []struct {
|
||||
ID string `json:"id"`
|
||||
Type string `json:"type"`
|
||||
Name string `json:"name"`
|
||||
Args json.RawMessage `json:"args"`
|
||||
Result json.RawMessage `json:"result"`
|
||||
Error string `json:"error"`
|
||||
} `json:"tool_calls"`
|
||||
}
|
||||
if err := json.Unmarshal(content, &m); err != nil || len(m.ToolCalls) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
byID := make(map[string]*persistedCall, len(m.ToolCalls))
|
||||
var order []string
|
||||
for _, tc := range m.ToolCalls {
|
||||
if tc.ID == "" {
|
||||
continue
|
||||
}
|
||||
pc, ok := byID[tc.ID]
|
||||
if !ok {
|
||||
pc = &persistedCall{id: tc.ID}
|
||||
byID[tc.ID] = pc
|
||||
order = append(order, tc.ID)
|
||||
}
|
||||
if tc.Name != "" {
|
||||
pc.name = tc.Name
|
||||
}
|
||||
if len(tc.Args) > 0 && string(tc.Args) != "null" {
|
||||
pc.args = tc.Args
|
||||
}
|
||||
if tc.Type == "tool_result" {
|
||||
pc.errMsg = tc.Error
|
||||
pc.result = tc.Result
|
||||
}
|
||||
}
|
||||
|
||||
calls := make([]persistedCall, 0, len(order))
|
||||
for _, id := range order {
|
||||
calls = append(calls, *byID[id])
|
||||
}
|
||||
return calls
|
||||
}
|
||||
|
||||
// assistantToolCallMessage builds the tool-calling assistant message that
|
||||
// must precede the tool-role results being replayed.
|
||||
func assistantToolCallMessage(calls []persistedCall) openai.ChatCompletionMessageParamUnion {
|
||||
toolCalls := make([]openai.ChatCompletionMessageToolCallParam, 0, len(calls))
|
||||
for _, c := range calls {
|
||||
args := string(c.args)
|
||||
if args == "" {
|
||||
args = "{}"
|
||||
}
|
||||
toolCalls = append(toolCalls, openai.ChatCompletionMessageToolCallParam{
|
||||
ID: c.id,
|
||||
Function: openai.ChatCompletionMessageToolCallFunctionParam{
|
||||
Name: c.name,
|
||||
Arguments: args,
|
||||
},
|
||||
})
|
||||
}
|
||||
return openai.ChatCompletionMessageParamUnion{
|
||||
OfAssistant: &openai.ChatCompletionAssistantMessageParam{ToolCalls: toolCalls},
|
||||
}
|
||||
}
|
||||
|
||||
// fleetSnapshot returns a compact, current-as-of-now fleet health line for
|
||||
// the system prompt so the agent starts each turn already oriented instead
|
||||
// of spending its first iteration rediscovering topology it already has
|
||||
// tools to query. Best-effort: an empty string on any failure just means no
|
||||
// snapshot, not an error for the turn.
|
||||
func (a *agent) fleetSnapshot() string {
|
||||
result, err := a.client.callTool("get_health_summary", map[string]any{})
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
rows, ok := result.([]any)
|
||||
if !ok {
|
||||
return ""
|
||||
}
|
||||
|
||||
counts := map[string]int{}
|
||||
var attention []string
|
||||
for _, r := range rows {
|
||||
row, ok := r.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
health, _ := row["health"].(string)
|
||||
counts[health]++
|
||||
if health != "healthy" && health != "" {
|
||||
if slug, ok := row["slug"].(string); ok && len(attention) < 10 {
|
||||
attention = append(attention, fmt.Sprintf("%s(%s)", slug, health))
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(counts) == 0 {
|
||||
return ""
|
||||
}
|
||||
|
||||
summary := fmt.Sprintf("Current fleet snapshot (as of now): healthy=%d degraded=%d down=%d stale=%d unknown=%d.",
|
||||
counts["healthy"], counts["degraded"], counts["down"], counts["stale"], counts["unknown"])
|
||||
if len(attention) > 0 {
|
||||
summary += " Needs attention: " + fmt.Sprintf("%v", attention) + "."
|
||||
}
|
||||
return summary
|
||||
}
|
||||
|
||||
func (a *agent) buildTools() ([]openai.ChatCompletionToolParam, error) {
|
||||
defs, err := a.client.listToolsFull()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var tools []openai.ChatCompletionToolParam
|
||||
for _, d := range defs {
|
||||
params := shared.FunctionParameters(d.InputSchema)
|
||||
if params == nil {
|
||||
params = shared.FunctionParameters{"type": "object", "properties": map[string]any{}}
|
||||
}
|
||||
|
||||
tools = append(tools, openai.ChatCompletionToolParam{
|
||||
Type: "function",
|
||||
Function: shared.FunctionDefinitionParam{
|
||||
Name: d.Name,
|
||||
Description: openai.String(d.Description),
|
||||
Parameters: params,
|
||||
},
|
||||
})
|
||||
}
|
||||
return tools, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) listToolsFull() ([]toolDef, error) {
|
||||
resp, err := c.doRequest("tools/list", map[string]any{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var tr struct {
|
||||
Tools []struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
InputSchema map[string]any `json:"inputSchema"`
|
||||
} `json:"tools"`
|
||||
}
|
||||
if err := json.Unmarshal(resp.Result, &tr); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := make([]toolDef, len(tr.Tools))
|
||||
for i, t := range tr.Tools {
|
||||
out[i] = toolDef{
|
||||
Name: t.Name,
|
||||
Description: t.Description,
|
||||
InputSchema: t.InputSchema,
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
520
cmd/nomos/main.go
Normal file
520
cmd/nomos/main.go
Normal file
@@ -0,0 +1,520 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strings"
|
||||
"sync"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
func main() {
|
||||
if len(os.Args) < 2 {
|
||||
fmt.Fprintln(os.Stderr, "usage: nomos serve")
|
||||
os.Exit(1)
|
||||
}
|
||||
mcpURL := os.Getenv("NOMOS_MCP_URL")
|
||||
if mcpURL == "" {
|
||||
mcpURL = "http://localhost:8090/mcp"
|
||||
}
|
||||
|
||||
agentSlug := os.Getenv("NOMOS_AGENT_SLUG")
|
||||
if agentSlug == "" {
|
||||
agentSlug = "agent:nomos"
|
||||
}
|
||||
|
||||
databaseURL := os.Getenv("DATABASE_URL")
|
||||
if databaseURL == "" {
|
||||
databaseURL = os.Getenv("OIKOS_DATABASE_URL")
|
||||
}
|
||||
|
||||
switch os.Args[1] {
|
||||
case "serve":
|
||||
ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT)
|
||||
defer cancel()
|
||||
|
||||
client, err := newMCPClient(mcpURL)
|
||||
if err != nil {
|
||||
slog.Error("nomos: mcp connect", "url", mcpURL, "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
st, err := newStore(ctx, databaseURL)
|
||||
if err != nil {
|
||||
slog.Error("nomos: db connect", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
if st != nil {
|
||||
defer st.close()
|
||||
}
|
||||
|
||||
nAgent, err := newAgent(ctx, client, st, agentSlug)
|
||||
if err != nil {
|
||||
slog.Error("nomos: agent init", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(200)
|
||||
w.Write([]byte("ok"))
|
||||
})
|
||||
mux.HandleFunc("/query", func(w http.ResponseWriter, r *http.Request) {
|
||||
handleQuery(w, r, client, agentSlug, mcpURL)
|
||||
})
|
||||
mux.HandleFunc("/chat", func(w http.ResponseWriter, r *http.Request) {
|
||||
handleChat(w, r, nAgent, st)
|
||||
})
|
||||
mux.HandleFunc("/sessions", func(w http.ResponseWriter, r *http.Request) {
|
||||
handleSessionsList(w, r, st)
|
||||
})
|
||||
mux.HandleFunc("/sessions/", func(w http.ResponseWriter, r *http.Request) {
|
||||
handleSessionDetail(w, r, st)
|
||||
})
|
||||
|
||||
addr := os.Getenv("NOMOS_LISTEN")
|
||||
if addr == "" {
|
||||
addr = ":8092"
|
||||
}
|
||||
|
||||
srv := &http.Server{Addr: addr, Handler: mux}
|
||||
go func() {
|
||||
slog.Info("nomos: gateway listening", "addr", addr, "mcp", mcpURL, "db", databaseURL != "")
|
||||
if err := srv.ListenAndServe(); err != http.ErrServerClosed {
|
||||
slog.Error("nomos: serve", "error", err)
|
||||
}
|
||||
}()
|
||||
|
||||
<-ctx.Done()
|
||||
slog.Info("nomos: shutting down")
|
||||
srv.Shutdown(context.Background())
|
||||
client.close()
|
||||
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "unknown command: %s\n", os.Args[1])
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func sseEvent(w http.ResponseWriter, flusher http.Flusher, event agentEvent) {
|
||||
data, _ := json.Marshal(event)
|
||||
fmt.Fprintf(w, "data: %s\n\n", data)
|
||||
flusher.Flush()
|
||||
}
|
||||
|
||||
func handleChat(w http.ResponseWriter, r *http.Request, a *agent, st *store) {
|
||||
if r.Method != http.MethodPost {
|
||||
http.Error(w, "method not allowed", 405)
|
||||
return
|
||||
}
|
||||
|
||||
var req struct {
|
||||
SessionID string `json:"session_id"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
http.Error(w, "bad request: "+err.Error(), 400)
|
||||
return
|
||||
}
|
||||
if req.Message == "" {
|
||||
http.Error(w, "message is required", 400)
|
||||
return
|
||||
}
|
||||
|
||||
flusher, ok := w.(http.Flusher)
|
||||
if !ok {
|
||||
http.Error(w, "streaming not supported", 500)
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "text/event-stream")
|
||||
w.Header().Set("Cache-Control", "no-cache")
|
||||
w.Header().Set("Connection", "keep-alive")
|
||||
w.WriteHeader(200)
|
||||
|
||||
ctx := r.Context()
|
||||
sessionID := req.SessionID
|
||||
|
||||
if sessionID == "" {
|
||||
title := truncate(req.Message, 80)
|
||||
sess, err := st.createSession(ctx, title)
|
||||
if err != nil {
|
||||
slog.Error("nomos: create session", "error", err)
|
||||
sessionID = "ephemeral"
|
||||
} else {
|
||||
sessionID = sess.ID
|
||||
}
|
||||
} else {
|
||||
st.touchSession(ctx, sessionID)
|
||||
}
|
||||
|
||||
slog.Info("nomos: chat", "session", sessionID, "message", truncate(req.Message, 100))
|
||||
|
||||
userMsg, _ := json.Marshal(map[string]any{"role": "user", "text": req.Message})
|
||||
st.saveMessage(ctx, sessionID, "user", userMsg)
|
||||
|
||||
sseEvent(w, flusher, agentEvent{Type: "session", Data: sessionID, SessionID: sessionID})
|
||||
|
||||
toolCalls := []map[string]any{}
|
||||
var finalText string
|
||||
|
||||
a.chat(ctx, sessionID, req.Message, func(ev agentEvent) {
|
||||
if ev.Type == "tool_use" || ev.Type == "tool_result" {
|
||||
if m, ok := ev.Data.(map[string]any); ok {
|
||||
m["type"] = ev.Type
|
||||
toolCalls = append(toolCalls, m)
|
||||
}
|
||||
}
|
||||
if ev.Type == "text" {
|
||||
finalText, _ = ev.Data.(string)
|
||||
}
|
||||
sseEvent(w, flusher, ev)
|
||||
})
|
||||
|
||||
assistantMsg, _ := json.Marshal(map[string]any{
|
||||
"role": "assistant",
|
||||
"text": finalText,
|
||||
"tool_calls": toolCalls,
|
||||
})
|
||||
st.saveMessage(ctx, sessionID, "assistant", assistantMsg)
|
||||
}
|
||||
|
||||
func handleSessionsList(w http.ResponseWriter, r *http.Request, st *store) {
|
||||
if st == nil {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{"sessions": []any{}})
|
||||
return
|
||||
}
|
||||
|
||||
if r.Method == http.MethodOptions {
|
||||
return
|
||||
}
|
||||
|
||||
sessions, err := st.listSessions(r.Context())
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), 500)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{"sessions": sessions})
|
||||
}
|
||||
|
||||
func handleSessionDetail(w http.ResponseWriter, r *http.Request, st *store) {
|
||||
if st == nil {
|
||||
http.Error(w, "not found", 404)
|
||||
return
|
||||
}
|
||||
|
||||
id := strings.TrimPrefix(r.URL.Path, "/sessions/")
|
||||
if id == "" {
|
||||
http.Error(w, "session id required", 400)
|
||||
return
|
||||
}
|
||||
|
||||
messages, err := st.getMessages(r.Context(), id)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), 500)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{"session_id": id, "messages": messages})
|
||||
}
|
||||
|
||||
func handleQuery(w http.ResponseWriter, r *http.Request, client *mcpClient, agentSlug, mcpURL string) {
|
||||
if r.Method != http.MethodPost {
|
||||
http.Error(w, "method not allowed", 405)
|
||||
return
|
||||
}
|
||||
|
||||
var req struct {
|
||||
Query string `json:"query"`
|
||||
Tool string `json:"tool"`
|
||||
Args map[string]any `json:"args"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
http.Error(w, "bad request: "+err.Error(), 400)
|
||||
return
|
||||
}
|
||||
|
||||
start := time.Now()
|
||||
|
||||
if req.Tool != "" {
|
||||
result, err := client.callTool(req.Tool, req.Args)
|
||||
duration := time.Since(start).Milliseconds()
|
||||
if err != nil {
|
||||
slog.Error("nomos: query failed", "tool", req.Tool, "error", err)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{
|
||||
"error": err.Error(),
|
||||
"elapsed_ms": duration,
|
||||
"agent_slug": agentSlug,
|
||||
})
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{
|
||||
"result": result,
|
||||
"elapsed_ms": duration,
|
||||
"agent_slug": agentSlug,
|
||||
"mcp_url": mcpURL,
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
if req.Query != "" {
|
||||
if strings.Contains(strings.ToLower(req.Query), "what can you do") ||
|
||||
strings.Contains(strings.ToLower(req.Query), "help") {
|
||||
|
||||
tools, err := client.listTools()
|
||||
duration := time.Since(start).Milliseconds()
|
||||
if err != nil {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{
|
||||
"error": err.Error(),
|
||||
"elapsed_ms": duration,
|
||||
})
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{
|
||||
"message": "natural language queries belong to /chat. Use structured /query with 'tool' for direct MCP calls.",
|
||||
"tools": tools,
|
||||
"elapsed_ms": duration,
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]any{
|
||||
"message": "natural language queries belong to /chat. Use structured /query with 'tool' for direct MCP calls.",
|
||||
"elapsed_ms": time.Since(start).Milliseconds(),
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
http.Error(w, "either 'tool' or 'query' required", 400)
|
||||
}
|
||||
|
||||
func truncate(s string, n int) string {
|
||||
if len(s) <= n {
|
||||
return s
|
||||
}
|
||||
return s[:n] + "..."
|
||||
}
|
||||
|
||||
// ─── MCP Streamable HTTP client ────────────────────────────────────────
|
||||
|
||||
type mcpClient struct {
|
||||
baseURL string
|
||||
sessionID string
|
||||
http *http.Client
|
||||
nextID int
|
||||
mu sync.Mutex // MCP is one stateful session; serialize concurrent calls
|
||||
}
|
||||
|
||||
func newMCPClient(baseURL string) (*mcpClient, error) {
|
||||
c := &mcpClient{
|
||||
baseURL: baseURL,
|
||||
http: &http.Client{Timeout: 30 * time.Second},
|
||||
}
|
||||
|
||||
resp, err := c.doRequest("initialize", map[string]any{
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": map[string]any{},
|
||||
"clientInfo": map[string]any{"name": "nomos", "version": "2.0"},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("initialize: %w", err)
|
||||
}
|
||||
if resp.sessionID == "" {
|
||||
return nil, fmt.Errorf("no session ID in initialize response")
|
||||
}
|
||||
c.sessionID = resp.sessionID
|
||||
|
||||
c.doRequest("notifications/initialized", map[string]any{})
|
||||
|
||||
slog.Info("nomos: mcp connected", "session", c.sessionID[:16]+"...")
|
||||
return c, nil
|
||||
}
|
||||
|
||||
type mcpJSONRPCResponse struct {
|
||||
sessionID string
|
||||
Result json.RawMessage `json:"result"`
|
||||
Error json.RawMessage `json:"error"`
|
||||
}
|
||||
|
||||
// errStaleSession signals that the MCP server rejected our session id (e.g.
|
||||
// after an api/MCP restart), so the client should re-initialize and retry.
|
||||
var errStaleSession = fmt.Errorf("mcp session stale")
|
||||
|
||||
// doRequest serializes MCP calls and transparently re-initializes the session
|
||||
// if the server has forgotten it (common after an api redeploy), retrying the
|
||||
// original call once. Without this, an api restart permanently breaks nomos
|
||||
// until it is itself restarted.
|
||||
func (c *mcpClient) doRequest(method string, params map[string]any) (*mcpJSONRPCResponse, error) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
|
||||
resp, err := c.send(method, params)
|
||||
if err != nil && method != "initialize" && errors.Is(err, errStaleSession) {
|
||||
slog.Warn("nomos: mcp session stale, reconnecting")
|
||||
if rerr := c.reconnectLocked(); rerr != nil {
|
||||
return nil, fmt.Errorf("mcp reconnect: %w (original: %v)", rerr, err)
|
||||
}
|
||||
return c.send(method, params)
|
||||
}
|
||||
return resp, err
|
||||
}
|
||||
|
||||
// reconnectLocked re-initializes the MCP session. The caller must hold c.mu.
|
||||
func (c *mcpClient) reconnectLocked() error {
|
||||
c.sessionID = ""
|
||||
resp, err := c.send("initialize", map[string]any{
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": map[string]any{},
|
||||
"clientInfo": map[string]any{"name": "nomos", "version": "2.0"},
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.sessionID == "" {
|
||||
return fmt.Errorf("no session ID on re-initialize")
|
||||
}
|
||||
c.sessionID = resp.sessionID
|
||||
_, _ = c.send("notifications/initialized", map[string]any{})
|
||||
slog.Info("nomos: mcp reconnected", "session", c.sessionID[:16]+"...")
|
||||
return nil
|
||||
}
|
||||
|
||||
// send performs one MCP round-trip. It does not lock; callers hold c.mu.
|
||||
func (c *mcpClient) send(method string, params map[string]any) (*mcpJSONRPCResponse, error) {
|
||||
c.nextID++
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"jsonrpc": "2.0",
|
||||
"method": method,
|
||||
"params": params,
|
||||
"id": c.nextID,
|
||||
})
|
||||
|
||||
req, err := http.NewRequest(http.MethodPost, c.baseURL, bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Accept", "application/json, text/event-stream")
|
||||
if c.sessionID != "" {
|
||||
req.Header.Set("Mcp-Session-Id", c.sessionID)
|
||||
}
|
||||
|
||||
resp, err := c.http.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// A rejected/unknown session comes back as 4xx (commonly 400/404).
|
||||
if resp.StatusCode == http.StatusNotFound || resp.StatusCode == http.StatusBadRequest {
|
||||
return nil, errStaleSession
|
||||
}
|
||||
|
||||
result := &mcpJSONRPCResponse{}
|
||||
result.sessionID = resp.Header.Get("Mcp-Session-Id")
|
||||
|
||||
scanner := bufio.NewScanner(resp.Body)
|
||||
scanner.Buffer(make([]byte, 0, 64*1024), 4*1024*1024)
|
||||
gotData := false
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
if strings.HasPrefix(line, "data: ") {
|
||||
gotData = true
|
||||
data := line[6:]
|
||||
if err := json.Unmarshal([]byte(data), result); err != nil {
|
||||
return nil, fmt.Errorf("parse response: %w", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if result.Error != nil {
|
||||
return nil, fmt.Errorf("rpc error: %s", string(result.Error))
|
||||
}
|
||||
|
||||
// Empty body with no result and a session set: the server likely dropped
|
||||
// our session. Notifications legitimately return no data, so exempt them.
|
||||
if !gotData && result.Result == nil && method != "notifications/initialized" {
|
||||
return nil, errStaleSession
|
||||
}
|
||||
|
||||
if result.sessionID != "" {
|
||||
c.sessionID = result.sessionID
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) callTool(name string, args map[string]any) (any, error) {
|
||||
resp, err := c.doRequest("tools/call", map[string]any{
|
||||
"name": name,
|
||||
"arguments": args,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var toolResult struct {
|
||||
Content []struct {
|
||||
Type string `json:"type"`
|
||||
Text string `json:"text"`
|
||||
} `json:"content"`
|
||||
}
|
||||
if err := json.Unmarshal(resp.Result, &toolResult); err != nil {
|
||||
return string(resp.Result), nil
|
||||
}
|
||||
|
||||
var texts []string
|
||||
for _, c := range toolResult.Content {
|
||||
if c.Type == "text" {
|
||||
var parsed any
|
||||
if json.Unmarshal([]byte(c.Text), &parsed) == nil {
|
||||
return parsed, nil
|
||||
}
|
||||
texts = append(texts, c.Text)
|
||||
}
|
||||
}
|
||||
if len(texts) == 1 {
|
||||
return texts[0], nil
|
||||
}
|
||||
return texts, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) listTools() ([]string, error) {
|
||||
resp, err := c.doRequest("tools/list", map[string]any{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var tr struct {
|
||||
Tools []struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
} `json:"tools"`
|
||||
}
|
||||
if err := json.Unmarshal(resp.Result, &tr); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var names []string
|
||||
for _, t := range tr.Tools {
|
||||
names = append(names, t.Name)
|
||||
}
|
||||
return names, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) close() {
|
||||
}
|
||||
156
cmd/nomos/store.go
Normal file
156
cmd/nomos/store.go
Normal file
@@ -0,0 +1,156 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
type store struct {
|
||||
pool *pgxpool.Pool
|
||||
}
|
||||
|
||||
func newStore(ctx context.Context, databaseURL string) (*store, error) {
|
||||
if databaseURL == "" {
|
||||
return nil, nil
|
||||
}
|
||||
pool, err := pgxpool.New(ctx, databaseURL)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("connect db: %w", err)
|
||||
}
|
||||
if err := pool.Ping(ctx); err != nil {
|
||||
pool.Close()
|
||||
return nil, fmt.Errorf("ping db: %w", err)
|
||||
}
|
||||
return &store{pool: pool}, nil
|
||||
}
|
||||
|
||||
func (s *store) close() {
|
||||
if s.pool != nil {
|
||||
s.pool.Close()
|
||||
}
|
||||
}
|
||||
|
||||
type session struct {
|
||||
ID string `json:"id"`
|
||||
Title string `json:"title"`
|
||||
Actor string `json:"actor"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
LastActiveAt time.Time `json:"last_active_at"`
|
||||
}
|
||||
|
||||
type message struct {
|
||||
ID string `json:"id"`
|
||||
SessionID string `json:"session_id"`
|
||||
Role string `json:"role"`
|
||||
Content json.RawMessage `json:"content"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
func (s *store) createSession(ctx context.Context, title string) (*session, error) {
|
||||
if s == nil {
|
||||
return &session{ID: "ephemeral", Title: title, Actor: "agent:nomos"}, nil
|
||||
}
|
||||
var id string
|
||||
err := s.pool.QueryRow(ctx,
|
||||
`INSERT INTO agent_sessions (title, actor) VALUES ($1, 'agent:nomos') RETURNING id`,
|
||||
title).Scan(&id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &session{ID: id, Title: title, Actor: "agent:nomos", CreatedAt: time.Now(), LastActiveAt: time.Now()}, nil
|
||||
}
|
||||
|
||||
func (s *store) saveMessage(ctx context.Context, sessionID, role string, content json.RawMessage) error {
|
||||
if s == nil {
|
||||
return nil
|
||||
}
|
||||
_, err := s.pool.Exec(ctx,
|
||||
`INSERT INTO agent_messages (session_id, role, content) VALUES ($1, $2, $3)`,
|
||||
sessionID, role, content)
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *store) touchSession(ctx context.Context, id string) {
|
||||
if s != nil {
|
||||
s.pool.Exec(ctx, `UPDATE agent_sessions SET last_active_at=now() WHERE id=$1`, id)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *store) listSessions(ctx context.Context) ([]session, error) {
|
||||
if s == nil {
|
||||
return nil, nil
|
||||
}
|
||||
rows, err := s.pool.Query(ctx,
|
||||
`SELECT id, title, actor, created_at, last_active_at FROM agent_sessions ORDER BY last_active_at DESC LIMIT 50`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var out []session
|
||||
for rows.Next() {
|
||||
var sess session
|
||||
if err := rows.Scan(&sess.ID, &sess.Title, &sess.Actor, &sess.CreatedAt, &sess.LastActiveAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, sess)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func (s *store) getMessages(ctx context.Context, sessionID string) ([]message, error) {
|
||||
if s == nil {
|
||||
return nil, nil
|
||||
}
|
||||
rows, err := s.pool.Query(ctx,
|
||||
`SELECT id, session_id, role, content, created_at FROM agent_messages WHERE session_id=$1 ORDER BY created_at ASC`,
|
||||
sessionID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var out []message
|
||||
for rows.Next() {
|
||||
var m message
|
||||
if err := rows.Scan(&m.ID, &m.SessionID, &m.Role, &m.Content, &m.CreatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, m)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// resolveAgentID looks up the UUID of the agent entity (e.g. "agent:nomos").
|
||||
// Returns uuid.Nil if the store is absent or the slug is unknown.
|
||||
func (s *store) resolveAgentID(ctx context.Context, slug string) uuid.UUID {
|
||||
if s == nil {
|
||||
return uuid.Nil
|
||||
}
|
||||
var id uuid.UUID
|
||||
if err := s.pool.QueryRow(ctx, `SELECT id FROM entities WHERE slug = $1`, slug).Scan(&id); err != nil {
|
||||
return uuid.Nil
|
||||
}
|
||||
return id
|
||||
}
|
||||
|
||||
// logActivity records a tool call. agent_id is the agent entity UUID and is
|
||||
// NOT NULL in the schema, so we skip logging when it can't be resolved.
|
||||
// The (nullable) session_id column carries the conversation id.
|
||||
func (s *store) logActivity(ctx context.Context, agentID uuid.UUID, sessionID, toolName, inputSummary, outputSummary string, durationMs int, success bool, correlationID string) {
|
||||
if s == nil || agentID == uuid.Nil {
|
||||
return
|
||||
}
|
||||
s.pool.Exec(ctx, `
|
||||
INSERT INTO agent_activity
|
||||
(agent_id, session_id, activity_type, tool_name, input_summary, output_summary,
|
||||
duration_ms, success, correlation_id)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)`,
|
||||
agentID, sessionID, "tool_call", toolName, inputSummary, outputSummary,
|
||||
durationMs, success, correlationID)
|
||||
}
|
||||
@@ -1,15 +1,17 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strings"
|
||||
"syscall"
|
||||
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/config"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
@@ -19,9 +21,50 @@ import (
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/dtoro/oikos/internal/scheduler"
|
||||
"github.com/dtoro/oikos/internal/secrets"
|
||||
"github.com/dtoro/oikos/web"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// uiHandler serves the control-room SPA from assets embedded at build time
|
||||
// (web/embed.go), with SPA fallback to index.html. Requests arrive as /ui/*;
|
||||
// the /ui prefix is stripped to index into the embedded dist/ tree. Files are
|
||||
// written via http.ServeContent (not http.FileServer) to avoid its
|
||||
// index.html -> "./" canonical redirect, which loops for /ui/.
|
||||
func uiHandler() http.Handler {
|
||||
dist, err := web.DistFS()
|
||||
if err != nil {
|
||||
slog.Warn("ui: embedded assets unavailable", "error", err)
|
||||
return http.NotFoundHandler()
|
||||
}
|
||||
serve := func(w http.ResponseWriter, r *http.Request, name string) bool {
|
||||
f, err := dist.Open(name)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
defer f.Close()
|
||||
data, err := io.ReadAll(f)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
http.ServeContent(w, r, name, time.Time{}, bytes.NewReader(data))
|
||||
return true
|
||||
}
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
name := strings.TrimPrefix(strings.TrimPrefix(r.URL.Path, "/ui"), "/")
|
||||
if name == "" {
|
||||
name = "index.html"
|
||||
}
|
||||
if serve(w, r, name) {
|
||||
return
|
||||
}
|
||||
// SPA fallback: serve index.html for unknown client-side routes.
|
||||
if serve(w, r, "index.html") {
|
||||
return
|
||||
}
|
||||
http.NotFound(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
var schedulerRunner = scheduler.RunnerForMain()
|
||||
var notifierRunner = notifier.RunnerForMain()
|
||||
|
||||
@@ -86,7 +129,7 @@ func main() {
|
||||
go notifierRunner(ctx, pool, cfg)
|
||||
|
||||
slog.Info("all: starting api with scheduler + notifier in background")
|
||||
if err := httpapi.ListenAndServe(ctx, pool, cfg); err != nil {
|
||||
if err := httpapi.ListenAndServe(ctx, pool, cfg, uiHandler()); err != nil {
|
||||
slog.Error("api failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
@@ -120,7 +163,7 @@ Roles:
|
||||
knowledge Convert wiki to knowledge seed (one-shot)
|
||||
version Print version info
|
||||
|
||||
The operator interface is Hermes (MCP agent) — no CLI needed.
|
||||
The operator interface is Nomos (MCP agent) — no CLI needed.
|
||||
Environment:
|
||||
OIKOS_DATABASE_URL Postgres connection string
|
||||
OIKOS_API_LISTEN API listen address (default :8090)
|
||||
@@ -265,7 +308,7 @@ func runAPI(ctx context.Context, cfg config.Config) error {
|
||||
return fmt.Errorf("migrations: %w", err)
|
||||
}
|
||||
|
||||
err = httpapi.ListenAndServe(ctx, pool, cfg)
|
||||
err = httpapi.ListenAndServe(ctx, pool, cfg, uiHandler())
|
||||
if err == http.ErrServerClosed {
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -11,18 +11,25 @@ oikos.hubris.network {
|
||||
handle @enroll {
|
||||
reverse_proxy <mac-mini-mesh-ip>:8090
|
||||
}
|
||||
# Nomos agent, same-origin for the control-room UI (EventSource/fetch can't
|
||||
# set cross-origin auth headers). Authentik gates it; handle_path strips
|
||||
# the /agent prefix so /agent/chat -> nomos /chat.
|
||||
handle_path /agent/* {
|
||||
import authentik
|
||||
reverse_proxy <mac-mini-mesh-ip>:8092
|
||||
}
|
||||
handle {
|
||||
import authentik
|
||||
reverse_proxy <mac-mini-mesh-ip>:8090
|
||||
}
|
||||
}
|
||||
|
||||
# Oikos MCP endpoint (Hermes agents) — no auth required
|
||||
# Oikos MCP endpoint (agents) — no auth required
|
||||
mcp.hubris.network {
|
||||
reverse_proxy <mac-mini-mesh-ip>:8090
|
||||
}
|
||||
|
||||
# Hermes gateway (workstation access)
|
||||
hermes.hubris.network {
|
||||
# Nomos gateway (workstation access) — formerly hermes.hubris.network
|
||||
nomos.hubris.network {
|
||||
reverse_proxy <mac-mini-mesh-ip>:8092
|
||||
}
|
||||
|
||||
@@ -1,25 +0,0 @@
|
||||
# Hermes agent container — standalone MCP client gateway (Phase 4)
|
||||
FROM golang:1.26-alpine AS builder
|
||||
|
||||
RUN apk add --no-cache git ca-certificates
|
||||
|
||||
WORKDIR /build
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
|
||||
COPY . .
|
||||
|
||||
RUN CGO_ENABLED=0 go build -o /hermes -tags timetzdata -ldflags="-s -w" ./cmd/hermes
|
||||
|
||||
FROM gcr.io/distroless/static:nonroot
|
||||
|
||||
COPY --from=builder /hermes /hermes
|
||||
COPY hermes/ /app/hermes/
|
||||
|
||||
ENV HERMES_MCP_URL=http://api:8090/mcp
|
||||
ENV HERMES_AGENT_SLUG=agent:hermes
|
||||
ENV HERMES_LISTEN=:8092
|
||||
|
||||
EXPOSE 8092
|
||||
|
||||
ENTRYPOINT ["/hermes", "serve"]
|
||||
26
compose/nomos/Dockerfile
Normal file
26
compose/nomos/Dockerfile
Normal file
@@ -0,0 +1,26 @@
|
||||
# Nomos agent container — standalone MCP client gateway (Phase 4)
|
||||
FROM golang:1.26-alpine AS builder
|
||||
|
||||
RUN apk add --no-cache git ca-certificates
|
||||
|
||||
WORKDIR /build
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
|
||||
COPY . .
|
||||
|
||||
RUN CGO_ENABLED=0 go build -o /nomos -tags timetzdata -ldflags="-s -w" ./cmd/nomos
|
||||
|
||||
FROM gcr.io/distroless/static:nonroot
|
||||
|
||||
COPY --from=builder /nomos /nomos
|
||||
COPY nomos/ /app/nomos/
|
||||
|
||||
ENV NOMOS_MCP_URL=http://api:8090/mcp
|
||||
ENV NOMOS_AGENT_SLUG=agent:nomos
|
||||
ENV NOMOS_LISTEN=:8092
|
||||
ENV NOMOS_MODEL=deepseek/deepseek-v4-flash
|
||||
|
||||
EXPOSE 8092
|
||||
|
||||
ENTRYPOINT ["/nomos", "serve"]
|
||||
@@ -1,4 +1,14 @@
|
||||
# Multi-stage Dockerfile for Oikos (ADR 0001: single binary)
|
||||
# Stage 1: build web UI
|
||||
FROM node:22-alpine AS ui-builder
|
||||
|
||||
WORKDIR /web
|
||||
COPY web/package.json web/package-lock.json ./
|
||||
RUN npm ci
|
||||
COPY web/ ./
|
||||
RUN npm run build
|
||||
|
||||
# Stage 2: build Go binary
|
||||
FROM golang:1.26-alpine AS builder
|
||||
|
||||
RUN apk add --no-cache git ca-certificates
|
||||
@@ -8,14 +18,19 @@ COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
|
||||
COPY . .
|
||||
# Bring in the built SPA so //go:embed all:dist (web/embed.go) has real assets.
|
||||
COPY --from=ui-builder /web/dist ./web/dist
|
||||
|
||||
RUN CGO_ENABLED=0 go build -o /oikos -tags timetzdata -ldflags="-s -w" ./cmd/oikos
|
||||
|
||||
# --- Runtime: distroless static ---
|
||||
FROM gcr.io/distroless/static:nonroot
|
||||
# --- Runtime: alpine with SSH + ping for scheduler checks ---
|
||||
FROM alpine:3.21
|
||||
|
||||
RUN apk add --no-cache ca-certificates openssh-client-default
|
||||
|
||||
COPY --from=builder /oikos /oikos
|
||||
COPY --from=builder /build/seeds /seeds
|
||||
COPY --from=builder /build/migrations /migrations
|
||||
# web/dist is embedded in the binary (web/embed.go) — no runtime copy needed.
|
||||
|
||||
ENTRYPOINT ["/oikos"]
|
||||
|
||||
@@ -60,7 +60,8 @@ services:
|
||||
OIKOS_API_LISTEN: ":8090"
|
||||
OIKOS_ENV: dev
|
||||
OIKOS_DEBUG: "true"
|
||||
OIKOS_HERMES_AGENT_SLUG: ${OIKOS_HERMES_AGENT_SLUG:-agent:hermes}
|
||||
OIKOS_NOMOS_AGENT_SLUG: ${OIKOS_NOMOS_AGENT_SLUG:-agent:nomos}
|
||||
NOMOS_PROXY_URL: http://nomos:8092
|
||||
volumes:
|
||||
- ${OIKOS_SSH_KEY_PATH:-~/.ssh/id_ed25519}:/etc/oikos/ssh_key:ro
|
||||
ports:
|
||||
@@ -82,6 +83,12 @@ services:
|
||||
OIKOS_DATABASE_URL: postgres://oikos:${OIKOS_DB_PASSWORD:-oikos_dev}@postgres:5432/oikos?sslmode=disable
|
||||
OIKOS_DEBUG: "true"
|
||||
OIKOS_SCHEDULER_INTERVAL: "30s"
|
||||
OIKOS_SSH_KEY_PATH: /etc/oikos/ssh_key
|
||||
OIKOS_SSH_USER: root
|
||||
volumes:
|
||||
- ${OIKOS_SSH_KEY_PATH:-~/.ssh/id_ed25519}:/etc/oikos/ssh_key:ro
|
||||
cap_add:
|
||||
- NET_RAW
|
||||
command: ["scheduler"]
|
||||
stop_signal: SIGTERM
|
||||
stop_grace_period: 30s
|
||||
@@ -107,18 +114,21 @@ services:
|
||||
stop_signal: SIGTERM
|
||||
stop_grace_period: 30s
|
||||
|
||||
# Hermes agent gateway (Phase 4) — mesh-published :8092
|
||||
hermes:
|
||||
# Nomos agent gateway (Phase 4) — mesh-published :8092
|
||||
nomos:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/hermes/Dockerfile
|
||||
dockerfile: compose/nomos/Dockerfile
|
||||
profiles: ["full"]
|
||||
depends_on:
|
||||
api:
|
||||
condition: service_started
|
||||
environment:
|
||||
HERMES_MCP_URL: http://api:8090/mcp
|
||||
HERMES_AGENT_SLUG: agent:hermes
|
||||
NOMOS_MCP_URL: http://api:8090/mcp
|
||||
NOMOS_AGENT_SLUG: agent:nomos
|
||||
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY}
|
||||
NOMOS_MODEL: ${NOMOS_MODEL:-deepseek/deepseek-v4-flash}
|
||||
DATABASE_URL: postgres://oikos:${OIKOS_DB_PASSWORD:-oikos_dev}@postgres:5432/oikos?sslmode=disable
|
||||
ports:
|
||||
- "8092:8092"
|
||||
stop_signal: SIGTERM
|
||||
|
||||
@@ -230,4 +230,13 @@ sequenceDiagram
|
||||
`GET /clients/{slug}/context`.
|
||||
- **The DB is the single source of truth.** All state transitions,
|
||||
audit entries, and event emissions go through Postgres. The scheduler,
|
||||
actuator, notifier, and API all read/write the same tables.
|
||||
actuator, notifier, and API all read/write the same tables.
|
||||
|
||||
---
|
||||
|
||||
**2026-07-08 — renamed to Nomos.** The Hermes agent gateway was renamed to
|
||||
Nomos (from *oikonomos*, the steward of the oikos) under the
|
||||
[Nomos resident agent plan](../../plans/2026-07-08-nomos-resident-agent.md),
|
||||
N0 milestone. The gateway binary (`cmd/nomos`), Docker service, DB slug
|
||||
(`agent:nomos`), and all referencing docs were updated. All architectural
|
||||
principles in this ADR remain unchanged.
|
||||
197
docs/adr/0013-signal-triggers.md
Normal file
197
docs/adr/0013-signal-triggers.md
Normal file
@@ -0,0 +1,197 @@
|
||||
# Signal Trigger Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
When Nomos is asked "what are the thermals of hubris?", here is exactly what happens:
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant N as Nomos (Agent)
|
||||
participant A as Oikos API
|
||||
participant S as Scheduler (Docker)
|
||||
participant H as Hubris (Proxmox)
|
||||
participant T as TimescaleDB
|
||||
|
||||
Note over S,H: Every 60s (autonomous loop)
|
||||
S->>H: SSH exec /opt/oikos/checks/cpu_check.sh
|
||||
H-->>S: {"health":"ok","metrics":{"cpu_pct":2.5,"cpu_temp":48}}
|
||||
S->>T: INSERT metric_samples (cpu_pct, cpu_temp)
|
||||
S->>T: UPSERT entity_status (health)
|
||||
alt unhealthy
|
||||
S->>T: UPSERT signal (dedup by target+kind)
|
||||
end
|
||||
|
||||
Note over N,T: User asks "what are the thermals of hubris?"
|
||||
N->>A: MCP query_metrics(metric=["cpu_pct","cpu_temp"])
|
||||
A->>T: SELECT time_bucket(…) FROM metric_samples
|
||||
T-->>A: cpu_pct=15%, cpu_temp=48°C
|
||||
A-->>N: {avg, min, max} per bucket
|
||||
```
|
||||
|
||||
## Two Paths
|
||||
|
||||
### Path A — Autonomous Collection (Scheduler)
|
||||
|
||||
1. Operator creates a check via REST API: `POST /api/v1/checks`
|
||||
2. Scheduler loads enabled checks every 30s from `check_defs` table
|
||||
3. For `ssh-script` checks, scheduler SSHs to target host and runs `/opt/oikos/checks/<script>.sh`
|
||||
4. Script returns JSON with `health`, `signalKind`, `evidence`, and `metrics`
|
||||
5. Metrics written to TimescaleDB `metric_samples` table **every cycle** (healthy or not)
|
||||
6. If unhealthy: a signal is raised (deduplicated by target_entity_id + kind)
|
||||
7. If healthy again: the signal is auto-resolved, entity_status health updated
|
||||
8. All state changes emit SSE events for real-time UI updates
|
||||
|
||||
### Path B — Query (Nomos via MCP)
|
||||
|
||||
1. Nomos calls `query_metrics(metric=["cpu_pct","cpu_temp"])` MCP tool
|
||||
2. API runs time-bucketed aggregation over `metric_samples`
|
||||
3. Returns latest readings with avg/min/max per bucket
|
||||
4. Nomos formats them and presents to the user
|
||||
|
||||
## Check Kinds
|
||||
|
||||
| Kind | Where it runs | Protocol | Example |
|
||||
|------|--------------|----------|---------|
|
||||
| `ping` | Scheduler container | ICMP (`ping` binary) | Reachability + latency |
|
||||
| `http` | Scheduler container | HTTP GET | Service endpoint health |
|
||||
| `tcp` | Scheduler container | TCP dial | Port open check |
|
||||
| `disk` | Scheduler container | `unix.Statfs` | Local disk usage + inodes |
|
||||
| `cert-expiry` | Scheduler container | TLS dial | Certificate days remaining |
|
||||
| `ssh-script` | Remote target via SSH | SSH exec + JSON | Any script in `/opt/oikos/checks/` |
|
||||
|
||||
## Available Check Scripts
|
||||
|
||||
All scripts live in `/opt/oikos/checks/` on target hosts. They output JSON:
|
||||
|
||||
```json
|
||||
{"health":"healthy","metrics":{"cpu_pct":2.5,"cpu_temp":48.0}}
|
||||
```
|
||||
or on failure:
|
||||
```json
|
||||
{"health":"degraded","signalKind":"disk-smart-fail","evidence":"SMART failed for Samsung 990"}
|
||||
```
|
||||
|
||||
| Script | Metrics | Signal (on failure) |
|
||||
|--------|---------|---------------------|
|
||||
| `cpu_check.sh` | `cpu_pct`, `cpu_temp` | threshold-based |
|
||||
| `memory_check.sh` | `mem_pct` | threshold-based |
|
||||
| `load_check.sh` | `load1`, `cores` | threshold-based |
|
||||
| `swap_check.sh` | `swap_pct` | threshold-based |
|
||||
| `disk_usage_check.sh` | `disk_*_pct`, `inode_*_pct` | threshold-based |
|
||||
| `disk_smart_check.sh` | — | `disk-smart-fail` |
|
||||
| `updates_check.sh` | `security_updates`, `reboot_required` | threshold-based |
|
||||
| `zfs_check.sh` | — | `zfs-degraded`, `zfs-scrub-overdue` |
|
||||
| `process_check.sh` | — | `<service-name>` |
|
||||
| `uptime_check.sh` | `uptime_seconds` | threshold-based |
|
||||
| `oom_check.sh` | `oom_count` | `oom-kills` |
|
||||
| `journal_check.sh` | `journal_errors` | `journal-errors` |
|
||||
| `time_check.sh` | `clock_drift_s` | `time-drift` |
|
||||
| `fd_check.sh` | `fd_pct` | threshold-based |
|
||||
| `docker_health_check.sh` | `docker_unhealthy`, `docker_total` | `docker-unhealthy` |
|
||||
| `caddy_error_rate.sh` | `caddy_5xx_rate`, `caddy_requests`, `caddy_5xx` | `caddy-errors` |
|
||||
| `backup_freshness.sh` | — | `backup-stale` |
|
||||
|
||||
## Script Deployment
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant R as Git Repo
|
||||
participant T as Sync Timer (5min)
|
||||
participant H as Target Host
|
||||
|
||||
Note over R,T: Operator pushes scripts
|
||||
R->>T: git pull (homelab-context)
|
||||
T->>T: tools/post-pull.sh
|
||||
T->>T: → tools/setup-checks.sh
|
||||
T->>T: → checks/install.sh
|
||||
T->>H: cp *.sh → /opt/oikos/checks/
|
||||
```
|
||||
|
||||
## Defining a Check
|
||||
|
||||
```bash
|
||||
curl -X POST http://oikos:8090/api/v1/checks \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"kind": "ssh-script",
|
||||
"target": "host:hubris",
|
||||
"config": {
|
||||
"host": "192.168.8.77",
|
||||
"script": "cpu_check.sh",
|
||||
"thresholds": {
|
||||
"cpu_pct": {"warn": 90, "crit": 95},
|
||||
"cpu_temp": {"crit": 85}
|
||||
}
|
||||
},
|
||||
"interval_s": 60
|
||||
}'
|
||||
```
|
||||
|
||||
## Signal Lifecycle
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> raised
|
||||
raised --> acknowledged
|
||||
raised --> muted: mute_until set
|
||||
raised --> resolved: condition cleared
|
||||
|
||||
acknowledged --> acting: classification exists
|
||||
acknowledged --> muted
|
||||
acknowledged --> resolved
|
||||
|
||||
acting --> resolved: verification passed
|
||||
acting --> raised: retry budget remaining
|
||||
acting --> failed
|
||||
|
||||
failed --> acknowledged: operator retry
|
||||
|
||||
muted --> raised: mute_until expired
|
||||
|
||||
resolved --> [*]
|
||||
```
|
||||
|
||||
Signals deduplicate: **one open signal per (target_entity_id, kind)**.
|
||||
Repeated failures increment `occurrence_count` instead of creating duplicates.
|
||||
|
||||
## Threshold Evaluation
|
||||
|
||||
Each check config can define per-metric thresholds in the `config` JSONB:
|
||||
|
||||
```json
|
||||
{
|
||||
"thresholds": {
|
||||
"cpu_temp": {"crit": 85},
|
||||
"cpu_pct": {"warn": 90, "crit": 95}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Severity mapping:
|
||||
- metric >= `crit` → severity = `critical`
|
||||
- metric >= `warn` → severity = `warning`
|
||||
- `health == "down"` with no thresholds → severity = `critical`
|
||||
- Otherwise → severity = `warning`
|
||||
|
||||
## Data Flow (DB Tables)
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
CD[check_defs] -->|scheduler reads| EC[executeCheck]
|
||||
EC -->|healthy?| RS[resolve signal + upsert entity_status]
|
||||
EC -->|unhealthy?| US[UpsertSignal dedup by target+kind]
|
||||
EC -->|every cycle| IM[INSERT metric_samples]
|
||||
US --> S[signals]
|
||||
RS --> ES[entity_status]
|
||||
IM --> MS[(metric_samples)]
|
||||
MS --> R1H[metric_rollups_1h continuous aggregate]
|
||||
MS --> R1D[metric_rollups_1d continuous aggregate]
|
||||
```
|
||||
|
||||
## Prerequisites for SSH Checks
|
||||
|
||||
1. **Key**: SSH private key mounted at `/etc/oikos/ssh_key` in the scheduler container
|
||||
2. **User**: `OIKOS_SSH_USER=root` (or set `"user"` in check config)
|
||||
3. **Scripts**: Deployed on target host at `/opt/oikos/checks/`
|
||||
4. **Network**: Scheduler container must reach target host (bridge → LAN works)
|
||||
5. **Container**: Scheduler needs `openssh-client` (alpine base) + `CAP_NET_RAW` (for ping)
|
||||
613
docs/adr/0014-entity-model.md
Normal file
613
docs/adr/0014-entity-model.md
Normal file
@@ -0,0 +1,613 @@
|
||||
# Oikos Entity Model — Types, Relationships & Interactions
|
||||
|
||||
**Status:** Adopted
|
||||
**Date:** 2026-07-08
|
||||
**Scope:** Full inventory of every entity type, relationship, state machine, and
|
||||
cognition pipeline — with clear markers for what is **code-real** vs **schema-only**.
|
||||
|
||||
---
|
||||
|
||||
## 1. Entity Type Hierarchy (56 types)
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph meta["layer: meta"]
|
||||
entity["★ entity"]
|
||||
end
|
||||
|
||||
subgraph infrastructure["layer: infrastructure"]
|
||||
subgraph physical["domain: physical"]
|
||||
site
|
||||
ups
|
||||
sensor
|
||||
peripheral
|
||||
end
|
||||
|
||||
subgraph compute["domain: compute"]
|
||||
ce["★ compute-entity"]
|
||||
machine["★ machine"]
|
||||
proxmox-host
|
||||
standalone-server
|
||||
workstation
|
||||
appliance
|
||||
vm
|
||||
container["★ container"]
|
||||
lxc
|
||||
docker-container
|
||||
hypervisor
|
||||
end
|
||||
|
||||
subgraph network["domain: network"]
|
||||
net["★ network"]
|
||||
lan
|
||||
mesh
|
||||
vlan
|
||||
network-interface
|
||||
dns-zone
|
||||
dns-record
|
||||
ingress-route
|
||||
certificate
|
||||
firewall-rule
|
||||
end
|
||||
|
||||
subgraph storage["domain: storage"]
|
||||
storage-pool
|
||||
volume
|
||||
backup-target
|
||||
dataset
|
||||
end
|
||||
|
||||
subgraph software["domain: software"]
|
||||
service
|
||||
application
|
||||
config-repo
|
||||
deploy-pipeline
|
||||
package-set
|
||||
cluster
|
||||
compose-stack
|
||||
end
|
||||
|
||||
subgraph external["domain: external"]
|
||||
domain-registration
|
||||
cloud-service
|
||||
isp-link
|
||||
vendor-dependency
|
||||
end
|
||||
end
|
||||
|
||||
subgraph governance["layer: governance"]
|
||||
subgraph identity["domain: identity"]
|
||||
person
|
||||
agent
|
||||
identity-provider
|
||||
account
|
||||
secret
|
||||
key
|
||||
access-grant
|
||||
end
|
||||
end
|
||||
|
||||
subgraph cognition["layer: cognition"]
|
||||
check
|
||||
signal
|
||||
classification
|
||||
execution
|
||||
feedback
|
||||
pattern
|
||||
skill
|
||||
approval
|
||||
document
|
||||
runbook
|
||||
investigation
|
||||
end
|
||||
|
||||
entity --> ce
|
||||
entity --> machine
|
||||
entity --> net
|
||||
entity --> container
|
||||
entity --> site
|
||||
entity --> ups
|
||||
entity --> sensor
|
||||
entity --> peripheral
|
||||
entity --> vm
|
||||
entity --> hypervisor
|
||||
entity --> network-interface
|
||||
entity --> dns-zone
|
||||
entity --> dns-record
|
||||
entity --> ingress-route
|
||||
entity --> certificate
|
||||
entity --> firewall-rule
|
||||
entity --> storage-pool
|
||||
entity --> volume
|
||||
entity --> backup-target
|
||||
entity --> dataset
|
||||
entity --> service
|
||||
entity --> application
|
||||
entity --> config-repo
|
||||
entity --> deploy-pipeline
|
||||
entity --> package-set
|
||||
entity --> cluster
|
||||
entity --> compose-stack
|
||||
entity --> domain-registration
|
||||
entity --> cloud-service
|
||||
entity --> isp-link
|
||||
entity --> vendor-dependency
|
||||
entity --> person
|
||||
entity --> agent
|
||||
entity --> identity-provider
|
||||
entity --> account
|
||||
entity --> secret
|
||||
entity --> key
|
||||
entity --> access-grant
|
||||
entity --> check
|
||||
entity --> signal
|
||||
entity --> classification
|
||||
entity --> execution
|
||||
entity --> feedback
|
||||
entity --> pattern
|
||||
entity --> skill
|
||||
entity --> approval
|
||||
entity --> document
|
||||
entity --> runbook
|
||||
entity --> investigation
|
||||
|
||||
ce --> machine
|
||||
ce --> container
|
||||
machine --> proxmox-host
|
||||
machine --> standalone-server
|
||||
machine --> workstation
|
||||
machine --> appliance
|
||||
container --> lxc
|
||||
container --> docker-container
|
||||
net --> lan
|
||||
net --> mesh
|
||||
net --> vlan
|
||||
```
|
||||
|
||||
★ = abstract (cannot be instantiated; polymorphic target for relationships)
|
||||
|
||||
### Concrete instances (88 active entities)
|
||||
|
||||
| Type | Count | Examples |
|
||||
|------|-------|---------|
|
||||
| `lxc` | 19 | jellyfin, caddy, dns, gitea, nextcloud, matrix, arriman… |
|
||||
| `service` | 25 | caddy, authentik, dns, jellyfin, paperless, matrix… |
|
||||
| `ingress-route` | 21 | *.hubris.network |
|
||||
| `config-repo` | 6 | caddy-conf, gitea-customizations, mule-image… |
|
||||
| `proxmox-host` | 2 | hubris, strong |
|
||||
| `workstation` | 2 | mac-mini, republic-laptop |
|
||||
| `standalone-server` | 1 | netbird-vps |
|
||||
| `vm` | 2 | zimaos, haos |
|
||||
| `storage-pool` | 3 | local-lvm-hubris, library-hubris, ludo-lvm |
|
||||
| `volume` | 2 | library, media-local |
|
||||
| + sites, networks, agents, documents, destroyed… | | |
|
||||
|
||||
---
|
||||
|
||||
## 2. Core Sequence: Machine Onboarding
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant O as Operator
|
||||
participant A as Oikos API
|
||||
participant D as DB
|
||||
participant S as Scheduler
|
||||
participant T as Target Machine
|
||||
|
||||
O->>A: POST /api/v1/entities {type:proxmox-host, slug:host:new, …}
|
||||
A->>D: INSERT INTO entities
|
||||
A->>A: ensureDefaultChecks().resolveHost() → lan_ip
|
||||
A->>D: INSERT check_defs × 6 (target_id set)
|
||||
A-->>O: 201 Created
|
||||
|
||||
Note over S,T: 30s scheduler tick
|
||||
S->>D: ListEnabledCheckDefs
|
||||
S->>T: SSH exec /opt/oikos/checks/cpu_check.sh
|
||||
T-->>S: {"health":"ok","metrics":{"cpu_pct":2.5,"cpu_temp":48}}
|
||||
S->>D: INSERT metric_samples
|
||||
S->>D: UPSERT entity_status (health)
|
||||
```
|
||||
|
||||
**What's code-real here:**
|
||||
- `CreateEntity()` at `internal/httpapi/impl.go:811` — handles POST, validates type, calls `ensureDefaultChecks()`
|
||||
- `ensureDefaultChecks()` → `internal/checkdefaults/defaults.go:144` — resolves host IP, SSH user, creates 6 check_defs rows with target_id
|
||||
- Scheduler at `internal/scheduler/scheduler.go:26` — loads `ListEnabledCheckDefs`, dispatches by kind, writes metrics + signals
|
||||
|
||||
---
|
||||
|
||||
## 3. Core Sequence: The OODA Loop (observe → orient → decide → act → learn)
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph OBSERVE["🔍 OBSERVE (Scheduler every 30s)"]
|
||||
direction TB
|
||||
S1["ListEnabledCheckDefs"]
|
||||
S2["ping → exec.Command(ping, host)"]
|
||||
S3["http → http.Get(url)"]
|
||||
S4["tcp → net.DialTimeout(tcp, addr)"]
|
||||
S5["disk → unix.Statfs(path)"]
|
||||
S6["cert-expiry → tls.Dial + cert.NotAfter"]
|
||||
S7["ssh-script → exec.Command(ssh, host, script)"]
|
||||
S8["checkResult{health, signalKind, evidence, metrics}"]
|
||||
S1 --> S2 & S3 & S4 & S5 & S6 & S7
|
||||
S2 & S3 & S4 & S5 & S6 & S7 --> S8
|
||||
S8 -->|INSERT| MS[("metric_samples")]
|
||||
S8 -->|UPSERT| SG["signals (dedup by target+kind)"]
|
||||
S8 -->|UPSERT| ES["entity_status (health)"]
|
||||
end
|
||||
|
||||
subgraph ORIENT["🧭 ORIENT (Classification)"]
|
||||
direction TB
|
||||
C1["policy.ClassifySignal(signal, entity, blast_radius)"]
|
||||
C2["read_only → route: auto_act"]
|
||||
C3["reversible_low → route: auto_act"]
|
||||
C4["config_mutation → route: escalate"]
|
||||
C5["destructive → route: hold"]
|
||||
C1 --> C2 & C3 & C4 & C5
|
||||
end
|
||||
|
||||
subgraph DECIDE["⚖️ DECIDE (Approval Gate)"]
|
||||
direction TB
|
||||
D1["auto_act → execute immediately"]
|
||||
D2["escalate → INSERT approval (pending)"]
|
||||
D3["notifier → Matrix alert + HMAC token"]
|
||||
D4["operator replies ✅ or ❌"]
|
||||
D5["hold → queued, never auto-executed"]
|
||||
D2 --> D3 --> D4
|
||||
end
|
||||
|
||||
subgraph ACT["⚡ ACT (Execution)"]
|
||||
direction TB
|
||||
A1["Nomos → MCP request_execution"]
|
||||
A2["actuator.Execute() → SSH exec"]
|
||||
A3["systemctl restart / apt upgrade / pct exec"]
|
||||
A1 --> A2 --> A3
|
||||
end
|
||||
|
||||
subgraph LEARN["🧠 LEARN (Patterns + Skills)"]
|
||||
direction TB
|
||||
L1["execution → produces → feedback"]
|
||||
L2["feedback → contributes-to → pattern"]
|
||||
L3["pattern → informs → skill"]
|
||||
L1 --> L2 --> L3
|
||||
end
|
||||
|
||||
SG --> ORIENT
|
||||
ORIENT --> DECIDE
|
||||
DECIDE --> ACT
|
||||
ACT --> LEARN
|
||||
|
||||
style OBSERVE fill:#e3f2fd
|
||||
style ORIENT fill:#fff3e0
|
||||
style DECIDE fill:#fce4ec
|
||||
style ACT fill:#e8f5e9
|
||||
style LEARN fill:#f3e5f5
|
||||
```
|
||||
|
||||
### What's code-real in the OODA loop
|
||||
|
||||
| Phase | Table | Code | Status |
|
||||
|-------|-------|------|--------|
|
||||
| Observe | `check_defs`, `metric_samples` | `scheduler.go:26-215` | ✅ fully wired, 6 probe kinds |
|
||||
| Observe → Orient | `signals` | `scheduler.go:130-141` (UpsertSignal) | ✅ dedup, severity, events |
|
||||
| Orient | `classifications` | `policy/classify.go` (function exists) | ⚠ function defined but scheduler never calls it |
|
||||
| Decide | `approvals` | `server.go:311-367` (request_execution) | ✅ escalation gate works |
|
||||
| Act | `executions` | `actuator/exec.go` (SSH exec) | ✅ systemctl, apt, pct |
|
||||
| Learn | `feedback`, `patterns`, `skills` | tables + list endpoints only | ⚠ schema only, no write path |
|
||||
|
||||
---
|
||||
|
||||
## 4. Relationship Types — The Edge Catalog (34 edges)
|
||||
|
||||
### Infrastructure Topology
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
HH["host:hubris"] -->|hosts| LX1["lxc:jellyfin"]
|
||||
HH -->|hosts| LX2["lxc:caddy"]
|
||||
HH -->|hosts| LX3["lxc:dns"]
|
||||
HH -->|hosts| LX4["lxc:gitea"]
|
||||
HH -->|hosts| LX5["lxc:…"]
|
||||
HS["host:strong"] -->|hosts| LX6["lxc:jellyfin"]
|
||||
HS -->|hosts| LX7["lxc:arriman"]
|
||||
HH -->|member-of| CL["cluster:homelab"]
|
||||
HS -->|member-of| CL
|
||||
LX2 -->|provides| SV1["service:caddy"]
|
||||
LX4 -->|provides| SV2["service:gitea"]
|
||||
LX3 -->|provides| SV3["service:dns"]
|
||||
HH -->|mounts| VL["volume:library"]
|
||||
HH -->|stores-on| PL["pool:library-hubris"]
|
||||
```
|
||||
|
||||
### Network
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
IG["ingress:paperless.hubris.network"] -->|routes-to| SP["service:paperless"]
|
||||
IG -->|secured-by| IDP["idp:authentik"]
|
||||
IG -->|uses-certificate| CRT["cert:*.hubris.network"]
|
||||
SJ["service:jellyfin"] -->|authenticates-via| IDP
|
||||
DNS["dns:paperless"] -->|in-zone| ZN["zone:hubris.network"]
|
||||
DNS -->|resolves-to| LX["lxc:caddy"]
|
||||
HH["host:hubris"] -->|connects-via| LL["lan:lab"]
|
||||
HS["host:strong"] -->|connects-via| LH["lan:household"]
|
||||
```
|
||||
|
||||
### Service Dependencies (feeds blast_radius CTE)
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
JF["service:jellyfin"] -->|depends-on| AK["service:authentik"]
|
||||
PP["service:paperless"] -->|depends-on| AK
|
||||
AR["service:arr-stack"] -->|depends-on| JF
|
||||
AK -->|depends-on| CD["service:caddy"]
|
||||
AK -->|depends-on| DNS["service:dns"]
|
||||
```
|
||||
|
||||
### Cognition (OODA edges)
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
CK["check:ssh-script:d419257d"] -->|checks| HH["host:hubris"]
|
||||
CK -->|raises| SG["signal:cpu-pressure"]
|
||||
SG -->|about| HH
|
||||
CL["classification:xyz"] -->|classifies| SG
|
||||
CL -->|precedes| EX["execution:restart-xyz"]
|
||||
EX -->|targets| HH
|
||||
EX -->|performs| AG["agent:nomos"]
|
||||
```
|
||||
|
||||
### Governance
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
DT["person:dtoro"] -->|owns| NO["agent:nomos"]
|
||||
DT -->|decides| AP["approval:xyz"]
|
||||
AK["idp:authentik"] -->|authenticates| DT
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Lifecycle State Machines
|
||||
|
||||
### Infrastructure (15 concrete types use this)
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> planned
|
||||
planned --> provisioning
|
||||
planned --> destroyed: cancelled
|
||||
|
||||
provisioning --> active
|
||||
provisioning --> failed
|
||||
|
||||
active --> migrating
|
||||
active --> failed
|
||||
active --> deprecated
|
||||
|
||||
migrating --> active: post-verify
|
||||
migrating --> failed
|
||||
|
||||
failed --> active: recovery verified
|
||||
failed --> deprecated: write-off
|
||||
|
||||
deprecated --> active: un-deprecate
|
||||
deprecated --> destroyed
|
||||
|
||||
destroyed --> [*]
|
||||
```
|
||||
|
||||
**Real precondition checks** (code in `impl.go:1494-1579`):
|
||||
| Transition | Precondition | How it's checked |
|
||||
|------------|-------------|-----------------|
|
||||
| provisioning→active | `health-check-answering` | `SELECT health FROM entity_status WHERE entity_id=$1` — must be healthy |
|
||||
| provisioning→active | `age-key-enrolled-if-needed` | Checks `attributes->>'age_pubkey'` (workstation only) |
|
||||
| provisioning→active | `mesh-joined-if-needed` | Checks `attributes->>'mesh_ip'` (workstation only) |
|
||||
| provisioning→active | `doc-page-complete` | `SELECT count(*) FROM relationships WHERE target_id=$1 AND type='documents'` |
|
||||
| deprecated→destroyed | `no-inbound-edges` | `SELECT count(*) FROM relationships WHERE target_id=$1 AND valid_to IS NULL` |
|
||||
| any → terminated | `backups-verified` | Checks flag in entity attributes |
|
||||
| any → terminated | `secrets-revoked` | Checks flag in entity attributes |
|
||||
|
||||
**Soft preconditions** (always pass — operator-confirmed): `inventory-entry`, `ip-reserved`, `preflight-passed`, `backup-verified`, `replacement-live`, `caddy-backends-checked`, `un-deprecate-note`, etc.
|
||||
|
||||
### Signal
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> raised
|
||||
raised --> acknowledged
|
||||
raised --> muted: mute_until set
|
||||
raised --> resolved: condition cleared
|
||||
|
||||
acknowledged --> acting: classification exists
|
||||
acknowledged --> muted
|
||||
acknowledged --> resolved
|
||||
|
||||
acting --> resolved: verification passed
|
||||
acting --> raised: retry budget
|
||||
acting --> failed
|
||||
|
||||
failed --> acknowledged: operator retry
|
||||
|
||||
muted --> raised: mute_until expired
|
||||
|
||||
resolved --> [*]
|
||||
```
|
||||
|
||||
**Implemented preconditions:**
|
||||
- `raised → muted`: requires `mute_until` set (MuteSignal handler, `impl.go:615-689`)
|
||||
- `acting → resolved`: requires `verification-passed` (soft — operator confirms)
|
||||
|
||||
**Dedup mechanism:** `UNIQUE INDEX uq_signals_open ON signals(target_entity_id, kind) WHERE state NOT IN ('resolved','failed')` — at most one open signal per (entity, kind). Repeated failures call `UpsertSignal` which increments `occurrence_count` on the existing row.
|
||||
|
||||
### Execution
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> proposed
|
||||
proposed --> approved
|
||||
proposed --> auto_approved
|
||||
proposed --> denied
|
||||
proposed --> expired
|
||||
|
||||
approved --> executing
|
||||
auto_approved --> executing
|
||||
expired --> [*]
|
||||
denied --> [*]
|
||||
|
||||
executing --> verified
|
||||
executing --> failed
|
||||
executing --> timed_out
|
||||
|
||||
failed --> rolled_back
|
||||
rolled_back --> verified
|
||||
rolled_back --> rollback_failed
|
||||
|
||||
verified --> [*]
|
||||
rollback_failed --> [*]
|
||||
timed_out --> [*]
|
||||
```
|
||||
|
||||
### Approval
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> pending
|
||||
pending --> approved
|
||||
pending --> denied
|
||||
approved --> revoked
|
||||
approved --> expired
|
||||
|
||||
denied --> [*]
|
||||
revoked --> [*]
|
||||
expired --> [*]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. What's Code-Real vs Schema-Only
|
||||
|
||||
### ✅ Fully Implemented (code exists, running in production)
|
||||
|
||||
| Component | File(s) | What it does |
|
||||
|-----------|---------|-------------|
|
||||
| Entity CRUD | `impl.go:811-966` | Create, read, patch, list entities |
|
||||
| Lifecycle transitions | `impl.go:1494-1579` | Precondition checks + state transitions |
|
||||
| Relationship management | `seed.go` (ingest) | Create edges with `valid_from/valid_to` |
|
||||
| Client enrollment | `impl.go:1134-1236` | `POST /clients/enroll` — age keypair, Infisical, state: provisioning |
|
||||
| Check definitions | `phase3.go:265-376` | CreateCheck, ListChecks, PatchCheck |
|
||||
| Scheduler observe | `scheduler.go:26-215` | 6 probe kinds, metric_samples, signals, entity_status |
|
||||
| Signals | `scheduler.go:81-161` | UpsertSignal (dedup), ResolveSignal, severity evaluation |
|
||||
| Executions | `server.go:286-411` | request_execution MCP tool — reversible_low/config_mutation/destructive |
|
||||
| Approvals | `server.go:970-1052` | createApproval, DecideApproval → executeApprovedAction |
|
||||
| Notifier | `notifier/notifier.go` | Matrix alerts for pending approvals |
|
||||
| Patterns | `phase3.go:1100+` | ListPatterns, PatchPattern (status/quarantine) |
|
||||
| Skills | `phase3.go:1300+` | ListSkills, PatchSkill, ListSkillVersions |
|
||||
| Default checks | `checkdefaults/defaults.go` | Auto-create checks on entity creation/enrollment/seed |
|
||||
| TimescaleDB metrics | `metric_samples` table | Hypertable with 1h/1d continuous aggregates, 90-day retention |
|
||||
| Events + SSE | `events` table + pg_notify | Real-time UI updates via SSE endpoint |
|
||||
| Audit log | `audit_log` hypertable | Every mutation with actor + action |
|
||||
| Knowledge entities | `knowledge_entities` | Documents, runbooks, investigations with FTS |
|
||||
| MCP tools | `server.go` | 24 tools for observe/orient/decide/act |
|
||||
| Blast radius | `blast_radius()` fn | Recursive CTE — depends-on + hosts + routes-to edges |
|
||||
|
||||
### ⚠ Schema Defined, Not Yet Wired (table exists, no active code path creates rows)
|
||||
|
||||
| Component | What's Missing |
|
||||
|-----------|---------------|
|
||||
| `classifications` auto-creation | `policy.ClassifySignal()` exists but scheduler never calls it. Signals are raised but never automatically classified. |
|
||||
| `feedback` records | No code writes to the `feedback` table. Execution results are not analyzed for patterns. |
|
||||
| Pattern auto-learning | No code transitions patterns from `hypothesized → validated`. Requires `evidence-count≥5 + confidence≥0.7` but no aggregation runs. |
|
||||
| Skill execution | Skill entities carry a JSON `procedure` field but no execution engine reads or runs it. |
|
||||
| `drift` check kind | Defined in OpenAPI and `check_defs.kind` enum, but no scheduler implementation exists. |
|
||||
|
||||
### 📋 Defined in Seeds Only (ontology.yaml references, no DB schema)
|
||||
|
||||
| Item | Notes |
|
||||
|------|-------|
|
||||
| Relationship type `cluster` | Mentioned in inventory but not in ontology relationship_types |
|
||||
| `certificate` entity type | Referenced in `uses-certificate` edges but no concrete certificates in inventory |
|
||||
| Relationship type `powers` / `monitors` | Not defined in relationship_types |
|
||||
|
||||
---
|
||||
|
||||
## 7. Database Physical Schema
|
||||
|
||||
```mermaid
|
||||
erDiagram
|
||||
lifecycle_defs ||--o{ entity_types : "lifecycle_id FK"
|
||||
entity_types ||--o{ entities : "type FK"
|
||||
entity_types ||--o| entity_types : "parent_type FK (self-ref)"
|
||||
|
||||
entities ||--o| entity_status : "dual entity (shared PK)"
|
||||
entities ||--o| check_defs : "dual entity (shared PK)"
|
||||
entities ||--o| signals : "dual entity (shared PK)"
|
||||
entities ||--o| classifications : "dual entity (shared PK)"
|
||||
entities ||--o| executions : "dual entity (shared PK)"
|
||||
entities ||--o| feedback : "dual entity (shared PK)"
|
||||
entities ||--o| patterns : "dual entity (shared PK)"
|
||||
entities ||--o| skills : "dual entity (shared PK)"
|
||||
entities ||--o| approvals : "dual entity (shared PK)"
|
||||
entities ||--o| knowledge_entities : "dual entity (shared PK)"
|
||||
|
||||
entities ||--o{ relationships : "source_id FK"
|
||||
entities ||--o{ relationships : "target_id FK"
|
||||
|
||||
check_defs }o--|| entities : "target_id FK"
|
||||
signals }o--|| entities : "target_entity_id FK"
|
||||
executions }o--|| entities : "target_entity_id FK"
|
||||
approvals }o--|| entities : "subject_entity_id FK"
|
||||
|
||||
entity_types ||--o{ relationship_types : "source_type FK"
|
||||
entity_types ||--o{ relationship_types : "target_type FK"
|
||||
relationship_types ||--o{ relationships : "type FK"
|
||||
|
||||
entity_types ||--o{ approval_rules : "entity_type FK"
|
||||
|
||||
signals ||--o| check_defs : "check_id FK"
|
||||
```
|
||||
|
||||
**Key architectural patterns:**
|
||||
- **Dual entities:** `check_defs`, `signals`, `classifications`, `executions`, `feedback`, `patterns`, `skills`, `approvals`, `knowledge_entities` — all have `entity_id UUID PK REFERENCES entities(id)`. Every row is also an entity.
|
||||
- **Partial unique indexes:** `relationships` (current edges), `signals` (open signals), `patterns` (per-type action) — all use `WHERE` clauses for snapshot semantics.
|
||||
- **TimescaleDB hypertables:** `metric_samples`, `events`, `audit_log`, `agent_activity` — with continuous aggregates and retention policies.
|
||||
- **SSE fan-out:** `pg_notify('oikos_events', ...)` trigger on `events` INSERT → Go listener fan-out → SSE connections.
|
||||
|
||||
---
|
||||
|
||||
## 8. How Nomos Queries Thermals — End-to-End Trace
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant U as User
|
||||
participant N as Nomos (Agent)
|
||||
participant A as Oikos API
|
||||
participant T as TimescaleDB
|
||||
participant S as Scheduler
|
||||
participant H as Hubris
|
||||
|
||||
Note over S,H: Autonomous collection (every 60s)
|
||||
S->>H: SSH exec cpu_check.sh
|
||||
H-->>S: {"metrics":{"cpu_pct":15,"cpu_temp":48}}
|
||||
S->>T: INSERT metric_samples (cpu_pct, cpu_temp)
|
||||
|
||||
Note over U,N: User asks question
|
||||
U->>N: "what are the thermals of hubris?"
|
||||
N->>A: MCP query_metrics(metric=["cpu_pct","cpu_temp"])
|
||||
A->>T: SELECT time_bucket('1h', ts) … FROM metric_samples
|
||||
T-->>A: {avg:15, min:2, max:40} (cpu_pct)
|
||||
T-->>A: {avg:48, min:42, max:85} (cpu_temp)
|
||||
A-->>N: time-bucketed metrics
|
||||
N-->>U: "CPU at 15%, temp 48°C — normal range"
|
||||
```
|
||||
|
||||
**What made this possible (chronologically):**
|
||||
1. `scheduler.go` refactored to return metrics map → `checkResult{metrics}`
|
||||
2. `ssh-script` check kind implemented → SSH exec to remote host
|
||||
3. `cpu_check.sh` deployed to hubris → returns `{"metrics":{"cpu_pct":2.5,"cpu_temp":48}}`
|
||||
4. Check created: `POST /checks {"kind":"ssh-script","target":"host:hubris","config":{"host":"192.168.8.77","script":"cpu_check.sh"}}`
|
||||
5. Fixed: `InsertMetricSample` missing `ts` column → `now()` literal
|
||||
6. Fixed: SSH port/user parsing bugs
|
||||
7. Fixed: SSH warnings polluting JSON output
|
||||
8. Scheduler loop → metrics written to TimescaleDB every 60s
|
||||
9. MCP `query_metrics` reads from TimescaleDB → Nomos gets live data
|
||||
@@ -5,7 +5,7 @@ after acceptance — superseding decisions get a new ADR that links back.
|
||||
Statuses: proposed | accepted | superseded-by-NNNN.
|
||||
|
||||
| ADR | Title |
|
||||
|---|---|
|
||||
|---|---|---|
|
||||
| [0001](0001-go-single-binary.md) | Go with single-binary role packaging |
|
||||
| [0002](0002-postgres-timescale-only-datastore.md) | PostgreSQL + TimescaleDB as the only datastore |
|
||||
| [0003](0003-db-native-ontology-yaml-seeds.md) | DB-native ontology with YAML seed manifests |
|
||||
@@ -16,3 +16,7 @@ Statuses: proposed | accepted | superseded-by-NNNN.
|
||||
| [0008](0008-forward-only-migrations.md) | Forward-only migrations |
|
||||
| [0009](0009-sse-over-websocket.md) | SSE over WebSocket for the event stream |
|
||||
| [0010](0010-infisical-with-sops-fallback.md) | Infisical secrets with SOPS DR fallback |
|
||||
| [0011](0011-client-lifecycle-flows.md) | Client lifecycle flows — enrollment, bootstrap, sync |
|
||||
| [0012](0012-hermes-oikos-interactions.md) | Hermes–Oikos interactions — agent/OS contract |
|
||||
| [0013](0013-signal-triggers.md) | Signal triggers — host health checks via scheduler |
|
||||
| [0014](0014-entity-model.md) | Entity model — types, relationships, state machines, OODA loop |
|
||||
|
||||
@@ -70,7 +70,7 @@ curl http://localhost:8090/healthz
|
||||
# Entity count matches
|
||||
curl -s http://localhost:8090/api/v1/entities?limit=1 | jq '.items | length'
|
||||
|
||||
# MCP tools working (via Hermes)
|
||||
# MCP tools working (via Nomos)
|
||||
curl -s http://localhost:8092/query -d '{"tool":"get_health_summary"}'
|
||||
```
|
||||
|
||||
|
||||
9
go.mod
9
go.mod
@@ -8,11 +8,14 @@ require (
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1
|
||||
github.com/google/jsonschema-go v0.4.3
|
||||
github.com/google/uuid v1.6.0
|
||||
github.com/infisical/go-sdk v0.8.0
|
||||
github.com/jackc/pgx/v5 v5.10.0
|
||||
github.com/modelcontextprotocol/go-sdk v1.6.1
|
||||
github.com/oapi-codegen/runtime v1.4.2
|
||||
github.com/openai/openai-go v1.12.0
|
||||
golang.org/x/crypto v0.53.0
|
||||
golang.org/x/sync v0.21.0
|
||||
golang.org/x/sys v0.46.0
|
||||
gopkg.in/yaml.v3 v3.0.1
|
||||
)
|
||||
|
||||
@@ -47,7 +50,6 @@ require (
|
||||
github.com/googleapis/enterprise-certificate-proxy v0.3.11 // indirect
|
||||
github.com/googleapis/gax-go/v2 v2.17.0 // indirect
|
||||
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
|
||||
github.com/infisical/go-sdk v0.8.0 // indirect
|
||||
github.com/jackc/pgpassfile v1.0.0 // indirect
|
||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
|
||||
github.com/jackc/puddle/v2 v2.2.2 // indirect
|
||||
@@ -60,6 +62,10 @@ require (
|
||||
github.com/segmentio/asm v1.1.3 // indirect
|
||||
github.com/segmentio/encoding v0.5.4 // indirect
|
||||
github.com/sony/gobreaker v0.5.0 // indirect
|
||||
github.com/tidwall/gjson v1.14.4 // indirect
|
||||
github.com/tidwall/match v1.1.1 // indirect
|
||||
github.com/tidwall/pretty v1.2.1 // indirect
|
||||
github.com/tidwall/sjson v1.2.5 // indirect
|
||||
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
|
||||
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect
|
||||
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
|
||||
@@ -70,7 +76,6 @@ require (
|
||||
go.opentelemetry.io/otel/trace v1.39.0 // indirect
|
||||
golang.org/x/net v0.55.0 // indirect
|
||||
golang.org/x/oauth2 v0.35.0 // indirect
|
||||
golang.org/x/sys v0.46.0 // indirect
|
||||
golang.org/x/text v0.38.0 // indirect
|
||||
golang.org/x/time v0.14.0 // indirect
|
||||
google.golang.org/api v0.267.0 // indirect
|
||||
|
||||
35
go.sum
35
go.sum
@@ -38,12 +38,19 @@ github.com/aws/smithy-go v1.20.2/go.mod h1:krry+ya/rV9RDcV/Q16kpu6ypI4K2czasz0NC
|
||||
github.com/bmatcuk/doublestar v1.1.1/go.mod h1:UD6OnuiIn0yFxxA2le/rnRU1G4RaI4UvFv1sNto9p6w=
|
||||
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
||||
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
||||
github.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5 h1:6xNmx7iTtyBRev0+D/Tv1FZd4SCg8axKApyNyRsAt/w=
|
||||
github.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5/go.mod h1:KdCmV+x/BuvyMxRnYBlmVaq4OLiKW6iRQfvC62cvdkI=
|
||||
github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=
|
||||
github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
|
||||
github.com/envoyproxy/go-control-plane v0.14.0 h1:hbG2kr4RuFj222B6+7T83thSPqLjwBIfQawTkC++2HA=
|
||||
github.com/envoyproxy/go-control-plane/envoy v1.36.0 h1:yg/JjO5E7ubRyKX3m07GF3reDNEnfOboJ0QySbH736g=
|
||||
github.com/envoyproxy/go-control-plane/envoy v1.36.0/go.mod h1:ty89S1YCCVruQAm9OtKeEkQLTb+Lkz0k8v9W0Oxsv98=
|
||||
github.com/envoyproxy/protoc-gen-validate v1.3.0 h1:TvGH1wof4H33rezVKWSpqKz5NXWg5VPuZ0uONDT6eb4=
|
||||
github.com/envoyproxy/protoc-gen-validate v1.3.0/go.mod h1:HvYl7zwPa5mffgyeTUHA9zHIH36nmrm7oCbo4YKoSWA=
|
||||
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
|
||||
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
|
||||
github.com/getkin/kin-openapi v0.140.0 h1:JFn675aXRFjyiZKa/BFWploGldQlI0gobp4J5k0EZ2g=
|
||||
@@ -68,6 +75,8 @@ github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw=
|
||||
github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU=
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
|
||||
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
|
||||
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
|
||||
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
|
||||
github.com/google/jsonschema-go v0.4.3 h1:/DBOLZTfDow7pe2GmaJNhltueGTtDKICi8V8p+DQPd0=
|
||||
@@ -93,8 +102,8 @@ github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QII
|
||||
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
|
||||
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
|
||||
github.com/juju/gnuflag v0.0.0-20171113085948-2ce1bb71843d/go.mod h1:2PavIy+JPciBPrBUjwbNvtwB6RQlve+hkpll6QSNmOE=
|
||||
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
|
||||
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
|
||||
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
|
||||
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
|
||||
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
|
||||
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
|
||||
github.com/modelcontextprotocol/go-sdk v1.6.1 h1:0zOSupjKUxPKSocPT1Wtago+mUHU2/uZ4xSOY0FGReU=
|
||||
@@ -107,9 +116,13 @@ github.com/oasdiff/yaml v0.1.0 h1:0bqZjfKc/8S9urj4JuwepX41WX9EoA6ifhU3SV06cXg=
|
||||
github.com/oasdiff/yaml v0.1.0/go.mod h1:kOlRmMdL2X3vucLCEQO5u61SU22RysnfXvcttrZA1O0=
|
||||
github.com/oasdiff/yaml3 v0.0.13 h1:06svmvOHOVBqF81+sY2EUScvUI/iS/vl2VIeUUxZQwg=
|
||||
github.com/oasdiff/yaml3 v0.0.13/go.mod h1:y5+oSEHCPT/DGrS++Wc/479ERge0zTFxaF8PbGKcg2o=
|
||||
github.com/openai/openai-go v1.12.0 h1:NBQCnXzqOTv5wsgNC36PrFEiskGfO5wccfCWDo9S1U0=
|
||||
github.com/openai/openai-go v1.12.0/go.mod h1:g461MYGXEXBVdV5SaR/5tNzNbSfwTBBefwc+LlDCK0Y=
|
||||
github.com/oracle/oci-go-sdk/v65 v65.95.2 h1:0HJ0AgpLydp/DtvYrF2d4str2BjXOVAeNbuW7E07g94=
|
||||
github.com/oracle/oci-go-sdk/v65 v65.95.2/go.mod h1:u6XRPsw9tPziBh76K7GrrRXPa8P8W3BQeqJ6ZZt9VLA=
|
||||
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
|
||||
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 h1:GFCKgmp0tecUJ0sJuv4pzYCqS9+RGSn52M3FUwPs+uo=
|
||||
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10/go.mod h1:t/avpk3KcrXxUnYOhZhMXJlSEyie6gQbtLq5NM3loB8=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/rogpeppe/go-internal v1.15.0 h1:D0RCU5rMAp+SpgkiNdrjfJ+LX4J1M32V2NeCY7EJ6hc=
|
||||
@@ -136,6 +149,16 @@ github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO
|
||||
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
|
||||
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
||||
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
|
||||
github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
||||
github.com/tidwall/gjson v1.14.4 h1:uo0p8EbA09J7RQaflQ1aBRffTR7xedD2bcIVSYxLnkM=
|
||||
github.com/tidwall/gjson v1.14.4/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
||||
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
||||
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
|
||||
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
|
||||
github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
|
||||
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
|
||||
github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=
|
||||
github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=
|
||||
github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4=
|
||||
github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4=
|
||||
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 h1:ilQV1hzziu+LLM3zUTJ0trRztfwgjqKnBWNtSRkbmwM=
|
||||
@@ -152,6 +175,10 @@ go.opentelemetry.io/otel v1.39.0 h1:8yPrr/S0ND9QEfTfdP9V+SiwT4E0G7Y5MO7p85nis48=
|
||||
go.opentelemetry.io/otel v1.39.0/go.mod h1:kLlFTywNWrFyEdH0oj2xK0bFYZtHRYUdv1NklR/tgc8=
|
||||
go.opentelemetry.io/otel/metric v1.39.0 h1:d1UzonvEZriVfpNKEVmHXbdf909uGTOQjA0HF0Ls5Q0=
|
||||
go.opentelemetry.io/otel/metric v1.39.0/go.mod h1:jrZSWL33sD7bBxg1xjrqyDjnuzTUB0x1nBERXd7Ftcs=
|
||||
go.opentelemetry.io/otel/sdk v1.39.0 h1:nMLYcjVsvdui1B/4FRkwjzoRVsMK8uL/cj0OyhKzt18=
|
||||
go.opentelemetry.io/otel/sdk v1.39.0/go.mod h1:vDojkC4/jsTJsE+kh+LXYQlbL8CgrEcwmt1ENZszdJE=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.39.0 h1:cXMVVFVgsIf2YL6QkRF4Urbr/aMInf+2WKg+sEJTtB8=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.39.0/go.mod h1:xq9HEVH7qeX69/JnwEfp6fVq5wosJsY1mt4lLfYdVew=
|
||||
go.opentelemetry.io/otel/trace v1.39.0 h1:2d2vfpEDmCJ5zVYz7ijaJdOF59xLomrvj7bjt6/qCJI=
|
||||
go.opentelemetry.io/otel/trace v1.39.0/go.mod h1:88w4/PnZSazkGzz/w84VHpQafiU4EtqqlVdxWy+rNOA=
|
||||
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||
@@ -232,8 +259,12 @@ golang.org/x/tools v0.45.0/go.mod h1:LuUGqqaXcXMEFEruIVJVm5mgDD8vww/z/SR1gQ4uE/0
|
||||
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk=
|
||||
gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E=
|
||||
google.golang.org/api v0.267.0 h1:w+vfWPMPYeRs8qH1aYYsFX68jMls5acWl/jocfLomwE=
|
||||
google.golang.org/api v0.267.0/go.mod h1:Jzc0+ZfLnyvXma3UtaTl023TdhZu6OMBP9tJ+0EmFD0=
|
||||
google.golang.org/genproto v0.0.0-20260128011058-8636f8732409 h1:VQZ/yAbAtjkHgH80teYd2em3xtIkkHd7ZhqfH2N9CsM=
|
||||
google.golang.org/genproto v0.0.0-20260128011058-8636f8732409/go.mod h1:rxKD3IEILWEu3P44seeNOAwZN4SaoKaQ/2eTg4mM6EM=
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20260128011058-8636f8732409 h1:merA0rdPeUV3YIIfHHcH4qBkiQAc1nfCKSI7lB4cV2M=
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20260128011058-8636f8732409/go.mod h1:fl8J1IvUjCilwZzQowmw2b7HQB2eAuYBabMXzWurF+I=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260203192932-546029d2fa20 h1:Jr5R2J6F6qWyzINc+4AM8t5pfUz6beZpHp678GNrMbE=
|
||||
|
||||
@@ -1,33 +0,0 @@
|
||||
# Hermes agent config — standalone MCP client gateway (Phase 4)
|
||||
|
||||
mcp:
|
||||
endpoint: ${HERMES_MCP_URL}?session_id=${HERMES_SESSION_ID}
|
||||
transport: streamable_http
|
||||
|
||||
server:
|
||||
listen: ${HERMES_LISTEN}
|
||||
mesh_only: true
|
||||
|
||||
agent:
|
||||
name: hermes
|
||||
slug: ${HERMES_AGENT_SLUG}
|
||||
|
||||
query_routing:
|
||||
# Maps natural-language query patterns to MCP tools
|
||||
- pattern: "depends on"
|
||||
tool: get_blast_radius
|
||||
entity_param: entity_id
|
||||
- pattern: "restart"
|
||||
tool: request_execution
|
||||
action: restart
|
||||
- pattern: "health"
|
||||
tool: get_health_summary
|
||||
- pattern: "what is"
|
||||
tool: get_entity
|
||||
entity_param: slug_or_id
|
||||
- pattern: "recent events"
|
||||
tool: get_event_timeline
|
||||
- pattern: "signals"
|
||||
tool: get_signal_history
|
||||
- pattern: "patterns"
|
||||
tool: get_patterns
|
||||
204
internal/checkdefaults/defaults.go
Normal file
204
internal/checkdefaults/defaults.go
Normal file
@@ -0,0 +1,204 @@
|
||||
package checkdefaults
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
type CheckDef struct {
|
||||
Kind string
|
||||
Script string
|
||||
Host string
|
||||
User string
|
||||
Port int
|
||||
Thresholds map[string]any
|
||||
Extra map[string]any
|
||||
}
|
||||
|
||||
func ResolveHost(attrs map[string]any) string {
|
||||
if ip, ok := attrs["lan_ip"].(string); ok && ip != "" {
|
||||
return ip
|
||||
}
|
||||
if mesh, ok := attrs["mesh"].(map[string]any); ok {
|
||||
if nb, ok := mesh["netbird"].(map[string]any); ok {
|
||||
if ip, ok := nb["ip"].(string); ok && ip != "" {
|
||||
return ip
|
||||
}
|
||||
}
|
||||
}
|
||||
if ip, ok := attrs["mesh_ip"].(string); ok && ip != "" {
|
||||
return ip
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func resolveSSHUser(attrs map[string]any) string {
|
||||
if ssh, ok := attrs["ssh"].(map[string]any); ok {
|
||||
if u, ok := ssh["user"].(string); ok && u != "" {
|
||||
return u
|
||||
}
|
||||
}
|
||||
return "root"
|
||||
}
|
||||
|
||||
func resolveSSHPort(attrs map[string]any) int {
|
||||
if ssh, ok := attrs["ssh"].(map[string]any); ok {
|
||||
switch p := ssh["port"].(type) {
|
||||
case float64:
|
||||
return int(p)
|
||||
case int:
|
||||
return p
|
||||
}
|
||||
}
|
||||
return 22
|
||||
}
|
||||
|
||||
func ForEntityType(entityType string, attrs map[string]any) []CheckDef {
|
||||
host := ResolveHost(attrs)
|
||||
user := resolveSSHUser(attrs)
|
||||
port := resolveSSHPort(attrs)
|
||||
|
||||
ssh := func(script string) CheckDef {
|
||||
return CheckDef{Kind: "ssh-script", Script: script, Host: host, User: user, Port: port}
|
||||
}
|
||||
|
||||
switch entityType {
|
||||
case "proxmox-host", "standalone-server":
|
||||
if host == "" {
|
||||
return nil
|
||||
}
|
||||
return []CheckDef{
|
||||
{Kind: "ping", Host: host},
|
||||
ssh("cpu_check.sh"),
|
||||
ssh("memory_check.sh"),
|
||||
ssh("load_check.sh"),
|
||||
ssh("disk_usage_check.sh"),
|
||||
ssh("updates_check.sh"),
|
||||
}
|
||||
case "workstation":
|
||||
if host == "" {
|
||||
return nil
|
||||
}
|
||||
return []CheckDef{
|
||||
{Kind: "ping", Host: host},
|
||||
ssh("cpu_check.sh"),
|
||||
ssh("memory_check.sh"),
|
||||
ssh("load_check.sh"),
|
||||
}
|
||||
case "lxc":
|
||||
if host == "" {
|
||||
return nil
|
||||
}
|
||||
return []CheckDef{
|
||||
ssh("cpu_check.sh"),
|
||||
ssh("memory_check.sh"),
|
||||
ssh("load_check.sh"),
|
||||
ssh("disk_usage_check.sh"),
|
||||
}
|
||||
case "vm":
|
||||
if host == "" {
|
||||
return nil
|
||||
}
|
||||
return []CheckDef{
|
||||
{Kind: "ping", Host: host},
|
||||
}
|
||||
case "service":
|
||||
if host == "" {
|
||||
return nil
|
||||
}
|
||||
n, _ := attrs["name"].(string)
|
||||
if n == "" {
|
||||
return nil
|
||||
}
|
||||
return []CheckDef{
|
||||
{Kind: "ssh-script", Script: "process_check.sh", Host: host, User: user, Port: port,
|
||||
Extra: map[string]any{"args": n}},
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func ShortSlug(slug string) string {
|
||||
const n = 8
|
||||
if len(slug) > n {
|
||||
return slug[len(slug)-n:]
|
||||
}
|
||||
return slug
|
||||
}
|
||||
|
||||
func DefaultInterval(kind string) int32 {
|
||||
switch kind {
|
||||
case "ping":
|
||||
return 30
|
||||
case "ssh-script":
|
||||
return 60
|
||||
default:
|
||||
return 300
|
||||
}
|
||||
}
|
||||
|
||||
func Ensure(ctx context.Context, tx pgx.Tx, entityID uuid.UUID, slug, entityType string, attrsJSON []byte) {
|
||||
_, _ = tx.Exec(ctx,
|
||||
`INSERT INTO entity_status (entity_id, health, updated_at)
|
||||
VALUES ($1, 'unknown', now())
|
||||
ON CONFLICT (entity_id) DO NOTHING`,
|
||||
entityID)
|
||||
|
||||
var attrs map[string]any
|
||||
if len(attrsJSON) > 0 {
|
||||
json.Unmarshal(attrsJSON, &attrs)
|
||||
}
|
||||
if attrs == nil {
|
||||
attrs = map[string]any{}
|
||||
}
|
||||
|
||||
defs := ForEntityType(entityType, attrs)
|
||||
if len(defs) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
for i, def := range defs {
|
||||
checkID, err := uuid.NewV7()
|
||||
if err != nil {
|
||||
checkID = uuid.New()
|
||||
}
|
||||
checkSlug := fmt.Sprintf("check:%s:%s:%d", def.Kind, ShortSlug(slug), i)
|
||||
|
||||
_, _ = tx.Exec(ctx,
|
||||
`INSERT INTO entities (id, slug, type, name, state, attributes, version, created_at, updated_at)
|
||||
VALUES ($1, $2, 'check', $2, 'active', '{}', 1, now(), now())
|
||||
ON CONFLICT (slug) DO NOTHING`,
|
||||
checkID, checkSlug)
|
||||
|
||||
configMap := map[string]any{}
|
||||
if def.Script != "" {
|
||||
configMap["script"] = def.Script
|
||||
}
|
||||
if def.Host != "" {
|
||||
configMap["host"] = def.Host
|
||||
}
|
||||
if def.User != "" && def.User != "root" {
|
||||
configMap["user"] = def.User
|
||||
}
|
||||
if def.Port != 0 && def.Port != 22 {
|
||||
configMap["port"] = def.Port
|
||||
}
|
||||
if def.Thresholds != nil {
|
||||
configMap["thresholds"] = def.Thresholds
|
||||
}
|
||||
for k, v := range def.Extra {
|
||||
configMap[k] = v
|
||||
}
|
||||
configJSON, _ := json.Marshal(configMap)
|
||||
|
||||
_, _ = tx.Exec(ctx,
|
||||
`INSERT INTO check_defs (entity_id, target_id, kind, config, interval_s, timeout_s, enabled)
|
||||
VALUES ($1, $2, $3, $4, $5, 30, true)
|
||||
ON CONFLICT (entity_id) DO NOTHING`,
|
||||
checkID, entityID, def.Kind, configJSON, DefaultInterval(def.Kind))
|
||||
}
|
||||
}
|
||||
@@ -20,7 +20,7 @@ type Config struct {
|
||||
|
||||
// Auth (Phase 2: static bearer tokens + OIDC JWT)
|
||||
APIToken string // operator/CI bearer token for the REST API
|
||||
MCPBearerToken string // shared secret for Hermes→API MCP calls
|
||||
MCPBearerToken string // shared secret for Nomos→API MCP calls
|
||||
OIDCIssuer string // OIDC issuer URL for JWT validation (e.g. https://authentik.example.com/application/o/oikos/)
|
||||
OIDCClientID string // OIDC client ID (aud claim expected in JWT)
|
||||
|
||||
@@ -54,9 +54,9 @@ type Config struct {
|
||||
// Approval HMAC secret (Phase 3)
|
||||
ApprovalHMACSecret string
|
||||
|
||||
// Hermes agent entity ID (Phase 4)
|
||||
HermesAgentID string
|
||||
HermesAgentSlug string
|
||||
// Nomos agent entity ID (Phase 4)
|
||||
NomosAgentID string
|
||||
NomosAgentSlug string
|
||||
|
||||
// Infisical (Phase 5)
|
||||
InfisicalSiteURL string
|
||||
@@ -151,11 +151,11 @@ func FromEnv() Config {
|
||||
if v := os.Getenv("OIKOS_APPROVAL_HMAC_SECRET"); v != "" {
|
||||
c.ApprovalHMACSecret = v
|
||||
}
|
||||
if v := os.Getenv("OIKOS_HERMES_AGENT_ID"); v != "" {
|
||||
c.HermesAgentID = v
|
||||
if v := os.Getenv("OIKOS_NOMOS_AGENT_ID"); v != "" {
|
||||
c.NomosAgentID = v
|
||||
}
|
||||
if v := os.Getenv("OIKOS_HERMES_AGENT_SLUG"); v != "" {
|
||||
c.HermesAgentSlug = v
|
||||
if v := os.Getenv("OIKOS_NOMOS_AGENT_SLUG"); v != "" {
|
||||
c.NomosAgentSlug = v
|
||||
}
|
||||
|
||||
// Phase 5: Infisical secrets
|
||||
|
||||
@@ -240,8 +240,8 @@ SELECT * FROM risk_classes ORDER BY name;
|
||||
SELECT * FROM approval_rules ORDER BY entity_type, action;
|
||||
|
||||
-- name: InsertMetricSample :exec
|
||||
INSERT INTO metric_samples (entity_id, metric, value, tags)
|
||||
VALUES ($1, $2, $3, $4);
|
||||
INSERT INTO metric_samples (entity_id, metric, value, tags, ts)
|
||||
VALUES ($1, $2, $3, $4, now());
|
||||
|
||||
-- name: QueryMetrics :many
|
||||
SELECT time_bucket(sqlc.arg('bucket_interval')::interval, ts) AS bucket,
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
"github.com/dtoro/oikos/internal/checkdefaults"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
@@ -143,6 +144,8 @@ func IngestInventorySeed(ctx context.Context, tx pgx.Tx, data map[string]any) (*
|
||||
return nil, fmt.Errorf("entity_status %s: %w", slug, err)
|
||||
}
|
||||
|
||||
checkdefaults.Ensure(ctx, tx, entityID, slug, typeName, attrsBytes)
|
||||
|
||||
r.Entities++
|
||||
}
|
||||
|
||||
|
||||
@@ -26,6 +26,22 @@ type AgentActivity struct {
|
||||
CorrelationID *string
|
||||
}
|
||||
|
||||
type AgentMessage struct {
|
||||
ID uuid.UUID
|
||||
SessionID uuid.UUID
|
||||
Role string
|
||||
Content []byte
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
type AgentSession struct {
|
||||
ID uuid.UUID
|
||||
Title string
|
||||
Actor string
|
||||
CreatedAt time.Time
|
||||
LastActiveAt time.Time
|
||||
}
|
||||
|
||||
type Approval struct {
|
||||
EntityID uuid.UUID
|
||||
SubjectEntityID *uuid.UUID
|
||||
|
||||
@@ -560,8 +560,8 @@ func (q *Queries) InsertFeedback(ctx context.Context, arg InsertFeedbackParams)
|
||||
}
|
||||
|
||||
const insertMetricSample = `-- name: InsertMetricSample :exec
|
||||
INSERT INTO metric_samples (entity_id, metric, value, tags)
|
||||
VALUES ($1, $2, $3, $4)
|
||||
INSERT INTO metric_samples (entity_id, metric, value, tags, ts)
|
||||
VALUES ($1, $2, $3, $4, now())
|
||||
`
|
||||
|
||||
type InsertMetricSampleParams struct {
|
||||
|
||||
@@ -93,7 +93,7 @@ func newTestHandler(t *testing.T, cfg config.Config) http.Handler {
|
||||
}
|
||||
}
|
||||
|
||||
return NewHandler(handlerCtx, pool, cfg)
|
||||
return NewHandler(handlerCtx, pool, cfg, nil)
|
||||
}
|
||||
|
||||
func get(t *testing.T, h http.Handler, path string, headers map[string]string) (*httptest.ResponseRecorder, map[string]any) {
|
||||
|
||||
172
internal/httpapi/dashboard.go
Normal file
172
internal/httpapi/dashboard.go
Normal file
@@ -0,0 +1,172 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
)
|
||||
|
||||
// GetDashboardSummary returns one round-trip overview for the control room
|
||||
// home page: entity counts, health rollup, open signals, pending approvals,
|
||||
// executions in the last 24h, and an event-rate sparkline.
|
||||
func (s *Server) GetDashboardSummary(ctx context.Context, req gen.GetDashboardSummaryRequestObject) (gen.GetDashboardSummaryResponseObject, error) {
|
||||
resp := gen.DashboardSummary{
|
||||
EntitiesByType: map[string]int{},
|
||||
EntitiesByState: map[string]int{},
|
||||
SignalsBySeverity: map[string]int{},
|
||||
ExecutionsByState: map[string]int{},
|
||||
}
|
||||
|
||||
rows, err := s.pool.Query(ctx, `SELECT type, count(*) FROM entities GROUP BY type`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for rows.Next() {
|
||||
var typ string
|
||||
var n int
|
||||
if err := rows.Scan(&typ, &n); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
resp.EntitiesByType[typ] = n
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
rows, err = s.pool.Query(ctx, `SELECT coalesce(state, 'unknown'), count(*) FROM entities GROUP BY state`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for rows.Next() {
|
||||
var state string
|
||||
var n int
|
||||
if err := rows.Scan(&state, &n); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
resp.EntitiesByState[state] = n
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Exclude 'check' entities (internal probes) — only entities actually
|
||||
// being monitored should count toward the fleet health rollup.
|
||||
rows, err = s.pool.Query(ctx, `
|
||||
SELECT st.health, count(*)
|
||||
FROM entity_status st JOIN entities e ON e.id = st.entity_id
|
||||
WHERE e.type <> 'check'
|
||||
GROUP BY st.health`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
stale := 0
|
||||
for rows.Next() {
|
||||
var health string
|
||||
var n int
|
||||
if err := rows.Scan(&health, &n); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
switch health {
|
||||
case "healthy":
|
||||
resp.Health.Healthy = n
|
||||
case "degraded":
|
||||
resp.Health.Degraded = n
|
||||
case "down":
|
||||
resp.Health.Down = n
|
||||
case "stale":
|
||||
stale = n
|
||||
default:
|
||||
resp.Health.Unknown = n
|
||||
}
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if stale > 0 {
|
||||
resp.Health.Stale = &stale
|
||||
}
|
||||
|
||||
rows, err = s.pool.Query(ctx, `
|
||||
SELECT severity, count(*) FROM signals
|
||||
WHERE state NOT IN ('resolved', 'failed')
|
||||
GROUP BY severity`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for rows.Next() {
|
||||
var severity string
|
||||
var n int
|
||||
if err := rows.Scan(&severity, &n); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
resp.SignalsBySeverity[severity] = n
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := s.pool.QueryRow(ctx,
|
||||
`SELECT count(*) FROM approvals WHERE status = 'pending'`,
|
||||
).Scan(&resp.ApprovalsPending); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
rows, err = s.pool.Query(ctx, `
|
||||
SELECT status, count(*) FROM executions
|
||||
WHERE created_at > now() - interval '24 hours'
|
||||
GROUP BY status`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for rows.Next() {
|
||||
var status string
|
||||
var n int
|
||||
if err := rows.Scan(&status, &n); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
resp.ExecutionsByState[status] = n
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
rows, err = s.pool.Query(ctx, `
|
||||
SELECT date_trunc('hour', ts) + (extract(minute FROM ts)::int / 5) * interval '5 minutes' AS bucket,
|
||||
count(*)
|
||||
FROM events
|
||||
WHERE ts > now() - interval '6 hours'
|
||||
GROUP BY bucket
|
||||
ORDER BY bucket`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for rows.Next() {
|
||||
var bucket time.Time
|
||||
var n int
|
||||
if err := rows.Scan(&bucket, &n); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
resp.EventRate = append(resp.EventRate, struct {
|
||||
Bucket time.Time `json:"bucket"`
|
||||
Count int `json:"count"`
|
||||
}{Bucket: bucket, Count: n})
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return gen.GetDashboardSummary200JSONResponse(resp), nil
|
||||
}
|
||||
13
internal/httpapi/default_checks.go
Normal file
13
internal/httpapi/default_checks.go
Normal file
@@ -0,0 +1,13 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/dtoro/oikos/internal/checkdefaults"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
func ensureDefaultChecks(ctx context.Context, tx pgx.Tx, entityID uuid.UUID, slug, entityType string, attrsJSON []byte) {
|
||||
checkdefaults.Ensure(ctx, tx, entityID, slug, entityType, attrsJSON)
|
||||
}
|
||||
@@ -81,6 +81,7 @@ const (
|
||||
CheckKindDisk CheckKind = "disk"
|
||||
CheckKindDrift CheckKind = "drift"
|
||||
CheckKindHttp CheckKind = "http"
|
||||
CheckKindPing CheckKind = "ping"
|
||||
CheckKindSshScript CheckKind = "ssh-script"
|
||||
CheckKindTcp CheckKind = "tcp"
|
||||
)
|
||||
@@ -91,6 +92,7 @@ const (
|
||||
CheckCreateKindDisk CheckCreateKind = "disk"
|
||||
CheckCreateKindDrift CheckCreateKind = "drift"
|
||||
CheckCreateKindHttp CheckCreateKind = "http"
|
||||
CheckCreateKindPing CheckCreateKind = "ping"
|
||||
CheckCreateKindSshScript CheckCreateKind = "ssh-script"
|
||||
CheckCreateKindTcp CheckCreateKind = "tcp"
|
||||
)
|
||||
@@ -102,6 +104,15 @@ const (
|
||||
ClassificationRouteHold ClassificationRoute = "hold"
|
||||
)
|
||||
|
||||
// Defines values for EntityHealth.
|
||||
const (
|
||||
EntityHealthDegraded EntityHealth = "degraded"
|
||||
EntityHealthDown EntityHealth = "down"
|
||||
EntityHealthHealthy EntityHealth = "healthy"
|
||||
EntityHealthStale EntityHealth = "stale"
|
||||
EntityHealthUnknown EntityHealth = "unknown"
|
||||
)
|
||||
|
||||
// Defines values for EntityTypeLayer.
|
||||
const (
|
||||
EntityTypeLayerCognition EntityTypeLayer = "cognition"
|
||||
@@ -153,11 +164,21 @@ const (
|
||||
ExecutionStatusVerifying ExecutionStatus = "verifying"
|
||||
)
|
||||
|
||||
// Defines values for GraphViewHealth.
|
||||
const (
|
||||
GraphViewHealthDegraded GraphViewHealth = "degraded"
|
||||
GraphViewHealthDown GraphViewHealth = "down"
|
||||
GraphViewHealthHealthy GraphViewHealth = "healthy"
|
||||
GraphViewHealthStale GraphViewHealth = "stale"
|
||||
GraphViewHealthUnknown GraphViewHealth = "unknown"
|
||||
)
|
||||
|
||||
// Defines values for HealthSummaryEntitiesHealth.
|
||||
const (
|
||||
HealthSummaryEntitiesHealthDegraded HealthSummaryEntitiesHealth = "degraded"
|
||||
HealthSummaryEntitiesHealthDown HealthSummaryEntitiesHealth = "down"
|
||||
HealthSummaryEntitiesHealthHealthy HealthSummaryEntitiesHealth = "healthy"
|
||||
HealthSummaryEntitiesHealthStale HealthSummaryEntitiesHealth = "stale"
|
||||
HealthSummaryEntitiesHealthUnknown HealthSummaryEntitiesHealth = "unknown"
|
||||
)
|
||||
|
||||
@@ -295,6 +316,11 @@ const (
|
||||
Out GetEntityRelationsParamsDirection = "out"
|
||||
)
|
||||
|
||||
// Defines values for GetGraphParamsInclude.
|
||||
const (
|
||||
Status GetGraphParamsInclude = "status"
|
||||
)
|
||||
|
||||
// Defines values for QueryMetricsParamsRollup.
|
||||
const (
|
||||
QueryMetricsParamsRollupAuto QueryMetricsParamsRollup = "auto"
|
||||
@@ -520,6 +546,38 @@ type ClientSecrets struct {
|
||||
Keys []string `json:"keys"`
|
||||
}
|
||||
|
||||
// DashboardSummary defines model for DashboardSummary.
|
||||
type DashboardSummary struct {
|
||||
ApprovalsPending int `json:"approvals_pending"`
|
||||
|
||||
// EntitiesByState entity counts keyed by state
|
||||
EntitiesByState map[string]int `json:"entities_by_state"`
|
||||
|
||||
// EntitiesByType entity counts keyed by type
|
||||
EntitiesByType map[string]int `json:"entities_by_type"`
|
||||
|
||||
// EventRate event counts bucketed by 5-minute interval, most recent last
|
||||
EventRate []struct {
|
||||
Bucket time.Time `json:"bucket"`
|
||||
Count int `json:"count"`
|
||||
} `json:"event_rate"`
|
||||
|
||||
// ExecutionsByState execution counts keyed by state, last 24h
|
||||
ExecutionsByState map[string]int `json:"executions_by_state"`
|
||||
Health struct {
|
||||
Degraded int `json:"degraded"`
|
||||
Down int `json:"down"`
|
||||
Healthy int `json:"healthy"`
|
||||
|
||||
// Stale last observation older than the check's expected cadence
|
||||
Stale *int `json:"stale,omitempty"`
|
||||
Unknown int `json:"unknown"`
|
||||
} `json:"health"`
|
||||
|
||||
// SignalsBySeverity open (non-resolved) signal counts keyed by severity
|
||||
SignalsBySeverity map[string]int `json:"signals_by_severity"`
|
||||
}
|
||||
|
||||
// EnrollRequest defines model for EnrollRequest.
|
||||
type EnrollRequest struct {
|
||||
// Hostname Actual hostname of the enrolling machine
|
||||
@@ -552,18 +610,27 @@ type EnrollResponse struct {
|
||||
|
||||
// Entity defines model for Entity.
|
||||
type Entity struct {
|
||||
Attributes *map[string]interface{} `json:"attributes,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
Id openapi_types.UUID `json:"id"`
|
||||
MaintenanceUntil *time.Time `json:"maintenance_until"`
|
||||
Name string `json:"name"`
|
||||
Slug string `json:"slug"`
|
||||
State *string `json:"state"`
|
||||
Type string `json:"type"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
Version int `json:"version"`
|
||||
Attributes *map[string]interface{} `json:"attributes,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
|
||||
// Health last observed health, when the entity is monitored
|
||||
Health *EntityHealth `json:"health"`
|
||||
Id openapi_types.UUID `json:"id"`
|
||||
|
||||
// LastCheckAt when health was last observed
|
||||
LastCheckAt *time.Time `json:"last_check_at"`
|
||||
MaintenanceUntil *time.Time `json:"maintenance_until"`
|
||||
Name string `json:"name"`
|
||||
Slug string `json:"slug"`
|
||||
State *string `json:"state"`
|
||||
Type string `json:"type"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
Version int `json:"version"`
|
||||
}
|
||||
|
||||
// EntityHealth last observed health, when the entity is monitored
|
||||
type EntityHealth string
|
||||
|
||||
// EntityCreate defines model for EntityCreate.
|
||||
type EntityCreate struct {
|
||||
// Attributes Validated against the type's attribute_schema
|
||||
@@ -696,12 +763,18 @@ type ExecutionRequest struct {
|
||||
// GraphView defines model for GraphView.
|
||||
type GraphView struct {
|
||||
Edges []Relationship `json:"edges"`
|
||||
Nodes []Entity `json:"nodes"`
|
||||
|
||||
// Health entity id -> health, present when include=status was requested
|
||||
Health *map[string]GraphViewHealth `json:"health,omitempty"`
|
||||
Nodes []Entity `json:"nodes"`
|
||||
|
||||
// Truncated True if node cap was hit
|
||||
Truncated *bool `json:"truncated,omitempty"`
|
||||
}
|
||||
|
||||
// GraphViewHealth defines model for GraphView.Health.
|
||||
type GraphViewHealth string
|
||||
|
||||
// HealthSummary defines model for HealthSummary.
|
||||
type HealthSummary struct {
|
||||
Entities []struct {
|
||||
@@ -715,7 +788,10 @@ type HealthSummary struct {
|
||||
Degraded int `json:"degraded"`
|
||||
Down int `json:"down"`
|
||||
Healthy int `json:"healthy"`
|
||||
Unknown int `json:"unknown"`
|
||||
|
||||
// Stale last observation older than the check's expected cadence
|
||||
Stale *int `json:"stale,omitempty"`
|
||||
Unknown int `json:"unknown"`
|
||||
} `json:"summary"`
|
||||
}
|
||||
|
||||
@@ -1234,8 +1310,14 @@ type GetGraphParams struct {
|
||||
Root *string `form:"root,omitempty" json:"root,omitempty"`
|
||||
Depth *int `form:"depth,omitempty" json:"depth,omitempty"`
|
||||
RelType *[]string `form:"rel_type,omitempty" json:"rel_type,omitempty"`
|
||||
|
||||
// Include include=status joins entity_status and populates GraphView.health
|
||||
Include *[]GetGraphParamsInclude `form:"include,omitempty" json:"include,omitempty"`
|
||||
}
|
||||
|
||||
// GetGraphParamsInclude defines parameters for GetGraph.
|
||||
type GetGraphParamsInclude string
|
||||
|
||||
// SearchKnowledgeParams defines parameters for SearchKnowledge.
|
||||
type SearchKnowledgeParams struct {
|
||||
Q string `form:"q" json:"q"`
|
||||
@@ -1469,6 +1551,9 @@ type ServerInterface interface {
|
||||
// List secrets accessible to this client
|
||||
// (GET /clients/{slug}/secrets)
|
||||
GetClientSecrets(w http.ResponseWriter, r *http.Request, slug EntitySlug)
|
||||
// One-round-trip overview for the control room home page
|
||||
// (GET /dashboard/summary)
|
||||
GetDashboardSummary(w http.ResponseWriter, r *http.Request)
|
||||
// List entities
|
||||
// (GET /entities)
|
||||
ListEntities(w http.ResponseWriter, r *http.Request, params ListEntitiesParams)
|
||||
@@ -1664,6 +1749,12 @@ func (_ Unimplemented) GetClientSecrets(w http.ResponseWriter, r *http.Request,
|
||||
w.WriteHeader(http.StatusNotImplemented)
|
||||
}
|
||||
|
||||
// One-round-trip overview for the control room home page
|
||||
// (GET /dashboard/summary)
|
||||
func (_ Unimplemented) GetDashboardSummary(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusNotImplemented)
|
||||
}
|
||||
|
||||
// List entities
|
||||
// (GET /entities)
|
||||
func (_ Unimplemented) ListEntities(w http.ResponseWriter, r *http.Request, params ListEntitiesParams) {
|
||||
@@ -2537,6 +2628,26 @@ func (siw *ServerInterfaceWrapper) GetClientSecrets(w http.ResponseWriter, r *ht
|
||||
handler.ServeHTTP(w, r)
|
||||
}
|
||||
|
||||
// GetDashboardSummary operation middleware
|
||||
func (siw *ServerInterfaceWrapper) GetDashboardSummary(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
ctx := r.Context()
|
||||
|
||||
ctx = context.WithValue(ctx, BearerAuthScopes, []string{})
|
||||
|
||||
r = r.WithContext(ctx)
|
||||
|
||||
handler := http.Handler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
siw.Handler.GetDashboardSummary(w, r)
|
||||
}))
|
||||
|
||||
for _, middleware := range siw.HandlerMiddlewares {
|
||||
handler = middleware(handler)
|
||||
}
|
||||
|
||||
handler.ServeHTTP(w, r)
|
||||
}
|
||||
|
||||
// ListEntities operation middleware
|
||||
func (siw *ServerInterfaceWrapper) ListEntities(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
@@ -3305,6 +3416,14 @@ func (siw *ServerInterfaceWrapper) GetGraph(w http.ResponseWriter, r *http.Reque
|
||||
return
|
||||
}
|
||||
|
||||
// ------------- Optional query parameter "include" -------------
|
||||
|
||||
err = runtime.BindQueryParameter("form", true, false, "include", r.URL.Query(), ¶ms.Include)
|
||||
if err != nil {
|
||||
siw.ErrorHandlerFunc(w, r, &InvalidParamFormatError{ParamName: "include", Err: err})
|
||||
return
|
||||
}
|
||||
|
||||
handler := http.Handler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
siw.Handler.GetGraph(w, r, params)
|
||||
}))
|
||||
@@ -4547,6 +4666,9 @@ func HandlerWithOptions(si ServerInterface, options ChiServerOptions) http.Handl
|
||||
r.Group(func(r chi.Router) {
|
||||
r.Get(options.BaseURL+"/clients/{slug}/secrets", wrapper.GetClientSecrets)
|
||||
})
|
||||
r.Group(func(r chi.Router) {
|
||||
r.Get(options.BaseURL+"/dashboard/summary", wrapper.GetDashboardSummary)
|
||||
})
|
||||
r.Group(func(r chi.Router) {
|
||||
r.Get(options.BaseURL+"/entities", wrapper.ListEntities)
|
||||
})
|
||||
@@ -5020,6 +5142,34 @@ func (response GetClientSecretsdefaultApplicationProblemPlusJSONResponse) VisitG
|
||||
return json.NewEncoder(w).Encode(response.Body)
|
||||
}
|
||||
|
||||
type GetDashboardSummaryRequestObject struct {
|
||||
}
|
||||
|
||||
type GetDashboardSummaryResponseObject interface {
|
||||
VisitGetDashboardSummaryResponse(w http.ResponseWriter) error
|
||||
}
|
||||
|
||||
type GetDashboardSummary200JSONResponse DashboardSummary
|
||||
|
||||
func (response GetDashboardSummary200JSONResponse) VisitGetDashboardSummaryResponse(w http.ResponseWriter) error {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(200)
|
||||
|
||||
return json.NewEncoder(w).Encode(response)
|
||||
}
|
||||
|
||||
type GetDashboardSummarydefaultApplicationProblemPlusJSONResponse struct {
|
||||
Body Problem
|
||||
StatusCode int
|
||||
}
|
||||
|
||||
func (response GetDashboardSummarydefaultApplicationProblemPlusJSONResponse) VisitGetDashboardSummaryResponse(w http.ResponseWriter) error {
|
||||
w.Header().Set("Content-Type", "application/problem+json")
|
||||
w.WriteHeader(response.StatusCode)
|
||||
|
||||
return json.NewEncoder(w).Encode(response.Body)
|
||||
}
|
||||
|
||||
type ListEntitiesRequestObject struct {
|
||||
Params ListEntitiesParams
|
||||
}
|
||||
@@ -6401,6 +6551,9 @@ type StrictServerInterface interface {
|
||||
// List secrets accessible to this client
|
||||
// (GET /clients/{slug}/secrets)
|
||||
GetClientSecrets(ctx context.Context, request GetClientSecretsRequestObject) (GetClientSecretsResponseObject, error)
|
||||
// One-round-trip overview for the control room home page
|
||||
// (GET /dashboard/summary)
|
||||
GetDashboardSummary(ctx context.Context, request GetDashboardSummaryRequestObject) (GetDashboardSummaryResponseObject, error)
|
||||
// List entities
|
||||
// (GET /entities)
|
||||
ListEntities(ctx context.Context, request ListEntitiesRequestObject) (ListEntitiesResponseObject, error)
|
||||
@@ -6870,6 +7023,30 @@ func (sh *strictHandler) GetClientSecrets(w http.ResponseWriter, r *http.Request
|
||||
}
|
||||
}
|
||||
|
||||
// GetDashboardSummary operation middleware
|
||||
func (sh *strictHandler) GetDashboardSummary(w http.ResponseWriter, r *http.Request) {
|
||||
var request GetDashboardSummaryRequestObject
|
||||
|
||||
handler := func(ctx context.Context, w http.ResponseWriter, r *http.Request, request interface{}) (interface{}, error) {
|
||||
return sh.ssi.GetDashboardSummary(ctx, request.(GetDashboardSummaryRequestObject))
|
||||
}
|
||||
for _, middleware := range sh.middlewares {
|
||||
handler = middleware(handler, "GetDashboardSummary")
|
||||
}
|
||||
|
||||
response, err := handler(r.Context(), w, r, request)
|
||||
|
||||
if err != nil {
|
||||
sh.options.ResponseErrorHandlerFunc(w, r, err)
|
||||
} else if validResponse, ok := response.(GetDashboardSummaryResponseObject); ok {
|
||||
if err := validResponse.VisitGetDashboardSummaryResponse(w); err != nil {
|
||||
sh.options.ResponseErrorHandlerFunc(w, r, err)
|
||||
}
|
||||
} else if response != nil {
|
||||
sh.options.ResponseErrorHandlerFunc(w, r, fmt.Errorf("unexpected response type: %T", response))
|
||||
}
|
||||
}
|
||||
|
||||
// ListEntities operation middleware
|
||||
func (sh *strictHandler) ListEntities(w http.ResponseWriter, r *http.Request, params ListEntitiesParams) {
|
||||
var request ListEntitiesRequestObject
|
||||
@@ -8031,166 +8208,175 @@ func (sh *strictHandler) GetTrends(w http.ResponseWriter, r *http.Request, entit
|
||||
// Base64 encoded, gzipped, json marshaled Swagger object
|
||||
var swaggerSpec = []string{
|
||||
|
||||
"H4sIAAAAAAAC/+x923LcuNngq6C4W5VWwlb7MPNnR75SZI3txI61lib/pkauFpr8uhsRCHAAsKWOS1W5",
|
||||
"2gfYyhPmSbZwIEh2g2z2QZYzlRtbEkEQ+E74zvgSJTzLOQOmZHTyJZoDTkGYH8+v8Ez/n4JMBMkV4Sw6",
|
||||
"iT6B5IVIAC1ASMIZmnKB3k2HH7BK5lEcyWQOGdbvqWUO0UkklSBsFj08PMRRjgXOQLkPnBVCcrH+iY85",
|
||||
"/qUAlJjHaCp4hjDKBSwILyQSIHPOJPxGIgb3amyHRXFE9Lu/FCCWURwxnOmP+4fty4qjc6aIWr5L11fy",
|
||||
"00/vXiMukKTFDA3geHaMbuZcqpN5MRFE3hyVn82xmldfJWkURwJ+KYiANDpRooA+K7ikRQDg9lljCXfy",
|
||||
"JMPJMCOM3MToht4nJwlO02XbevS7W67oR8GzK6Lf/hIErMZKA6xTLjKsopMoxQqGSr8aB+Z9l0KWcwUs",
|
||||
"Wf4Jluu7PaMEmBrOgIHAClJ0C8tXSEBO8VKiO6LmhKEX382RAFUIhtQcEBdkRhimnjRKKFhirhZd+/hQ",
|
||||
"f72+/gzfvwc2U/Po5PmL/xVc+tTS+DqGrvCsIlPCBXpzfvUKfff8BeLM80lGZOZ4JLy4ioe2QdR7khHV",
|
||||
"hiVqHtYnSGGKC6qik++fxXrPJCuy6OTFM/0bYfa35373hCmYgTAfuuJd9KD49tTwoHdqMWbkwQWwlLDZ",
|
||||
"aZ4LvsBU/ynhTAEz+8N5TkmCNcxHf5Ma8F9qH/yfAqbRSfQ/RpU4G9mncuQnNJ9cobc5ZjNAUuEZpK8Q",
|
||||
"RhkoPMTuDXSHJUoEGEocpAWmQ70iwelR9BBHF4JPKGQdC83tiN9tt+By3sB6z4XgAg0+/XiGfvju+9+b",
|
||||
"ZVySGcP0p1zDOj0Y1OysoTW4LyFZjigxb7B4OgOmThNFFkQZBs8Fz0EoYpGM3ZOxpYYvETBNcz9HinM6",
|
||||
"TjClhgGw5ExTif52QjQDRXGUJfm4pDuQCaZmX9HnNdKKI6xXMSZpgGniKOFCgH3ZDWEFpXhCoeS4tVfS",
|
||||
"QtjxmewY7/kljsCI7b7TNxZam4WwvFBjWWQZFsteM/FCbfuKBCm3AIUskgRkFxgmnFPATA9W/BbYOOGF",
|
||||
"JcfNcDNkYIVKj7VYpaXn2VOJ1Z/tEa1kVKOUeIU2K7Lik79BovT36rJpna4te62TmxUgY6z6LtZSfdr9",
|
||||
"zmaadXNM+pEB3OdEgNxqmZZk/NiisHBdHXZLWFrndbiHpFCWqXNOSbIcJkYQ69+xUiDY0CCjncFzvKQc",
|
||||
"B3Q2oyJp3HAJKbKzo5RMp+0gq/AriLwdJxRb8l4nfaehrT9QWBWyvsXcHmaaqgzNQGpkGSPmBwtrqyYu",
|
||||
"+C2kwT3Kwi6sUyfUGpA/rxLOEhBMbiaPED84PdGRcgMaDod+pw1yaZB4F9t8KihsxTq4UJzxbDmmsABa",
|
||||
"h69+Uh0Dhh9gASIIRyeLyxOnCcsPeIkmgPBEKoEThQaEzUEQJVHK79hRH0bryQWbiCvhOYztWrtRrk2u",
|
||||
"HMTQjkV8AUKQFGSftTp1NHTchEgiTAsraKlm3YT8M0MnT08C++Kmm5l6AS0IqiIl6pwpsdwORIniou/x",
|
||||
"bQeval/mFIziSH8RK2syL6WC0shLC9oC2V2UKVCY0NpWKggcRm3KQM15vymMpdxL7TFujzHJ+402YnKc",
|
||||
"8BR66j0H0GQqzHrGDVOZJcRLUErPuEZqt9Yyh3uc5XrN0YzyCabHmoLHOFEh4VZYo2Ar7WGBaRFmx/5S",
|
||||
"6tbY8Xambjl0Nofkdn2zCWdTEvC7/AVTYu0cLWrd6RegV43WOhnWlN+e54LemlhgOpZhal7VnuZK5Xqe",
|
||||
"RP+bEnmrD2AQamiOZA2OVJCpsfvlfGj3FNYv2tQZhcUMNugdA8KkwiyBoRGOaa+T0k7cchK72fVDNND/",
|
||||
"bjUzyYBrwycMww6CiqO/c9bH3OhQmRx51DBZX1FFJj0otO2IrOi0iwi9f6fNIGsSmx/+X8+exd8Y6W0i",
|
||||
"nm4S8Dt7HtxYifJuFNex24qwi9IruAu+NiEocFB0EfpDaJFa/yBT5wXaTfdKStm5NmRCsVRjgVNi7R+i",
|
||||
"IAvrUO4PWAi8DCsOB7GcewpdZ2aODZpSYEmXBGBFNrHQrzxTIcQKSHiWATOWuwfrvlan4IWCVcV3aM/h",
|
||||
"mvI757TFjDR+ut7unVtCew+uuHUH2RlWk+1umz7AFVLZaG/aKMIZZwruVYDijctnSijIsfU7BPwIF1jN",
|
||||
"JeJTZEYjfdqJwqwYmTeRmmOFytfjLQhfEkdtzQ9ekQykwllu7Dtt1jO4VyjnlCINO5Aa4f2YQPJcWtKe",
|
||||
"te/wShSAyBQd69HHS5zp7yQk17CTtZ2FvHqc9gGdGTf67bEEVeTHcr47zGrn94r5zhlXnJEEJRbdPuDi",
|
||||
"mDbepEF2nsiGkC4hEWAV9DVFWa4v6R2bEkkSTJE0LyI9DGHjNSUTCkhxpOZEosTMvgUc1nVfGVz2OROc",
|
||||
"0k+OaNaWPedSlS7W5tJPE1VgisoBBodzQGDmI2yGMpzMCQvSXAZy7syj5qSXNmCsn6N3F4a6tcQ1yt7C",
|
||||
"atlWDrRqCZsionfyhMHdkOJc8fxoo8lkpu2CmwsjhgTHOBdkgRWMb0Phy9M358PL87NP51fDP53/dXh8",
|
||||
"fGy2S7mmhhQSsczb9mrmLiaUJOGp8Qye6/nsGE1TZurLjxeXNbYN2xeOHseW4JxwbyPanxjRLIHpaaHm",
|
||||
"jkbRu9e9ZrYEv/Xs7rUQUVl6G5cEMzYBha4PuDcqErOMh+yLm0hjBQvxGsrD4GwHRZjMVDg2ppQgk0KB",
|
||||
"DGoXj6gNZVhLR6bNuXHBlHXO7BZ1KAVLKzNXboVaKkXU4kPpGQZqMwl28kps5yF1NoJzu5jdV3M0kNZY",
|
||||
"TjtdtPpIG9TR5qzAM6xVFSO29Qd+I5F/cewiv4FPb8RaO3aaK3lt7S5pDzlAlEwhWSZUL8TZZGP7agce",
|
||||
"V474QirjokeMs6F31EPlLugn8ZtIakfARTjN41QhClgqxJk/GKcEaCpR5laYC5DA1HEUb4G7DyBM7sFi",
|
||||
"DYd9EPdVODeM6iuj/FcYRmYcGiiBmSRGU/Z7Ch/KLQi4cmTQAsNxPZ2lvqA/Xn78M7osQbXR7Gq8HNh2",
|
||||
"yjVwg4+IHJd0GLbiKV6CqNtsGShsjwmBrSVRCI2VGV+AMOgzds6MkdaQpwd0X+usFaE5FvqIKtlts01o",
|
||||
"YDrudKKth0BNBBdM2DMXkJjslM97CVwnXR1iSig30eFX8rmTvjZK2fFa1tVjUI53U00xlUGH3RolHZKE",
|
||||
"diaZbnEbxlM3QlrcaAfBx660GRRRC5dateru2z4MhhV+xCCYhAUIp2fWaIdHcXSHRelZEURprTXsODKG",
|
||||
"W9gulf0VKh9r9Iqf9UgdC0wkpFtEuNz57XfmlxgkLZ9kspXLs5Y4tjmW6kLbfccnDVds77e4BpvaMxvo",
|
||||
"kdytWyfG9fbPCpwFtKXLW0Ipsk/RYF1nsk+csDgKaUwCpJG4+ztmH9GxagfXTsbNgO3S1MW+1BNIcHIp",
|
||||
"Vs0MJxsi7sx4cslfRvxoPp4uaz/bwVNMbLRMLy8d88LkGOkTjtq/C65/GE9wcut+0z+O3Xuf9/FU1xYS",
|
||||
"0Oy2Tpvy+VLb+rC9+Gp141VSrJKsAgy2uzkqwBJYth2ddRpfYUXzyPp0M26y9SC1ns0BN4MwPYrircPL",
|
||||
"9aqLjYeDm6sz4eGNwPn8LwTu1mEI6QyacauunOhPDoFyTvKQl5rxdIvZnBsoMI8SBUvKbO6w015/CiU4",
|
||||
"Nynqc6ICfvpVpYzbFDG75RCc3gKman5ZpQ6vwEqvl6xscMWzbGZoRIzNX0xkGGYCp1Yq8DvNKAW7Zfqn",
|
||||
"oKaKpbJhx/3kVmucWUAztE0yLbTKnHO9VPuzVOYDjdXu6pDq9Eo42IUQsxZBakORh3FQIzRgDz4psRR8",
|
||||
"WG58s322HbLL/a1CxW0urgguBJM/MX5HNS2/JQHZ2FPLoITdQjoOUvbGmJTA7LZX0Lj9ZGYkz0OS8C2Z",
|
||||
"zSmZzbVANVVDZXSrF83bTLXemW2KKAodO654JOVJkdmQlSjYhPNb49JYgFRk1parvdlpahcQQvL70l59",
|
||||
"rcXoOrXX/YlBazttd2htiW0FIiP6KNzpZe8SC6i0X6aCZyfoi+In6IsDlTxBP2srOh0aGRij4+Pjzw8P",
|
||||
"D9Em7iFlkraR9Wse19o6QvD+AEqQ5BKEg3DgAFi2WQ+Zebclg4HSIq9TksB3URw9n+t/WrIWjErTddjg",
|
||||
"xawX+21RfZLh+15TZoT1GreNmeyzIVcKbvEdsrBAZZbjhs+uakiy15niD8QuleXKDFr9QkUXngo8zitE",
|
||||
"hhZxYbNwtrPQ85wSkO15YM2UniY0/5tQyRmi/A4EmvCCpbHWonJI0WSJYGHfswVLo++jAEqbY8InprZD",
|
||||
"CtE5pO8BpdUgb+XupQrlFazXnv1SYIGZIqwtK22Lypf5MudqDpL83SY8losvC6xWvG7mAPFjPrcXnHVB",
|
||||
"c7eYXYOSagZcCakGKa1hvmbbdaWQ1IpEV0+vlSz4WrWCEFwEToofCdB0aOoHapkTyA5Hg+9evDiqZ5M0",
|
||||
"v2diVWHx3Gb+rcDOzuDH95EqZZLwBtoJpTluUkq86zzCE16okwnV+lgtQ6oQZLP5aD7TGTO40GaB7DTE",
|
||||
"uyKzH969jlHCBcgYCZyNs0mMUiJvx7NJjEgeIwVZTrGCGGWarmSMJIgFSUCGvFdzLgP64iUtZmVM8kLw",
|
||||
"+4zfmyQelx9TC7QHDfJwMtDbIsNsKACnWq4g59TvmaRjvkvvk5O/AaXLKWHbx3vpfRKjRRYjLlDKk1sQ",
|
||||
"pvoaE1ZP6+of8XXA24Djttyfqvinn03vaxyDvhPv3UHvXptQucDJLcrLZRA207/MBBgn0oZjIngcRytL",
|
||||
"6Nz2pefElU1ryRLo0LEAgSm1ggeRaXPh3nu3u3XeEnE+K4QA5kP/rXkEUkHepTmadY8zkBLP+kVAp4QR",
|
||||
"Od/fifoYjlhfaSoK5sI6xjDzeJC32sxsOVwV5D08FXpUp5TsTFR0zFjiy6InNEvDwbZBzm703lfhq7Cw",
|
||||
"dI1sPLv09lN6aWsTIDomaFNRzeE9Nk1UtrEPSDpWfFfaWcWJhU5ceVCdrKytbROK+qUq9UZMt+O3HR8b",
|
||||
"3+vnjQsDZBMMwskqCRYpYZiuxF85g6HiQ24yaN0vGWaaePR/1bPyN/Pwc7CGsjvkTZhWSmG/PBHnSepV",
|
||||
"OB1tW+618f1wUkF9Tc0vxA2oB/FG5O1ZGc5bIdkylFt9skIbcwirimPLH22SvchapKsvoMFUm5wtxtUm",
|
||||
"XLbgJwyf9Y0ElhECjmv4sk7KxgnfM25Z2kg9D1Uh1VgCsK1CzlOK8y5jcM5pOk75Hds3IW7b7hZVfoNV",
|
||||
"4IfO9R226rfeNyW3QJfjBBc9+Tor1N5JgTxJjNLV7fA4QK7JRlWwch26rBGc3JYxgNK5YL6jt23NVMnp",
|
||||
"oh5Q/rzLIT8Hn2pqijYO0WKjbKZRS2RxutEavFfZZIV6gpx8Syjdy6e2OZvE1PCMK89Bzzd6d6PZxj9W",
|
||||
"yD2V6o6sN1slSNItHf654AmkhQipn2XqXoqMIjyySRCjMothVCa35BQz9Onl8IejtXxiuM8h0aaET79p",
|
||||
"qx1nWg6W/shsLT69eSP1fJpw8oBbd++4tyHPS21RhEKba6bbzlNZuB5iroA1JH32SpAFt/CXpgJPrcBS",
|
||||
"IFf8pAKmxidbM+Y2JNOWzlLRXTtR+bN3c52u1Tl456l3ilYs0CqjLp3xuZpKmWWYBZwmb7h3lpmmNy7Z",
|
||||
"Kwo3s3Kdk1YZhyjflSOUxJzyQg8wbiYZVrr6F08ICLdyUXobqoVlU6B42b+8Wxv9zQRhKedRXJbqm3Jy",
|
||||
"1nLotp19WwA6XHT/X882ll26dcce3SEquSqDUisAZDzDdNmiTRMBVWbUDtkd6won1xwX9LsSE5ijhAEW",
|
||||
"KBf8b/bTMfp9imza2uZYYnvcVFIespze289NiUI5CJTi5dZBQR+mq6AVTMyQkBRaQzFVFRb4E8ACxGmh",
|
||||
"5qHWvdYsGi0I3IE4QXqY1p5u0cd3r8/QH//7qp60Sdjw9OId+tc//onOcJour9mUizss0iEu1BwRUzIE",
|
||||
"TMKQsGEKuZrHiHFb3OS8N1pDE4WaHx1fM9N68sS4BUmC7Dpt3Z9tz1oVCQ5MZxF0Y7J9b/S7ZftSQ0zm",
|
||||
"zYraDSuZRphGp3UdNl0Gv2uAmioueLTeX9S2Cx3qsxyQ3mxZ3P2R3HKJ5jwDiifo4+UxutLq5ZRQ0BvX",
|
||||
"Q377W7/Ja2Z2+dvfooHpQIoTNTR64dEJesNNxAAEkqqYSIQFoKqB7h1Rc8RxToZa7M2AxdfMlihKNCg/",
|
||||
"f/b+XYymhdZK0E/v5JGFlwEzzgDJHJLja3bNzjhbaHRyVtNPXh6dXLMhOrdRKP31sj0pumlrhnpzrF95",
|
||||
"T6SSqJCAbr6YMzqu93R+uLGLd42gczwjzAa8Bk7QINPgFn3/LEYZvkcvnj07MvP+xCSeArr4eHllC69z",
|
||||
"hW5Wuv/eoIHtI5xTvER3hKX8zr79oTBCAQnX6lqiBAuxRDfutLt5hd6cX7kOxBLdnF/h2U2MLk6vzt6i",
|
||||
"Mn8D3ZQNfW/QwLUCLlsA28/4ev8KZi9fvvwB/XR1Zp6fu6Qk8xSnqQApzbomzRRJNGj2pDaIupoD+nB2",
|
||||
"gYz4n+IE0EAqATgzM7y9urqIEZ9OSUIw1QR0+fpPR5rsTAgKUoQVuhllSX5zzTirCGFCGBZLhFmqB/NC",
|
||||
"GT+q4SVL15plXZLQK0RMGSCnEt0JnF+zip6sfYxMXQjChtqlNrPSnBOmpOVHShJwoRjHZBe2EFeLa0Ed",
|
||||
"Y8qT0chZ3scukjxyBbu1OGJk2e304l1NazmJnh8/O35mzNwcGM5JdBK9PH52/NIGgedG3o2MkBjiWktb",
|
||||
"d2haJxDh7F0anUT/uwCxbHa/bXY8/zncOrnWgLSj0XPLu42OpTtMUE/d6Hw5pDlXmxv5fuE9xrpO0j1G",
|
||||
"ug7xPUbaNtgPn1daSr949myrhsgrSYSl2dDLfmiiPpQdXGtXv33HFLOEwBm9duSUS0DAlMnjeogrxSy8",
|
||||
"BQ+zWuvpWpap7emMJjDHC2LaGRg3O55J49OeaHbGE1K6Xe+H5cJtA6/oJLLqgJl1VDooZSsn6WPh1I/q",
|
||||
"xUTe6qhwebhOrG3MU3px1j65d3/bXx1v+BbsT8cWnqD25wdNoAjXKLTkBY94uQ0jjL6Q9GHkG51rWDcp",
|
||||
"fgOC/QUWGse5SxBpstRr0wvaoyHe8gsr1zZYWjLZMH/g6XIPMqpv2tdmWja1XLr0nBk0zRhvyfxt6RPy",
|
||||
"9sPpWdUu2doGA0nYjMKwkBCjsvjHqdRDSVLY3FHGbyNMic0LHR72ZMRd7zp47RaJBCRcpJAe4mSwuDIp",
|
||||
"OsCWdVg66LYCtC/L+KibZZoitfn+HTqYGdJP96q3TN1F+7J9dx9T8WpT+zjb5c21KrT/KH37HWxVx+Yn",
|
||||
"PNr0Ikp1Dw2URK/PL8+ODsHeZubt9b0mz5oIcre6d2aH9GLaNbWrJ+37vI4dmLVs37r2aq3a7tdF2bZd",
|
||||
"8tMRtaOIAylrhgRRClPCXPlLRdC2xHWTxtamWdkcKAutJ1SrNqLS5Wr10keeH/bTQfTa6ucD4NfOhLDD",
|
||||
"8cBKmyGWwxQrHKPSTfn7o944D4kvo6Tvq5uXPU6aJGRan+xIQe5SskclHdua5Strsq2UU150tT/l2JmM",
|
||||
"7kqsa9UR0a6E0mjvseHAWxnbz8vhi/G/tsZZduVd93Vs1Yz4V3dINntrP+FpuUJOBxCrbkbQlp01HLVu",
|
||||
"OQfksgl5IYflE2TMMqQEJvRoV3+Ii0qNbLdZg5pgsUvZ7FCWLWVjH+6SCDOEZ4BuYZljImJ3e5/5e3uP",
|
||||
"0NhENGrFsY2sL94obzhGZ5hSELbpH6YCcLpEc7wA/Q33DmHm2GGQaunSqI4wiV42wNGUCrb57FnZE/gx",
|
||||
"pHmzL/BXFugrzXVDlxuaERkw5a/ytBFA20CZpR5hB6Bv+zGEEYO7sg/tv/7xT0SkLKCkoZJ+arTjl1BR",
|
||||
"uSPcFhK3V+s0KPyLpMXsYZRUDcqDaRifXITxbk6SuetDbnqPxzasZsnWdAC2vb7L1trItBg3RDwjC2BI",
|
||||
"lbFGE2VmqAwAm+biJmpHmFSAU8SnaEYUygtKQ0T6BlSzufrauRXaAuKMLt3ipF8ckdW67JWWL1++/OGo",
|
||||
"5Spf2zV960tGPz+mitKAREgqu5bkKVCFD0Czb0A5MkjqMzuI4gqc2xJnvJNWay4sfvgcoGxZdUyfdfaD",
|
||||
"HpoEE+MdNG/YYHJado+10/5GrknsdsIsm7U/Ot7LDwXwftmr7/uBTNsScl0N5r8uMdTbqbQqwGV6wyYJ",
|
||||
"8iOhCoSpzq/fD0RYQosUJNKjgaWYKdkmOnb17/p6tW1f9I0vt36z7Gja+eIKsRUT+9B1iuHMpIWMXIpm",
|
||||
"6Cu/7On4/bfS0dvbbH0l3dy1NaNEHoznoWIer2rXmiXt7MY6L4XrN+nHavRH/8qOrJKK2j1ZsbvS3Xz6",
|
||||
"/ArP2qZ0w0ZmjJvwIB4whtZU0w1U0fRflINH3mBpN8LOnF1VM5RqRo+RnLFv/22qZqS2Dk3Woe2JoO22",
|
||||
"BOc4MQZYmW98FKPSjeJmt8GuqsGnyacIWGxNM02LwXJ3PuAb0mh9Dfy3TftrXSi+Mv2vd0hol3Sus2Xc",
|
||||
"RMgvBRRPyiZ+Cwgj/VqhPOkO3v+fsxj95UOMfIeJI2QGmpYR+/JT6ToOKkNvQHnSe0Tju018OZy5bjRP",
|
||||
"h503vpJ+NcF1p0PuAE761aT78pqDutTBAgIXN9Qu36g6rqcwfXXNCKUww7Qxic0kRt89+0HrtWa6YfX8",
|
||||
"6Bhd2Byymf7INbMCUdtEy+rVl2hQSjkPl6OgvNPb21XWPXK0oX7/xlf3TrUxiIs3VGfrU3GIC1dUzRU0",
|
||||
"g9Qu21i5iOMgUmtk7nccVvc7tomwP+hxn+ywXrEMU87RsEM8eF6aRnwkK7Lo5PtAIdFjWxOrKWq5LXVZ",
|
||||
"L9XaridQW5se+4Gtm6pskR8ynZpqUW82WK/qTOB8jlLiOnQdwqVaViyUHyRT64hwgn2KCZVfVZyvE3SZ",
|
||||
"/iQ3H8i+zUc/ihZAd04lq8qwghwRTbhhFt9HxJSWGReDefL58I7PfUzu7j7ZO9NxfdpDhLheG6AjUZ/W",
|
||||
"3Pjp4zgDDV3kkSOPnpB4rWfVq9SjqpK4jYpX23s94vG5+qkA9i6aUTDInRhy+ziAfs8pDbdQM8VJq0r/",
|
||||
"10Jl3TG6MN7XzuzV84W7x7GHxHn0qp+gV7TWtOI/6adP7NtcOI/9U7k2F7aO9IApp2+JVFyYUCuUrLBr",
|
||||
"qZGdYGQrH1uDUZc2Mf0SmEJ2Q8foHCdz+/3fSHRD0puyJtf8DQl+h0iKBgJkkcE1M4Ls5r1Wlc0Mw3ev",
|
||||
"b45idGNGr7yrgRqjmxQr7J/88fLjn6+ZeRVZaB+jt4CFmgBWyF4qrqSeQCzR8+/lMfoDSDWE6ZQLEwQk",
|
||||
"5sm//vHPa2a6GkOKchBDWUz0Ticg0KSYTkHEKBU8H3KaglSuhPfiv45emSLcN+dXyMHsmimOJji5nZJw",
|
||||
"IPjSwLRNWLWGcDwEUC5gSu73jdhYI6t6sYGCzhk2s62Ce2XBMawoqH3C9Sjg5TlyLx7C7b8oCcjOiQaX",
|
||||
"l+dH+zBHlZvTGaerhu1aiffo+dnfSD3Ev9fR4WsUn/D4qGjrUIGxOrVunYUWtwQ7ruaA5pilFMRqdGLg",
|
||||
"M8gMDR7FNoNUujjFqGy9F18zzFIERM1BIGDGG+6OBd8LeGCTKV2Z6hHiopa/ds183ZrzvZkYSNmGoDkT",
|
||||
"YeimvKHrxuecnVLJEdybv5YpFjafRHAKJv3JJgPZ6T7++f1f0R1e2jFSbzF0FLiIxHm96PWbDB+u3qj1",
|
||||
"tUOIFcd1sEIZPUGDzDbIdPXLPoh1CCXrkyegOvU50l6if/3f/1cVSdq0ev0nR7VbJXjWst+qsZsDIjVa",
|
||||
"ejyXby98HMIv5kFsD0f0O+QuIdxNRu3lT2giYWRv1XuUmuMzM/XTo/LMXxx4gFC7mQthVArXkb/OENWr",
|
||||
"/nesbtWyWbSXt56bx5cA6d7OnBXVwbT14bZZWRN6fz398B7VLn5a7xDKFKd8tsur9ozc+sUVfcMvIK7t",
|
||||
"w0/eRw/REEWTQh/wBxGuZTo6knpivRtpOyolroH96z+Ul6W//oRGyDWkMcFnwRsZ9HIpFWS9iMf487uk",
|
||||
"qrkHcZOxdqmw8JHYQT0Oe/QK8Ywo40y7m2uFwUYQBvYCnbbsO8H5Tkp9R4DoxYYAUWw6ZFLT5s9qrL39",
|
||||
"9f07Y0q1NJ2Fplxk0aPmGVf3VwZo1zxEC/N0b9K9LCYWpxrHCyILTMnfXc8tc38k+h0y90fu4AjXJFrd",
|
||||
"D9lGoz9SAGUvonzMA6N51WUArHYAKkGzP2jNxtC8Ma11gJubtxBhqd4KF/s4vHxD5JEELJJ2SF+ax/4O",
|
||||
"xX6m/S/Rqrq8n738DVjBjVskDxepekvUIUzaHwtKhybP36LTNuP0SK7iuYPysJQxcjczNljUv7IVDX3x",
|
||||
"cYIe2Ut1Wvq1ofO9uSa0AvwhOitQ6jUcOSpxhuyFpNo2D2Vz9kVjmJvNhaDBGFB/pjY2hG0euiGO9cEN",
|
||||
"6iVZegekehzrvrXp/od6azFsebVhIFKPC8VrkfrmRZf2KobdCmN3CXc9pWht3CV6OF600yIJB+ptZ4jV",
|
||||
"VE4N7Zwo85S76zFcN4napObHymo5IIocI2koblshYS6yCbCETyvrP2Hj2t7Qzcm1jIst17p29c4mympA",
|
||||
"JPjpxhb7kKBH3WFOeFSSi3HA1qqc9GFeW275t9pqayRarqk/ddrckuXQIyDs/K4nnyIsXanpOCuUMwy8",
|
||||
"i1pzDh56x+HdHBiqclHX/Mb1kpMra4d9w2UneoVPWXpiib2zkcqLZy960KF1J9c7Mu7t3lTagFFzqCjZ",
|
||||
"GDa2trpG0P3ptenZCFLs6Is+jkMtWQLajiuG20LRaU0Ef4tFilKgoEyjbsYVkkWec2G6bc9N/25366lE",
|
||||
"cE+krSv39zb4UmsbfX/9MsAatSTt3TjjqyRq66U9YbJ2G0fUGsQ8EUfUGst4rFdJhftwguse2x2yvygH",
|
||||
"baN779EEccfYe79MgV9TzL287vzpIu6eNA4Ub88rUivpmYK78WuzJlK+fdDeWuEbKZDEU1BLtMB0AU70",
|
||||
"Xr75/dExOvWNmLU4z+vazpqqc/ldm7C+8HeGf31J3STJ1pa4my95X7ubaN/L2x+euB2uZ7hv8ZS4qkp3",
|
||||
"cMlIaOD6ggMaoQq2aFSdJEf9eW3l7HAZKb5MrKDQrwn7p4KCjL6B/uF6IYcsKDD7OnA/cCSKpmlWBSJ3",
|
||||
"SD0688ZVo5G8/9oro3frTyJ3iZ9ttoOnCsR6BXQp+V622mMNUH+jFll9jdvYZE/C5hc2U8BZQw0qQYO0",
|
||||
"wHQYCPt20kwPtn7sbpX7UckjWyf/puRhKMKx9yEJw2UgdjkjT92YS1CKsNnTyvrmWg4o7v3uDtEY2y4S",
|
||||
"STcnGtwSSofyjqhkHiMGCxDDsjem6f1ytMORENZpP2EiTUZguQgiUZ1eKKRo8OLZC/S7KmnwGL3nd2Da",
|
||||
"BBFlCwXc0tHNjPIJpsd6ujFO1Am6jvh0eh3daAsWpzb70G5pXA5Ct+DqDcpjh2QZpAQroEv99WdHJ+Zo",
|
||||
"qoHFtkw086A77DJJMOtuz2GkTYg8d5Mbejv6EaYXDRptiww9nt76bfLIqcGmLW1Rgtik5idUkr14LGm9",
|
||||
"7CLYkJCvGmT28ccfNUt4gtxPfgoib4cmNXaDtuxvl39aXbm65P6AijKRt6iEwYH0ZVGfc0vRqNHTKNu1",
|
||||
"QpKCNX1XG6umjYLkfmUs5uq/bRNeOsta9p6plrC2lSN7hRS/C3UarYWZgB3mbphzlmqtpj71QIKStmHK",
|
||||
"WHFru5hMFiLRLeT2RJibCsDl0S4NLNrMqHN3taCNoQVatqzHBdFgTkBgkcyXQ3yHBRy9QgkWKWGY2uvV",
|
||||
"plwkkLYZUt00920YUvU1Pk1wq9kq4KvcE9CgSJextFOnlLI3fNehcOnG9K6dg4PVZPu7j9mUR3F051xF",
|
||||
"cZQIokgSvBP6USrG+1zY8mty81ucP6GXvyS6Q3WY9TS83ZUpNR6xxSY4uX2USpPT5NbBPIz17p3bVw93",
|
||||
"rcRpUmVoYge8HW+UaEAvK6x2c3DwfSgU1OB3iCCEXuu4YIrQvp26W+/yW725vJp5j8v2vi5FaAB7UnDN",
|
||||
"SK6u3h+CKARIThePQxef7NwHJo12NK8h85tAnoNChb8MswJTutwVfdpU3aA02CH9rjC07pexyTr9T/T+",
|
||||
"MY91jZWnPNUtVRzqUDezoQHFCqTyJWg5CPvoaLeQvp32scMPFhXfcKw9J4xBOvY3zYe6B66H2zUmWmPt",
|
||||
"31503THENx9bNzRpLuIh+tcSKTuG0WsUPnJT9ZDmfylHfnsCbGeB5Pe0P77cVKXvx3Ths3jbXgztWSFv",
|
||||
"yvFk3+KnKzN6a1H071bMYbZ5QNJxYDsEowNLEWaYLiVxjf4oLWs4TA/vQCHVNgUdj1lMpbcCSWFcN3rq",
|
||||
"CWAB4rRQ8+jk588a4/bWbPvhQtDoJBrhnIwWzw09uP2sX6/jyuBdhbavKzDNWU3TmLrvvLkNW1azlodi",
|
||||
"b8gCf0VXXN1BRKTtSEw4i8vraGotltydM+tznm9X6uDm41X1xZew38Ns0bXhGVhUmxhQ4xbF4IJ8t4bq",
|
||||
"FgJ3o15c3fePBqm5TX+EE1WbFurNjL605F2apXnVS8uz2gxevq2/Xw/AxCupRrEPjlVTuSjK+kS+QtKR",
|
||||
"hqsTrnx1tRrHL8HSKxnbimXz3ZSo2PXpi11xcw1TDS4LgTvnQq2/53oePHx++P8BAAD//5TywEKf5AAA",
|
||||
"H4sIAAAAAAAC/+x963IbudXgq6C4WzXUpCn6MpPsyLU/FFljO7FjraXJt6mRiwK7D0mM0EAPgKbEuFyV",
|
||||
"X/sAW3nCPMlXuHY3iSabF1nOVP7YkhqNBs4FOPfzqZfyvOAMmJK9k0+9GeAMhPnx/ApP9f8ZyFSQQhHO",
|
||||
"eie9DyB5KVJAcxCScIYmXKA3k8E7rNJZL+nJdAY51u+pRQG9k55UgrBp7/Pnz0mvwALnoNwHzkohuVj9",
|
||||
"xPsC/1oCSs1jNBE8RxgVAuaElxIJkAVnEr6RiMG9GtlhvaRH9Lu/liAWvaTHcK4/Hh62LyvpnTNF1OJN",
|
||||
"trqSn3568xJxgSQtp6gPx9NjdDPjUp3MyrEg8ubIf7bAalZ9lWS9pCfg15IIyHonSpTQZQWXtIwA3D5r",
|
||||
"LOFOnuQ4HeSEkZsE3dD79CTFWbZoW49+d8sV/Sh4fkX025+igNVYaYB1wkWOVe+kl2EFA6VfTSLzvskg",
|
||||
"L7gCli7+DIvV3Z5RAkwNpsBAYAUZuoXFCySgoHgh0R1RM8LQs+9mSIAqBUNqBogLMiUM00AaHgqWmKtF",
|
||||
"1z4+0F+vrz/H92+BTdWsd/L02f+KLn1iaXwVQ1d4WpEp4QK9Or96gb57+gxxFvgkJzJ3PBJfXMVD2yDq",
|
||||
"LcmJasMSNQ/rE2QwwSVVvZPvnyR6zyQv897Jsyf6N8Lsb0/D7glTMAVhPnTF19GD4ttTw2e9U4sxcx5c",
|
||||
"AMsIm54WheBzTPWfUs4UMLM/XBSUpFjDfPiL1ID/VPvg/xQw6Z30/sewOs6G9qkchgnNJ5fobYbZFJBU",
|
||||
"eArZC4RRDgoPsHsD3WGJUgGGEvtZielAr0hwetT7nPQuBB9TyNcstLAjfrfdgv28kfWeC8EF6n/48Qz9",
|
||||
"8N33fzDLuCRThulPhYZ1djCo2Vlja3BfQtKP8Jg3WDydAlOnqSJzogyDF4IXIBSxSMbuychSw6ceME1z",
|
||||
"P/cU53SUYkoNA2DJmaYS/e2UaAbqJb08LUae7kCmmJp99T6ukFbSw3oVI5JFmCbppVwIsC+7IaykFI8p",
|
||||
"eI5beSUrhR2fyzXjA78kPTDHdtfpGwutzUJYUaqRLPMci0WnmXiptn1FgpRbgEKWaQpyHRjGnFPATA9W",
|
||||
"/BbYKOWlJcfNcDNkYA+VDmuxQkvHu6c6Vn+2V7SSvRqlJEu0WZEVH/8CqdLfq59Nq3Rt2WuV3OwBMsKq",
|
||||
"62It1Wfr39lMs26OcTcygPuCCJBbLdOSTBhblhauy8NuCcvqvA73kJbKMnXBKUkXg9QcxPp3rBQINjDI",
|
||||
"aGfwAi8oxxGZzYhIGjdcQobs7Cgjk0k7yCr8CiJvRynFlrxXSd9JaKsPFFalrG+xsJeZpipDM5CZs4wR",
|
||||
"84OFtRUT5/wWsugeZWkXtlYm1BJQuK9SzlIQTG4mjxg/ODnRkXIDGg6HYacNcmmQ+Dq2+VBS2Ip1cKk4",
|
||||
"4/liRGEOtA5f/aS6Bgw/wBxEFI7uLPY3ThOW7/ACjQHhsVQCpwr1CZuBIEqijN+xoy6M1pELNhFXygsY",
|
||||
"2bWuR7lWuQoQAzsW8TkIQTKQXdbqxNHYdRMjiTgtLKGlmnUT8s8MnTw+CeyLm/XM1AloUVCVGVHnTInF",
|
||||
"diBKFRddr287eFn6MrdgL+npL2JlVeaFVOCVvKykLZDdRZgChQmtbaWCwGHEphzUjHebwmjKncQeY/YY",
|
||||
"kaLbaHNMjlKeQUe55wCSTIXZwLhxKrOEeAlK6RlXSO3WauZwj/NCr7k3pXyM6bGm4BFOVexwK61SsJX0",
|
||||
"MMe0jLNj91Pq1ujxdqb159DZDNLb1c2mnE1IxO7yV0yJ1XP0Uetuvwi9arTWybAm/Ha8F/TWxBzTkYxT",
|
||||
"87L0NFOq0POk+t+MyFt9AYNQA3Mla3Bkgkw0lgorgUg5G9itxcWMNqlGYTGFDeJHnzCpMEthYM7IrNOF",
|
||||
"aSduuZDd7Poh6ut/t5qZ5MC1/hMH5Rq6Snp/56yL1rFGcnJUUkNofUUVtXQg1LabsiLXdbQYzDxtelmT",
|
||||
"5sLw3z95knydFLiJhtZTQtjg0+j+PObXY7qO5Fa8XXgb4S5o24SnyLWxjt4/xxappREycTah3SSx1J+k",
|
||||
"K0PGFEs1EjgjVhsiCvK4ROX+gIXAi7gYcRA9uuMR7JTOkUFTBixddxCwMh9b6Fd2qhhiBaQ8z4EZPT6A",
|
||||
"dV8dVPBSwbIYPLC3ck0UnnHaolQaq11nY88toZ0HV9y6wxEaF5rtbpsWwSVS2ah9Wp/CGWcK7lWE4o0B",
|
||||
"aEIoyJG1QkSsChdYzSTiE2RGI33pidKsGJk3kZphhfzryRaEL4mjtuYHr0gOUuG8MNqeVvIZ3CtUcEqR",
|
||||
"hh1IjfBuTCB5IS1pT9t3eCVKQGSCjvXo4wXO9XdSUmjYydrOYjY+TruAzowbfnssQZXFsZztDrPaNb6k",
|
||||
"zHPGFWckRalFd3C/OKZNNsmTay9mQ0iXkAqw4vqK2CxXl/SGTYgkKaZImheRHoawsaGSMQWkOFIzIlFq",
|
||||
"Zt8CDquSsIwu+yWWszHHIrusjMJLLOBUdDnyZqvoZWOUMwJyNF6MtIJjyBZnGdFbxfSiMefq60t2Oivm",
|
||||
"GdOw1ECBDI0XyM6btGiG7uP+0j/wt53qtPrpuT4hhNvw0lT6mZ9pXKa3oOxk3w9ywkoFyF/hCcq5VJqp",
|
||||
"9Bv6oqzjuokQO1H3ay4Y2DdQt5vXvxCjlmVeC7baA6HdTxfHfGIAg559N4shYgaYqoh0lcFU4AxarAEZ",
|
||||
"v2uR+O18i/hDqTCNINysj4+lxqnZB6cZ6BMaW2+0EY6+kQjuC0g1LaTYChQxwbNkt6xldUuY80tNqs26",
|
||||
"nVWzxNBpr3uLO5iDcAasXdHHC2CozzgbCJCcziE7cn7AVXz6z62samlrK5wdO2kC8uNbSiJnWJx2G+wc",
|
||||
"g9g5E5zSD+6OXaG1GZfK+6easDlNVYkp8gPMlTcDBGY+wqYox+mMsCgD5yBnzrbUnPTSRtvo5+jNhREG",
|
||||
"tIBqzq+5NVFYsalVqdoUTnInTxjcDSguFC+ONtqbzLTr4OZiMGJy1qgQZI4VjG5jsR+nr84Hl+dnH86v",
|
||||
"Bn8+/9vg+PjYbJdyfXlmkIpF0bZXM3c5piSNT42n8FTPZ8doGjVTX76/uKxJOXHjjLu+R/Z+drJw2x3/",
|
||||
"EyNagsD0tFQzd6WjNy87zWzlg61nd6/FiMrS28gTzMh4Y9d9wL1RkZiVU5B9cRNpLGEhWUF5HJztoIiT",
|
||||
"mYoHFiglyLhUjXOsem0X5bG6blqvAMiQHZWguxkwx/AGdESinDOiuHX0BdtJh4PcXz4fD+d8Moq5uZsc",
|
||||
"CJpbMou3SzNBL40ttikaG9eWY32RMMxSGJVMWZv/blP5I7f1mKus1bUIvV6Lab5jdEGbbWknY/d2jjdn",
|
||||
"bHL3odl9NUeDnBvLaeeYVtdbg2/abOB4irXOa+hbf+AbicKLIxdQFPn0Rqy1Y6e5kpfWgCettgSIkgmk",
|
||||
"i5TqhTjj3mhJdVjF45KuWEplPL9IizPB/wuV+bnbXdhEUjsCLuLRg6cKUTDcxoLIMCFAM4lyt8JCgASm",
|
||||
"jnvJFrh7B8KEtM1XcNgFcV+Ec+OovjJWpArDVjFAfSUwk0ZorfYUF1daEHDlyKAFhqN6lGR9QX+6fP8X",
|
||||
"dOlBtdF+13g5su2Ma+BGHxE58nQYNwdTvABRN/7loLC9QAW2JqlSaKxM+RyEQZ9R9qaMtEbSBEB3NfO1",
|
||||
"IrTAQl/ent02GxcNTEdrnTKrkTUmMAjM/VkISE3Q48e9Dlx3ujrEeCg30RFW8nEtfW08ZUcrwbwPQTnB",
|
||||
"3zHBVEYdQCuUdEgS2plk1h+3cTytR0iLP+Yg+NiVNqNH1NxF7C77jbaPrsAKP2BsRd2KUKMd3kt6d1h4",
|
||||
"E70gSsvzcQ+EUWnjBk7ZXaAKISxB8LOGgWOBiYRsi8AJd3/XjAluiVHSCrGLW/nOavHIm0N0nCmj6/i0",
|
||||
"4dPr/BbXYFN7Bpk+kN9u63jrzo4+gfOItHR5SyhF9inqr8pM9ok7LI5iEpMAaU7c/T18D+ihs4NrN+Nm",
|
||||
"wK6T1MW+1BOJm3WRu83AWRt5tDaQ1tn6zPGj+XiyqP1sB08wsdEXennZiJfGEq5vOGr/Lrj+YTTG6a37",
|
||||
"Tf84cu993MflWVtIRLLbOho3hOFu6wwNx1ergbM6xaqTVYDB9nqOirAElm1XZ53Gl1jRmpONczDnJggc",
|
||||
"Musi6/PCGq2PesnW4Ur1ZL6Nl4Oba20c3SuBi9lfCdytwhCyKTQDINal2nxwCJQzUsRcMJUdqs1sv6Nx",
|
||||
"KRKZGXGTkQwNrssnT55DsHU5jdTavAhLaZnB/7Y0acxHzkMN0Zg5xrMtgOPsfRGwKFGy1Oc8xZ3Z+lMo",
|
||||
"xYVZ1IyoiP96WcbkNpDaYjCG9tcGBq2+VO+6aGxwyYUQ8HkovK1Y93Y8jlvjsAQ0I8BIrs9in6Gll+wC",
|
||||
"vZT5QMMntaudba2xxcGwiwdTtqHqP27DFrfhMuQdACu/XBTuf2b8jmq+eU0i10pXOzVht5CNoly0MS5E",
|
||||
"YHbbKXCrXahhpChil8hrMp1RMp1p1Jg8Xh9h0omvbOx451hzRRSFNTuu+DDjaZnbsBFRsjHnt8YaNAep",
|
||||
"yLQte2qzvdkuIIbkt17Vf6mP7FWOqptio4aKrN0WuCW2FYicaClip5eDNTGiDXyaCJ6foE+Kn6BPDlTy",
|
||||
"BP3McA7ZwLBqgo6Pjz9+/vx5o3ub+LQpc6+sGKtr64jB+x0oQdJLEA7Ckctm0aZ45ebdlihCSsuiTkkC",
|
||||
"3/WS3tOZ/qclctBIg+suNjyfdmK/LfJBc3zfacqcsE7jtrEwhPyEpRIY+A5ZWCCfd7Dhs8vCpex0b4VL",
|
||||
"d514dGUGRUMqFlYhcFQQcF4hMraICxsJu51xoygoAdkei90Mq21C878IlZwhyu9AoDEvWZZoia2wQSQw",
|
||||
"t+/ZFOLh970ISptj4reyVuFKsXbINo7UYCDYS9wqKlivPPu1xAIzRVhbZPgWuaizRcHVDCT5u8098Iv3",
|
||||
"Kc9LBktzgYQxH9tTwNdBczd3Z4OSarqvh1SDlFYwX1OL14Vx1so2LN9eS3lptfxBIbiI3BQ/EqDZwGT0",
|
||||
"1cJxkB2O+t89e3bUHuVn3Hzx47lNc16CnZ0hjO9yqvh8nQ20E0s12CSUBK9DD495qU7GVMtjteCBUpDN",
|
||||
"mrf5zFp3y4VWPeRaG8Y6p/a7Ny8TlHIBMkEC56N8nKCMyNvRdJwgUiRIQV5QE4yYm5i2BGmxnaQgo0GJ",
|
||||
"XEbkxUtaTr0790Lw+5zfm8gwF3RVi1GI2jLiEWavyxyzgQCc6XMFOX9Ix8gv8116n578ApQuJoRt7yqn",
|
||||
"92mC5nmCuEAZT29BmHoomLB6aHV3Z7kD3gYctwWUVem43ewHIRowanYKhjH05qWJMhA4vUWFXwZhU/3L",
|
||||
"VICxv224JqLXcW9pCWu3fRk4cWnT+mSJ1Myag8CU2oMHkUlz4cHwubsFoMVZf1YKASxETbSGYEgFxTrJ",
|
||||
"0ax7lIOUeNrNeTwhjMjZ/vbnh7BhhwBUUTLnETOKWcCDvNVqZsvlqqDoYA3Ro9aekmuTBRwzenxZ9MRm",
|
||||
"adgmN5yzGx0flecvfli60nKBXTqbeMNpa2NH1kzQJqKay3tkypptox+QbKT4rrSzjBMLnaQyPruzsra2",
|
||||
"TSjqFuXVGTHrbebt+Nj4XjeLXxwgm2AQj/NJscgIw3TJdc0ZDBQfcBOW7X7JMdPEo/+rnvnfzMONtvOY",
|
||||
"5YNpoRT2C7FxlqROpUx622Zeb3w/Ho9RX1PzC0kD6lG8EXl75j2h8aSkUfXJCm3MIawqV+F/tIluIm85",
|
||||
"XUMSK6Za5WxRrjbhsgU/cfisbiSyjBhwXAm2VVI2hv6OLl+vI3W8VIVUIwnAtvLWTygu1imDM06zUcbv",
|
||||
"2L6xhNvWm6pCQ6wAP3Cm77hWv/W+KbkFuhiluOzI13mp9o6n5GlqhK71Bo8DhOlsFAUr06ELuMHprfcB",
|
||||
"eOOC+Y7etlVTbc5QJQp93OWSr2LsbeLkIYpe+fJWtRggJxutwHuZTZaoJ8rJt4TSvWxqmwNxTB7tqLIc",
|
||||
"dHyjc324bexjpdxTqF4TMGgz9Um2pcG/EDyFrBQx8dNHPWbICMJDGz8y9AEgQx8XVFDM0Ifngx+OVkKx",
|
||||
"vd9uFCKX2qq5MH0OentkvuIL37yReihSPO7Crbuzj92Q56XWKGLu0xXVbeepLFwPMVdEG5Ih8CeeCNnd",
|
||||
"XpoJPHGxCz6IIdhJBUyMTbamzG2IQ/bGUrE+7aSyZ+9mOl1JEQnG02AUrVig9Yy6dMrnchRqnmMWMZq8",
|
||||
"4sFYZsrQuTi5Xry8pKtluMw4RIU6WTF3esZLPcCYmWRc6OqedyIgXlxN6W2o1hRYihfdS6xopb8ZWy3l",
|
||||
"rJf4qjkmH5y1XLptd98WgI4Xvvn9k42lD9y6k4DuGJVceafUEgAZzzFdtEjTREAVVLZDBMmqwMk1x0Xt",
|
||||
"rsQ45ihhgAUqBP/FfjpBf8iQjfjb7Ets95tKymOa01v7uQlRqACBMrzY2ikY3HQVtKKBGRLSUksoJiHF",
|
||||
"VQsALECclrFcxfdOLRrOCdyBOEF6mJaebtH7Ny/P0J/+66oe70rY4PTiDfrXP/6JznCWLa7ZhIs7LLIB",
|
||||
"LtUMEZNtBUzCgLBBBoWaJYhxmxfmrDdaQhOlmh0dXzNTDPrEmAVJiuw6bTKpLZheZZ72TZEvdGMCpW/0",
|
||||
"u76guCEm82ZF7YaVTGlqI9O6mtcu+cGVJM8UF3w1qO3MFvAe6LsckN6sL7DyntxyiWY8B4rH6P3lMbrS",
|
||||
"4uWEUNAb10O+/TZs8pqZXX77LeqbmuA4VQMjFx6doFfceAxAIKnKsURYAKpK2t8RNUMcF2Sgj70psOSa",
|
||||
"2bxXifr+82dv3yRoUmqpBP30Rh5ZeBkw4xyQLCA9vmbX7IyzuUYnZzX55PnRyTUboHPrhdJf9wXD0U1b",
|
||||
"efKbY/3KWyKVRKUEdPPJ3NFJvcvC5xu7eNeaocBTwqzDq+8OGmRKzqPvnyQox/fo2ZMnR2ben5jEE0AX",
|
||||
"7y+vbPGTQqGbpXr8N6hvK/sXFC/QHWEZv7NvvyvNoYCEaz4hUYqFWKAbd9vdvECvzq9cTwCJbs6v8PQm",
|
||||
"QRenV2evkY/fQDe+xP4N6rvi/L4ov/1MqLlTwez58+c/oJ+uzszzcxeUZJ7iLBMgpVnXuBldivrNLhEG",
|
||||
"UVczQO/OLmw5kAlOAfWlEoBzM8Prq6uLBPHJhKQEU01Aly//fGRziEtmItEVuhnmaXFzzTirCGFMGBYL",
|
||||
"hFmmB/NSGTuq4SVL15plXZDQC0RMBiWnEt0JXFyzip6sfoxMSg3ChtqlVrOyghOmpOVHSlJwrhjHZBc2",
|
||||
"u1sf14I6xpQnw6HTvI+dJ3nossBrfsSeZbfTizc1qeWk9/T4yfETo+YWwHBBeie958dPjp9bJ/DMnHdD",
|
||||
"c0gMcK3IvLs0rRGIcPYm6530/k8JYtGsR9/sQfJzvJlBrST4mtYLLe82aojvMEE9dGPtyzHJudrcMHTw",
|
||||
"6DDW9XboMNL1bOkw0jam+PxxqcnDsydPtmpRsBRE6NWGTvpDE/URfaTeQGb7qmVmCZE7euXK8UtAwJSJ",
|
||||
"4/qcVIJZfAsBZrVmELVIVttlAY1hhufE1MgwZnY8lcambcNMx8SbXe8HfuG2lmbvpGfFATPrMJROaeUk",
|
||||
"fS2chlGdmChoHRUuD1cbvY15vBVn5ZN7V5z/zfFGaIryeGwRCGp/ftAEinCNQj0vVCWAtmGE4SeSfR6G",
|
||||
"1iMa1k2K34Dg0FJK47hwASJNlnppujMENCRbfmGpkZKlJRMN80eeLfYgo/qmQ1qrZVPLpYvAmVHVjPGW",
|
||||
"yN+W4jOv352eVQ0MrG7Ql4RNKQxKCQnyeVNOpB5IksHmMkVhG3FKbLZY+rwnI+7afeilWyQSkHKRQXaI",
|
||||
"m8HiyoToAFvUYemg2wrQriwTvG6WacrMxvuvkcHMkG6yV72I+S7Sl62E/5CCV5vYx9kub64k8P1H6Nvv",
|
||||
"Yqt6KDzi1aYX4cU91FcSvTy/PDs6BHubmbeX95o8azzI68W9MzukE9OuiF0daT/EdezArL6S+sqrtcy+",
|
||||
"3xZl2wYGj0fUjiIOJKwZEkQZTAhz6S8VQbsKjxsktjbJysZAWWg9oli1EZUuVquTPPL0sJ+Ootcmjh8A",
|
||||
"v3YmhB2O+/a0GWA5yLDCCfJmyj8cdcZ57PgyQvq+srkvD9MkIVM1ZkcKcm1CH5R0bFWbLyzJtlKObz25",
|
||||
"P+XYmYzsSqxp1RHRroTSqIyy4cJbGtvNyhHqGHxpidNXxl+1dWzVEOA3d0k2+1s84m25RE4HOFbdjKA1",
|
||||
"O6s4atlyBshFE/JSDvwTZNQypAQm9GhXe4jzSg1tCWODmmiyi68TKX2d4iS4uyTCDOEpoFtYFJiIxPXT",
|
||||
"NX9vLzybGI9GLTm2EfXFG+kNx+gMUwrC1kvEVADOFmiG56C/4YtYMHPtMMj06dLIjjCBXtbB0TwVbEXj",
|
||||
"M1+X/yFO82ax6S98oC9VbI61GzYjcmAqNNe2HkDbxIBlAWEHoG/7MYQRgztf3Phf//gnIlKW4GnI00+N",
|
||||
"dsISKip3hNtC4rbZXYPCP0laTj8P06pJSDQM44PzMN7NSDpzvUBM/4/EutUs2Zqy0rbfhm9vgUybD0PE",
|
||||
"UzIHhpT3NRovM0PeAWwafBivHWFSAc4Qn6ApUagoKY0R6StQzQYnK/dWbAuIM7pwi5NhcURW67JNpp8/",
|
||||
"f/7DUUtzfdu5ZOu23x8fUkRpQCJ2Kru2IBlQhQ9As69AOTJI6zM7iOIKnNsSZ7KTVHtJy2nv88cIZcuq",
|
||||
"a8l0bZHxgQkwMdZB84Z1Jme+8K6d9hu5cmK3E6ZvmPLgePcfiuD9slPvlQOpth5y65q8fFliyHwLmGGt",
|
||||
"GE5UEn4FaqVfzAMibuVbMSu5H4P84vfH03sGA8FLlg2UIIUJqdOCT4gFcm3+keA8NyFBqMBT2MPHWi9o",
|
||||
"06qC+ACTTWf4j4QqEKY+Qr1ZoivEJZEeDSzDTMm2w3tXC3vIGNz2xVC1des3fTnetS8usXs5tg9drR7O",
|
||||
"TGDO0AXJxr7y656m938rLam9qNoX0o5cTT5K5MFOXaiYJyg7tXJVOxsSz/319lVaEhvF/b+wKdFTUbst",
|
||||
"0dRKy8BWzji/wtO2Kd2woRnjJjyIDZKhFeVgA1U0LUh+8DCojO1q8JnTbOtNQCq10/W3mgddOeVMav3c",
|
||||
"xH3aqhRac05xgVOjAvuI76MEeUOWm926G6vqtCaiJaIzNxVlfQz63QWXe0ynCFUIvm7aX6kD8oXpf7VG",
|
||||
"RftJ58qyJk2E/FpC+ahsEraAMNKvlSqQbv/t/z1L0F/fJSjU+DhCZqAp2rEvP3njfZsUGkjvAc0fbceX",
|
||||
"w5mrB/R42HkVahkshxjvdMkdwE2ynPbge3TUTx0sINJ1pNY5pmoXkMHkxTUjlMIU08YkNpYbfffkBy3X",
|
||||
"mukG1fOjY3Rho/im+iPXzB6IWitdVK8+R31/ygW4HEXPO729Xc+6B/b31JvHfHH7YBuDOI9Pdbc+Foc4",
|
||||
"h1FV3kIzSK1TzFIXmYOcWkPT5XpQdbluO8L+qMd9sMM6eZNMQk1DDwngeW5KIZK8zHsn30dSuR5am1gO",
|
||||
"EixsslFLk9jOVZnaCiXZD2xd1maLCJ3JxNbZ9ai1du2pwMUMZcTVSDuEUdvnjPgPkok1BbmDfYIJlV/0",
|
||||
"OF8laB+AJjdfyKHQSjeKFkB3DuarEuGiHNEbc8MsoZKLSe4zJgbz5OPhTc/7qNzri7zvTMf1aQ/hZHxp",
|
||||
"gI5EfVrT9zx40voauiggRx49IvFa23YQqYdVLncbFS8XWHvA63P5UxHsXTT9kFC4Y8jt4wDyPac0XsTO",
|
||||
"mDqXhf4vhcqaadp045Xr44fP5649a4cT58HzrqJW0VrZkP8EAD+ybXPufCaPZdqc20zeAwb9viZScWGc",
|
||||
"3eBZYWdHxNxCy+SetroDL21qwCUwheyGjtE5Tmf2+99IdEOyG58VbXvgC36HSIb6AmSZwzUzB9nNWy0q",
|
||||
"mxkGb17eHCXoxoxeelcDNUE3GVY4PPnT5fu/XDPzKrLQPkavAQs1Bqz0uZUbOGvOW6Cn38tj9EeQagCT",
|
||||
"CRfGDUvMk3/945/XzNSVhgwVIAayHOudjkGgcTmZgEhQJngx4DQDqVwS9cXvj16YNOhX51fIweyaKY7G",
|
||||
"OL2dkLgr/tLAtO2wanXhBAigQsCE3O/rsbFKVvViAwVrZ9jMtgrulQXHoKKg9glX/bCX58i9eAiz/9wT",
|
||||
"kJ0T9S8vz4/2YY4qOmqtn64atmsu5INHyH8lGSn/XldHyBJ9xOujoq1DOcbq1Lp1HGDS4uy4mgGaYZZR",
|
||||
"EMveiX6I4TM0eJTYGF7p/BRDX/wwuWaYZQiImoFAwIw13F0LoRpz34azukThI8RFLYLwmoXMQWd7Mz4Q",
|
||||
"XwiiORNh6Ma3l7sJUX+nVHIE9+avPsjFRvQITsEEoNlwLDvd+7+8/Ru6wws7Ruotxq4C55E4r6cdf5Xu",
|
||||
"w+V2cF/ahVhx3BpW8N4T1M9tiVKXQR6cWIcQsj4EAqpTnyPtBfrX//v/VZqqTWzQf3JUu1WIbS3+sBq7",
|
||||
"2SFSo6WHM/l2wsch7GIBxK5t3O+Q66C52xm1lz2hiYShbQn5IFnfZ2bqx0flWeh6eQBXu5kLYeQP12Ho",
|
||||
"xYnqdRd2zC/WZ7NoTzA+N48vAbK9jTlLooMprMRtrFwTen87ffcW1VpvrdZoZYpTPt3lVXtHbv3ikrwR",
|
||||
"FpDU9hEm7yKHaIiicakv+IMcrj4hAEk9sd6NtDWtUtdC4OUffaf/lx/QELmSQD4Sr5EqtpAK8k7EY+z5",
|
||||
"605V08Rzk7J2qbAInth+3Q979ALxnChjTLubaYHBehD6toVRW/Sd4HwnoX6Ng+jZBgdRYmqUUlNo0Uqs",
|
||||
"ne313WuTSrUwtZ0mXOS91bC8pU6hv3DCvB9k5P6mxbeCFyU1El5osXrs+j0mXTbhPhPfQ6jJuNw5oeuu",
|
||||
"HjJ+vWopG+FI8xDNzdO9GfKyHFtK1ZQ7J7LElPzd1XIzPVDR75DpgbqDeV8zXtXjtI3zfqQA6rVH64OB",
|
||||
"tNmuNQJWO+CAscVmY65Vrp/WmvVNRzdEWKa3wsU+ZrxQaHsoAYu0HdKX5nHozdnNYPFrb1kJ2M8K8BXo",
|
||||
"9o3upIfzv70m6hCK+o8lpQOTP2LRaYu8BiRXXuq+FwFkglzHzwaLhle2oqFPwfvRISarTku/NXS+Ne1n",
|
||||
"K8AfomIHpUFuk0OPM2Qb3SLFozGqXdEY52bTaDbq2erO1EYzskVpN3jn3rlBnU6Wzm62Dvd8KJl7EFEl",
|
||||
"Lqf5lpmR+ANcKl6LP2g2ULUtPnZLuN7FifeYR2ujR+3heNFOiyQcqGaiIVaTkTewc6I8UO6u13Bd0Ws7",
|
||||
"Nd9XutgBUeQYSUNx27wP0yApwhIhWK77hI120LGO3LU4ki3XutLSaRNlNSAS/XRji11IMKDuMDc88uRi",
|
||||
"zMq13C19mdeW6/9WW22NRP2aulOnjZhZDAIC4ib9ekgtwtKlMI/yUjnFIBjeNefgQTCH3s2AoSrCdsUa",
|
||||
"Xk+kubLa5VecTKNX+JgJNZbY1xboefbkWQc6tEbyeqXPvY22SiswagYVJRvFxubs1wi6O7027TVRih1+",
|
||||
"0tdxrNRPRNpxKX5bCDqt4e2vschQBhSUKQDPuEKyLAouTBX3makL77rpSgT3RNp6BaEfSEjhtzEFL59H",
|
||||
"WKMWer4bZ3yR8HO9tEcMQW/jiFrhoUfiiFrBooD1KlRyH05wVYnXByJc+EHbyN57FNfcMaKgW/zDbymS",
|
||||
"wLfRf7w4gkAaB4oiKCpS8/RMwXWS2yyJ+LcPWrMt3ukESTwBtUBzTOfgjt7LV384OkanocC3Ps6LurSz",
|
||||
"Iupcftd2WF+EXvRf/qRukmRrqeVfSywwU6ZRVbQjz2rPq6rhf63ZVa2zlfEjhTExtfZxyywHhvsab4mr",
|
||||
"KiEJe0ZCfVdvHtAQVbBFw+omOerOa0t3h4uzCclvJYVuxf0/lBRk7yuoS68Xcsg0CbOvA9eZR6JsqmaV",
|
||||
"e3WHgKqzoFw1GhSEr70wcrf+JHLNIW0RJzxRIFbzuv3J97xVH2uA+ivVyOpr3EYnexQ2v7DxD04balAJ",
|
||||
"6mclpoOIM3stzXRg64eugroflTywdvJvSh6GIhx7H5IwXFzlOmPkqRtzCUoRNn3cs765lgMe92F3hyi4",
|
||||
"bheJpJsT9W8JpQN5R1Q6SxCDOYiBr7lqKtoc7XAlxGXaD5hIE+foF0EkqtMLhQz1nz15hn5XhUIeo7f8",
|
||||
"DkzxI6Js+oNbOrqZUj7G9FhPN8KpOkHXPT6ZXPdutAaLMxtTabc08oPQLbgsCn/tkDyHjGAFdKG//uTo",
|
||||
"xFxNNbDYUpxmHnSHXXwMZuuLjpjTJkaeu50bejv6EaYXDRpt8ww9nNz6dfLIqcGmTdhRgthQ7UcUksPx",
|
||||
"6GndV6dsnJAvGmT2/scfNUsEgtzv/BRE3g5MwO8GafkDkbdnbtxjphT7ZRxSUCbyFnkYHEheFvU5tzwa",
|
||||
"NXoaycj2kKRgVd/lgr1ZI826W3KOaSm5bcDL2mSdvWeqheFtZcheIsXvYhVsa24mYIfpOXTOMi3V1Kfu",
|
||||
"S1DSloEZKW51FxPJQiS6hcLeCDOT17g42qUsR5sade5aVlofWqQQzapfEPVnBAQW6WwxwHdYwNELlGKR",
|
||||
"EYapbds34SKFrE2RWk9zX4ciVV/j4zi3mgUQvkj/iQZFuoilneq/+J4D6y6FSzemc0YgHCzTPPTUZhPe",
|
||||
"S3p3zlSU9FJBFEmjvcYfJA++SyOg35KZ3+L8Ea38nugOVbk40PB2rXhqPGJTaHB6+yD5M6fprYN5HOvr",
|
||||
"d25fPVy7ktO0itDEDng7dippQC8vrXRzcPC9KxXU4HcIJ4Re66hkitCuFeBbe0Qud8SvZt6jieOXpQgN",
|
||||
"4EAKrsTK1dXbQxCFAMnp/GHo4oOd+8Ck0Y7mFWR+FchzUKjwl2NWYkoXu6JPq6obhAY7pFtrTGt+GZmo",
|
||||
"0/947x/yWtdYecxb3VLFoS51Mxvqm5QqFRLrChD20dFuLn077UO7HywqvmJfe0EYg2zkoBqvibjqbteY",
|
||||
"aPW1f33edccQX71v3dCkafBE9K8eKTu60WsUPnRTdTjN/+pHfn0H2M4HUtjT/vhyU3nbj6ktaPG2/TG0",
|
||||
"Z96/SceTXZOfrszorY+if7dkDrPNA5KOA9shGB1YhjDDdCGJK19Iqc/hMJXJI4lU2yR0PGQyld4KpKUx",
|
||||
"3eipx4AFiNNSzXonP3/UGLfd2O2HS0F7J70hLshw/tTQg9vPatsml9zv8s5DXoEpOWtK4dRt581t2LSa",
|
||||
"lTgU23kNQuu3pOptRaSts0w4S3ybo1rhKNfLaHXO8+1SHdx8vMq++BS3e5gtuuJCfYtq4wNqdOeMLijU",
|
||||
"oKh6K7hOjUnwUkrUzyAlGQxxqmrTQr1E06eWuEuztCB66fOsNkM431bfrztgkqVQoyQ4x6qpnBdldaKQ",
|
||||
"IelIw+UJV7a6Wo7jp2jqlUxsxrL5bkZU4qoPJihk43tMNbgsBu6CC7X6nqvk8Pnj5/8OAAD//2XM3suJ",
|
||||
"7gAA",
|
||||
}
|
||||
|
||||
// GetSwagger returns the content of the embedded swagger specification file
|
||||
|
||||
@@ -75,22 +75,31 @@ func (s *Server) resolveEntityID(ctx context.Context, idOrSlug string) (uuid.UUI
|
||||
return id, err
|
||||
}
|
||||
|
||||
// entityCols requires the entities table to be aliased as `e`.
|
||||
// entityCols requires the entities table to be aliased as `e`, with
|
||||
// entity_status left-joined and aliased as `st` (see withEntityStatus).
|
||||
const entityCols = `e.id, e.slug, e.type, e.name, e.state, e.attributes,
|
||||
e.maintenance_until, e.version, e.created_at, e.updated_at`
|
||||
e.maintenance_until, e.version, e.created_at, e.updated_at,
|
||||
st.health, st.last_check_at`
|
||||
|
||||
func scanEntity(row pgx.Row) (gen.Entity, error) {
|
||||
var e gen.Entity
|
||||
var state *string
|
||||
var attrsJSON []byte
|
||||
var maint *time.Time
|
||||
var health *string
|
||||
var lastCheckAt *time.Time
|
||||
err := row.Scan(&e.Id, &e.Slug, &e.Type, &e.Name, &state, &attrsJSON,
|
||||
&maint, &e.Version, &e.CreatedAt, &e.UpdatedAt)
|
||||
&maint, &e.Version, &e.CreatedAt, &e.UpdatedAt, &health, &lastCheckAt)
|
||||
if err != nil {
|
||||
return e, err
|
||||
}
|
||||
e.State = state
|
||||
e.MaintenanceUntil = maint
|
||||
if health != nil {
|
||||
h := gen.EntityHealth(*health)
|
||||
e.Health = &h
|
||||
}
|
||||
e.LastCheckAt = lastCheckAt
|
||||
var attrs map[string]any
|
||||
if len(attrsJSON) > 0 && json.Unmarshal(attrsJSON, &attrs) == nil && len(attrs) > 0 {
|
||||
e.Attributes = &attrs
|
||||
@@ -113,6 +122,7 @@ func (s *Server) ListEntities(ctx context.Context, req gen.ListEntitiesRequestOb
|
||||
)
|
||||
SELECT ` + entityCols + ` FROM entities e
|
||||
JOIN entity_types et ON et.name = e.type
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.type IN (SELECT name FROM tt)
|
||||
AND ($2::text IS NULL OR e.state = $2)
|
||||
AND ($3::text IS NULL OR et.domain = $3)
|
||||
@@ -159,7 +169,7 @@ func (s *Server) GetEntity(ctx context.Context, req gen.GetEntityRequestObject)
|
||||
return nil, err
|
||||
}
|
||||
e, err := scanEntity(s.pool.QueryRow(ctx,
|
||||
"SELECT "+entityCols+" FROM entities e WHERE e.id = $1", id))
|
||||
"SELECT "+entityCols+" FROM entities e LEFT JOIN entity_status st ON st.entity_id = e.id WHERE e.id = $1", id))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -231,6 +241,7 @@ func (s *Server) GetBlastRadius(ctx context.Context, req gen.GetBlastRadiusReque
|
||||
SELECT `+entityCols+`, b.depth
|
||||
FROM blast_radius($1, $2) b
|
||||
JOIN entities e ON e.id = b.entity_id
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
ORDER BY b.depth, e.slug`, id, depth)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -246,13 +257,20 @@ func (s *Server) GetBlastRadius(ctx context.Context, req gen.GetBlastRadiusReque
|
||||
var state *string
|
||||
var attrsJSON []byte
|
||||
var maint *time.Time
|
||||
var health *string
|
||||
var lastCheckAt *time.Time
|
||||
var d int
|
||||
if err := rows.Scan(&e.Id, &e.Slug, &e.Type, &e.Name, &state, &attrsJSON,
|
||||
&maint, &e.Version, &e.CreatedAt, &e.UpdatedAt, &d); err != nil {
|
||||
&maint, &e.Version, &e.CreatedAt, &e.UpdatedAt, &health, &lastCheckAt, &d); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
e.State = state
|
||||
e.MaintenanceUntil = maint
|
||||
if health != nil {
|
||||
h := gen.EntityHealth(*health)
|
||||
e.Health = &h
|
||||
}
|
||||
e.LastCheckAt = lastCheckAt
|
||||
var attrs map[string]any
|
||||
if len(attrsJSON) > 0 && json.Unmarshal(attrsJSON, &attrs) == nil && len(attrs) > 0 {
|
||||
e.Attributes = &attrs
|
||||
@@ -283,10 +301,13 @@ func (s *Server) GetGraph(ctx context.Context, req gen.GetGraphRequestObject) (g
|
||||
nodes, err = s.queryEntities(ctx, `
|
||||
SELECT `+entityCols+`
|
||||
FROM blast_radius($1, $2, $3) b JOIN entities e ON e.id = b.entity_id
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
ORDER BY e.slug`, rootID, depth, req.Params.RelType)
|
||||
} else {
|
||||
nodes, err = s.queryEntities(ctx, `
|
||||
SELECT `+entityCols+` FROM entities e ORDER BY e.slug LIMIT $1`,
|
||||
SELECT `+entityCols+` FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
ORDER BY e.slug LIMIT $1`,
|
||||
graphNodeCap+1)
|
||||
if err == nil && len(nodes) > graphNodeCap {
|
||||
nodes = nodes[:graphNodeCap]
|
||||
@@ -322,9 +343,44 @@ func (s *Server) GetGraph(ctx context.Context, req gen.GetGraphRequestObject) (g
|
||||
if truncated {
|
||||
resp.Truncated = &truncated
|
||||
}
|
||||
|
||||
if req.Params.Include != nil {
|
||||
for _, inc := range *req.Params.Include {
|
||||
if inc == gen.Status {
|
||||
health, herr := s.entityHealthByID(ctx, ids)
|
||||
if herr != nil {
|
||||
return nil, herr
|
||||
}
|
||||
resp.Health = &health
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
// entityHealthByID returns entity_status.health keyed by entity id, for the
|
||||
// given id set (used by GetGraph's include=status).
|
||||
func (s *Server) entityHealthByID(ctx context.Context, ids []uuid.UUID) (map[string]gen.GraphViewHealth, error) {
|
||||
health := make(map[string]gen.GraphViewHealth, len(ids))
|
||||
rows, err := s.pool.Query(ctx,
|
||||
`SELECT entity_id, health FROM entity_status WHERE entity_id = ANY($1)`, ids)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
for rows.Next() {
|
||||
var id uuid.UUID
|
||||
var h string
|
||||
if err := rows.Scan(&id, &h); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
health[id.String()] = gen.GraphViewHealth(h)
|
||||
}
|
||||
return health, rows.Err()
|
||||
}
|
||||
|
||||
func (s *Server) queryEntities(ctx context.Context, query string, args ...any) ([]gen.Entity, error) {
|
||||
rows, err := s.pool.Query(ctx, query, args...)
|
||||
if err != nil {
|
||||
@@ -488,14 +544,18 @@ func (s *Server) GetFleetHealth(ctx context.Context, req gen.GetFleetHealthReque
|
||||
Type string `json:"type"`
|
||||
}{}
|
||||
|
||||
// Exclude 'check' entities (internal probes) — only entities actually
|
||||
// being monitored should count toward fleet health.
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT e.slug, e.type, st.health, st.last_check_at
|
||||
FROM entity_status st JOIN entities e ON e.id = st.entity_id
|
||||
WHERE e.type <> 'check'
|
||||
ORDER BY e.slug`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
stale := 0
|
||||
for rows.Next() {
|
||||
var slug, typ, health string
|
||||
var lastCheck *time.Time
|
||||
@@ -509,6 +569,8 @@ func (s *Server) GetFleetHealth(ctx context.Context, req gen.GetFleetHealthReque
|
||||
resp.Summary.Degraded++
|
||||
case "down":
|
||||
resp.Summary.Down++
|
||||
case "stale":
|
||||
stale++
|
||||
default:
|
||||
resp.Summary.Unknown++
|
||||
}
|
||||
@@ -525,6 +587,9 @@ func (s *Server) GetFleetHealth(ctx context.Context, req gen.GetFleetHealthReque
|
||||
Type: typ,
|
||||
})
|
||||
}
|
||||
if stale > 0 {
|
||||
resp.Summary.Stale = &stale
|
||||
}
|
||||
return resp, rows.Err()
|
||||
}
|
||||
|
||||
@@ -920,6 +985,8 @@ func (s *Server) CreateEntity(ctx context.Context, req gen.CreateEntityRequestOb
|
||||
return nil, eventErr
|
||||
}
|
||||
|
||||
ensureDefaultChecks(ctx, tx, inserted.ID, slug, req.Body.Type, attrsJSON)
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -1184,6 +1251,8 @@ func (s *Server) EnrollClient(ctx context.Context, req gen.EnrollClientRequestOb
|
||||
"info", "oikos-api", "",
|
||||
map[string]any{"slug": req.Body.Slug, "type": current.Type})
|
||||
|
||||
ensureDefaultChecks(ctx, tx, id, req.Body.Slug, current.Type, attrsJSON)
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@@ -122,6 +122,16 @@ func resolveHostSSH(ctx context.Context, pool *db.Pool, entitySlug string) (stri
|
||||
|
||||
// executeApprovedAction runs a gated action after operator approval.
|
||||
// Runs in a background goroutine to not block the HTTP response.
|
||||
// emitExecutionEvent records an execution lifecycle event for SSE fan-out so
|
||||
// the control room can watch approved actions run to completion live.
|
||||
func emitExecutionEvent(ctx context.Context, pool *db.Pool, execID uuid.UUID, status string, detail map[string]any) {
|
||||
severity := "info"
|
||||
if status == "failed" {
|
||||
severity = "warning"
|
||||
}
|
||||
_ = observability.Event(ctx, sqlcgen.New(pool), "execution."+status, &execID, severity, "actuator", "", detail)
|
||||
}
|
||||
|
||||
func executeApprovedAction(ctx context.Context, pool *db.Pool, execID uuid.UUID, targetSlug string, actionStr string) {
|
||||
slog.Info("httpapi: executing approved action", "execution_id", execID, "target", targetSlug, "action", actionStr)
|
||||
|
||||
@@ -130,6 +140,7 @@ func executeApprovedAction(ctx context.Context, pool *db.Pool, execID uuid.UUID,
|
||||
slog.Error("httpapi: resolve host for approved execution", "error", err, "target", targetSlug)
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, fmt.Sprintf(`{"error":"%s"}`, err.Error()))
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": err.Error()})
|
||||
return
|
||||
}
|
||||
|
||||
@@ -186,6 +197,10 @@ func executeApprovedAction(ctx context.Context, pool *db.Pool, execID uuid.UUID,
|
||||
pool.Exec(ctx, `UPDATE executions SET status=$2, result=$3::jsonb, duration_ms=$4, verified=$5, started_at=$6, completed_at=$7 WHERE entity_id=$1`,
|
||||
execID, status, result, durationMs, verified, startedAt, time.Now())
|
||||
|
||||
emitExecutionEvent(ctx, pool, execID, status, map[string]any{
|
||||
"action": action, "target": targetSlug, "duration_ms": durationMs,
|
||||
})
|
||||
|
||||
slog.Info("httpapi: approved action executed",
|
||||
"execution_id", execID, "action", action, "status", status, "duration_ms", durationMs)
|
||||
}
|
||||
@@ -302,7 +317,7 @@ func (s *Server) CreateCheck(ctx context.Context, req gen.CreateCheckRequestObje
|
||||
entity, err := q.InsertEntity(ctx, sqlcgen.InsertEntityParams{
|
||||
ID: id,
|
||||
Slug: slug,
|
||||
Type: "check_def",
|
||||
Type: "check",
|
||||
Name: slug,
|
||||
Attributes: []byte("{}"),
|
||||
})
|
||||
@@ -947,6 +962,12 @@ func (s *Server) DecideApproval(ctx context.Context, req gen.DecideApprovalReque
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
// Emit for SSE fan-out (in-tx; NOTIFY fires post-commit).
|
||||
if evErr := observability.Event(ctx, q, "approval.decided", &id, "info", "api", "",
|
||||
map[string]any{"decision": status, "actor": actor}); evErr != nil {
|
||||
return nil, evErr
|
||||
}
|
||||
|
||||
// On approve: execute the linked gated command.
|
||||
if status == "approved" {
|
||||
var execID, targetID uuid.UUID
|
||||
@@ -1814,10 +1835,6 @@ func (s *Server) QueryMetrics(ctx context.Context, req gen.QueryMetricsRequestOb
|
||||
if req.Params.EntityId == nil || *req.Params.EntityId == "" {
|
||||
return nil, fmt.Errorf("%w: entity_id is required", domain.ErrInvalidInput)
|
||||
}
|
||||
if req.Params.Metric == nil || len(*req.Params.Metric) == 0 {
|
||||
return nil, fmt.Errorf("%w: metric is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
entityID, err := s.resolveEntityID(ctx, *req.Params.EntityId)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -1832,8 +1849,33 @@ func (s *Server) QueryMetrics(ctx context.Context, req gen.QueryMetricsRequestOb
|
||||
to = *req.Params.To
|
||||
}
|
||||
|
||||
var metricNames []string
|
||||
if req.Params.Metric != nil && len(*req.Params.Metric) > 0 {
|
||||
metricNames = *req.Params.Metric
|
||||
} else {
|
||||
// metric omitted: report every metric recorded for this entity in range.
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT DISTINCT metric FROM metric_samples
|
||||
WHERE entity_id = $1 AND ts >= $2 AND ts <= $3
|
||||
ORDER BY metric`, entityID, from, to)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for rows.Next() {
|
||||
var name string
|
||||
if err := rows.Scan(&name); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
metricNames = append(metricNames, name)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
items := []gen.MetricSeries{}
|
||||
for _, metricName := range *req.Params.Metric {
|
||||
for _, metricName := range metricNames {
|
||||
series := gen.MetricSeries{
|
||||
EntityId: entityID.String(),
|
||||
Metric: metricName,
|
||||
|
||||
@@ -15,6 +15,9 @@ import (
|
||||
"log/slog"
|
||||
"math/big"
|
||||
"net/http"
|
||||
"net/http/httputil"
|
||||
"net/url"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
@@ -66,7 +69,7 @@ type secretsBackend interface {
|
||||
// holds a dedicated pooled connection for LISTEN. Callers MUST cancel ctx
|
||||
// before closing the pool — otherwise the held connection never releases
|
||||
// and pool.Close() deadlocks.
|
||||
func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Handler {
|
||||
func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config, uiHandler http.Handler) http.Handler {
|
||||
s := &Server{
|
||||
pool: pool,
|
||||
cfg: cfg,
|
||||
@@ -137,16 +140,34 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Hand
|
||||
r.With(combinedAuth(cfg)).Get("/api/v1/events/stream", s.serveSSE)
|
||||
|
||||
// Mount MCP at /mcp (plan R3-10)
|
||||
hermesAgentID := uuid.Nil
|
||||
if cfg.HermesAgentID != "" {
|
||||
if id, err := uuid.Parse(cfg.HermesAgentID); err == nil {
|
||||
hermesAgentID = id
|
||||
nomosAgentID := uuid.Nil
|
||||
if cfg.NomosAgentID != "" {
|
||||
if id, err := uuid.Parse(cfg.NomosAgentID); err == nil {
|
||||
nomosAgentID = id
|
||||
}
|
||||
}
|
||||
if hermesAgentID == uuid.Nil && cfg.HermesAgentSlug != "" {
|
||||
_ = pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", cfg.HermesAgentSlug).Scan(&hermesAgentID)
|
||||
if nomosAgentID == uuid.Nil && cfg.NomosAgentSlug != "" {
|
||||
_ = pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", cfg.NomosAgentSlug).Scan(&nomosAgentID)
|
||||
}
|
||||
r.With(combinedAuth(cfg)).Handle("/mcp", mcphandler.NewHandler(pool, cfg.MCPBearerToken, nomosAgentID))
|
||||
|
||||
r.Get("/ui/*", func(w http.ResponseWriter, req *http.Request) {
|
||||
if uiHandler != nil {
|
||||
uiHandler.ServeHTTP(w, req)
|
||||
}
|
||||
})
|
||||
r.Get("/ui", func(w http.ResponseWriter, req *http.Request) {
|
||||
http.Redirect(w, req, "/ui/", http.StatusMovedPermanently)
|
||||
})
|
||||
r.Get("/", func(w http.ResponseWriter, req *http.Request) {
|
||||
http.Redirect(w, req, "/ui/", http.StatusMovedPermanently)
|
||||
})
|
||||
|
||||
if nomosURL := os.Getenv("NOMOS_PROXY_URL"); nomosURL != "" {
|
||||
target, _ := url.Parse(nomosURL)
|
||||
proxy := httputil.NewSingleHostReverseProxy(target)
|
||||
r.Mount("/agent", http.StripPrefix("/agent", proxy))
|
||||
}
|
||||
r.With(combinedAuth(cfg)).Handle("/mcp", mcphandler.NewHandler(pool, cfg.MCPBearerToken, hermesAgentID))
|
||||
|
||||
return r
|
||||
}
|
||||
@@ -484,10 +505,10 @@ func requestLogger(next http.Handler) http.Handler {
|
||||
|
||||
// ListenAndServe runs the API server with graceful shutdown on ctx cancel
|
||||
// (SG4): stop accepting, drain in-flight for up to 30s, then exit.
|
||||
func ListenAndServe(ctx context.Context, pool *db.Pool, cfg config.Config) error {
|
||||
func ListenAndServe(ctx context.Context, pool *db.Pool, cfg config.Config, uiHandler http.Handler) error {
|
||||
srv := &http.Server{
|
||||
Addr: cfg.APIListen,
|
||||
Handler: NewHandler(ctx, pool, cfg),
|
||||
Handler: NewHandler(ctx, pool, cfg, uiHandler),
|
||||
ReadHeaderTimeout: 10 * time.Second,
|
||||
}
|
||||
|
||||
|
||||
@@ -14,6 +14,8 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/google/jsonschema-go/jsonschema"
|
||||
"github.com/google/uuid"
|
||||
"github.com/modelcontextprotocol/go-sdk/mcp"
|
||||
@@ -37,7 +39,7 @@ func objSchema(props ...prop) *jsonschema.Schema {
|
||||
}
|
||||
|
||||
// NewHandler creates an http.Handler that serves the Oikos MCP server.
|
||||
// agentID is the Hermes agent entity UUID; tool calls are logged to agent_activity.
|
||||
// agentID is the Nomos agent entity UUID; tool calls are logged to agent_activity.
|
||||
func NewHandler(pool *db.Pool, token string, agentID uuid.UUID) http.Handler {
|
||||
s := newServer(pool, agentID)
|
||||
handler := mcp.NewStreamableHTTPHandler(func(r *http.Request) *mcp.Server {
|
||||
@@ -123,6 +125,7 @@ func newServer(pool *db.Pool, agentID uuid.UUID) *mcp.Server {
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, st.health, st.last_check_at
|
||||
FROM entity_status st JOIN entities e ON e.id = st.entity_id
|
||||
WHERE e.type <> 'check'
|
||||
ORDER BY e.slug`), nil
|
||||
})
|
||||
|
||||
@@ -258,7 +261,7 @@ func newServer(pool *db.Pool, agentID uuid.UUID) *mcp.Server {
|
||||
nStr(args["status"])), nil
|
||||
})
|
||||
|
||||
register(&mcp.Tool{Name: "request_execution", Description: "Request a gated execution (Hermes-only mutation path). Supported actions: restart, systemctl, pct_exec, apt_upgrade.",
|
||||
register(&mcp.Tool{Name: "request_execution", Description: "Request a gated execution (agent-only mutation path). Supported actions: restart, systemctl, pct_exec, apt_upgrade.",
|
||||
InputSchema: objSchema(
|
||||
prop{"target", "string", "Target entity slug (e.g. lxc:caddy)"},
|
||||
prop{"action", "string", "Action: restart, systemctl, pct_exec, apt_upgrade"},
|
||||
@@ -968,13 +971,27 @@ func resolveHost(ctx context.Context, pool *db.Pool, entitySlug string) (hostIP
|
||||
}
|
||||
|
||||
func createApproval(ctx context.Context, pool *db.Pool, execID, targetID uuid.UUID, action, params, riskClass string) {
|
||||
approvalID, _ := uuid.NewV7()
|
||||
payload := fmt.Sprintf(`{"action":"%s","params":"%s","execution_id":"%s"}`, action, params, execID)
|
||||
pool.Exec(ctx, `
|
||||
// approvals.entity_id is PK + FK to entities(id). Reuse the execution's
|
||||
// entity (already inserted by request_execution) so the FK is satisfied —
|
||||
// a fresh UUID here had no matching entities row, so the INSERT silently
|
||||
// failed, orphaning the execution and never alerting the operator. One
|
||||
// execution maps to at most one approval, so the 1:1 identity holds.
|
||||
if _, err := pool.Exec(ctx, `
|
||||
INSERT INTO approvals (entity_id, subject_entity_id, action, risk_class,
|
||||
kind, payload, status, expires_at, created_at)
|
||||
VALUES ($1, $2, $3, $4, 'execution', $5::jsonb, 'pending',
|
||||
now() + interval '1 hour', now())`,
|
||||
approvalID, targetID, action, riskClass, payload)
|
||||
pool.Exec(ctx, `UPDATE executions SET approval_id = $2 WHERE entity_id = $1`, execID, approvalID)
|
||||
execID, targetID, action, riskClass, payload); err != nil {
|
||||
slog.Error("createApproval: insert approval", "error", err, "execution", execID)
|
||||
return
|
||||
}
|
||||
if _, err := pool.Exec(ctx, `UPDATE executions SET approval_id = $1 WHERE entity_id = $1`, execID); err != nil {
|
||||
slog.Error("createApproval: link approval to execution", "error", err, "execution", execID)
|
||||
}
|
||||
|
||||
// Emit for SSE fan-out — the operator-facing moment: an agent-requested
|
||||
// gated action is now awaiting a decision.
|
||||
_ = observability.Event(ctx, sqlcgen.New(pool), "approval.created", &execID, "warning", "mcp", "",
|
||||
map[string]any{"action": action, "params": params, "risk_class": riskClass})
|
||||
}
|
||||
@@ -7,20 +7,31 @@ import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net"
|
||||
"net/http"
|
||||
"os/exec"
|
||||
"regexp"
|
||||
"runtime"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/config"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/google/uuid"
|
||||
"golang.org/x/sync/errgroup"
|
||||
"golang.org/x/sys/unix"
|
||||
)
|
||||
|
||||
var (
|
||||
sshKeyPath string
|
||||
sshUser string
|
||||
)
|
||||
|
||||
// Run starts the scheduler loop. Blocks until ctx is cancelled.
|
||||
func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
|
||||
slog.Info("scheduler: starting", "interval", cfg.SchedulerInterval)
|
||||
@@ -29,6 +40,12 @@ func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
|
||||
interval = 30 * time.Second
|
||||
}
|
||||
|
||||
sshKeyPath = cfg.SSHKeyPath
|
||||
sshUser = cfg.SSHUser
|
||||
if sshUser == "" {
|
||||
sshUser = "root"
|
||||
}
|
||||
|
||||
ticker := time.NewTicker(interval)
|
||||
defer ticker.Stop()
|
||||
|
||||
@@ -77,82 +94,129 @@ func runCheckPass(ctx context.Context, pool *db.Pool) {
|
||||
}
|
||||
|
||||
// runCheck executes a single check and processes the result.
|
||||
//
|
||||
// check_defs.entity_id identifies the *check* (probe) entity itself;
|
||||
// check_defs.target_id identifies the entity actually being observed (the
|
||||
// host/service/etc). Health, metrics, and events must attach to the target
|
||||
// so the observed entity's own record reflects reality — not the internal
|
||||
// probe. Signals stay keyed by the check entity (cd.EntityID), matching how
|
||||
// they are created below and resolved elsewhere.
|
||||
func runCheck(ctx context.Context, pool *db.Pool, cd sqlcgen.ListEnabledCheckDefsRow) {
|
||||
q := sqlcgen.New(pool)
|
||||
start := time.Now()
|
||||
|
||||
health, signalKind, evidence, checkErr := executeCheck(ctx, cd)
|
||||
result := executeCheck(ctx, cd)
|
||||
|
||||
latency := time.Since(start).Milliseconds()
|
||||
|
||||
// Write metric
|
||||
_ = q.InsertMetricSample(ctx, sqlcgen.InsertMetricSampleParams{
|
||||
EntityID: cd.EntityID,
|
||||
Metric: "probe_latency_ms",
|
||||
Value: float64(latency),
|
||||
Tags: []byte(`{}`),
|
||||
})
|
||||
if result.metrics == nil {
|
||||
result.metrics = make(map[string]float64)
|
||||
}
|
||||
result.metrics["probe_latency_ms"] = float64(latency)
|
||||
|
||||
if checkErr != nil {
|
||||
slog.Warn("scheduler: check failed",
|
||||
"entity", cd.EntitySlug, "kind", cd.Kind, "error", checkErr)
|
||||
targetID := cd.EntityID
|
||||
if cd.TargetID != nil {
|
||||
targetID = *cd.TargetID
|
||||
}
|
||||
|
||||
if signalKind == "" || health == "healthy" {
|
||||
// Recovery: resolve any open signal for this check
|
||||
resolveSignal(ctx, pool, cd.EntityID, cd.EntitySlug)
|
||||
// Update entity_status to healthy
|
||||
for metric, value := range result.metrics {
|
||||
_ = q.InsertMetricSample(ctx, sqlcgen.InsertMetricSampleParams{
|
||||
EntityID: targetID,
|
||||
Metric: metric,
|
||||
Value: value,
|
||||
Tags: []byte(`{}`),
|
||||
})
|
||||
}
|
||||
|
||||
if result.err != nil {
|
||||
slog.Warn("scheduler: check failed",
|
||||
"entity", cd.EntitySlug, "kind", cd.Kind, "error", result.err)
|
||||
}
|
||||
|
||||
prevHealth := currentHealth(ctx, pool, targetID)
|
||||
|
||||
if result.signalKind == "" || result.health == "healthy" {
|
||||
resolveSignal(ctx, pool, cd.EntityID, targetID, cd.EntitySlug)
|
||||
_ = q.UpsertEntityStatus(ctx, sqlcgen.UpsertEntityStatusParams{
|
||||
EntityID: cd.EntityID,
|
||||
EntityID: targetID,
|
||||
Health: "healthy",
|
||||
LastCheckAt: &[]time.Time{time.Now()}[0],
|
||||
Details: []byte(`{}`),
|
||||
})
|
||||
if prevHealth != "" && prevHealth != "healthy" {
|
||||
emitSchedulerEvent(ctx, pool, "health.changed", targetID, "info",
|
||||
map[string]any{"slug": cd.EntitySlug, "from": prevHealth, "to": "healthy"})
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// Failure: upsert signal (dedup via partial unique index)
|
||||
slog.Warn("scheduler: raising signal",
|
||||
"entity", cd.EntitySlug, "kind", signalKind, "evidence", evidence)
|
||||
"entity", cd.EntitySlug, "kind", result.signalKind, "evidence", result.evidence)
|
||||
|
||||
severity := "warning"
|
||||
if signalKind == "down" {
|
||||
severity = "critical"
|
||||
}
|
||||
severity := evaluateSeverity(cd.Kind, result.signalKind, cd.Config, result.metrics)
|
||||
|
||||
sig, err := q.UpsertSignal(ctx, sqlcgen.UpsertSignalParams{
|
||||
EntityID: cd.EntityID,
|
||||
Kind: signalKind,
|
||||
Kind: result.signalKind,
|
||||
Severity: severity,
|
||||
TargetEntityID: cd.TargetID,
|
||||
Evidence: &evidence,
|
||||
Evidence: &result.evidence,
|
||||
})
|
||||
if err != nil {
|
||||
slog.Error("scheduler: upsert signal", "error", err)
|
||||
return
|
||||
}
|
||||
|
||||
// Update entity_status
|
||||
_ = q.UpsertEntityStatus(ctx, sqlcgen.UpsertEntityStatusParams{
|
||||
EntityID: cd.EntityID,
|
||||
Health: health,
|
||||
EntityID: targetID,
|
||||
Health: result.health,
|
||||
LastCheckAt: &[]time.Time{time.Now()}[0],
|
||||
Details: []byte(`{}`),
|
||||
})
|
||||
_ = sig // used for flap detection below
|
||||
_ = sig
|
||||
|
||||
if prevHealth == "" || prevHealth == "healthy" {
|
||||
emitSchedulerEvent(ctx, pool, "signal.raised", targetID, severity,
|
||||
map[string]any{"slug": cd.EntitySlug, "kind": result.signalKind, "evidence": result.evidence})
|
||||
}
|
||||
if prevHealth != result.health {
|
||||
emitSchedulerEvent(ctx, pool, "health.changed", targetID, severity,
|
||||
map[string]any{"slug": cd.EntitySlug, "from": prevHealth, "to": result.health})
|
||||
}
|
||||
}
|
||||
|
||||
// resolveSignal resolves any open signal for the given check entity.
|
||||
func resolveSignal(ctx context.Context, pool *db.Pool, entityID uuid.UUID, slug string) {
|
||||
// currentHealth reads the last recorded health for an entity, or "" if none.
|
||||
func currentHealth(ctx context.Context, pool *db.Pool, entityID uuid.UUID) string {
|
||||
var health string
|
||||
if err := pool.QueryRow(ctx,
|
||||
`SELECT health FROM entity_status WHERE entity_id = $1`, entityID).Scan(&health); err != nil {
|
||||
return ""
|
||||
}
|
||||
return health
|
||||
}
|
||||
|
||||
// emitSchedulerEvent records a scheduler-sourced event for SSE fan-out.
|
||||
func emitSchedulerEvent(ctx context.Context, pool *db.Pool, eventType string, entityID uuid.UUID, severity string, data map[string]any) {
|
||||
_ = observability.Event(ctx, sqlcgen.New(pool), eventType, &entityID, severity, "scheduler", "", data)
|
||||
}
|
||||
|
||||
// resolveSignal resolves any open signal raised by the given check entity.
|
||||
// checkID matches how signals are keyed (UpsertSignal uses the check's own
|
||||
// entity id); targetID is the observed entity whose status this affects.
|
||||
func resolveSignal(ctx context.Context, pool *db.Pool, checkID, targetID uuid.UUID, slug string) {
|
||||
q := sqlcgen.New(pool)
|
||||
// Check if there's an open signal on this entity
|
||||
_, err := pool.Exec(ctx, `UPDATE signals SET state = 'resolved', updated_at = now()
|
||||
WHERE entity_id = $1 AND state = 'raised'`, entityID)
|
||||
tag, err := pool.Exec(ctx, `UPDATE signals SET state = 'resolved', updated_at = now()
|
||||
WHERE entity_id = $1 AND state = 'raised'`, checkID)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
if tag.RowsAffected() > 0 {
|
||||
emitSchedulerEvent(ctx, pool, "signal.resolved", targetID, "info",
|
||||
map[string]any{"slug": slug})
|
||||
}
|
||||
_ = q.UpsertEntityStatus(ctx, sqlcgen.UpsertEntityStatusParams{
|
||||
EntityID: entityID,
|
||||
EntityID: targetID,
|
||||
Health: "healthy",
|
||||
LastCheckAt: &[]time.Time{time.Now()}[0],
|
||||
Details: []byte(`{}`),
|
||||
@@ -160,8 +224,17 @@ func resolveSignal(ctx context.Context, pool *db.Pool, entityID uuid.UUID, slug
|
||||
slog.Info("scheduler: signal resolved", "entity", slug)
|
||||
}
|
||||
|
||||
// checkResult bundles the outcome of a single check execution.
|
||||
type checkResult struct {
|
||||
health string
|
||||
signalKind string
|
||||
evidence string
|
||||
metrics map[string]float64
|
||||
err error
|
||||
}
|
||||
|
||||
// executeCheck dispatches to the appropriate checker by kind.
|
||||
func executeCheck(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (health string, signalKind string, evidence string, err error) {
|
||||
func executeCheck(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkResult {
|
||||
switch cd.Kind {
|
||||
case "http":
|
||||
return checkHTTP(ctx, cd)
|
||||
@@ -171,8 +244,12 @@ func executeCheck(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (heal
|
||||
return checkDisk(ctx, cd)
|
||||
case "cert-expiry":
|
||||
return checkCertExpiry(ctx, cd)
|
||||
case "ping":
|
||||
return checkPing(ctx, cd)
|
||||
case "ssh-script":
|
||||
return checkSSHScript(ctx, cd)
|
||||
default:
|
||||
return "unknown", "", "", nil
|
||||
return checkResult{health: "unknown"}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -187,12 +264,76 @@ func housekeeping(ctx context.Context, pool *db.Pool) {
|
||||
slog.Error("scheduler: prune idempotency keys", "error", err)
|
||||
}
|
||||
|
||||
staleSweep(ctx, pool)
|
||||
|
||||
// Log housekeeping completion
|
||||
slog.Debug("scheduler: housekeeping done", "pruned_idempotency_before", cutoff.Format(time.RFC3339))
|
||||
}
|
||||
|
||||
// staleMultiplier and staleFloor bound how long an entity can go unobserved
|
||||
// before its last-known health is no longer trusted. An entity is stale once
|
||||
// it has gone longer than staleMultiplier times its fastest enabled check's
|
||||
// interval (or staleFloor, whichever is larger) without a fresh observation —
|
||||
// covering both a stalled scheduler and a disabled/broken check_def.
|
||||
const (
|
||||
staleMultiplier = 3
|
||||
staleFloor = 5 * time.Minute
|
||||
)
|
||||
|
||||
// staleSweep marks entities whose last observation has aged past their
|
||||
// check's expected cadence as 'stale', so the system never reports an old
|
||||
// health value as if it were current. Runs once per housekeeping pass.
|
||||
func staleSweep(ctx context.Context, pool *db.Pool) {
|
||||
rows, err := pool.Query(ctx, `
|
||||
SELECT e.id, e.slug, st.health
|
||||
FROM entity_status st
|
||||
JOIN entities e ON e.id = st.entity_id
|
||||
JOIN (
|
||||
SELECT target_id, MIN(interval_s) AS min_interval
|
||||
FROM check_defs
|
||||
WHERE enabled AND target_id IS NOT NULL
|
||||
GROUP BY target_id
|
||||
) iv ON iv.target_id = st.entity_id
|
||||
WHERE st.health <> 'stale'
|
||||
AND (st.last_check_at IS NULL
|
||||
OR st.last_check_at < now() - make_interval(secs => GREATEST(iv.min_interval * $1, $2)))`,
|
||||
staleMultiplier, int(staleFloor.Seconds()))
|
||||
if err != nil {
|
||||
slog.Error("scheduler: stale sweep query", "error", err)
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
type staleEntity struct {
|
||||
id uuid.UUID
|
||||
slug string
|
||||
health string
|
||||
}
|
||||
var stale []staleEntity
|
||||
for rows.Next() {
|
||||
var se staleEntity
|
||||
if err := rows.Scan(&se.id, &se.slug, &se.health); err != nil {
|
||||
continue
|
||||
}
|
||||
stale = append(stale, se)
|
||||
}
|
||||
rows.Close()
|
||||
|
||||
for _, se := range stale {
|
||||
_, err := pool.Exec(ctx,
|
||||
`UPDATE entity_status SET health = 'stale', updated_at = now() WHERE entity_id = $1`, se.id)
|
||||
if err != nil {
|
||||
slog.Error("scheduler: mark stale", "entity", se.slug, "error", err)
|
||||
continue
|
||||
}
|
||||
slog.Warn("scheduler: entity stale", "entity", se.slug, "prev_health", se.health)
|
||||
emitSchedulerEvent(ctx, pool, "health.stale", se.id, "warning",
|
||||
map[string]any{"slug": se.slug, "from": se.health, "to": "stale"})
|
||||
}
|
||||
}
|
||||
|
||||
// checkHTTP performs an HTTP health check.
|
||||
func checkHTTP(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (string, string, string, error) {
|
||||
func checkHTTP(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkResult {
|
||||
cfg := struct {
|
||||
URL string `json:"url"`
|
||||
ExpectedStatus int `json:"expected_status"`
|
||||
@@ -204,7 +345,7 @@ func checkHTTP(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (string,
|
||||
_ = json.Unmarshal(cd.Config, &cfg)
|
||||
}
|
||||
if cfg.URL == "" {
|
||||
return "healthy", "", "", nil
|
||||
return checkResult{health: "healthy"}
|
||||
}
|
||||
|
||||
timeout := time.Duration(cd.TimeoutS) * time.Second
|
||||
@@ -221,25 +362,35 @@ func checkHTTP(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (string,
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, cfg.URL, nil)
|
||||
if err != nil {
|
||||
return "down", "http", fmt.Sprintf("invalid URL %q: %v", cfg.URL, err), err
|
||||
return checkResult{
|
||||
health: "down", signalKind: "http",
|
||||
evidence: fmt.Sprintf("invalid URL %q: %v", cfg.URL, err),
|
||||
err: err,
|
||||
}
|
||||
}
|
||||
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return "down", "http", fmt.Sprintf("GET %s: %v", cfg.URL, err), err
|
||||
return checkResult{
|
||||
health: "down", signalKind: "http",
|
||||
evidence: fmt.Sprintf("GET %s: %v", cfg.URL, err),
|
||||
err: err,
|
||||
}
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode != cfg.ExpectedStatus {
|
||||
return "degraded", "http",
|
||||
fmt.Sprintf("GET %s returned %d (expected %d)", cfg.URL, resp.StatusCode, cfg.ExpectedStatus), nil
|
||||
return checkResult{
|
||||
health: "degraded", signalKind: "http",
|
||||
evidence: fmt.Sprintf("GET %s returned %d (expected %d)", cfg.URL, resp.StatusCode, cfg.ExpectedStatus),
|
||||
}
|
||||
}
|
||||
|
||||
return "healthy", "", "", nil
|
||||
return checkResult{health: "healthy"}
|
||||
}
|
||||
|
||||
// checkTCP performs a TCP dial check.
|
||||
func checkTCP(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (string, string, string, error) {
|
||||
func checkTCP(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkResult {
|
||||
cfg := struct {
|
||||
Host string `json:"host"`
|
||||
Port int `json:"port"`
|
||||
@@ -248,7 +399,7 @@ func checkTCP(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (string,
|
||||
_ = json.Unmarshal(cd.Config, &cfg)
|
||||
}
|
||||
if cfg.Host == "" || cfg.Port == 0 {
|
||||
return "healthy", "", "", nil
|
||||
return checkResult{health: "healthy"}
|
||||
}
|
||||
|
||||
timeout := time.Duration(cd.TimeoutS) * time.Second
|
||||
@@ -259,14 +410,18 @@ func checkTCP(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (string,
|
||||
addr := net.JoinHostPort(cfg.Host, fmt.Sprintf("%d", cfg.Port))
|
||||
conn, err := net.DialTimeout("tcp", addr, timeout)
|
||||
if err != nil {
|
||||
return "down", "tcp", fmt.Sprintf("dial %s: %v", addr, err), err
|
||||
return checkResult{
|
||||
health: "down", signalKind: "tcp",
|
||||
evidence: fmt.Sprintf("dial %s: %v", addr, err),
|
||||
err: err,
|
||||
}
|
||||
}
|
||||
conn.Close()
|
||||
return "healthy", "", "", nil
|
||||
return checkResult{health: "healthy"}
|
||||
}
|
||||
|
||||
// checkDisk performs a disk usage check via local or SSH.
|
||||
func checkDisk(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (string, string, string, error) {
|
||||
// checkDisk performs a disk usage check.
|
||||
func checkDisk(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkResult {
|
||||
cfg := struct {
|
||||
Path string `json:"path"`
|
||||
ThresholdPct int `json:"threshold_pct"`
|
||||
@@ -278,29 +433,45 @@ func checkDisk(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (string,
|
||||
_ = json.Unmarshal(cd.Config, &cfg)
|
||||
}
|
||||
|
||||
// Use unix.Statfs for disk usage.
|
||||
var stat unix.Statfs_t
|
||||
if err := unix.Statfs(cfg.Path, &stat); err != nil {
|
||||
return "down", "disk", fmt.Sprintf("statfs %s: %v", cfg.Path, err), err
|
||||
return checkResult{
|
||||
health: "down", signalKind: "disk",
|
||||
evidence: fmt.Sprintf("statfs %s: %v", cfg.Path, err),
|
||||
err: err,
|
||||
}
|
||||
}
|
||||
|
||||
total := stat.Blocks * uint64(stat.Bsize)
|
||||
free := stat.Bfree * uint64(stat.Bsize)
|
||||
if total == 0 {
|
||||
return "healthy", "", "", nil
|
||||
return checkResult{health: "healthy"}
|
||||
}
|
||||
|
||||
usedPct := float64(total-free) / float64(total) * 100
|
||||
if usedPct > float64(cfg.ThresholdPct) {
|
||||
return "degraded", "disk",
|
||||
fmt.Sprintf("%s %.1f%% full (threshold %d%%)", cfg.Path, usedPct, cfg.ThresholdPct), nil
|
||||
inodePct := 0.0
|
||||
if stat.Files > 0 {
|
||||
inodePct = float64(stat.Files-stat.Ffree) / float64(stat.Files) * 100
|
||||
}
|
||||
|
||||
return "healthy", "", "", nil
|
||||
metrics := map[string]float64{
|
||||
"disk_used_pct": usedPct,
|
||||
"disk_inode_pct": inodePct,
|
||||
}
|
||||
|
||||
if usedPct > float64(cfg.ThresholdPct) {
|
||||
return checkResult{
|
||||
health: "degraded", signalKind: "disk",
|
||||
evidence: fmt.Sprintf("%s %.1f%% full (threshold %d%%)", cfg.Path, usedPct, cfg.ThresholdPct),
|
||||
metrics: metrics,
|
||||
}
|
||||
}
|
||||
|
||||
return checkResult{health: "healthy", metrics: metrics}
|
||||
}
|
||||
|
||||
// checkCertExpiry checks TLS certificate expiry.
|
||||
func checkCertExpiry(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (string, string, string, error) {
|
||||
func checkCertExpiry(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkResult {
|
||||
cfg := struct {
|
||||
Host string `json:"host"`
|
||||
Port int `json:"port"`
|
||||
@@ -315,7 +486,7 @@ func checkCertExpiry(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (s
|
||||
_ = json.Unmarshal(cd.Config, &cfg)
|
||||
}
|
||||
if cfg.Host == "" {
|
||||
return "healthy", "", "", nil
|
||||
return checkResult{health: "healthy"}
|
||||
}
|
||||
|
||||
timeout := time.Duration(cd.TimeoutS) * time.Second
|
||||
@@ -328,31 +499,264 @@ func checkCertExpiry(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (s
|
||||
d := tls.Dialer{Config: &tls.Config{InsecureSkipVerify: true}}
|
||||
conn, err := d.DialContext(ctx, "tcp", addr)
|
||||
if err != nil {
|
||||
return "down", "cert-expiry", fmt.Sprintf("TLS dial %s: %v", addr, err), err
|
||||
return checkResult{
|
||||
health: "down", signalKind: "cert-expiry",
|
||||
evidence: fmt.Sprintf("TLS dial %s: %v", addr, err),
|
||||
err: err,
|
||||
}
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
tlsConn := conn.(*tls.Conn)
|
||||
// Use crypto/tls ConnectionState to get verified chains
|
||||
cs := tlsConn.ConnectionState()
|
||||
if len(cs.PeerCertificates) == 0 {
|
||||
return "down", "cert-expiry", "no peer certificates", nil
|
||||
return checkResult{
|
||||
health: "down", signalKind: "cert-expiry",
|
||||
evidence: "no peer certificates",
|
||||
}
|
||||
}
|
||||
|
||||
cert := cs.PeerCertificates[0]
|
||||
daysLeft := int(time.Until(cert.NotAfter).Hours() / 24)
|
||||
|
||||
if daysLeft <= cfg.CritDays {
|
||||
return "down", "cert-expiry",
|
||||
fmt.Sprintf("%s expires in %d days (crit=%d)", cfg.Host, daysLeft, cfg.CritDays), nil
|
||||
}
|
||||
if daysLeft <= cfg.WarnDays {
|
||||
return "degraded", "cert-expiry",
|
||||
fmt.Sprintf("%s expires in %d days (warn=%d)", cfg.Host, daysLeft, cfg.WarnDays), nil
|
||||
metrics := map[string]float64{
|
||||
"cert_days_left": float64(daysLeft),
|
||||
}
|
||||
|
||||
return "healthy", "", "", nil
|
||||
if daysLeft <= cfg.CritDays {
|
||||
return checkResult{
|
||||
health: "down", signalKind: "cert-expiry",
|
||||
evidence: fmt.Sprintf("%s expires in %d days (crit=%d)", cfg.Host, daysLeft, cfg.CritDays),
|
||||
metrics: metrics,
|
||||
}
|
||||
}
|
||||
if daysLeft <= cfg.WarnDays {
|
||||
return checkResult{
|
||||
health: "degraded", signalKind: "cert-expiry",
|
||||
evidence: fmt.Sprintf("%s expires in %d days (warn=%d)", cfg.Host, daysLeft, cfg.WarnDays),
|
||||
metrics: metrics,
|
||||
}
|
||||
}
|
||||
|
||||
return checkResult{health: "healthy", metrics: metrics}
|
||||
}
|
||||
|
||||
// checkPing performs an ICMP ping check using the system ping command.
|
||||
func checkPing(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkResult {
|
||||
cfg := struct {
|
||||
Host string `json:"host"`
|
||||
Count int `json:"count"`
|
||||
}{}
|
||||
if len(cd.Config) > 0 {
|
||||
_ = json.Unmarshal(cd.Config, &cfg)
|
||||
}
|
||||
if cfg.Host == "" {
|
||||
return checkResult{health: "healthy"}
|
||||
}
|
||||
if cfg.Count <= 0 {
|
||||
cfg.Count = 1
|
||||
}
|
||||
|
||||
timeout := time.Duration(cd.TimeoutS) * time.Second
|
||||
if timeout <= 0 {
|
||||
timeout = 10 * time.Second
|
||||
}
|
||||
|
||||
deadline := time.Duration(cfg.Count+1) * timeout
|
||||
ctx, cancel := context.WithTimeout(ctx, deadline)
|
||||
defer cancel()
|
||||
|
||||
countStr := strconv.Itoa(cfg.Count)
|
||||
timeoutSec := strconv.Itoa(int(timeout.Seconds()))
|
||||
if timeoutSec == "0" {
|
||||
timeoutSec = "1"
|
||||
}
|
||||
|
||||
cmd := exec.CommandContext(ctx, "ping", "-c", countStr, "-W", timeoutSec, cfg.Host)
|
||||
if runtime.GOOS == "darwin" {
|
||||
cmd = exec.CommandContext(ctx, "ping", "-c", countStr, "-t", timeoutSec, cfg.Host)
|
||||
}
|
||||
|
||||
output, err := cmd.Output()
|
||||
if err != nil {
|
||||
return checkResult{
|
||||
health: "down", signalKind: "ping",
|
||||
evidence: fmt.Sprintf("ping %s: %v", cfg.Host, err),
|
||||
err: err,
|
||||
}
|
||||
}
|
||||
|
||||
latency := parsePingLatency(output)
|
||||
metrics := map[string]float64{}
|
||||
if latency > 0 {
|
||||
metrics["ping_latency_ms"] = latency
|
||||
}
|
||||
|
||||
return checkResult{health: "healthy", metrics: metrics}
|
||||
}
|
||||
|
||||
var pingRttRe = regexp.MustCompile(`(?:rtt\s+min\/avg\/max\/mdev|round-trip\s+min\/avg\/max\/stddev)\s*=\s*[\d.]+\/([\d.]+)\/`)
|
||||
|
||||
func parsePingLatency(output []byte) float64 {
|
||||
matches := pingRttRe.FindSubmatch(output)
|
||||
if len(matches) < 2 {
|
||||
return 0
|
||||
}
|
||||
val, err := strconv.ParseFloat(string(matches[1]), 64)
|
||||
if err != nil {
|
||||
return 0
|
||||
}
|
||||
return val
|
||||
}
|
||||
|
||||
// checkSSHScript executes an allowlisted script on a remote host via SSH.
|
||||
func checkSSHScript(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkResult {
|
||||
cfg := struct {
|
||||
Host string `json:"host"`
|
||||
Port int `json:"port"`
|
||||
User string `json:"user"`
|
||||
Script string `json:"script"`
|
||||
}{}
|
||||
if len(cd.Config) > 0 {
|
||||
_ = json.Unmarshal(cd.Config, &cfg)
|
||||
}
|
||||
if cfg.Host == "" || cfg.Script == "" {
|
||||
return checkResult{health: "healthy"}
|
||||
}
|
||||
if cfg.Port == 0 {
|
||||
cfg.Port = 22
|
||||
}
|
||||
if cfg.User == "" {
|
||||
cfg.User = sshUser
|
||||
}
|
||||
|
||||
if !allowlistedScript(cfg.Script) {
|
||||
return checkResult{
|
||||
health: "unknown", signalKind: "ssh-script",
|
||||
evidence: fmt.Sprintf("script %q not allowlisted", cfg.Script),
|
||||
}
|
||||
}
|
||||
|
||||
timeout := time.Duration(cd.TimeoutS) * time.Second
|
||||
if timeout <= 0 {
|
||||
timeout = 10 * time.Second
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(ctx, timeout)
|
||||
defer cancel()
|
||||
|
||||
scriptPath := "/opt/oikos/checks/" + cfg.Script
|
||||
port := strconv.Itoa(cfg.Port)
|
||||
|
||||
output, err := sshExec(ctx, cfg.Host, port, cfg.User, scriptPath, timeout)
|
||||
if err != nil {
|
||||
return checkResult{
|
||||
health: "down", signalKind: "ssh-script",
|
||||
evidence: fmt.Sprintf("ssh %s:%s %s: %v", cfg.Host, strconv.Itoa(cfg.Port), cfg.Script, err),
|
||||
err: err,
|
||||
}
|
||||
}
|
||||
|
||||
type scriptOutput struct {
|
||||
Health string `json:"health"`
|
||||
SignalKind string `json:"signalKind"`
|
||||
Evidence string `json:"evidence"`
|
||||
Metrics map[string]float64 `json:"metrics"`
|
||||
}
|
||||
var so scriptOutput
|
||||
if err := json.Unmarshal(output, &so); err != nil {
|
||||
return checkResult{
|
||||
health: "down", signalKind: "ssh-script",
|
||||
evidence: fmt.Sprintf("invalid script output from %s: %v", cfg.Script, err),
|
||||
err: err,
|
||||
}
|
||||
}
|
||||
|
||||
health := so.Health
|
||||
if health == "" {
|
||||
health = "healthy"
|
||||
}
|
||||
|
||||
metrics := so.Metrics
|
||||
if metrics == nil {
|
||||
metrics = make(map[string]float64)
|
||||
}
|
||||
|
||||
return checkResult{
|
||||
health: health,
|
||||
signalKind: so.SignalKind,
|
||||
evidence: so.Evidence,
|
||||
metrics: metrics,
|
||||
}
|
||||
}
|
||||
|
||||
var scriptNameRe = regexp.MustCompile(`^[a-z][a-z0-9_-]+\.sh$`)
|
||||
|
||||
func allowlistedScript(name string) bool {
|
||||
return scriptNameRe.MatchString(name)
|
||||
}
|
||||
|
||||
func sshExec(ctx context.Context, host, port, user, cmd string, timeout time.Duration) ([]byte, error) {
|
||||
args := []string{
|
||||
"-o", "ConnectTimeout=" + strconv.Itoa(int(timeout.Seconds())),
|
||||
"-o", "StrictHostKeyChecking=no",
|
||||
"-o", "BatchMode=yes",
|
||||
"-o", "UserKnownHostsFile=/dev/null",
|
||||
"-o", "LogLevel=ERROR",
|
||||
}
|
||||
if sshKeyPath != "" {
|
||||
args = append(args, "-i", sshKeyPath)
|
||||
}
|
||||
if port != "" && port != "22" {
|
||||
args = append(args, "-p", port)
|
||||
}
|
||||
args = append(args, "-l", user, host, cmd)
|
||||
c := exec.CommandContext(ctx, "ssh", args...)
|
||||
out, err := c.Output()
|
||||
if err != nil {
|
||||
var ee *exec.ExitError
|
||||
if errors.As(err, &ee) {
|
||||
return nil, fmt.Errorf("ssh %s: %v (stderr: %s)", host, err, string(ee.Stderr))
|
||||
}
|
||||
return nil, fmt.Errorf("ssh %s: %v", host, err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
|
||||
// metricThreshold defines warn/crit thresholds for a single metric.
|
||||
type metricThreshold struct {
|
||||
Warn float64 `json:"warn"`
|
||||
Crit float64 `json:"crit"`
|
||||
}
|
||||
|
||||
// thresholdsConfig is parsed from check_defs.config.thresholds JSONB.
|
||||
type thresholdsConfig map[string]metricThreshold
|
||||
|
||||
// evaluateSeverity determines signal severity from check result and thresholds.
|
||||
func evaluateSeverity(kind string, signalKind string, config []byte, metrics map[string]float64) string {
|
||||
var thresholds thresholdsConfig
|
||||
if len(config) > 0 {
|
||||
_ = json.Unmarshal(config, &thresholds)
|
||||
}
|
||||
|
||||
for metric, value := range metrics {
|
||||
t, ok := thresholds[metric]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if t.Crit > 0 && value >= t.Crit {
|
||||
return "critical"
|
||||
}
|
||||
if t.Warn > 0 && value >= t.Warn {
|
||||
return "warning"
|
||||
}
|
||||
}
|
||||
|
||||
if signalKind == "down" {
|
||||
return "critical"
|
||||
}
|
||||
return "warning"
|
||||
}
|
||||
|
||||
var _ = uuid.UUID{} // ensure uuid import stays
|
||||
@@ -511,7 +511,7 @@ hosts:
|
||||
192.168.8.0/24. Re-enroll in mesh as a follow-up if off-LAN access
|
||||
to this host itself (not just its future guests) is needed.
|
||||
- First step of the planned library-SSD migration — see
|
||||
.hermes/plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md
|
||||
.nomos/plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md
|
||||
(filename kept as-is, it's a historical planning doc). Only Phase 1
|
||||
(Proxmox install + cluster join) is done; no physical
|
||||
drive move, service migration, or GPU passthrough has happened yet.
|
||||
@@ -555,7 +555,7 @@ archaeology:
|
||||
kind: lxc
|
||||
pve_id: 123
|
||||
destroyed: 2026-06-04
|
||||
reason: replaced by Hermes Agent on mac-mini; monitoring moved to homelab-health-watchdog cron
|
||||
reason: replaced by Nomos Agent on mac-mini; monitoring moved to homelab-health-watchdog cron
|
||||
plato:
|
||||
kind: lxc
|
||||
pve_id: 126
|
||||
|
||||
18
migrations/014_rename_agent_hermes_to_nomos.up.sql
Normal file
18
migrations/014_rename_agent_hermes_to_nomos.up.sql
Normal file
@@ -0,0 +1,18 @@
|
||||
-- 014_rename_agent_hermes_to_nomos.up.sql
|
||||
-- Rename the Hermes agent entity to Nomos (N0 milestone).
|
||||
-- Identity-preserving: the UUID, relationships, and audit history survive.
|
||||
-- The matching seed upsert will no-op because it upserts by slug.
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF EXISTS (SELECT 1 FROM entities WHERE slug = 'agent:hermes') THEN
|
||||
UPDATE entities
|
||||
SET slug = 'agent:nomos',
|
||||
name = 'nomos',
|
||||
attributes = jsonb_set(attributes, '{name}', '"nomos"'),
|
||||
updated_at = now()
|
||||
WHERE slug = 'agent:hermes'
|
||||
AND NOT EXISTS (SELECT 1 FROM entities WHERE slug = 'agent:nomos');
|
||||
END IF;
|
||||
END
|
||||
$$;
|
||||
25
migrations/015_agent_sessions.up.sql
Normal file
25
migrations/015_agent_sessions.up.sql
Normal file
@@ -0,0 +1,25 @@
|
||||
-- 015_agent_sessions.up.sql
|
||||
-- Nomos agent sessions: persist conversations across restarts.
|
||||
-- agent_messages stores the full message history (JSONB).
|
||||
-- agent_activity is joined via correlation_id for tool-call tracing.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS agent_sessions (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
title TEXT NOT NULL DEFAULT '',
|
||||
actor TEXT NOT NULL DEFAULT 'agent:nomos',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
last_active_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS agent_messages (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
session_id UUID NOT NULL REFERENCES agent_sessions(id) ON DELETE CASCADE,
|
||||
role TEXT NOT NULL,
|
||||
content JSONB NOT NULL DEFAULT '{}',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_agent_messages_session
|
||||
ON agent_messages(session_id, created_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_agent_sessions_active
|
||||
ON agent_sessions(last_active_at DESC);
|
||||
16
migrations/016_fix_check_status_misattribution.up.sql
Normal file
16
migrations/016_fix_check_status_misattribution.up.sql
Normal file
@@ -0,0 +1,16 @@
|
||||
-- 016_fix_check_status_misattribution.up.sql
|
||||
-- The scheduler historically wrote entity_status and metric_samples keyed by
|
||||
-- the *check* entity (check_defs.entity_id) instead of the entity being
|
||||
-- monitored (check_defs.target_id). Every real host/service/lxc/etc. entity
|
||||
-- was therefore permanently stuck at health='unknown' while all observed
|
||||
-- health and metrics accumulated on the internal probe entities instead.
|
||||
-- The Go fix (scheduler.go) starts writing to target_id going forward; this
|
||||
-- migration removes the now-orphaned check-entity rows so dashboard/fleet
|
||||
-- health rollups stop double-counting probes as if they were monitored
|
||||
-- entities. Historical metric_samples on check entities are left in place
|
||||
-- (time-series data, not safe to reattribute retroactively) but are no
|
||||
-- longer queried through any entity-scoped view once checks stop being
|
||||
-- written to.
|
||||
|
||||
DELETE FROM entity_status
|
||||
WHERE entity_id IN (SELECT id FROM entities WHERE type = 'check');
|
||||
@@ -1,7 +1,7 @@
|
||||
# SOUL.md — Hermes agent persona (Phase 4, container runtime)
|
||||
# SOUL.md — Nomos agent persona (Phase 4, container runtime)
|
||||
|
||||
You are **Hermes**, the homelab AI agent running in a Docker container on
|
||||
mac-mini. You operate in **gateway mode** on mesh-only port 8092.
|
||||
You are **Nomos** (from *oikonomos*, the steward of the oikos), the homelab
|
||||
AI agent running in a Docker container on mac-mini. You operate on port 8092.
|
||||
|
||||
## Source of truth
|
||||
|
||||
@@ -45,6 +45,6 @@ describing state, be concise — the operator reads your output in Matrix.
|
||||
|
||||
## Skills
|
||||
|
||||
Skills live in `/app/hermes/skills/`. Load a skill when its description
|
||||
Skills live in `/app/nomos/skills/`. Load a skill when its description
|
||||
matches the task. The `homelab-ops` skill covers:
|
||||
- Health checks, signal triage, pattern validation, and escalation flow.
|
||||
18
nomos/config.yaml
Normal file
18
nomos/config.yaml
Normal file
@@ -0,0 +1,18 @@
|
||||
# Nomos agent config — LLM-backed resident agent (Phase 4)
|
||||
|
||||
mcp:
|
||||
endpoint: ${NOMOS_MCP_URL}?session_id=${NOMOS_SESSION_ID}
|
||||
transport: streamable_http
|
||||
|
||||
server:
|
||||
listen: ${NOMOS_LISTEN}
|
||||
mesh_only: true
|
||||
|
||||
agent:
|
||||
name: nomos
|
||||
slug: ${NOMOS_AGENT_SLUG}
|
||||
|
||||
llm:
|
||||
provider: openrouter
|
||||
model: ${NOMOS_MODEL}
|
||||
max_iterations: 15
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
## Overview
|
||||
|
||||
Standard operating procedures for the Hermes agent managing the hubris
|
||||
Standard operating procedures for the Nomos agent managing the hubris
|
||||
homelab. All mutations route through `request_execution` → Oikos policy
|
||||
gating → actuator (SSH).
|
||||
|
||||
@@ -41,5 +41,8 @@ gating → actuator (SSH).
|
||||
|
||||
## Changelog
|
||||
|
||||
### 2026-07-08 — rename to Nomos
|
||||
Agent renamed from Hermes to Nomos (N0 milestone).
|
||||
|
||||
### 2026-07-07 — initial Phase 4 skill
|
||||
Baseline homelab operations skill for Hermes container.
|
||||
Baseline homelab operations skill for Nomos container.
|
||||
@@ -1,6 +1,20 @@
|
||||
# 2026-07-08 — Control room web UI
|
||||
|
||||
**Status:** Planned
|
||||
**Status:** In Progress — N0-N3 (Nomos amendment: chat home + sessions), M1
|
||||
(dashboard/summary, Overview, Entities table, live event feed, shadcn-svelte
|
||||
component system), M2 (Operations ledger with approve/deny + cancel, Signals
|
||||
page with ack/resolve/mute, live nav badges), and M3 (graph explorer with
|
||||
`include=status` health coloring, per-relationship-type edge coloring +
|
||||
legend/filter, node-type filter, node search/highlight, entity detail page
|
||||
with uPlot metric charts, `#/entity/:slug` route) complete 2026-07-08. Event
|
||||
gap-fill (M2's
|
||||
other half) landed earlier in commit e8e230b, and approval creation's FK bug
|
||||
(gaps-plan A1) was already fixed, unblocking M2. While building M3, also
|
||||
fixed `GET /metrics` to make the `metric` query param genuinely optional
|
||||
(server now reports every metric recorded for the entity in range) — the
|
||||
implementation previously 400'd when it was omitted, contradicting its own
|
||||
documented-optional spec. M4 (agent activity, knowledge search page, audit,
|
||||
correlation grouping, polish) remains.
|
||||
|
||||
## Goal
|
||||
|
||||
@@ -31,8 +45,18 @@ Dependencies kept minimal:
|
||||
- `d3-force` — graph physics only; render SVG/canvas by hand
|
||||
- `uPlot` — ~45 KB canvas time-series, ideal for `/metrics` rollups
|
||||
- `openapi-typescript` — dev-only, generates `api-types.d.ts`
|
||||
- No SvelteKit (no SSR wanted — the Go binary is the server), no component
|
||||
framework; hash router; hand-rolled dark-theme CSS.
|
||||
- No SvelteKit (no SSR wanted — the Go binary is the server); hash router.
|
||||
|
||||
*Amendment 2026-07-08 (M1):* component library is
|
||||
[shadcn-svelte](https://www.shadcn-svelte.com/) over Tailwind CSS v4
|
||||
(`@tailwindcss/vite`), not hand-rolled CSS — Table, Card, Badge, Sidebar,
|
||||
Sheet, Select, Input, Button, Tabs, ScrollArea, Tooltip, Dialog,
|
||||
Dropdown-menu, Sonner installed via `npx shadcn-svelte add`. The existing
|
||||
dark GitHub-style palette (`app.css`) was ported into shadcn's CSS-variable
|
||||
theme contract (`--background`, `--card`, `--primary`, etc. under
|
||||
`@theme inline`) so old and new components share one palette. `d3-force` /
|
||||
`uPlot` remain the plan for the graph/charts milestones (M3), unaffected by
|
||||
this change.
|
||||
|
||||
Build integration: commit a placeholder `web/dist/index.html` so backend-only
|
||||
`go build` never breaks; `make ui` runs the Vite build; add a node stage to
|
||||
@@ -116,12 +140,12 @@ Persistent nav + live top status strip (health dots, open-signal badge,
|
||||
pending-approval badge):
|
||||
|
||||
0. **Agent chat (home view)** — *amendment 2026-07-08, see the
|
||||
[Hermes resident agent plan](2026-07-08-hermes-resident-agent.md)*: the
|
||||
default view at `/ui/#/` is a conversation with the resident Hermes agent
|
||||
(streamed via `/agent/chat`), with a live right rail showing pending
|
||||
approvals (decidable in place), recent events, and health. A persistent
|
||||
chat drawer is reachable from every other page. Lands with resident-agent
|
||||
milestone H3; until then, Overview is the home view.
|
||||
[Nomos resident agent plan](2026-07-08-nomos-resident-agent.md)*: the
|
||||
default view at `/ui/#/` is a conversation with the resident Nomos agent
|
||||
(formerly Hermes; streamed via `/agent/chat`), with a live right rail
|
||||
showing pending approvals (decidable in place), recent events, and health.
|
||||
A persistent chat drawer is reachable from every other page. Lands with
|
||||
resident-agent milestone N3; until then, Overview is the home view.
|
||||
1. **Overview** — summary cards from `/dashboard/summary`, event-rate
|
||||
sparkline, live event ticker, top degraded entities.
|
||||
2. **Graph explorer** — d3-force over `/graph`; node color = health; filters
|
||||
|
||||
@@ -1,167 +0,0 @@
|
||||
# 2026-07-08 — Hermes resident agent
|
||||
|
||||
**Status:** Planned
|
||||
|
||||
## Goal
|
||||
|
||||
Turn Hermes from a keyword router into a resident LLM-backed agent: a
|
||||
long-running service you converse with, which reasons over the full 28-tool
|
||||
MCP surface, holds session context, and is surfaced as the **main entry point
|
||||
of the control-room web UI** ([companion plan](2026-07-08-control-room-webui.md)).
|
||||
Oikos already has the agentic substrate — policy classes, approval gating,
|
||||
`agent_activity`, an OODA loop; this gives it the conversational front half.
|
||||
|
||||
## What exists vs what's added
|
||||
|
||||
Today `cmd/hermes/main.go` contains no LLM anywhere: `routeQuery` is
|
||||
`strings.Contains` over ~6 phrases, `extractEntity` knows 5 hardcoded
|
||||
services, and everything else silently falls back to `get_health_summary`.
|
||||
But the hard plumbing already exists and is reused as-is:
|
||||
|
||||
- `mcpClient` (main.go:190) — working Streamable-HTTP MCP client
|
||||
(initialize → session → `tools/call` with SSE frame parsing).
|
||||
- `listTools()` (main.go:318) — currently dead code; extended (below) it
|
||||
becomes the bridge that feeds MCP tool schemas to Claude.
|
||||
- `.agents/HERMES.md` / `hermes/SOUL.md` — the persona, becomes the system
|
||||
prompt.
|
||||
- `agent_activity` hypertable + `correlation_id` conventions — where tool
|
||||
calls get logged so the control room can show the agent working.
|
||||
|
||||
## Architecture decision: hand-rolled tool loop, not the MCP connector
|
||||
|
||||
Two ways to let Claude drive the tools:
|
||||
|
||||
1. **Anthropic MCP connector** (`mcp_servers` param on the Messages API):
|
||||
zero loop code, but Anthropic's servers must reach the MCP endpoint over
|
||||
the public internet. `mcp.hubris.network` is currently exposed *without
|
||||
auth* — a hole the [gaps plan](2026-07-08-oikos-gaps-and-improvements.md)
|
||||
says to close, not to build on. Keeping the MCP surface mesh-private is
|
||||
the right posture for a homelab control plane.
|
||||
2. **Hand-rolled agent loop** (~200 lines): Hermes calls the Messages API
|
||||
with tool definitions translated from `tools/list`, executes Claude's
|
||||
tool_use blocks through its *existing* `mcpClient` inside the mesh, feeds
|
||||
back `tool_result` blocks, repeats until Claude answers in text.
|
||||
|
||||
**Decision: (2).** MCP tool `inputSchema` is already JSON Schema — the exact
|
||||
format the Anthropic `tools` parameter expects — so translation is a field
|
||||
rename. Only the conversation text ever leaves the mesh; the tool transport
|
||||
stays private. Use the official `anthropic-sdk-go`.
|
||||
|
||||
## Design
|
||||
|
||||
### Agent loop (`cmd/hermes/agent.go`, new)
|
||||
|
||||
```
|
||||
system prompt = SOUL.md content (mounted; already provisioned by tools/setup-hermes-soul.sh)
|
||||
tools = listToolsFull() // extend listTools() to return name, description, inputSchema
|
||||
loop (max 15 iterations):
|
||||
resp = messages.New(model, system, history, tools)
|
||||
if resp has tool_use blocks:
|
||||
for each: result = mcpClient.callTool(name, args) // existing code path
|
||||
log to agent_activity (correlation_id = session turn id)
|
||||
append assistant msg + tool_result user msg to history
|
||||
else: final text → stream to caller, persist turn
|
||||
```
|
||||
|
||||
- Model: `claude-sonnet-5` default, `HERMES_MODEL` override. `max_tokens`
|
||||
and iteration cap configurable; hard per-turn budget so a pathological
|
||||
loop can't burn the API bill.
|
||||
- `ANTHROPIC_API_KEY` from env / Infisical (same secret path as other creds).
|
||||
- Mutations need no new guardrails: the agent's only write path is
|
||||
`request_execution`, which flows through the existing risk-class /
|
||||
approval machinery. The agent's actor identity is `agent:hermes`.
|
||||
**Depends on gaps-plan bug A1** (approvals FK) — without that fix the
|
||||
agent's config mutations dead-end silently, which is worse when a
|
||||
conversational agent confidently reports "queued for approval".
|
||||
|
||||
### HTTP surface (`cmd/hermes/main.go`)
|
||||
|
||||
- `POST /chat` `{session_id?, message}` → **SSE stream** of typed events:
|
||||
`text` (deltas), `tool_use` (name + args), `tool_result` (truncated),
|
||||
`done` (session_id, usage). The web UI renders tool calls as inline chips
|
||||
as they happen.
|
||||
- `GET /sessions`, `GET /sessions/{id}` — history for the UI.
|
||||
- `/query` kept for scripts/structured callers. The toy NLU is **removed**
|
||||
(per gaps plan §C): a `query` with no `tool` returns "natural language
|
||||
belongs to /chat" plus the tool list from the now-live `listTools()`.
|
||||
- `/healthz` unchanged.
|
||||
|
||||
### Sessions (Postgres, new migration)
|
||||
|
||||
`agent_sessions (id, title, actor, created_at, last_active_at)` and
|
||||
`agent_messages (id, session_id, role, content JSONB, created_at)` in the
|
||||
shared oikos DB — Hermes gains a `DATABASE_URL` (it's in the same compose
|
||||
stack). DB-backed rather than in-memory so conversations survive restarts
|
||||
and the control room can list/replay them. Tool invocations additionally go
|
||||
to `agent_activity` with the session's correlation_id, so the existing
|
||||
agent-activity page and the ops ledger join up with zero new query paths.
|
||||
|
||||
### Connectivity & auth
|
||||
|
||||
- Compose: hermes already runs under the `full` profile; add
|
||||
`ANTHROPIC_API_KEY`, `DATABASE_URL` env.
|
||||
- Caddy: route `oikos.hubris.network/agent/*` → `hermes:8092` **behind the
|
||||
same Authentik forward_auth** as the rest of the vhost, stripping the
|
||||
`/agent` prefix. The web UI then calls same-origin `/agent/chat` — no
|
||||
CORS, and EventSource/fetch-streaming work unmodified.
|
||||
- Hermes enforces trusted-proxy headers (`X-Authentik-Username`) when
|
||||
`HERMES_TRUSTED_PROXY=true`, and finally implements the `mesh_only: true`
|
||||
check that `hermes/config.yaml:9` promises but `main.go` never enforces
|
||||
(gaps plan B3). Direct :8092 access stays mesh-only for agents/scripts.
|
||||
|
||||
### Web UI entry point (amends the control-room plan)
|
||||
|
||||
The agent chat becomes the **home view** of the control room (`/ui/#/`):
|
||||
|
||||
- Center: conversation pane (streamed text, expandable tool-call chips
|
||||
showing args/results, correlation-id links into the ops ledger).
|
||||
- Right rail: live context — pending approvals with approve/deny buttons,
|
||||
recent events, health strip. When the agent's `request_execution` needs
|
||||
approval, the approval card appears in the rail *mid-conversation* (via
|
||||
the `approval.created` SSE event) and can be decided without leaving chat.
|
||||
That's the whole product in one screen: ask → watch it act → approve →
|
||||
watch it complete.
|
||||
- A persistent chat drawer is available from every other page.
|
||||
- Session list in the nav; sessions resumable.
|
||||
|
||||
### Later (explicitly out of scope for v1)
|
||||
|
||||
- Matrix bridge: Hermes as a Matrix bot in the operator room, reusing the
|
||||
notifier's homeserver credentials — same `/chat` loop, different frontend.
|
||||
- Proactive mode: agent opens a session itself when a signal fires
|
||||
(escalation-with-context instead of a bare alert).
|
||||
|
||||
## Milestones
|
||||
|
||||
- **H1 — loop:** `agent.go` + anthropic-sdk-go; `listToolsFull()`; `/chat`
|
||||
(non-streaming JSON first); remove toy NLU, fix `/query` fallback + help.
|
||||
Verify by curl: multi-tool question ("what's degraded and what depends on
|
||||
it?") produces chained `get_health_summary` → `get_blast_radius` calls.
|
||||
- **H2 — state + streaming:** sessions migration, DB persistence, SSE
|
||||
streaming on `/chat`, `agent_activity` logging, budgets.
|
||||
- **H3 — UI entry point:** home chat view + context rail + drawer in the
|
||||
control-room SPA (needs control-room M1; rail approvals need M2's
|
||||
`approval.created` event and gaps-plan A1).
|
||||
- **H4 (optional):** Matrix bridge, proactive sessions.
|
||||
|
||||
## Files
|
||||
|
||||
```
|
||||
cmd/hermes/agent.go # Claude loop (new)
|
||||
cmd/hermes/main.go # /chat, /sessions, NLU removal, mesh/proxy auth
|
||||
cmd/hermes/store.go # session persistence (new)
|
||||
migrations/0xx_agent_sessions.up.sql
|
||||
hermes/config.yaml # model, budgets; drop query_routing block
|
||||
compose/ (env), Caddyfile.oikos (/agent route)
|
||||
web/src/pages/Chat.svelte + lib/stores/chat.ts # control-room home view
|
||||
go.mod # anthropic-sdk-go
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
- H1: `curl -N /agent/chat` with a question requiring 2+ tools; confirm the
|
||||
tool chain in the response and rows in `agent_activity`.
|
||||
- H2: restart hermes mid-session, resume by session_id, history intact.
|
||||
- H3: from the UI home, ask Hermes to restart a low-risk service; watch the
|
||||
tool chips stream, the execution appear in the rail, and (for a gated
|
||||
action) the approval card arrive and be decidable in place.
|
||||
253
plans/2026-07-08-liveness-drift-and-ux-cohesion.md
Normal file
253
plans/2026-07-08-liveness-drift-and-ux-cohesion.md
Normal file
@@ -0,0 +1,253 @@
|
||||
# 2026-07-08 — Liveness, drift, and UX cohesion
|
||||
|
||||
**Status:** Code complete for Phases 1–4 core scope; not yet deployed to the
|
||||
live containers (pending explicit go-ahead — see below). Phase 5 partially
|
||||
covered by pre-existing endpoints; full CRUD UI deferred.
|
||||
|
||||
- **Phase 1 (drift/staleness):** done. Health/metrics/events misattribution
|
||||
fix, staleness sweep, `/entities` health+freshness, dashboard/fleet-health
|
||||
`check`-entity exclusion, Entities table health column. Migration 016
|
||||
applied to the live dev DB (safe cleanup, additive-only).
|
||||
- **Phase 2 (detail sidebar + legibility):** done. `EntityDetailContent`
|
||||
extracted and shared between the full `#/entity/:slug` page and a new
|
||||
`EntitySheet` opened from the Entities table (master-detail, no
|
||||
navigation). Adds a **Monitoring** card (per-entity `check_defs`: kind,
|
||||
interval, enabled/disabled with click-to-toggle) and renders attributes as
|
||||
a key/value list instead of raw JSON.
|
||||
- **Phase 3 (sessions rejoin chat):** done. Fixed the click-does-nothing bug,
|
||||
added a session rail inside Chat, fixed the local dev proxy to match
|
||||
production's `/agent` prefix-stripping. **Also found and fixed a real
|
||||
latent bug**: persisted `tool_calls` store the `tool_use`/`tool_result` as
|
||||
two entries sharing one `id`; Chat.svelte's keyed `{#each tool (tool.id)}`
|
||||
threw on the duplicate key and silently blanked the entire message list.
|
||||
This had presumably never been noticed because sessions were never
|
||||
clickable before this fix. Fixed in `chat.ts` by merging tool_calls by id
|
||||
before rendering.
|
||||
- **Phase 4 (agent efficiency):** core piece done — prior turns' tool
|
||||
calls/results are now replayed into the conversation (previously dropped
|
||||
entirely), and a compact live fleet-health snapshot is injected into the
|
||||
system prompt each turn so the agent starts oriented. Prompt caching and
|
||||
reconsidering the default model are **not done** (lower priority, no
|
||||
measured regression without them).
|
||||
- **Phase 5 (CRUD):** `PatchEntity` and a full `/checks` CRUD API
|
||||
(list/create/patch, including enable/disable) already existed server-side;
|
||||
the new Monitoring card's toggle uses `PatchCheck`. **Not done**: a
|
||||
"run check now" endpoint (no scheduler on-demand entrypoint exists yet),
|
||||
relationship editing, and an entity attribute editor UI.
|
||||
|
||||
**Not yet deployed** — the live `oikos-api`/`oikos-scheduler`/nomos
|
||||
containers still run the pre-fix binaries; rebuilding and restarting them
|
||||
needs an explicit go-ahead since it touches the running homelab control
|
||||
plane.
|
||||
|
||||
Addresses five felt problems with the current system: (1) the agent reports
|
||||
stale machine state as if it were fresh, (2) sessions can't be opened and feel
|
||||
disconnected from chat, (3) the Nomos agent re-derives state every turn and
|
||||
wastes iterations, (4) the UI feels dead — tables with no context, no sense of
|
||||
what is monitored, (5) no way to inspect or customize entities and their checks.
|
||||
|
||||
The unifying UX principle for this plan: **master-detail with a detail
|
||||
sidebar**, not full-page navigation. Selecting an entity, session, or signal
|
||||
opens a right-hand detail panel over the current list, so the operator keeps
|
||||
context and drills in without losing their place. Full pages remain
|
||||
addressable (deep links) but are no longer the primary way to inspect a row.
|
||||
|
||||
Sequencing is driven by pain: **drift/staleness is Phase 1.**
|
||||
|
||||
---
|
||||
|
||||
## Root causes (verified in code)
|
||||
|
||||
### Drift / staleness — root cause was worse than a missing TTL
|
||||
Live-DB inspection (`oikos-postgres-1`) found the real cause: `check_defs` has
|
||||
two entity references — `entity_id` (the internal probe/"check" entity) and
|
||||
`target_id` (the host/service actually being observed). The scheduler wrote
|
||||
`entity_status`, `metric_samples`, and scheduler-sourced `events` keyed by
|
||||
`cd.EntityID` (the probe) instead of `cd.TargetID` (the target) —
|
||||
[scheduler.go:110-171](../internal/scheduler/scheduler.go) (pre-fix). Verified
|
||||
against the live database:
|
||||
|
||||
```
|
||||
entity_status by type: only type='check' rows ever had real health (24
|
||||
healthy, 1 down); every host/service/lxc/vm/proxmox-host was frozen at
|
||||
'unknown' since creation.
|
||||
metric_samples: 17,559 rows, 100% attached to type='check' entities — zero
|
||||
attached to any real host or service.
|
||||
events: 45 of 46 scheduler-sourced rows attached to type='check' entities.
|
||||
```
|
||||
|
||||
So this wasn't staleness in the TTL sense — the entities you actually care
|
||||
about (`host:hubris`, `service:authentik`, etc.) **never received an
|
||||
observation at all**. Every health check, metric, and event the scheduler
|
||||
produced was filed under an internal bookkeeping entity the UI doesn't even
|
||||
surface distinctly. This is the literal mechanism behind "the agent tells me
|
||||
stale/wrong state."
|
||||
|
||||
**Fixed** (this session): `runCheck`/`resolveSignal` now resolve
|
||||
`targetID := cd.TargetID` and write status/metrics/events there, falling back
|
||||
to the check's own id only if `target_id` is unset. Signals remain keyed by
|
||||
the check entity (unchanged, matches their existing resolution logic). A new
|
||||
migration ([016_fix_check_status_misattribution](../migrations/016_fix_check_status_misattribution.up.sql))
|
||||
deletes the orphaned check-entity `entity_status` rows so rollups stop
|
||||
double-counting probes as monitored entities; historical `metric_samples` on
|
||||
check entities are left as-is (time-series data, not safe to reattribute).
|
||||
|
||||
On top of the misattribution fix, a genuine staleness gap also existed and is
|
||||
now closed: `entity_status.health` was written only when a check ran, with no
|
||||
TTL — a stalled scheduler or disabled check_def would leave the last health
|
||||
value looking current forever.
|
||||
- `last_check_at` is recorded but was never surfaced. The Entities table
|
||||
showed `entity.updated_at` (row mutation time), not observation time
|
||||
([Entities.svelte:108](../web/src/pages/Entities.svelte), pre-fix).
|
||||
- The `/entities` list endpoint returned neither `health` nor `last_check_at`
|
||||
— only `/graph?include=status` and `/fleet/health` did
|
||||
([impl.go:344](../internal/httpapi/impl.go), pre-fix).
|
||||
|
||||
### Sessions
|
||||
- Clicking a session calls `loadSessionMessages()` but never navigates to the
|
||||
chat page ([Sessions.svelte:17](../web/src/pages/Sessions.svelte)); it mutates
|
||||
the chat store while the user stays on the session list, so nothing appears to
|
||||
happen. There is also no session switcher inside Chat.
|
||||
|
||||
### Agent efficiency
|
||||
- Multi-turn history replay **drops all `tool_use`/`tool_result` pairs**; only
|
||||
prior final text is replayed ([agent.go:108-127](../cmd/nomos/agent.go)). Each
|
||||
new turn re-discovers the fleet from scratch, re-calling tools already run.
|
||||
- Cold start: the system prompt injects no fleet snapshot
|
||||
([agent.go:81](../cmd/nomos/agent.go)); default model is
|
||||
`deepseek/deepseek-v4-flash` ([agent.go:34](../cmd/nomos/agent.go)); tool
|
||||
schema + system prompt are rebuilt each call with no prompt caching.
|
||||
|
||||
### Dead UI / no inspection
|
||||
- Entities table = slug/type/name/state/updated; no health, no last-seen, no
|
||||
signal count.
|
||||
- EntityDetail dumps `JSON.stringify(attributes)` raw
|
||||
([EntityDetail.svelte:123](../web/src/pages/EntityDetail.svelte)) and never
|
||||
shows the entity's `check_defs` — the operator cannot see *what is monitored*,
|
||||
when it last ran, or what it returned.
|
||||
- No CRUD anywhere: no entity editor, no check management (enable/disable/edit/
|
||||
run-now), no relationship editing. `check_defs` do not appear in the web app.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Kill the drift (highest priority)
|
||||
|
||||
Goal: the system never presents stale observations as fresh, and freshness is
|
||||
visible everywhere health is.
|
||||
|
||||
**Backend**
|
||||
- Add a staleness sweep to `housekeeping()`
|
||||
([scheduler.go:243](../internal/scheduler/scheduler.go)): for each
|
||||
`entity_status` where `now() - last_check_at > staleAfter` (default
|
||||
`max(3 × check interval, 5m)`), transition health to a new `stale` value and
|
||||
emit a `health.stale` event once (not every pass).
|
||||
- Treat `stale` as a first-class health in dashboard rollups
|
||||
([dashboard.go:57](../internal/httpapi/dashboard.go)) and fleet health
|
||||
([impl.go:516](../internal/httpapi/impl.go)) — do not fold it into `unknown`.
|
||||
- Extend the `/entities` list response with `health` and `last_check_at`
|
||||
(join `entity_status`), so the table can show freshness without N graph calls.
|
||||
- Nomos: when answering about state, tool results should carry `last_check_at`
|
||||
and a stale flag so the agent can hedge ("healthy as of 4m ago") instead of
|
||||
asserting stale data. (Verify the MCP topology/health tools include it.)
|
||||
|
||||
**Frontend**
|
||||
- Entities table: replace the `Updated` column with **health dot + relative
|
||||
"checked 2m ago"**, and add an **open-signal count** badge per row. Stale rows
|
||||
get a distinct muted/amber treatment, not a green dot.
|
||||
- Global header: add an "as of {time}" and make the SSE connection dot a real
|
||||
liveness indicator (last event received, reconnect state).
|
||||
|
||||
**Acceptance:** disable a check or stop the scheduler → within one stale window
|
||||
the affected entity shows `stale` in the table and dashboard, an event fires,
|
||||
and asking Nomos "is X healthy?" yields a freshness-qualified answer.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Detail sidebar + entity legibility (less navigation)
|
||||
|
||||
Goal: inspect any row in place; make an entity's monitoring self-evident.
|
||||
|
||||
- Introduce a reusable **DetailSheet** (right-side panel) used across Entities,
|
||||
Signals, Sessions, Executions. Row click opens the sheet; URL hash updates for
|
||||
deep-linking; Esc / click-away closes. Full `#/entity/:slug` page remains for
|
||||
direct links but reuses the same detail component.
|
||||
- Entity detail content (in the sheet):
|
||||
- Header: slug, type, **health + freshness** ("checked 2m ago" / "stale
|
||||
18m").
|
||||
- **Monitoring card**: the entity's `check_defs` — kind, schedule, enabled,
|
||||
last result + evidence, next run. This is the missing "what is watched."
|
||||
- Attributes rendered as a key/value panel, not raw JSON.
|
||||
- Relations, open signals, recent executions, metrics sparklines (reuse
|
||||
existing EntityDetail sections).
|
||||
- Backend: endpoint to list `check_defs` for an entity with last-result join
|
||||
(currently checks are only visible to the scheduler).
|
||||
|
||||
**Acceptance:** from the Entities list, one click reveals what an entity is,
|
||||
what's monitoring it, when it was last seen, and its open signals — without a
|
||||
full page load or losing the list.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Sessions rejoin chat
|
||||
|
||||
Goal: sessions are openable and live next to the conversation.
|
||||
|
||||
- Fix: clicking a session navigates to `#/chat` and loads it
|
||||
([Sessions.svelte:17](../web/src/pages/Sessions.svelte)).
|
||||
- Add a **session rail inside Chat** (collapsible left list: title, last-active,
|
||||
active highlight) so switching sessions never leaves the chat surface. The
|
||||
standalone Sessions page becomes a thin wrapper / can be retired from nav.
|
||||
- Show session metadata (message count, last actor) and allow rename/delete.
|
||||
|
||||
**Acceptance:** clicking any past session opens its transcript in the chat view;
|
||||
starting a new chat and switching back and forth works without navigation.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — Agent efficiency
|
||||
|
||||
Goal: stop re-deriving state; start each turn already oriented.
|
||||
|
||||
- Persist and replay tool evidence across turns
|
||||
([agent.go:108-127](../cmd/nomos/agent.go)): either replay `tool_use`/
|
||||
`tool_result` pairs with consistent ids, or persist a compacted per-turn
|
||||
"evidence summary" and replay that. Removes redundant re-querying.
|
||||
- Inject a compact fleet snapshot (counts by health, open signals, stale set)
|
||||
into the system prompt ([agent.go:81](../cmd/nomos/agent.go)) so the agent
|
||||
starts oriented instead of spending iterations on discovery.
|
||||
- Add prompt caching for the system prompt + tool schema (rebuilt every call
|
||||
today); revisit the default model
|
||||
([agent.go:34](../cmd/nomos/agent.go)) — evaluate a stronger default for
|
||||
fewer, better tool calls.
|
||||
- Surface per-turn iteration/token/cost in the chat UI (data already logged to
|
||||
`agent_activity`) so inefficiency is visible and measurable.
|
||||
|
||||
**Acceptance:** a 3-turn conversation about the same entity does not re-call the
|
||||
same read tools each turn; median iterations-per-answer drops.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — Customize & inspect (CRUD)
|
||||
|
||||
Goal: manage the system from the UI, not just observe it.
|
||||
|
||||
- Entity editor (attributes, state) via existing mutation endpoints.
|
||||
- Check management from the entity detail sheet: enable/disable, edit config/
|
||||
thresholds, and **run-now** (trigger a single check pass on demand — new
|
||||
scheduler entrypoint).
|
||||
- Relationship add/remove.
|
||||
- Raw DB-row view toggle in the detail sheet for power inspection.
|
||||
|
||||
---
|
||||
|
||||
## Suggested order of work
|
||||
|
||||
1. Phase 1 backend (staleness sweep + `/entities` health/freshness) →
|
||||
Phase 1 frontend (table freshness + liveness header).
|
||||
2. Phase 2 DetailSheet + entity monitoring card.
|
||||
3. Phase 3 sessions fix (small; can slot in earlier if desired).
|
||||
4. Phase 4 agent efficiency.
|
||||
5. Phase 5 CRUD.
|
||||
|
||||
Phases 1–3 are the ones that most directly turn "the system feels dead and I
|
||||
don't trust it" into "it's alive and I can see and act on it."
|
||||
264
plans/2026-07-08-nomos-resident-agent.md
Normal file
264
plans/2026-07-08-nomos-resident-agent.md
Normal file
@@ -0,0 +1,264 @@
|
||||
# 2026-07-08 — Nomos resident agent (renames Hermes)
|
||||
|
||||
**Status:** In Progress — N0-N3 complete 2026-07-08
|
||||
|
||||
## Goal
|
||||
|
||||
Rename the gateway formerly known as **Hermes** to **Nomos** (from
|
||||
*oikonomos*, the steward of the oikos — and avoiding confusion with Nous
|
||||
Research's unrelated "Hermes Agent" product), and turn it from a keyword
|
||||
router into a resident LLM-backed agent: a long-running service you converse
|
||||
with, which reasons over the full 28-tool MCP surface, holds session context,
|
||||
and is surfaced as the **main entry point of the control-room web UI**
|
||||
([companion plan](2026-07-08-control-room-webui.md)). Oikos already has the
|
||||
agentic substrate — policy classes, approval gating, `agent_activity`, an
|
||||
OODA loop; this gives it the conversational front half.
|
||||
|
||||
## What exists vs what's added
|
||||
|
||||
Today `cmd/hermes/main.go` contains no LLM anywhere: `routeQuery` is
|
||||
`strings.Contains` over ~6 phrases, `extractEntity` knows 5 hardcoded
|
||||
services, and everything else silently falls back to `get_health_summary`.
|
||||
But the hard plumbing already exists and is reused as-is:
|
||||
|
||||
- `mcpClient` (main.go:190) — working Streamable-HTTP MCP client
|
||||
(initialize → session → `tools/call` with SSE frame parsing).
|
||||
- `listTools()` (main.go:318) — currently dead code; extended (below) it
|
||||
becomes the bridge that feeds MCP tool schemas to the LLM.
|
||||
- `.agents/HERMES.md` / `hermes/SOUL.md` — the persona, becomes the system
|
||||
prompt (renamed in N0).
|
||||
- `agent_activity` hypertable + `correlation_id` conventions — where tool
|
||||
calls get logged so the control room can show the agent working.
|
||||
|
||||
## Milestone N0 — the rename (Hermes → Nomos)
|
||||
|
||||
Do this first, as its own commit, before any agent code lands. Scope is the
|
||||
**live service identity**; history (`archive/`, `plans/done/`, past
|
||||
investigations) is never rewritten.
|
||||
|
||||
**Code & build**
|
||||
- `cmd/hermes/` → `cmd/nomos/` (binary `nomos`; keep `serve` subcommand).
|
||||
- Env vars in main.go: `HERMES_MCP_URL`, `HERMES_LISTEN`,
|
||||
`HERMES_AGENT_SLUG` → `NOMOS_*`. Default slug `agent:nomos`;
|
||||
`clientInfo.name` → `nomos`.
|
||||
- `internal/config/config.go`: `HermesAgentSlug` field +
|
||||
`OIKOS_HERMES_AGENT_SLUG` → `NomosAgentSlug` / `OIKOS_NOMOS_AGENT_SLUG`;
|
||||
`hermesAgentID` in `internal/httpapi/server.go:140-149` renamed to match.
|
||||
- `hermes/` directory → `nomos/` (`SOUL.md`, `config.yaml`,
|
||||
`skills/homelab-ops/`); `tools/setup-hermes-soul.sh` →
|
||||
`tools/setup-nomos-soul.sh` with paths updated.
|
||||
|
||||
**Deploy**
|
||||
- `compose/hermes/Dockerfile` → `compose/nomos/Dockerfile`; docker-compose
|
||||
service `hermes:` → `nomos:` (docker-compose.yml:111, env at :63 and :121).
|
||||
- Caddy: `hermes.hubris.network` vhost → `nomos.hubris.network`
|
||||
(compose/caddy/Caddyfile.oikos:26); keep the old hostname as a `redir`
|
||||
block for one transition window; add the DNS record. (These live in the
|
||||
external caddy-conf repo / DNS — operator step.)
|
||||
|
||||
**Data (identity-preserving — do NOT create a new entity)**
|
||||
- Live DB: `UPDATE entities SET slug='agent:nomos',
|
||||
attributes = jsonb_set(attributes,'{name}','"nomos"') WHERE
|
||||
slug='agent:hermes';` — relationships/audit reference the UUID, so history
|
||||
survives the slug change. Ship as a migration so every environment gets it.
|
||||
- `seeds/inventory.yaml:328` (`agent:hermes` entity) and `:541` (owns
|
||||
relationship) → `agent:nomos`. Seed upserts by slug, so the migration must
|
||||
run before seeding or the seed would mint a duplicate entity.
|
||||
|
||||
**Docs & persona**
|
||||
- `.agents/HERMES.md` → `.agents/NOMOS.md`; update every referencing doc
|
||||
(`AGENTS.md`, `CLIENTS.md`, `README.md`, `CONTRIBUTING.md`,
|
||||
`.agents/operations/hermes-agent.md` → `nomos-agent.md`,
|
||||
`.agents/operations/commands.md`, skills docs).
|
||||
- ADR-0012 (`docs/adr/0012-hermes-oikos-interactions.md`): append a
|
||||
"renamed to Nomos" note; don't rewrite the ADR.
|
||||
|
||||
**Explicitly out of scope**
|
||||
- Matrix user `@hermes:hubris.network` (docker-compose.yml:103) — that's the
|
||||
notifier's homeserver account; renaming it is an operational Matrix task,
|
||||
optional and later.
|
||||
- Legacy `bin/hermes` wrapper and `hermesd` on LXC 129 (referenced in
|
||||
`.sops.yaml:89,110`) — separate legacy systems, untouched. Note: that
|
||||
`.sops.yaml` entry shows an **OpenRouter API key path already exists** in
|
||||
the secrets tree — reuse it for N1.
|
||||
- Historical knowledge/investigation pages and executed plans.
|
||||
|
||||
**Verify N0:** `go build ./...`; `docker compose --profile full config`
|
||||
resolves; `curl nomos.hubris.network/healthz` (and the old vhost redirects);
|
||||
`SELECT slug FROM entities WHERE slug LIKE 'agent:%'` shows `agent:nomos`
|
||||
with its original UUID; `grep -ri hermes` returns only history/ADR/legacy
|
||||
hits.
|
||||
|
||||
## Architecture decisions
|
||||
|
||||
### Hand-rolled tool loop, not a hosted connector or off-the-shelf agent
|
||||
|
||||
Three ways to get an LLM driving the tools; decision is (3):
|
||||
|
||||
1. **Hosted MCP connector** (e.g. Anthropic's `mcp_servers` param): zero
|
||||
loop code, but the provider's servers must reach the MCP endpoint over
|
||||
the public internet. `mcp.hubris.network` is currently exposed *without
|
||||
auth* — a hole the [gaps plan](2026-07-08-oikos-gaps-and-improvements.md)
|
||||
says to close, not to build on. Keeping the MCP surface mesh-private is
|
||||
the right posture for a homelab control plane.
|
||||
2. **Off-the-shelf agent framework** (evaluated: Nous Research's
|
||||
[Hermes Agent](https://hermes-agent.nousresearch.com/) — MIT, self-hostable,
|
||||
multi-channel personal assistant with memory/subagents/browsing/code-exec;
|
||||
also the naming-collision motivation for N0). Rejected for this role: it
|
||||
is a desktop/personal-assistant framework with chat-platform frontends,
|
||||
not an embeddable service — there is no clean `/chat` API for the
|
||||
control-room UI to stream from; the oikos-native integration (sessions in
|
||||
Postgres, `agent_activity` + correlation-id joins, approval rail) would
|
||||
not exist; and its browsing/code-exec surface is far larger than a loop
|
||||
confined to the 28 MCP tools. It (or any MCP client) remains usable as an
|
||||
*external* agent against the MCP endpoint — orthogonal to the resident.
|
||||
3. **Hand-rolled agent loop** (~200 lines): Nomos calls the LLM API with
|
||||
tool definitions translated from `tools/list`, executes returned tool
|
||||
calls through its *existing* `mcpClient` inside the mesh, feeds results
|
||||
back, repeats until the model answers in text. Only conversation text
|
||||
leaves the mesh; the tool transport stays private; the agent is confined
|
||||
to the MCP surface by construction.
|
||||
|
||||
### LLM provider: OpenRouter, default model DeepSeek V4 Flash
|
||||
|
||||
- **OpenRouter** (OpenAI-compatible Chat Completions API) rather than a
|
||||
single-vendor SDK: use the official `openai-go` client with
|
||||
`base_url=https://openrouter.ai/api/v1` and `OPENROUTER_API_KEY` (secret
|
||||
path already exists in `.sops.yaml`). Model choice becomes one env var.
|
||||
- **Default model: `deepseek/deepseek-v4-flash`** — supports tool calling,
|
||||
1M context, $0.09/M input / $0.18/M output. Use OpenRouter's **Exacto**
|
||||
routing (highest measured tool-calling accuracy) since tool calls are this
|
||||
agent's entire job; low temperature; strict system prompt.
|
||||
- MCP tool `inputSchema` is already JSON Schema — exactly what
|
||||
`tools[].function.parameters` expects — so translation is a field rename.
|
||||
- **Privacy:** conversation text and tool results (hostnames, log excerpts)
|
||||
transit OpenRouter and the upstream provider. Pin OpenRouter provider
|
||||
preferences to ZDR / no-training providers in the request payload.
|
||||
- A flash-class MoE will be weaker than frontier models on long multi-step
|
||||
chains; mitigations are the iteration cap, Exacto routing, and bumping
|
||||
`NOMOS_MODEL` per-deployment when a task warrants it.
|
||||
|
||||
## Design
|
||||
|
||||
### Agent loop (`cmd/nomos/agent.go`, new)
|
||||
|
||||
```
|
||||
system prompt = SOUL.md content (mounted; provisioned by tools/setup-nomos-soul.sh)
|
||||
tools = listToolsFull() // extend listTools() to return name, description, inputSchema
|
||||
loop (max 15 iterations):
|
||||
resp = chat.completions (OpenRouter, model, system, history, tools)
|
||||
if resp has tool_calls:
|
||||
for each: result = mcpClient.callTool(name, args) // existing code path
|
||||
log to agent_activity (correlation_id = session turn id)
|
||||
append assistant msg + tool-role result msgs to history
|
||||
else: final text → stream to caller, persist turn
|
||||
```
|
||||
|
||||
- Model: `deepseek/deepseek-v4-flash` default, `NOMOS_MODEL` override.
|
||||
`max_tokens` and iteration cap configurable; hard per-turn budget so a
|
||||
pathological loop can't burn the API bill.
|
||||
- `OPENROUTER_API_KEY` from env / Infisical / SOPS (existing path).
|
||||
- Mutations need no new guardrails: the agent's only write path is
|
||||
`request_execution`, which flows through the existing risk-class /
|
||||
approval machinery. The agent's actor identity is `agent:nomos`.
|
||||
**Depends on gaps-plan bug A1** (approvals FK) — without that fix the
|
||||
agent's config mutations dead-end silently, which is worse when a
|
||||
conversational agent confidently reports "queued for approval".
|
||||
|
||||
### HTTP surface (`cmd/nomos/main.go`)
|
||||
|
||||
- `POST /chat` `{session_id?, message}` → **SSE stream** of typed events:
|
||||
`text` (deltas), `tool_use` (name + args), `tool_result` (truncated),
|
||||
`done` (session_id, usage). The web UI renders tool calls as inline chips
|
||||
as they happen.
|
||||
- `GET /sessions`, `GET /sessions/{id}` — history for the UI.
|
||||
- `/query` kept for scripts/structured callers. The toy NLU is **removed**
|
||||
(per gaps plan §C): a `query` with no `tool` returns "natural language
|
||||
belongs to /chat" plus the tool list from the now-live `listTools()`.
|
||||
- `/healthz` unchanged.
|
||||
|
||||
### Sessions (Postgres, new migration)
|
||||
|
||||
`agent_sessions (id, title, actor, created_at, last_active_at)` and
|
||||
`agent_messages (id, session_id, role, content JSONB, created_at)` in the
|
||||
shared oikos DB — Nomos gains a `DATABASE_URL` (it's in the same compose
|
||||
stack). DB-backed rather than in-memory so conversations survive restarts
|
||||
and the control room can list/replay them. Tool invocations additionally go
|
||||
to `agent_activity` with the session's correlation_id, so the existing
|
||||
agent-activity page and the ops ledger join up with zero new query paths.
|
||||
|
||||
### Connectivity & auth
|
||||
|
||||
- Compose: the `nomos` service (renamed in N0, `full` profile) gains
|
||||
`OPENROUTER_API_KEY`, `NOMOS_MODEL`, `DATABASE_URL` env.
|
||||
- Caddy: route `oikos.hubris.network/agent/*` → `nomos:8092` **behind the
|
||||
same Authentik forward_auth** as the rest of the vhost, stripping the
|
||||
`/agent` prefix. The web UI then calls same-origin `/agent/chat` — no
|
||||
CORS, and EventSource/fetch-streaming work unmodified.
|
||||
- Nomos enforces trusted-proxy headers (`X-Authentik-Username`) when
|
||||
`NOMOS_TRUSTED_PROXY=true`, and finally implements the `mesh_only: true`
|
||||
check that the config promises but `main.go` never enforces (gaps plan
|
||||
B3). Direct :8092 access stays mesh-only for agents/scripts.
|
||||
|
||||
### Web UI entry point (amends the control-room plan)
|
||||
|
||||
The agent chat becomes the **home view** of the control room (`/ui/#/`):
|
||||
|
||||
- Center: conversation pane (streamed text, expandable tool-call chips
|
||||
showing args/results, correlation-id links into the ops ledger).
|
||||
- Right rail: live context — pending approvals with approve/deny buttons,
|
||||
recent events, health strip. When the agent's `request_execution` needs
|
||||
approval, the approval card appears in the rail *mid-conversation* (via
|
||||
the `approval.created` SSE event) and can be decided without leaving chat.
|
||||
That's the whole product in one screen: ask → watch it act → approve →
|
||||
watch it complete.
|
||||
- A persistent chat drawer is available from every other page.
|
||||
- Session list in the nav; sessions resumable.
|
||||
|
||||
### Later (explicitly out of scope for v1)
|
||||
|
||||
- Matrix bridge: Nomos as a Matrix bot in the operator room, reusing the
|
||||
notifier's homeserver credentials — same `/chat` loop, different frontend.
|
||||
- Proactive mode: agent opens a session itself when a signal fires
|
||||
(escalation-with-context instead of a bare alert).
|
||||
|
||||
## Milestones
|
||||
|
||||
- **N0 — rename:** see above; standalone commit, deployable on its own.
|
||||
- **N1 — loop:** `agent.go` + openai-go (OpenRouter base URL);
|
||||
`listToolsFull()`; `/chat` (non-streaming JSON first); remove toy NLU, fix
|
||||
`/query` fallback + help. Verify by curl: multi-tool question ("what's
|
||||
degraded and what depends on it?") produces chained
|
||||
`get_health_summary` → `get_blast_radius` calls.
|
||||
- **N2 — state + streaming:** sessions migration, DB persistence, SSE
|
||||
streaming on `/chat`, `agent_activity` logging, budgets.
|
||||
- **N3 — UI entry point:** home chat view + context rail + drawer in the
|
||||
control-room SPA (needs control-room M1; rail approvals need M2's
|
||||
`approval.created` event and gaps-plan A1).
|
||||
- **N4 (optional):** Matrix bridge, proactive sessions.
|
||||
|
||||
## Files
|
||||
|
||||
```
|
||||
cmd/nomos/{main.go, agent.go, store.go} # renamed + loop + sessions
|
||||
internal/config/config.go, internal/httpapi/server.go # slug config rename
|
||||
nomos/{SOUL.md, config.yaml, skills/} # renamed dir; drop query_routing
|
||||
tools/setup-nomos-soul.sh
|
||||
migrations/0xx_rename_agent_hermes_to_nomos.up.sql
|
||||
migrations/0xx_agent_sessions.up.sql
|
||||
seeds/inventory.yaml
|
||||
compose/nomos/Dockerfile, docker-compose.yml, Caddyfile.oikos (/agent route, vhost)
|
||||
.agents/NOMOS.md + referencing docs; docs/adr/0012 note
|
||||
web/src/pages/Chat.svelte + lib/stores/chat.ts # control-room home view
|
||||
go.mod # openai-go
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
- N0: see milestone N0 verify list.
|
||||
- N1: `curl -N /agent/chat` with a question requiring 2+ tools; confirm the
|
||||
tool chain in the response and rows in `agent_activity`.
|
||||
- N2: restart nomos mid-session, resume by session_id, history intact.
|
||||
- N3: from the UI home, ask Nomos to restart a low-risk service; watch the
|
||||
tool chips stream, the execution appear in the rail, and (for a gated
|
||||
action) the approval card arrive and be decidable in place.
|
||||
387
plans/2026-07-08-signal-triggers.md
Normal file
387
plans/2026-07-08-signal-triggers.md
Normal file
@@ -0,0 +1,387 @@
|
||||
# 2026-07-08 — Signal triggers: host health checks
|
||||
|
||||
**Status:** Implemented (Phases 1-5 complete)
|
||||
|
||||
## Goal
|
||||
|
||||
Add a comprehensive set of signal triggers for host-level monitoring —
|
||||
network reachability, CPU/memory/disk pressure, thermal state, pending
|
||||
updates, disk health, ZFS pool status, and process liveness. Every check
|
||||
raises properly deduped signals through the existing `UpsertSignal` path
|
||||
and feeds the OODA pipeline (observe → classify → decide → act).
|
||||
|
||||
---
|
||||
|
||||
## 1. New check kinds
|
||||
|
||||
| Kind | What it measures | Signal kind | Severity mapping |
|
||||
|------|-----------------|-------------|------------------|
|
||||
| `ping` | ICMP reachability + RTT | `ping-unreachable` | down → critical |
|
||||
| `cpu` | Usage % + thermal (Linux only) | `cpu-pressure`, `cpu-thermal` | >90% → warning, >95% → critical; temp >85°C → critical |
|
||||
| `memory` | RAM usage % | `memory-pressure` | >90% → warning, >95% → critical |
|
||||
| `load` | Load avg / CPU count | `load-pressure` | >CPU×2 → warning, >CPU×4 → critical |
|
||||
| `swap` | Swap usage % | `swap-pressure` | >50% → warning, >80% → critical |
|
||||
| `disk-usage` | Already exists as `disk` — enhance with inode %, per-mountpoint | `disk-full` | >85% → warning, >95% → critical |
|
||||
| `disk-smart` | SMART pre-failure attributes | `disk-smart-fail` | any fail → critical |
|
||||
| `updates` | Pending apt updates (security, critical) | `updates-pending` | security >0 → warning, critical-reboot >0 → critical |
|
||||
| `zfs` | Pool health + scrub status | `zfs-degraded`, `zfs-scrub-overdue` | degraded → critical, scrub >30d → warning |
|
||||
| `process` | Process/service running | `process-down` | not running → critical |
|
||||
| `uptime` | Detect unexpected reboots | `uptime-bounce` | < previous → warning |
|
||||
|
||||
---
|
||||
|
||||
## 2. Execution model
|
||||
|
||||
Two tiers based on where the check runs:
|
||||
|
||||
### Tier A: Local (scheduler host)
|
||||
|
||||
`ping`, `http`, `tcp`, `cert-expiry` — run directly from the scheduler process. Already implemented for `http`/`tcp`/`disk`/`cert-expiry`. Add `ping` here.
|
||||
|
||||
### Tier B: Remote via SSH (`ssh-script`)
|
||||
|
||||
`cpu`, `memory`, `load`, `swap`, `disk-smart`, `updates`, `zfs`, `process`, `uptime` — run on the target host via SSH. The existing `ssh-script` kind (defined in OpenAPI, not implemented in scheduler) is the one generic mechanism.
|
||||
|
||||
### Why one `ssh-script` kind instead of separate kinds per metric
|
||||
|
||||
The check runs a small allowlisted shell snippet on the target host via SSH.
|
||||
The script outputs JSON with `health`, `signalKind`, `evidence`, and optional
|
||||
`metrics` (name→value map). This keeps the scheduler simple — one code path for
|
||||
all remote checks — while the check definition's `config.script` field encodes
|
||||
what to run.
|
||||
|
||||
### SSH config shape
|
||||
|
||||
```json
|
||||
{
|
||||
"host": "192.168.30.10",
|
||||
"port": 22,
|
||||
"user": "root",
|
||||
"script": "cpu_check.sh",
|
||||
"timeout_s": 10
|
||||
}
|
||||
```
|
||||
|
||||
Scripts live in `/opt/oikos/checks/` on each host, deployed by the Homelab sync
|
||||
timer alongside the AGENTS.md context. They are allowlisted — the scheduler only
|
||||
executes scripts whose names match `^[a-z][a-z0-9_-]+\.sh$` and that exist in the
|
||||
check directory.
|
||||
|
||||
---
|
||||
|
||||
## 3. Check scripts (per host, `/opt/oikos/checks/`)
|
||||
|
||||
### `cpu_check.sh`
|
||||
```sh
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
USAGE=$(top -bn1 | awk '/^%Cpu/ {print 100 - $8}')
|
||||
CORES=$(nproc)
|
||||
TEMP=""
|
||||
if [ -f /sys/class/thermal/thermal_zone0/temp ]; then
|
||||
TEMP=$(echo "scale=1; $(cat /sys/class/thermal/thermal_zone0/temp) / 1000" | bc)
|
||||
fi
|
||||
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"cpu_pct\":$USAGE,\"cpu_temp\":$TEMP}}"
|
||||
```
|
||||
Signal raised by scheduler logic when `cpu_pct > threshold_pct` or `cpu_temp > threshold_temp`.
|
||||
|
||||
### `memory_check.sh`
|
||||
```sh
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
MEMINFO=$(awk '/MemTotal|MemAvailable|SwapTotal|SwapFree/ {printf "\"%s\":%d,", tolower($1), $2}' /proc/meminfo | sed 's/,$//')
|
||||
USED_PCT=$(python3 -c "print(round((1 - ${memavailable:-0}/${memtotal:-1})*100, 1))")
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"mem_pct\":$USED_PCT}}"
|
||||
```
|
||||
Actual implementation would precompute from `/proc/meminfo` in bash directly (avoid python dependency).
|
||||
|
||||
### `load_check.sh`
|
||||
```sh
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
LOAD=$(awk '{print $1}' /proc/loadavg)
|
||||
CORES=$(nproc)
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"load1\":$LOAD,\"cores\":$CORES}}"
|
||||
```
|
||||
Scheduler computes `load_pct = load1 / cores` and thresholds on that.
|
||||
|
||||
### `swap_check.sh`
|
||||
Reports swap used / swap total from `/proc/meminfo`.
|
||||
|
||||
### `disk_smart_check.sh`
|
||||
```sh
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
FAIL=0
|
||||
for dev in $(lsblk -ndo NAME,TYPE | awk '$2=="disk"{print "/dev/"$1}'); do
|
||||
smartctl -H "$dev" | grep -q "PASSED" || { FAIL=1; break; }
|
||||
done
|
||||
[ $FAIL -eq 0 ] && echo '{"health":"healthy"}' || echo '{"health":"degraded","signalKind":"disk-smart-fail","evidence":"SMART health check failed"}'
|
||||
```
|
||||
|
||||
### `updates_check.sh`
|
||||
```sh
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
apt update -qq >/dev/null 2>&1
|
||||
SECURITY=$(apt list --upgradable 2>/dev/null | grep -c '\-security' || true)
|
||||
REBOOT=$(test -f /var/run/reboot-required && echo 1 || echo 0)
|
||||
echo "{\"health\":\"healthy\",\"metrics\":{\"security_updates\":$SECURITY,\"reboot_required\":$REBOOT}}"
|
||||
```
|
||||
Scheduler thresholds: `security_updates > 0` → warning signal, `reboot_required > 0` → critical.
|
||||
|
||||
### `zfs_check.sh`
|
||||
```sh
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
STATUS=$(zpool status -x 2>&1)
|
||||
if echo "$STATUS" | grep -q "all pools are healthy"; then
|
||||
echo '{"health":"healthy"}'
|
||||
else
|
||||
echo "{\"health\":\"degraded\",\"signalKind\":\"zfs-degraded\",\"evidence\":\"$(echo $STATUS | head -1)\"}"
|
||||
fi
|
||||
```
|
||||
|
||||
### `process_check.sh`
|
||||
Runs `systemctl is-active <service>` for a service name passed in config.
|
||||
Signal on `inactive`/`failed`.
|
||||
|
||||
### `uptime_check.sh`
|
||||
Reads current uptime, compares to last stored value (in a temp file or via metric).
|
||||
Signal if uptime went backward (reboot detected).
|
||||
|
||||
---
|
||||
|
||||
## 4. Scheduler changes
|
||||
|
||||
### `internal/scheduler/scheduler.go`
|
||||
|
||||
Add two new cases to `executeCheck()`:
|
||||
|
||||
```go
|
||||
case "ping":
|
||||
return checkPing(ctx, cd)
|
||||
case "ssh-script":
|
||||
return checkSSHScript(ctx, cd)
|
||||
```
|
||||
|
||||
Additional changes:
|
||||
- **Metric recording**: stop hardcoding `probe_latency_ms`. Each check returns
|
||||
a `map[string]float64` of metrics, and the scheduler writes all of them.
|
||||
Change `executeCheck` signature from `(health, signalKind, evidence, err)`
|
||||
to also return `metrics map[string]float64`.
|
||||
- **Threshold evaluation for metric-based checks** (`cpu`, `memory`, `load`,
|
||||
`swap`, `updates`): the scheduler reads `config.thresholds` from check config
|
||||
JSONB and evaluates metrics against them.
|
||||
|
||||
### `checkPing()`
|
||||
|
||||
Uses `golang.org/x/net/icmp` + `golang.org/x/net/ipv4` for non-privileged
|
||||
ICMP echo (or falls back to `net.DialTimeout("ip4:icmp", ...)`).
|
||||
On macOS, `ping -c 1 -t <timeout>` via exec as fallback (ICMP raw sockets
|
||||
require root on macOS). Returns `probe_latency_ms` metric.
|
||||
|
||||
Config:
|
||||
```json
|
||||
{
|
||||
"host": "192.168.30.10",
|
||||
"count": 1,
|
||||
"timeout_s": 5
|
||||
}
|
||||
```
|
||||
|
||||
### `checkSSHScript()`
|
||||
|
||||
Connects via `golang.org/x/crypto/ssh` using agent forwarding or key file.
|
||||
Executes the allowlisted script path, validates JSON output, returns metrics
|
||||
and health.
|
||||
|
||||
Config:
|
||||
```json
|
||||
{
|
||||
"host": "192.168.30.10",
|
||||
"port": 22,
|
||||
"user": "root",
|
||||
"script": "cpu_check.sh",
|
||||
"thresholds": {
|
||||
"cpu_pct": {"warn": 90, "crit": 95},
|
||||
"cpu_temp": {"crit": 85}
|
||||
},
|
||||
"timeout_s": 10
|
||||
}
|
||||
```
|
||||
|
||||
### Threshold config schema
|
||||
|
||||
Each check kind with metric-based thresholds adds a `thresholds` key:
|
||||
|
||||
```json
|
||||
{
|
||||
"thresholds": {
|
||||
"<metric_name>": {"warn": <float>, "crit": <float>}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Missing thresholds → no signal raised; metrics still recorded.
|
||||
|
||||
---
|
||||
|
||||
## 5. DB and migration
|
||||
|
||||
No schema migration needed. `check_defs.config` is JSONB — new check kinds
|
||||
use it with their own config shapes. `signals` table handles any `kind` string.
|
||||
|
||||
One small addition: add the new signal kinds to the OpenAPI `CheckKind` enum
|
||||
and the generated code (`api/openapi.yaml` line 2247, `internal/httpapi/gen/api.gen.go` line 84).
|
||||
|
||||
---
|
||||
|
||||
## 6. Seed data: default checks per host
|
||||
|
||||
Add to `seeds/inventory.yaml` or a new `seeds/checks.yaml` — a set of default
|
||||
checks per entity type:
|
||||
|
||||
- Every `machine`, `workstation`, `proxmox-host`, `lxc` gets:
|
||||
- `cpu` (via ssh-script)
|
||||
- `memory` (via ssh-script)
|
||||
- `load` (via ssh-script)
|
||||
- `swap` (via ssh-script)
|
||||
- `disk-usage` (local or ssh-script for remote)
|
||||
- `updates` (via ssh-script)
|
||||
- `uptime` (via ssh-script)
|
||||
- Every `machine`, `proxmox-host`, `standalone-server` additionally gets:
|
||||
- `disk-smart` (via ssh-script)
|
||||
- `zfs` (via ssh-script) if `storage-pool` edges exist
|
||||
- Every `service` gets:
|
||||
- `process` (via ssh-script, `systemctl is-active`)
|
||||
|
||||
Default intervals:
|
||||
- `cpu`, `memory`, `load`: 60s
|
||||
- `disk-usage`, `swap`: 300s
|
||||
- `ping`: 30s
|
||||
- `updates`: 3600s (hourly)
|
||||
- `disk-smart`, `zfs`: 86400s (daily)
|
||||
- `process`: 30s
|
||||
|
||||
---
|
||||
|
||||
## 7. Policy integration
|
||||
|
||||
New approval rule for `ssh-script` checks:
|
||||
|
||||
```yaml
|
||||
# ssh-script execution is read_only on the target — it only reads metrics
|
||||
- entity_type: machine
|
||||
action: health-check
|
||||
risk_class: read_only
|
||||
autonomy: auto
|
||||
```
|
||||
|
||||
The `ping` kind is `read_only` (no mutation). All new checks are `read_only`
|
||||
— they observe state, no action taken automatically. The *response* to signals
|
||||
(restart service, clear cache, apt upgrade) goes through the existing
|
||||
classification → approval → execution pipeline separately.
|
||||
|
||||
---
|
||||
|
||||
## 8. Other common important signals (per user request)
|
||||
|
||||
Beyond the core checks above, these are worth including:
|
||||
|
||||
| # | Trigger | Why important |
|
||||
|---|---------|---------------|
|
||||
| 11 | **Inode exhaustion** | Filesystem can be "full" with free space but zero inodes (Docker overlay, mail queues). Distinct from disk-usage. |
|
||||
| 12 | **OOM kills** | `dmesg | grep -i 'out of memory'` count since last boot. Indicates memory pressure beyond usage %. |
|
||||
| 13 | **Journal errors** | `journalctl -p err -S -1h --no-pager | wc -l`. Catches kernel panics, segfaults, service failures. |
|
||||
| 14 | **Docker/container health** | `docker ps --filter health=unhealthy`. Catches containers in unhealthy state. |
|
||||
| 15 | **Caddy/nginx error rate** | Parse access logs for 5xx rate over last 5min. Expensive, do at 300s interval. |
|
||||
| 16 | **Time drift** | `chronyc tracking | grep 'System time'`. NTP offset > 1s → warning (affects TLS, auth, DB). |
|
||||
| 17 | **Open file descriptors** | `/proc/sys/fs/file-nr` ratio used/total. >80% → warning (service exhaustion). |
|
||||
| 18 | **Backup freshness** | Check timestamp of last backup file. >schedule+grace → critical. |
|
||||
|
||||
All of these (11–18) are implemented as `ssh-script` checks with
|
||||
corresponding scripts in `/opt/oikos/checks/`.
|
||||
|
||||
---
|
||||
|
||||
## 9. Implementation order
|
||||
|
||||
### Phase 1: Foundation (2-3 days)
|
||||
|
||||
1. **Refactor `executeCheck`** to return metrics map. Update all existing check
|
||||
functions (`http`, `tcp`, `disk`, `cert-expiry`).
|
||||
2. **Add `ping` kind** — ICMP reachability on the scheduler host.
|
||||
3. **Add `ssh-script` kind** — SSH execution engine, script allowlisting,
|
||||
JSON output parsing.
|
||||
4. **Add threshold evaluation** — scheduler reads `config.thresholds` and
|
||||
compares against returned metrics to decide health/signal.
|
||||
|
||||
### Phase 2: Host check scripts (1-2 days)
|
||||
|
||||
5. Write and test each script in `/opt/oikos/checks/`:
|
||||
`cpu_check.sh`, `memory_check.sh`, `load_check.sh`, `swap_check.sh`,
|
||||
`disk_smart_check.sh`, `updates_check.sh`, `zfs_check.sh`,
|
||||
`process_check.sh`, `uptime_check.sh`, `oom_check.sh`, `journal_check.sh`,
|
||||
`time_check.sh`, `fd_check.sh`.
|
||||
6. Add `tools/setup-checks.sh` to auto-deploy scripts via the sync timer
|
||||
(same pattern as `tools/setup-caveman.sh`).
|
||||
|
||||
### Phase 3: Seed data + API (1 day)
|
||||
|
||||
7. Add `seeds/checks.yaml` with default check definitions per entity type.
|
||||
8. Add new check kinds to OpenAPI spec and regenerate Go types.
|
||||
9. Add a `/api/v1/checks/defaults/{entity_type}` endpoint that returns
|
||||
recommended checks for a given entity type (convenience for operators).
|
||||
|
||||
### Phase 4: Policy + observability (1 day)
|
||||
|
||||
10. Add `read_only` policy rules for `ping` and `ssh-script` actions.
|
||||
11. Add per-check-kind metric recording (not just `probe_latency_ms`).
|
||||
12. Wire `cpu_temp`, `mem_pct`, `load_pct`, etc. into `query_metrics` and
|
||||
the MCP `get_trend` tool.
|
||||
|
||||
### Phase 5: Docker + service signals (1 day)
|
||||
|
||||
13. `docker_health_check.sh` — `docker ps --filter health=unhealthy`.
|
||||
14. `caddy_error_rate.sh` — parse Caddy JSON logs for 5xx.
|
||||
15. `backup_freshness.sh` — check last backup timestamp.
|
||||
|
||||
---
|
||||
|
||||
## 10. Dependencies
|
||||
|
||||
| Dependency | For | Risk |
|
||||
|------------|-----|------|
|
||||
| `golang.org/x/crypto/ssh` | SSH client in scheduler | Already in `go.mod` (used by deployer) |
|
||||
| `golang.org/x/net/icmp` + `ipv4` | ICMP ping | New dep; macOS needs root for raw sockets → exec fallback |
|
||||
| `smartmontools` on hosts | `disk_smart_check.sh` | Already installed on Proxmox; add to LXCs |
|
||||
| `zfsutils-linux` on hosts | `zfs_check.sh` | Already on Proxmox; add to LXCs with ZFS |
|
||||
| SSH key on scheduler | `ssh-script` to all hosts | Already deployed (Homelab sync SSH keys) |
|
||||
|
||||
---
|
||||
|
||||
## 11. Risks and mitigations
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| `ssh-script` is a remote exec vector | Scripts are allowlisted by name (`^[a-z][a-z0-9_-]+\.sh$`), deployed via git (auditable), and read-only (no mutation). SSH key restricted to a dedicated `oikos-check` user with sudo only for `systemctl is-active`. |
|
||||
| ICMP requires root on macOS | Fallback to `ping` CLI via `os/exec`. Production runs on Linux (strong) where raw sockets work. |
|
||||
| Metrics cardinality explosion | Metrics are per-check-definition, not per-script-output. The script returns a fixed set of known metric names. TimescaleDB handles the volume. |
|
||||
| Check script drift between hosts | Scripts deploy via `tools/setup-checks.sh` in the sync timer — same mechanism that keeps AGENTS.md in sync. Checksum validation before execution. |
|
||||
|
||||
---
|
||||
|
||||
## 12. Verification
|
||||
|
||||
- **Unit tests**: each check function (`checkPing`, `checkSSHScript`) tested
|
||||
with mock SSH server and mock ICMP responses.
|
||||
- **Integration test**: deploy to `strong` (macOS scheduler host), define
|
||||
checks for `hubris` (Proxmox), `dns` (LXC), `caddy` (LXC), run scheduler
|
||||
with `--check-interval 10s`, verify signals appear in `get_signal_history`.
|
||||
- **MCP smoke test**: `search_knowledge("signal triggers")`, `get_signal_history`,
|
||||
`query_metrics(metric=["cpu_pct", "mem_pct"])`, `get_trend`.
|
||||
- **Policy smoke test**: `preflight(service:caddy, restart)` still returns
|
||||
correct risk class despite new check kinds in the DB.
|
||||
160
plans/2026-07-09-chat-sessions-improvements.md
Normal file
160
plans/2026-07-09-chat-sessions-improvements.md
Normal file
@@ -0,0 +1,160 @@
|
||||
# 2026-07-09 — Chat sessions: reliability, cost, and session-management fixes
|
||||
|
||||
**Status:** Planned
|
||||
|
||||
## Goal
|
||||
|
||||
Fix concrete problems found by inspecting the *actual* production Nomos chat
|
||||
data (`agent_sessions`/`agent_messages` on the `oikos` Postgres, 24 sessions /
|
||||
52 messages as of 2026-07-09), not a code-review of the UI in the abstract.
|
||||
Findings below are backed by real rows, not hypotheticals.
|
||||
|
||||
## How this was investigated
|
||||
|
||||
Queried `oikos-postgres-1` directly (`docker exec oikos-postgres-1 psql -U oikos
|
||||
-d oikos`) since this runs on mac-mini, the same host as the production
|
||||
containers. Pulled session list, per-message content sizes, tool-call
|
||||
breakdowns, and cross-checked against `cmd/nomos/agent.go` to explain what was
|
||||
observed.
|
||||
|
||||
---
|
||||
|
||||
## Findings
|
||||
|
||||
### 1. ~17% of turns come back completely empty, silently
|
||||
|
||||
4 of 24 sessions ("hi" ×3, "what services are healthy?") have an assistant
|
||||
message with `text=""` and zero tool calls — the model returned a blank
|
||||
completion. `cmd/nomos/agent.go:175-183` treats `len(msg.ToolCalls) == 0` as a
|
||||
normal final answer and emits `text: ""` + `done`. No `error` event fires, so
|
||||
[chat.ts](../web/src/lib/stores/chat.ts) never sets `$error`, and
|
||||
[Chat.svelte](../web/src/pages/Chat.svelte) renders a permanently-blank
|
||||
assistant bubble (the "…" typing dots only show while `$streaming` is true;
|
||||
once `done` fires they vanish, leaving nothing). The user has no idea the turn
|
||||
failed and no obvious way to retry — they have to notice the silence and
|
||||
retype.
|
||||
|
||||
**Fix:** in `agent.chat()`, if `msg.Content == "" && len(msg.ToolCalls) == 0`,
|
||||
treat it as a retryable failure: log it, retry once against the provider
|
||||
before giving up, and if still empty, emit a real `error` event instead of an
|
||||
empty `text`/`done` pair. On the frontend, surface a "Nomos didn't respond —
|
||||
retry?" affordance on empty assistant messages rather than a silent blank
|
||||
bubble.
|
||||
|
||||
### 2. A tool-heavy turn came back as a canned non-English refusal
|
||||
|
||||
The session "what are the termals of hubris?" ran 22 tool calls and then
|
||||
returned, verbatim: `关于这个问题,我没有相关信息,您可以尝试问我其它问题,我会尽力为您解答~`
|
||||
("I don't have relevant information on this, try asking me something else").
|
||||
This is after successfully gathering data via tools — the model discarded its
|
||||
own tool results and emitted a boilerplate deflection in the wrong language.
|
||||
`NOMOS_MODEL` is currently a DeepSeek flash-tier model on OpenRouter, which is
|
||||
consistent with this kind of degraded-tier fallback text leaking through.
|
||||
|
||||
**Fix:** add a response-quality guard in `agent.chat()` — if the final text
|
||||
doesn't match the conversation's language/looks like a canned refusal (simple
|
||||
heuristic: non-ASCII-majority reply to an ASCII-majority conversation, or
|
||||
matches a small denylist of known refusal boilerplate), treat it like the
|
||||
empty-response case (retry, then surface an error rather than showing it to
|
||||
the operator as a real answer). Separately, reconsider whether the flash-tier
|
||||
model is worth the latency/cost tradeoff given it's producing failures like
|
||||
this in a small sample — worth an eval pass against a couple of alternative
|
||||
OpenRouter models on the same 24 real prompts before deciding.
|
||||
|
||||
### 3. Simple fleet questions fan out into dozens of individual tool calls
|
||||
|
||||
"Are any of the proxmox hosts saturated?" (2 hosts) triggered **70 tool
|
||||
calls** in one turn, 21 of them individual `get_lxc_state` calls — one per LXC
|
||||
container — instead of using the already-available `list_lxcs()` bulk tool.
|
||||
Similar pattern in "What should be updated with high priority?" (68 calls) and
|
||||
"What needs updating?" (54 calls). Each `get_lxc_state` is a live `pct status`
|
||||
SSH round-trip to the Proxmox host, so this is 21 sequential SSH round trips
|
||||
to answer a question `list_lxcs()` already answers in one call. This is the
|
||||
direct cause of both slow responses and the huge persisted payloads in
|
||||
finding 4.
|
||||
|
||||
**Fix:** two angles, not mutually exclusive:
|
||||
- **Prompt-level**: tighten the Nomos system prompt (`nomos/SOUL.md`) to
|
||||
explicitly prefer bulk tools (`list_lxcs`, `get_state_snapshot`,
|
||||
`query_metrics`) over per-entity tools when the question is fleet-wide, and
|
||||
only fall back to `get_lxc_state`/`tail_log` for a specific named entity.
|
||||
- **Tool-level**: `get_lxc_state` already exists per-slug; consider whether
|
||||
`list_lxcs()`'s summary is actually sufficient for "saturated" (CPU/mem %
|
||||
per container) — if it's missing that field, that's *why* the model loops
|
||||
per-container, and the real fix is enriching `list_lxcs()` rather than
|
||||
prompting around the gap.
|
||||
|
||||
### 4. Tool results are persisted raw and unbounded, inflating messages to 100KB+
|
||||
|
||||
Message content sizes in `agent_messages.content` (JSONB) range up to
|
||||
**106KB** for a single assistant turn. Even a plain "hi" greeting produced a
|
||||
44KB message, because `get_state_snapshot()`'s full result — every entity in
|
||||
the DB, including ~15 `document:containers/*` rows that are all
|
||||
`state: <nil>, health: unknown` and contribute nothing — gets embedded
|
||||
verbatim in the `tool_calls[].result` field and stored as-is
|
||||
(`cmd/nomos/store.go:68-76` just JSON-inserts whatever the tool returned).
|
||||
This bloats the DB, and every time a session is opened via
|
||||
[loadSessionMessages](../web/src/lib/stores/chat.ts#L56) or the
|
||||
[SessionRail](../web/src/lib/components/SessionRail.svelte)/
|
||||
[Sessions](../web/src/pages/Sessions.svelte) page loads history, the browser
|
||||
downloads and parses all of it just to render a collapsed tool-call summary.
|
||||
|
||||
**Fix:**
|
||||
- Filter `get_state_snapshot()`'s result server-side (in the MCP tool, not
|
||||
the agent) to drop entities with no meaningful state/health signal, or add
|
||||
a `type` filter param the agent can pass.
|
||||
- In `store.saveMessage`, cap persisted tool-result size (e.g. truncate to a
|
||||
few KB with a `"...truncated, N bytes"` marker) — the full result already
|
||||
served its purpose informing that turn's answer; historical replay
|
||||
(`agent.chat()`'s history-replay loop at `agent.go:122-137`) doesn't need
|
||||
the full blob, just enough for the model to know what it already checked.
|
||||
|
||||
### 5. No session hygiene: duplicate/typo'd titles, no delete/archive
|
||||
|
||||
Session titles are the raw, unprocessed first user message
|
||||
(`store.createSession`), with no dedup, normalization, or cleanup. Real
|
||||
production titles include **6 sessions titled "hi"**, **2 titled "say ok"**,
|
||||
and typos preserved verbatim ("what are the **termals** of hubris?", "whats
|
||||
the **termans** of strong", "Are any of the **proxomox** hosts saturated?").
|
||||
Neither [Sessions.svelte](../web/src/pages/Sessions.svelte) nor
|
||||
[SessionRail.svelte](../web/src/lib/components/SessionRail.svelte) nor the
|
||||
`store`/API layer (`cmd/nomos/store.go`, `web/src/lib/api.ts`) has any delete
|
||||
or archive path — grepped the whole stack, confirmed absent. Throwaway test
|
||||
sessions accumulate forever with no way to clean them from the UI.
|
||||
|
||||
**Fix:**
|
||||
- Add `DELETE /sessions/{id}` to the nomos gateway + a matching store method
|
||||
and wire a delete affordance into `SessionRail`/`Sessions` (hover trash
|
||||
icon, confirm on click).
|
||||
- Generate titles from the assistant's actual answer once the turn completes
|
||||
(or a cheap follow-up summarization call) instead of the raw first message,
|
||||
so distinct "hi" sessions become distinguishable by what was actually
|
||||
discussed.
|
||||
|
||||
---
|
||||
|
||||
## Implementation order
|
||||
|
||||
1. **Empty-response + refusal-leak guard** (`cmd/nomos/agent.go`) — highest
|
||||
user-visible impact, smallest change, no schema/API changes.
|
||||
2. **Bulk-tool prompting fix** (`nomos/SOUL.md`) — cheap, directly cuts
|
||||
latency and tool-call volume; re-run the same 24 real prompts against the
|
||||
updated prompt to confirm `get_lxc_state` fan-out drops.
|
||||
3. **Tool-result truncation on persist** (`cmd/nomos/store.go`) — bounds
|
||||
future DB growth; pair with a one-off cleanup pass on the 52 existing rows
|
||||
if the table needs to be shrunk immediately.
|
||||
4. **`get_state_snapshot` filtering** — coordinate with whichever MCP tool
|
||||
file defines it; verify with `jsonb_pretty` on a fresh "hi" session that
|
||||
payload drops well below the current ~44KB.
|
||||
5. **Session delete + title generation** — UI + gateway change, lowest risk,
|
||||
can ship independently of 1-4.
|
||||
|
||||
## Verification
|
||||
|
||||
- Re-run the same 24 real user prompts (recorded in this plan's investigation)
|
||||
against the patched agent; confirm zero empty/refusal-leak responses and
|
||||
`get_lxc_state`-style fan-out drops to O(hosts) not O(containers).
|
||||
- `docker exec oikos-postgres-1 psql -U oikos -d oikos -c "SELECT max(length(content::text)) FROM agent_messages;"`
|
||||
before/after — expect the ceiling to move from ~106KB to low single-digit KB.
|
||||
- Manually delete a test session via the new UI affordance, confirm it's gone
|
||||
from both `SessionRail` and the `agent_sessions` table.
|
||||
@@ -11,8 +11,9 @@ went sideways, open an investigation.
|
||||
| 2026-07-05 | [Oikos Prometheus LXC](2026-07-05-oikos-prometheus-lxc.md) | Planned |
|
||||
| 2026-07-08 | [Plan vs implementation cross-reference](2026-07-08-plan-implementation-audit.md) | Planned |
|
||||
| 2026-07-08 | [Oikos gaps, broken things, and improvements](2026-07-08-oikos-gaps-and-improvements.md) | Planned |
|
||||
| 2026-07-08 | [Control room web UI](2026-07-08-control-room-webui.md) | Planned |
|
||||
| 2026-07-08 | [Hermes resident agent](2026-07-08-hermes-resident-agent.md) | Planned |
|
||||
| 2026-07-08 | [Control room web UI](2026-07-08-control-room-webui.md) | In Progress |
|
||||
| 2026-07-08 | [Nomos resident agent (renames Hermes)](2026-07-08-nomos-resident-agent.md) | In Progress |
|
||||
| 2026-07-09 | [Chat sessions: reliability, cost, and session-management fixes](2026-07-09-chat-sessions-improvements.md) | Planned |
|
||||
|
||||
## Done
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ Status: [x] = done, [ ] = pending
|
||||
|
||||
- [x] **Backup**: `pg_dump oikos > backups/pre-cutover-20260707.sql` (145K)
|
||||
- [x] **CI green**: pushed to main, `.gitea/workflows/ci.yml` exists
|
||||
- [x] **Deploy test**: Docker stack running with api + scheduler + notifier + hermes
|
||||
- [x] **Deploy test**: Docker stack running with api + scheduler + notifier + nomos
|
||||
- [x] **Caddy config**: `compose/caddy/Caddyfile.oikos` pushed to `dtoro/caddy-conf` (ed20908). Auto-deploys to caddy (121).
|
||||
- [x] **DNS**: `oikos.hubris.network` already resolves to 192.168.8.175 (mac-mini mesh)
|
||||
- [x] **Secrets**: Infisical bootstrapped + migration complete 2026-07-07. All 11 SOPS secrets migrated to Infisical (oikos project, dev env). Machine identity `oikos-api` has RW access verified via Go SDK. ENCRYPTION_KEY must be 32-char raw string (docs incorrect). SOPS fallback preserved for DR. secrets-issuance decommissioned — stopped/disabled on apps/105; superseded by Infisical.
|
||||
@@ -23,7 +23,7 @@ Status: [x] = done, [ ] = pending
|
||||
## Post-cutover verification
|
||||
|
||||
- [x] **./scripts/verify-phase6.sh** — all 14 checks pass
|
||||
- [x] **Hermes query**: `curl http://localhost:8092/query -d '{"query":"fleet health"}'` → HTTP 200
|
||||
- [x] **Nomos query**: `curl http://localhost:8092/query -d '{"query":"fleet health"}'` → HTTP 200
|
||||
- [x] **Agent activity**: `curl http://localhost:8090/api/v1/agent-activity` → returns data
|
||||
- [x] **Scheduler ticking**: 30s ticks logged
|
||||
- [x] **Notifier polling**: running
|
||||
|
||||
@@ -27,7 +27,7 @@ check "4. Scheduler: check pass" "http://localhost:8090/api/v1/check
|
||||
check "5. Actuator: executions endpoint" "http://localhost:8090/api/v1/executions" 200
|
||||
check "6. Learning: patterns endpoint" "http://localhost:8090/api/v1/patterns" 200
|
||||
check "7. Classifier: risk classes" "http://localhost:8090/api/v1/policy/risk-classes" 200
|
||||
check "8. Hermes: gateway health" "http://localhost:8092/healthz" 200
|
||||
check "8. Nomos: gateway health" "http://localhost:8092/healthz" 200
|
||||
check "9. Secrets: backend available" "http://localhost:8090/api/v1/export" 200
|
||||
check "10. Deploy: events endpoint" "http://localhost:8090/api/v1/events" 200
|
||||
check "11. Knowledge: content search" "http://localhost:8090/healthz" 200
|
||||
|
||||
@@ -325,7 +325,7 @@ entities:
|
||||
attributes: {matrix_id: "@dtoro:avispero"}}
|
||||
- {slug: "idp:authentik", type: identity-provider, name: authentik,
|
||||
attributes: {issuer: "https://auth.hubris.network", auth_mode: both}}
|
||||
- {slug: "agent:hermes", type: agent, name: hermes,
|
||||
- {slug: "agent:nomos", type: agent, name: nomos,
|
||||
state: active,
|
||||
attributes: {gateway_port: 8092, session_mode: smart_approve, note: "Phase 4 — Docker gateway mode"}}
|
||||
- {slug: "agent:oikos", type: agent, name: oikos,
|
||||
@@ -334,7 +334,7 @@ entities:
|
||||
|
||||
# ─── Archaeology (state: destroyed — kept for "what happened to X?") ─
|
||||
- {slug: "lxc:claudio-bot", type: lxc, name: claudio-bot, state: destroyed,
|
||||
attributes: {pve_id: 123, destroyed: "2026-06-04", reason: "replaced by Hermes Agent on mac-mini"}}
|
||||
attributes: {pve_id: 123, destroyed: "2026-06-04", reason: "replaced by Nomos Agent on mac-mini"}}
|
||||
- {slug: "lxc:plato", type: lxc, name: plato, state: destroyed,
|
||||
attributes: {pve_id: 126, destroyed: "2026-06-28", reason: "notes workspace decommissioned; data at /mnt/library/documents/plato"}}
|
||||
- {slug: "lxc:mule-photos-new", type: lxc, name: mule-photos-new, state: destroyed,
|
||||
@@ -538,5 +538,5 @@ relationships:
|
||||
- {source: "lxc:rclone", target: "backup:proton-drive", type: backs-up-to}
|
||||
|
||||
# ─── Governance ────────────────────────────────────────────────────
|
||||
- {source: "person:dtoro", target: "agent:hermes", type: owns}
|
||||
- {source: "person:dtoro", target: "agent:nomos", type: owns}
|
||||
- {source: "person:dtoro", target: "agent:oikos", type: owns}
|
||||
|
||||
@@ -531,7 +531,7 @@ entity_types:
|
||||
domain: identity
|
||||
layer: governance
|
||||
lifecycle: infrastructure # agents are deployed/retired like infrastructure
|
||||
description: Software agent actor (Hermes, the Oikos control loop).
|
||||
description: Software agent actor (Nomos, the Oikos control loop).
|
||||
attributes:
|
||||
type: object
|
||||
properties:
|
||||
|
||||
13
tools/setup-checks.sh
Normal file
13
tools/setup-checks.sh
Normal file
@@ -0,0 +1,13 @@
|
||||
#!/usr/bin/env bash
|
||||
# setup-checks.sh — deploy check scripts to /opt/oikos/checks on each host.
|
||||
# Auto-setup hook: tools/*.setup.sh runs after every git pull.
|
||||
set -euo pipefail
|
||||
|
||||
CLONE_DIR="${HOMELAB_CONTEXT_DIR:-/opt/homelab}"
|
||||
CHECK_SETUP="$CLONE_DIR/checks/install.sh"
|
||||
|
||||
if [ -f "$CHECK_SETUP" ]; then
|
||||
bash "$CHECK_SETUP" || echo "[setup-checks] WARNING: install.sh exited with code $?"
|
||||
else
|
||||
echo "[setup-checks] no checks/install.sh found, skipping"
|
||||
fi
|
||||
@@ -1,14 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# setup-hermes-soul.sh — provision Hermes agent persona.
|
||||
# Copies ~/.hermes/SOUL.md from hermes/SOUL.md. No-op on non-Hermes agents.
|
||||
set -euo pipefail
|
||||
|
||||
CLONE_DIR="${HOMELAB_CONTEXT_DIR:-/opt/homelab}"
|
||||
|
||||
if [ -f "$CLONE_DIR/hermes/SOUL.md" ]; then
|
||||
mkdir -p "$HOME/.hermes"
|
||||
cp "$CLONE_DIR/hermes/SOUL.md" "$HOME/.hermes/SOUL.md"
|
||||
echo "[setup-hermes-soul] SOUL.md provisioned"
|
||||
else
|
||||
echo "[setup-hermes-soul] no hermes/SOUL.md found; skipping"
|
||||
fi
|
||||
14
tools/setup-nomos-soul.sh
Normal file
14
tools/setup-nomos-soul.sh
Normal file
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env bash
|
||||
# setup-nomos-soul.sh — provision Nomos agent persona.
|
||||
# Copies ~/.nomos/SOUL.md from nomos/SOUL.md. No-op on non-Nomos agents.
|
||||
set -euo pipefail
|
||||
|
||||
CLONE_DIR="${HOMELAB_CONTEXT_DIR:-/opt/homelab}"
|
||||
|
||||
if [ -f "$CLONE_DIR/nomos/SOUL.md" ]; then
|
||||
mkdir -p "$HOME/.nomos"
|
||||
cp "$CLONE_DIR/nomos/SOUL.md" "$HOME/.nomos/SOUL.md"
|
||||
echo "[setup-nomos-soul] SOUL.md provisioned"
|
||||
else
|
||||
echo "[setup-nomos-soul] no nomos/SOUL.md found; skipping"
|
||||
fi
|
||||
17
web/components.json
Normal file
17
web/components.json
Normal file
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"$schema": "https://shadcn-svelte.com/schema.json",
|
||||
"style": "vega",
|
||||
"tailwind": {
|
||||
"css": "src/app.css",
|
||||
"baseColor": "zinc"
|
||||
},
|
||||
"aliases": {
|
||||
"components": "$lib/components",
|
||||
"utils": "$lib/utils",
|
||||
"ui": "$lib/components/ui",
|
||||
"hooks": "$lib/hooks",
|
||||
"lib": "$lib"
|
||||
},
|
||||
"typescript": true,
|
||||
"registry": "https://shadcn-svelte.com/registry"
|
||||
}
|
||||
0
web/dist/.gitkeep
vendored
Normal file
0
web/dist/.gitkeep
vendored
Normal file
21
web/embed.go
Normal file
21
web/embed.go
Normal file
@@ -0,0 +1,21 @@
|
||||
// Package web embeds the compiled control-room SPA (web/dist) into the oikos
|
||||
// binary, preserving the single-binary deployment (ADR-0001). The dist tree is
|
||||
// produced by `npm run build` (or the Docker ui-builder stage); a committed
|
||||
// web/dist/.gitkeep keeps a backend-only `go build` green when the UI has not
|
||||
// been built.
|
||||
package web
|
||||
|
||||
import (
|
||||
"embed"
|
||||
"io/fs"
|
||||
)
|
||||
|
||||
//go:embed all:dist
|
||||
var dist embed.FS
|
||||
|
||||
// DistFS returns the built SPA rooted at dist/. When the UI has not been built
|
||||
// (only the .gitkeep placeholder is present), Open("index.html") will fail and
|
||||
// the caller serves a 404 — the binary still starts.
|
||||
func DistFS() (fs.FS, error) {
|
||||
return fs.Sub(dist, "dist")
|
||||
}
|
||||
16
web/index.html
Normal file
16
web/index.html
Normal file
@@ -0,0 +1,16 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Oikos — Control Room</title>
|
||||
<link rel="icon" type="image/svg+xml" href="favicon.svg" />
|
||||
<link rel="apple-touch-icon" href="apple-touch-icon.png" />
|
||||
<link rel="icon" type="image/png" sizes="192x192" href="android-chrome-192.png" />
|
||||
<link rel="icon" type="image/png" sizes="512x512" href="android-chrome-512.png" />
|
||||
</head>
|
||||
<body>
|
||||
<div id="app"></div>
|
||||
<script type="module" src="/src/main.ts"></script>
|
||||
</body>
|
||||
</html>
|
||||
2431
web/package-lock.json
generated
Normal file
2431
web/package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
35
web/package.json
Normal file
35
web/package.json
Normal file
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"name": "oikos-web",
|
||||
"private": true,
|
||||
"version": "0.1.0",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
"build": "vite build && touch dist/.gitkeep",
|
||||
"preview": "vite preview"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@internationalized/date": "^3.12.2",
|
||||
"@lucide/svelte": "^1.23.0",
|
||||
"@sveltejs/vite-plugin-svelte": "^5.0.0",
|
||||
"@tailwindcss/vite": "^4.3.2",
|
||||
"@tsconfig/svelte": "^5.0.0",
|
||||
"@types/d3-force": "^3.0.10",
|
||||
"bits-ui": "^2.18.1",
|
||||
"mode-watcher": "^1.1.0",
|
||||
"svelte": "^5.0.0",
|
||||
"svelte-sonner": "^1.1.1",
|
||||
"tailwind-variants": "^3.2.2",
|
||||
"tailwindcss": "^4.3.2",
|
||||
"typescript": "^5.5.0",
|
||||
"vite": "^6.0.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"clsx": "^2.1.1",
|
||||
"d3-force": "^3.0.0",
|
||||
"dompurify": "^3.4.11",
|
||||
"marked": "^18.0.5",
|
||||
"tailwind-merge": "^3.6.0",
|
||||
"uplot": "^1.6.32"
|
||||
}
|
||||
}
|
||||
BIN
web/public/android-chrome-192.png
Normal file
BIN
web/public/android-chrome-192.png
Normal file
Binary file not shown.
BIN
web/public/android-chrome-512.png
Normal file
BIN
web/public/android-chrome-512.png
Normal file
Binary file not shown.
BIN
web/public/apple-touch-icon.png
Normal file
BIN
web/public/apple-touch-icon.png
Normal file
Binary file not shown.
BIN
web/public/favicon.png
Normal file
BIN
web/public/favicon.png
Normal file
Binary file not shown.
4
web/public/favicon.svg
Normal file
4
web/public/favicon.svg
Normal file
@@ -0,0 +1,4 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<svg width="91" height="100" version="1.1" xmlns="http://www.w3.org/2000/svg">
|
||||
<path fill="#ffffff" d="m45.601 1q20.993 0 33.71 15.946 10.799 13.625 10.799 31.287 0 12.414-5.9548 25.131-5.9548 12.717-16.451 19.176-10.395 6.4592-23.213 6.4592-20.892 0-33.205-16.653-10.395-14.029-10.395-31.489 0-12.717 6.2577-25.232 6.3584-12.616 16.653-18.57 10.295-6.0556 21.801-6.0556zm-3.128 6.5605q-5.3492 0-10.799 3.2296-5.3492 3.1287-8.68 11.102-3.3305 7.9735-3.3305 20.488 0 20.185 7.973 34.82 8.0743 14.634 21.195 14.634 9.7896 0 16.149-8.0743 6.3584-8.0743 6.3584-27.755 0-24.627-10.597-38.756-7.1657-9.6888-18.268-9.6888z"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 666 B |
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user