fix: scheduler wrote health/metrics/events to probe entities, not targets
Problem: every host/service/lxc/etc. entity_status row was permanently stuck at 'unknown' since creation. Verified against the live DB: metric_samples had 17,559 rows, 100% attached to type='check' probe entities and 0% to any real monitored entity; only 25 check entities ever had real health written. check_defs.entity_id (the probe's own bookkeeping entity) and check_defs.target_id (the host/service actually being observed) were both real fields, but the scheduler wrote UpsertEntityStatus/InsertMetricSample/emitSchedulerEvent keyed by entity_id instead of target_id — so every check ran and every result was real, it just landed on the wrong row. This is the mechanism behind observed drift: the agent's dashboard/health tools reported the internal probes' state, never the actual fleet. Change: - scheduler.go: runCheck/resolveSignal now resolve targetID from cd.TargetID (falling back to the check's own id if unset) and write status/metrics/events there. Signals stay keyed by the check entity, unchanged, matching their existing resolution logic. - Added a staleness sweep to housekeeping(): an entity whose last observation is older than 3x its fastest enabled check's interval (floor 5m) is marked 'stale' and emits health.stale, so a stalled scheduler or disabled check_def can no longer look like current data forever. - migrations/016: deletes the now-orphaned check-entity entity_status rows so dashboard/fleet-health rollups stop double-counting probes as monitored entities. Historical metric_samples on check entities are left as-is (time-series data, not safe to reattribute). - openapi.yaml + regenerated gen code: Entity gains health/last_check_at; 'stale' added to the health enum everywhere it's used. - dashboard.go / GetFleetHealth / nomos's get_health_summary MCP tool: exclude type='check' entities from rollups. - nomos/agent.go: replay prior turns' tool_use/tool_result pairs into the conversation instead of dropping them (previously only final text was replayed, forcing the agent to re-derive fleet state every turn), and inject a compact live fleet-health snapshot into the system prompt each turn so it starts oriented instead of spending an iteration on discovery. Risk: config_mutation (schema-adjacent — new migration, no destructive DDL, additive DELETE only on orphaned rows). No behavior change until oikos-api/oikos-scheduler/nomos are rebuilt and redeployed. Verification: go build/vet clean across the repo. Ran this worktree's own API binary against the live dev Postgres on an alternate port (read-only from the live containers' perspective) and confirmed /api/v1/entities now returns health/last_check_at, and the dashboard health rollup dropped from double-counting to an honest 168 unmonitored entities (matches reality pre-deploy — the live scheduler hasn't run the fixed code yet). Confirmed check_defs.target_id correctly maps multiple checks to host:hubris via direct psql query. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1914,6 +1914,21 @@ components:
|
||||
updated_at:
|
||||
type: string
|
||||
format: date-time
|
||||
health:
|
||||
type: string
|
||||
description: last observed health, when the entity is monitored
|
||||
nullable: true
|
||||
enum:
|
||||
- healthy
|
||||
- degraded
|
||||
- down
|
||||
- unknown
|
||||
- stale
|
||||
last_check_at:
|
||||
type: string
|
||||
format: date-time
|
||||
nullable: true
|
||||
description: when health was last observed
|
||||
EntityCreate:
|
||||
type: object
|
||||
required:
|
||||
@@ -2020,6 +2035,7 @@ components:
|
||||
- degraded
|
||||
- down
|
||||
- unknown
|
||||
- stale
|
||||
EntityType:
|
||||
type: object
|
||||
required:
|
||||
@@ -3031,6 +3047,9 @@ components:
|
||||
type: integer
|
||||
unknown:
|
||||
type: integer
|
||||
stale:
|
||||
type: integer
|
||||
description: last observation older than the check's expected cadence
|
||||
entities:
|
||||
type: array
|
||||
items:
|
||||
@@ -3051,6 +3070,7 @@ components:
|
||||
- degraded
|
||||
- down
|
||||
- unknown
|
||||
- stale
|
||||
trend:
|
||||
type: string
|
||||
enum:
|
||||
@@ -3100,6 +3120,9 @@ components:
|
||||
type: integer
|
||||
unknown:
|
||||
type: integer
|
||||
stale:
|
||||
type: integer
|
||||
description: last observation older than the check's expected cadence
|
||||
signals_by_severity:
|
||||
type: object
|
||||
description: open (non-resolved) signal counts keyed by severity
|
||||
|
||||
@@ -108,11 +108,16 @@ func (a *agent) chat(ctx context.Context, sessionID, message string, emit func(a
|
||||
// Rebuild conversation context from persisted history so sessions are
|
||||
// multi-turn. The current user turn is saved by the HTTP handler before
|
||||
// this runs, so it is already included in the history for real sessions.
|
||||
// Intermediate tool_use/tool_result pairs are not replayed (their ids
|
||||
// must match exactly or the API rejects them); prior final answers carry
|
||||
// the salient context. Ephemeral sessions (no store) fall back to the
|
||||
// single incoming message.
|
||||
messages := []openai.ChatCompletionMessageParamUnion{openai.SystemMessage(a.system)}
|
||||
// Prior tool_use/tool_result pairs are replayed as a tool-calling
|
||||
// assistant message followed by matching tool-role results, so the agent
|
||||
// starts each turn already knowing what it already checked instead of
|
||||
// re-querying the same tools from scratch. Ephemeral sessions (no store)
|
||||
// fall back to the single incoming message.
|
||||
system := a.system
|
||||
if snapshot := a.fleetSnapshot(); snapshot != "" {
|
||||
system += "\n\n" + snapshot
|
||||
}
|
||||
messages := []openai.ChatCompletionMessageParamUnion{openai.SystemMessage(system)}
|
||||
history, _ := a.store.getMessages(ctx, sessionID)
|
||||
for _, m := range history {
|
||||
text := extractText(m.Content)
|
||||
@@ -120,6 +125,12 @@ func (a *agent) chat(ctx context.Context, sessionID, message string, emit func(a
|
||||
case "user":
|
||||
messages = append(messages, openai.UserMessage(text))
|
||||
case "assistant":
|
||||
if calls := extractToolCalls(m.Content); len(calls) > 0 {
|
||||
messages = append(messages, assistantToolCallMessage(calls))
|
||||
for _, c := range calls {
|
||||
messages = append(messages, openai.ToolMessage(c.resultText(), c.id))
|
||||
}
|
||||
}
|
||||
if text != "" {
|
||||
messages = append(messages, openai.AssistantMessage(text))
|
||||
}
|
||||
@@ -243,6 +254,140 @@ func extractText(content json.RawMessage) string {
|
||||
return m.Text
|
||||
}
|
||||
|
||||
// persistedCall is one merged tool_use+tool_result pair from a persisted
|
||||
// assistant message's tool_calls array. The store keeps them as two entries
|
||||
// sharing the same id (mirroring the SSE event pair); replay needs one
|
||||
// entry per id to build a valid tool-calling assistant message.
|
||||
type persistedCall struct {
|
||||
id string
|
||||
name string
|
||||
args json.RawMessage
|
||||
result json.RawMessage
|
||||
errMsg string
|
||||
}
|
||||
|
||||
func (c persistedCall) resultText() string {
|
||||
if c.errMsg != "" {
|
||||
return c.errMsg
|
||||
}
|
||||
if len(c.result) > 0 {
|
||||
return string(c.result)
|
||||
}
|
||||
return "null"
|
||||
}
|
||||
|
||||
// extractToolCalls parses and merges a persisted message's tool_calls array,
|
||||
// preserving first-seen order across ids.
|
||||
func extractToolCalls(content json.RawMessage) []persistedCall {
|
||||
var m struct {
|
||||
ToolCalls []struct {
|
||||
ID string `json:"id"`
|
||||
Type string `json:"type"`
|
||||
Name string `json:"name"`
|
||||
Args json.RawMessage `json:"args"`
|
||||
Result json.RawMessage `json:"result"`
|
||||
Error string `json:"error"`
|
||||
} `json:"tool_calls"`
|
||||
}
|
||||
if err := json.Unmarshal(content, &m); err != nil || len(m.ToolCalls) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
byID := make(map[string]*persistedCall, len(m.ToolCalls))
|
||||
var order []string
|
||||
for _, tc := range m.ToolCalls {
|
||||
if tc.ID == "" {
|
||||
continue
|
||||
}
|
||||
pc, ok := byID[tc.ID]
|
||||
if !ok {
|
||||
pc = &persistedCall{id: tc.ID}
|
||||
byID[tc.ID] = pc
|
||||
order = append(order, tc.ID)
|
||||
}
|
||||
if tc.Name != "" {
|
||||
pc.name = tc.Name
|
||||
}
|
||||
if len(tc.Args) > 0 && string(tc.Args) != "null" {
|
||||
pc.args = tc.Args
|
||||
}
|
||||
if tc.Type == "tool_result" {
|
||||
pc.errMsg = tc.Error
|
||||
pc.result = tc.Result
|
||||
}
|
||||
}
|
||||
|
||||
calls := make([]persistedCall, 0, len(order))
|
||||
for _, id := range order {
|
||||
calls = append(calls, *byID[id])
|
||||
}
|
||||
return calls
|
||||
}
|
||||
|
||||
// assistantToolCallMessage builds the tool-calling assistant message that
|
||||
// must precede the tool-role results being replayed.
|
||||
func assistantToolCallMessage(calls []persistedCall) openai.ChatCompletionMessageParamUnion {
|
||||
toolCalls := make([]openai.ChatCompletionMessageToolCallParam, 0, len(calls))
|
||||
for _, c := range calls {
|
||||
args := string(c.args)
|
||||
if args == "" {
|
||||
args = "{}"
|
||||
}
|
||||
toolCalls = append(toolCalls, openai.ChatCompletionMessageToolCallParam{
|
||||
ID: c.id,
|
||||
Function: openai.ChatCompletionMessageToolCallFunctionParam{
|
||||
Name: c.name,
|
||||
Arguments: args,
|
||||
},
|
||||
})
|
||||
}
|
||||
return openai.ChatCompletionMessageParamUnion{
|
||||
OfAssistant: &openai.ChatCompletionAssistantMessageParam{ToolCalls: toolCalls},
|
||||
}
|
||||
}
|
||||
|
||||
// fleetSnapshot returns a compact, current-as-of-now fleet health line for
|
||||
// the system prompt so the agent starts each turn already oriented instead
|
||||
// of spending its first iteration rediscovering topology it already has
|
||||
// tools to query. Best-effort: an empty string on any failure just means no
|
||||
// snapshot, not an error for the turn.
|
||||
func (a *agent) fleetSnapshot() string {
|
||||
result, err := a.client.callTool("get_health_summary", map[string]any{})
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
rows, ok := result.([]any)
|
||||
if !ok {
|
||||
return ""
|
||||
}
|
||||
|
||||
counts := map[string]int{}
|
||||
var attention []string
|
||||
for _, r := range rows {
|
||||
row, ok := r.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
health, _ := row["health"].(string)
|
||||
counts[health]++
|
||||
if health != "healthy" && health != "" {
|
||||
if slug, ok := row["slug"].(string); ok && len(attention) < 10 {
|
||||
attention = append(attention, fmt.Sprintf("%s(%s)", slug, health))
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(counts) == 0 {
|
||||
return ""
|
||||
}
|
||||
|
||||
summary := fmt.Sprintf("Current fleet snapshot (as of now): healthy=%d degraded=%d down=%d stale=%d unknown=%d.",
|
||||
counts["healthy"], counts["degraded"], counts["down"], counts["stale"], counts["unknown"])
|
||||
if len(attention) > 0 {
|
||||
summary += " Needs attention: " + fmt.Sprintf("%v", attention) + "."
|
||||
}
|
||||
return summary
|
||||
}
|
||||
|
||||
func (a *agent) buildTools() ([]openai.ChatCompletionToolParam, error) {
|
||||
defs, err := a.client.listToolsFull()
|
||||
if err != nil {
|
||||
|
||||
@@ -54,10 +54,17 @@ func (s *Server) GetDashboardSummary(ctx context.Context, req gen.GetDashboardSu
|
||||
return nil, err
|
||||
}
|
||||
|
||||
rows, err = s.pool.Query(ctx, `SELECT health, count(*) FROM entity_status GROUP BY health`)
|
||||
// Exclude 'check' entities (internal probes) — only entities actually
|
||||
// being monitored should count toward the fleet health rollup.
|
||||
rows, err = s.pool.Query(ctx, `
|
||||
SELECT st.health, count(*)
|
||||
FROM entity_status st JOIN entities e ON e.id = st.entity_id
|
||||
WHERE e.type <> 'check'
|
||||
GROUP BY st.health`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
stale := 0
|
||||
for rows.Next() {
|
||||
var health string
|
||||
var n int
|
||||
@@ -72,6 +79,8 @@ func (s *Server) GetDashboardSummary(ctx context.Context, req gen.GetDashboardSu
|
||||
resp.Health.Degraded = n
|
||||
case "down":
|
||||
resp.Health.Down = n
|
||||
case "stale":
|
||||
stale = n
|
||||
default:
|
||||
resp.Health.Unknown = n
|
||||
}
|
||||
@@ -80,6 +89,9 @@ func (s *Server) GetDashboardSummary(ctx context.Context, req gen.GetDashboardSu
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if stale > 0 {
|
||||
resp.Health.Stale = &stale
|
||||
}
|
||||
|
||||
rows, err = s.pool.Query(ctx, `
|
||||
SELECT severity, count(*) FROM signals
|
||||
|
||||
@@ -104,6 +104,15 @@ const (
|
||||
ClassificationRouteHold ClassificationRoute = "hold"
|
||||
)
|
||||
|
||||
// Defines values for EntityHealth.
|
||||
const (
|
||||
EntityHealthDegraded EntityHealth = "degraded"
|
||||
EntityHealthDown EntityHealth = "down"
|
||||
EntityHealthHealthy EntityHealth = "healthy"
|
||||
EntityHealthStale EntityHealth = "stale"
|
||||
EntityHealthUnknown EntityHealth = "unknown"
|
||||
)
|
||||
|
||||
// Defines values for EntityTypeLayer.
|
||||
const (
|
||||
EntityTypeLayerCognition EntityTypeLayer = "cognition"
|
||||
@@ -160,6 +169,7 @@ const (
|
||||
GraphViewHealthDegraded GraphViewHealth = "degraded"
|
||||
GraphViewHealthDown GraphViewHealth = "down"
|
||||
GraphViewHealthHealthy GraphViewHealth = "healthy"
|
||||
GraphViewHealthStale GraphViewHealth = "stale"
|
||||
GraphViewHealthUnknown GraphViewHealth = "unknown"
|
||||
)
|
||||
|
||||
@@ -168,6 +178,7 @@ const (
|
||||
HealthSummaryEntitiesHealthDegraded HealthSummaryEntitiesHealth = "degraded"
|
||||
HealthSummaryEntitiesHealthDown HealthSummaryEntitiesHealth = "down"
|
||||
HealthSummaryEntitiesHealthHealthy HealthSummaryEntitiesHealth = "healthy"
|
||||
HealthSummaryEntitiesHealthStale HealthSummaryEntitiesHealth = "stale"
|
||||
HealthSummaryEntitiesHealthUnknown HealthSummaryEntitiesHealth = "unknown"
|
||||
)
|
||||
|
||||
@@ -262,10 +273,10 @@ const (
|
||||
|
||||
// Defines values for TrendDirection.
|
||||
const (
|
||||
TrendDirectionDegrading TrendDirection = "degrading"
|
||||
TrendDirectionImproving TrendDirection = "improving"
|
||||
TrendDirectionStable TrendDirection = "stable"
|
||||
TrendDirectionUnknown TrendDirection = "unknown"
|
||||
Degrading TrendDirection = "degrading"
|
||||
Improving TrendDirection = "improving"
|
||||
Stable TrendDirection = "stable"
|
||||
Unknown TrendDirection = "unknown"
|
||||
)
|
||||
|
||||
// Defines values for ListApprovalsParamsStatus.
|
||||
@@ -557,6 +568,9 @@ type DashboardSummary struct {
|
||||
Degraded int `json:"degraded"`
|
||||
Down int `json:"down"`
|
||||
Healthy int `json:"healthy"`
|
||||
|
||||
// Stale last observation older than the check's expected cadence
|
||||
Stale *int `json:"stale,omitempty"`
|
||||
Unknown int `json:"unknown"`
|
||||
} `json:"health"`
|
||||
|
||||
@@ -598,7 +612,13 @@ type EnrollResponse struct {
|
||||
type Entity struct {
|
||||
Attributes *map[string]interface{} `json:"attributes,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
|
||||
// Health last observed health, when the entity is monitored
|
||||
Health *EntityHealth `json:"health"`
|
||||
Id openapi_types.UUID `json:"id"`
|
||||
|
||||
// LastCheckAt when health was last observed
|
||||
LastCheckAt *time.Time `json:"last_check_at"`
|
||||
MaintenanceUntil *time.Time `json:"maintenance_until"`
|
||||
Name string `json:"name"`
|
||||
Slug string `json:"slug"`
|
||||
@@ -608,6 +628,9 @@ type Entity struct {
|
||||
Version int `json:"version"`
|
||||
}
|
||||
|
||||
// EntityHealth last observed health, when the entity is monitored
|
||||
type EntityHealth string
|
||||
|
||||
// EntityCreate defines model for EntityCreate.
|
||||
type EntityCreate struct {
|
||||
// Attributes Validated against the type's attribute_schema
|
||||
@@ -765,6 +788,9 @@ type HealthSummary struct {
|
||||
Degraded int `json:"degraded"`
|
||||
Down int `json:"down"`
|
||||
Healthy int `json:"healthy"`
|
||||
|
||||
// Stale last observation older than the check's expected cadence
|
||||
Stale *int `json:"stale,omitempty"`
|
||||
Unknown int `json:"unknown"`
|
||||
} `json:"summary"`
|
||||
}
|
||||
@@ -8182,173 +8208,175 @@ func (sh *strictHandler) GetTrends(w http.ResponseWriter, r *http.Request, entit
|
||||
// Base64 encoded, gzipped, json marshaled Swagger object
|
||||
var swaggerSpec = []string{
|
||||
|
||||
"H4sIAAAAAAAC/+x963IbudXgq6C4WxVq0hR9mUl25Nofiqyxndix1tLk29TIRYHdhyRGaKAHQFNiXK7K",
|
||||
"r32ArTxhnuQrXLubRJPNiyzPVP7YkhqNBs45ODj386mX8rzgDJiSvZNPvRngDIT58fwKT/X/GchUkEIR",
|
||||
"znonvQ8geSlSQHMQknCGJlygN5PBO6zSWS/pyXQGOdbvqUUBvZOeVIKwae/z589Jr8AC56DcB85KIblY",
|
||||
"/cT7Av9SAkrNYzQRPEcYFQLmhJcSCZAFZxJ+JxGDezWyw3pJj+h3fylBLHpJj+Fcfzw8bF9W0jtniqjF",
|
||||
"m2x1JT/++OYl4gJJWk5RH46nx+hmxqU6mZVjQeTNkf9sgdWs+irJeklPwC8lEZD1TpQoocsKLmkZAbh9",
|
||||
"1ljCnTzJcTrICSM3Cbqh9+lJirNs0bYe/e6WK/pB8PyK6Lc/RQGrsdIA64SLHKveSS/DCgZKv5pE5n2T",
|
||||
"QV5wBSxd/AUWq7s9owSYGkyBgcAKMnQLixdIQEHxQqI7omaEoWffzpAAVQqG1AwQF2RKGKaBNDwULDFX",
|
||||
"i659fKC/Xl9/ju/fApuqWe/k6bP/FV36xNL4Koau8LQiU8IFenV+9QJ9+/QZ4iyck5zI3J2R+OKqM7QN",
|
||||
"ot6SnKg2LFHzsD5BBhNcUtU7+e5JovdM8jLvnTx7on8jzP72NOyeMAVTEOZDV3wdPSi+PTV81ju1GDP8",
|
||||
"4AJYRtj0tCgEn2Oq/5RypoCZ/eGioCTFGubDn6UG/KfaB/+ngEnvpPc/hhU7G9qnchgmNJ9corcZZlNA",
|
||||
"UuEpZC8QRjkoPMDuDXSHJUoFGErsZyWmA70iwelR73PSuxB8TCFfs9DCjvj9dgv280bWey4EF6j/4Ycz",
|
||||
"9P233/3RLOOSTBmmPxYa1tnBoGZnja3BfQlJP8Jj3mDxdApMnaaKzIkyB7wQvAChiEUydk9Glho+9YBp",
|
||||
"mvuppzinoxRTag4AlpxpKtHfTok+QL2kl6fFyNMdyBRTs6/exxXSSnpYr2JEssihSXopFwLsy24IKynF",
|
||||
"Ywr+xK28kpXCjs/lmvHhvCQ9MGy76/SNhdZmIawo1UiWeY7FotNMvFTbviJByi1AIcs0BbkODGPOKWCm",
|
||||
"Byt+C2yU8tKS42a4GTKwTKXDWqzQ0vHuqdjqT/aKVrJXo5RkiTYrsuLjnyFV+nt13rRK1/Z4rZKbZSAj",
|
||||
"rLou1lJ9tv6dzTTr5hh3IwO4L4gAudUyLcmEsWVp4bo87JawrH7W4R7SUtlDXXBK0sUgNYxY/46VAsEG",
|
||||
"BhntB7zAC8pxRGYzIpLGDZeQITs7yshk0g6yCr+CyNtRSrEl71XSdxLa6gOFVSnrWyzsZaapytAMZIaX",
|
||||
"MWJ+sLC2YuKc30IW3aMs7cLWyoRaAgr3VcpZCoLJzeQROw9OTnSk3ICGw2HYaYNcGiS+7th8KClsdXRw",
|
||||
"qTjj+WJEYQ60Dl/9pLoGzHmAOYgoHB0v9jdOE5bv8AKNAeGxVAKnCvUJm4EgSqKM37GjLget4ynYRFwp",
|
||||
"L2Bk17oe5VrlKkAM7FjE5yAEyUB2WasTR2PXTYwk4rSwhJZq1k3IPzN08vgksC9u1h+mTkCLgqrMiDpn",
|
||||
"Siy2A1GquOh6fdvBy9KXuQV7SU9/ESurMi+kAq/kZSVtgewuwhQoTGhtKxUEDiM25aBmvNsURlPuJPYY",
|
||||
"s8eIFN1GGzY5SnkGHeWeA0gyFWbDwY1TmSXES1BKz7hCardWM4d7nBd6zb0p5WNMjzUFj3CqYsyttErB",
|
||||
"VtLDHNMyfhy7c6lbo8fbmdbzobMZpLerm005m5CI3eVvmBKr52hW626/CL1qtNbJsCb8drwX9NbEHNOR",
|
||||
"jFPzsvQ0U6rQ86T634zIW30Bg1ADcyVrcGSCTIzeL2cDu6e4fNEmzigsprBB7ugTJhVmKQwMc8w63ZR2",
|
||||
"4pab2M2uH6K+/nermUkOXCs+cRiuIaik9w/Ouqgba0QmRx41TNZXVJFJBwptuyIrOl1HhMG+06aQNYkt",
|
||||
"DP/DkyfJV0Z6m4hnPQmEnT2NbsyjfD2K69htRdiFtwrugq9NCIpcFOsI/XNskVr+IBNnBdpN9ko971wZ",
|
||||
"MqZYqpHAGbH6D1GQx2Uo9wcsBF7EBYeDaM4dma5TM0cGTRmwdB0HYGU+ttCvLFMxxApIeZ4DM5p7AOu+",
|
||||
"WqfgpYJlwXdg7+Ga8DvjtEWNNHa6zuadW0I7D65O6w68My4m2902bYBLpLJR37RehDPOFNyrCMUbk8+E",
|
||||
"UJAja3eI2BEusJpJxCfIjEb6thOlWTEybyI1wwr515MtCF8SR23ND16RHKTCeWH0O63WM7hXqOCUIg07",
|
||||
"kBrh3Q6B5IW0pD1t3+GVKAGRCTrWo48XONffSUmhYSdrO4tZ9TjtAjozbvjNsQRVFsdytjvMavf3kvrO",
|
||||
"GVeckRSlFt3B4eIObbJJglx7IxtCuoRUgBXQVwRlubqkN2xCJEkxRdK8iPQwhI3VlIwpIMWRmhGJUjP7",
|
||||
"FnBYlX1ldNkvsZyNORbZZWUGXjoCTimXI2+oil42Rh0jIEfjxUirNIZscZYRvVVMLxpzrr6+ZJmz8p0x",
|
||||
"BksNFMjQeIHsvEmLLug+7i/9A3/bKUurn55rDiHchpem0s/8TOMyvQVlJ/tukBNWKkD+Ck9QzqXSh0q/",
|
||||
"oS/KOq6bCLETdb/mgkl9A3W7ef0LMWpZPmvBOnsgtPvp4phPDGDQs29nMUTMAFMVka4ymAqcQYv+n/G7",
|
||||
"FlHfzreIPyzZLWt5cwmqfpqkWoj7ajVLDNT2KrZwhTkIZ07aFbS8AIb6jLOBAMnpHLIj55VbhbX/3Mqq",
|
||||
"lra2cupiXCAgJr6lJMJf4nTVOGoxiJ0zwSn94O6/FTqYcam8t6gJm9NUlZgiP8BcRzNAYOYjbIpynM4I",
|
||||
"ix6uHOTMWXqak17a2Bf9HL25MBe1Fh4Nb5lbg4EVaVoVnk3BHXfyhMHdgOJC8eJoo/XHTLsObi4iIiYD",
|
||||
"jQpB5ljB6DYWiXH66nxweX724fxq8Jfzvw+Oj4/NdinXF1sGqVgUbXs1c5djStL41HgKT/V8doymUTP1",
|
||||
"5fuLy5oEEjeVuKt1ZO9OJ6e23b8/MqJvd0xPSzVz1y1687LTzPbu3np291qMqCy9jTzBjIxvdN0H3BsV",
|
||||
"iVkZAtkXN5HGEhaSFZTHwdkOijiZqbibXylBxqVq8LHqtQdU7HKs2SXDLIVRyZS1M+/mQPWMpfUwVxbS",
|
||||
"WlRYr8Uc3NGj3Wbd2MnAup2zx5k7HNc3u6/maCCtsZx2umh19zSoo83uiqdYa12GbesP/E6i8OLIBbFE",
|
||||
"Pr0Ra+3Yaa7kpTUhSSuvA6JkAukipXohzrw0WhJeV/G4pK2UUhlvI9KXdvA5QmX57Mbxm0hqR8BFPGLt",
|
||||
"VCEKWuriLFyMEwI0kyh3KywESGDquJdsgbt3IEwY1XwFh10Q90VObhzVV8aOUWHYiqaorwRm0ohm1Z7i",
|
||||
"l3ILAq4cGbTAcFSPzKsv6M+X7/+KLj2oNlqQGi9Htp1xDdzoIyJHng7jBkmKFyDq5qccFLbXhMDWKFIK",
|
||||
"jZUpn4Mw6DPqxpSR1uiNAOiuhqZWhBZY6CvKH7fN5i0D09Faf8BqNIcJRgEj7hcCUhNo93Evhuu4q0OM",
|
||||
"h3ITHWElH9fS10YuO1oJIH0IygkW9wmmMup7WKGkQ5LQziSznt3G8bQeIS0egYPgY1fajLKouYsSXfZc",
|
||||
"bO/Rxwo/oD+/rivXaIf3kt4dFt5ILIjSUmvcBm4Ut7iJTXYXqELYRBD8rPp7LDCRkG3hrHf3d01ldkuM",
|
||||
"klaIl9vKe1OLgd0cFuIU9q7j04ZXqfNbXINN7RnY+ECeo61jfDu7mgTOI9LS5S2hFNmnqL8qM9knjlkc",
|
||||
"xSQmAdJw3P19TA/oI7KDazfjZsCuk9TFvtQTidV00aLNYE0b7bI2eNNZtAz70ed4sqj9bAdPMLGOf728",
|
||||
"bMRLY4vVNxy1fxdc/zAa4/TW/aZ/HLn3Pu7jdKstJCLZbR0BGkI/t3XHBfbVasaruFjFWQUYbK8/UZEj",
|
||||
"gWXb1Vmn8aWjaI2mxj2VcxN4DJl10vR5YU2zR71k60iZegLZxsvBzbU2duuVwMXsbwTuVmEI2RSaLvh1",
|
||||
"6R0fHALljBQxJ0BleG8zToeYkK1s4fUowIiDhmRocF0+efIckJ038ZooupsBQ4SltMzgf1taNPk5zjcK",
|
||||
"0fgsxrMtgOKsWRFwKFGy1OfXxN2o+lMoxYVZ1IyoiOd0WbbkNmjXYi6G7tcGBq1ePG+Yb2xwyUAe8Lgv",
|
||||
"vkyghwkE2Y/9tkb+CGgGG5Fc816fBaSXan+Wynygsdpd7WprjSsOdl18ZrINRb9KR9UyVNzmKk9QFCZ/",
|
||||
"YfyOalp+TSIsvqOwRAm7hWwUpeyNUQICs9tOYTztAgYjRRFj6K/JdEbJdKbvBZPH6eMNOtG8jR3uHGus",
|
||||
"iKKwZsfVGcl4WuY2iECUbMz5rbHMzEEqMm3Lntls+7ULiCH5rVe7X2o2ukrtdbNo1GiQtdvltsS2ApET",
|
||||
"faPv9HKw7EUk808TwfMT9EnxE/TJgUqeoJ8YziEbGB6YoOPj44+fP3/e6FAlPm3G8PoVw3FtHTF4vwMl",
|
||||
"SHoJwkE4cgEs2pSg3LzbElNGaVnUKUngu17SezrT/7TEkRnJbN1lg+fTTsdvi3zAHN93mjInrNO4bbT9",
|
||||
"EJ++VAIB3yELC+Tjzjd8dlnQk53ulHAhrhNZrsygqBN/YYVzRwUB5xUiY4u4sHGR2xkaioISkO2Ruc0g",
|
||||
"yyY0/4tQyRmi/A4EGvOSZYmWogobtgBz+55NIR1+14ugtDkmfmNqdaoUa4d0vaC0GBSU9b1EoaKC9cqz",
|
||||
"X0osMFOEtcUJb5GLOFsUXM1Akn/YEHS/eJ/yumQ8NBdIGPOxPQV4HTR3cz02KKmmh3pINUhpBfM1FXVd",
|
||||
"UF8tbX/59lrKS6rljwnBReSm+IEAzQYmo6sWAILscNT/9tmzo/aYL+Nyi7PnNi12CXZ2hjC+C1fxaRsb",
|
||||
"aCcWeL5JKAkegB4e81KdjKmWx2oxq6Ugm7Vg85m1ro8LrRbItfaEdQ7md29eJijlAmSCBM5H+ThBGZG3",
|
||||
"o+k4QaRIkIK8oCY0LTdRVAmSIOYkBRkNUeMyIi9e0nLqXasXgt/n/N7EIrkwn1q8QNSuEI9pel3mmA0E",
|
||||
"4EzzFeR8Ex1jjcx36X168jNQupgQtr3bmt6nCZrnCeICZTy9BWHqYWDC6oG23R3XDngbcNwWwlSlY3bT",
|
||||
"6UP8WdQEFIxU6M1L4/EXOL1FhV8GYVP9y1SAsYVtuCai13FvaQlrt30ZTuLSpjVnidRMmoPAlFrGg8ik",
|
||||
"ufBghNxdO29xnJ+VQgALEQyt4RBSQbFOcjTrHuUgJZ52c+ROCCNytr8t+CHsySHkUZTMeaeMYhbwIG+1",
|
||||
"mtlyuSooOlgq9Ki1XHJt6Lg7jB5fFj2xWRp2wg18dqMTovLCxZmlKy0Wjktnc2vgtjaOY80EbSKqubxH",
|
||||
"pqzVNvoByUaK70o7yzix0EkqQ7DjlbW1bUJRt4irzohZb79ux8fG97pZ4+IA2QSDeMxNikVGGKZLbmTO",
|
||||
"YKD4gJtAYPdLjpkmHv1f9cz/Zh5utGfHLB9MC6WwX7iLsyR1KmXR2zYBd+P78diI+pqaX0gaUI/ijcjb",
|
||||
"M++VjKeojKpPVmhjDmFVuQL/o017EnkLdw0pjZhqlbNFudqEyxb8xOGzupHIMmLAcSW4VknZGOE7ul+9",
|
||||
"jtTxUhVSjSQA28pzPqG4WKcMzjjNRhm/Y/vG9W1bb6gK07AC/MCZvuNa/db7puQW6GKU4rLjuc5LtXds",
|
||||
"I09TI3StN3gcIGRmoyhYmQ5d8AtOb70PwBsXzHf0tq2aarNUKlHo4y6X/AxCxKxJoztE0SNf3qgWj+Nk",
|
||||
"oxV4Lx+TJeqJnuRbQuleNrXNQTEmq3JUWQ46vtG5Ptg29rFS7ilUrwnes3nbJNvS4F8InkJWipj46SMQ",
|
||||
"M2QE4aGN5Rj6YIyhj9EpKGbow/PB90crYdFwX0CqVYkQRdRWzYNpPujtkfmKf3rzRuphQfEYCLfuzn5v",
|
||||
"Q56XWqOIuTZXVLedp7JwPcRcEW1IhiCceOpdd3tpJvDExRP4wIJgJxUwMTbZmjK3ISbYG0vF+hSQyp69",
|
||||
"m+l0JV0jGE+DUbQ6Aq086tIpn8sRoXmOWcRo8ooHY5kpQ+Zi1nrx8oKult3ywSEq1EmKxWJnvNQDjJlJ",
|
||||
"xoWu7jkgAuLFtZTehmpNuqR40b3ghlb6m3HOUs56iS+eYrKDWcul23b3bQHoeBmUPzzZmAjv1p0EdMeo",
|
||||
"5Mo7pZYAyHiO6aJFmiYCqgCvHaI7VgVOrk9c1O5KjGOOEgZYoELwn+2nE/THDNnou82+xHa/qaQ8pjm9",
|
||||
"tZ+bEIUKECjDi62dgsFNV0ErGpghIS21hGKSQ1zuOGAB4rRUs1gxdasWDecE7kCcID1MS0+36P2bl2fo",
|
||||
"z/91VY89JWxwevEG/fuf/0JnOMsW12zCxR0W2QCXaoaIyXwCJmFA2CCDQs0SxLjN0XLWGy2hiVLNjo6v",
|
||||
"mSkGfGLMgiRFdp02fdEWzK5yHfum1hO6MUHLN/pdX1DaEJN5s6J2c5RMaWIj07qaxy4RwZWkzhQXfDXQ",
|
||||
"7MwWcB7ouxyQ3qwvt/Ge3HKJZjwHisfo/eUxutLi5YRQ0BvXQ775JmzympldfvMN6pua0DhVAyMXHp2g",
|
||||
"V9x4DEAgqcqxRFgAqkqa3xE1QxwXZKDZ3hRYcs1spqVEff/5s7dvEjQptVSCfnwjjyy8DJhxDkgWkB5f",
|
||||
"s2t2xtlco5Ozmnzy/Ojkmg3QufVC6a/7gtHopq089c2xfuUtkUqiUgK6+WTu6KReZf/zjV28K81f4Clh",
|
||||
"1uHVd4wGmZLj6LsnCcrxPXr25MmRmfdHJvEE0MX7yytbCqNQ6GapHvsN6tvK7gXFC3RHWMbv7NvvSsMU",
|
||||
"kHDNByRKsRALdONuu5sX6NX5lasJL9HN+RWe3iTo4vTq7DXy8RvoxpdYv0F9V5zdF2W3nwkVWCqYPX/+",
|
||||
"/Hv049WZeX7ugpLMU5xlAqQ06xo3Iz1Rv9klwCDqagbo3dmFLQ4xwSmgvlQCcG5meH11dZEgPpmQlGCq",
|
||||
"Cejy5V+ONNkZFxRkCCt0M8zT4uaacVYRwpgwLBYIs0wP5qUydlRzlixd6yPrgoReIGKyGTmV6E7g4ppV",
|
||||
"9GT1Y2TSWxA21C61mpUVnDAl7XmkJAXninGH7MLmE2t2Lag7mPJkOHSa97HzJA9d3nHNj9izx+304k1N",
|
||||
"ajnpPT1+cvzEqLkFMFyQ3knv+fGT4+fWCTwz/G5omMQA14qMu0vTGoEIZ2+y3knv/5QgFs165M0eFD/F",
|
||||
"i9nXSkKvKb3f8m6jhvQOE9RDN9a+HJOcq80NQweHDmNdbf8OI13Pjg4jbWOCzx+Xivw/e/JkqxL1S0GE",
|
||||
"Xm3opD80UR/RR+oNRLavYWWWELmjV64cvwQETJk4rs9JJZjFtxBgVmsGUIsytVX20RhmeE5MVQZjZsdT",
|
||||
"aWzaY32c8Zh4s+v9wC/cllTsnfSsOGBmHYZiHa0nSV8Lp2FUp0MUtI4Kl4erjd12eLwVZ+WTe1cc/82d",
|
||||
"jdAU4/GORSCo/c+DJlCEaxTqz0JVdGabgzD8RLLPw9B6QsO6SfEbEBxaCmkcFy5ApHmkXprq/AENyZZf",
|
||||
"WGqkY2nJRMP8iWeLPciovumQYmqPqT2li3Ayo6oZ4y2Rvy3lTl6/Oz2rCthb3aAvCZtSGJQSEuRzmJxI",
|
||||
"PZAkg82FccI24pTYbLHzec+DuGv3mZdukUhAykUG2SFuBosrE6IDbFGHpYNuK0C7HpngdbOHpsxsvP8a",
|
||||
"GcwM6SZ71YtY7yJ92UroDyl4tYl9nO3y5koy3X+Evv0utqqG/iNebXoRXtxDfSXRy/PLs6NDHG8z8/by",
|
||||
"XvPMGg/yenHvzA7pdGhXxK6OtB/iOnY4rL6g9sqrtWy73xZl2wL2j0fUjiIOJKwZEkQZTAhz6S8VQbua",
|
||||
"ghsktjbJysZAWWg9oli1EZUuVquTPPL0sJ+OotcmcR8Av3YmhB2O+5bbDLAcZFjhBHkz5R+POuM8xr6M",
|
||||
"kL6vbO5LtTRJyFRw2ZGCXJvIByUdW2HmC0uyrZTjWw/uTzl2JiO7EmtadUS0K6E0qpRsuPCWxnazcoSa",
|
||||
"Al9a4vR10ldtHVuVh//NXZLNbgePeFsukdMB2KqbEbRmZxVHLVvOALloQl7KgX+CjFqGlMCEHu1qD3Fe",
|
||||
"qaEtmmtQE0128TUbpa+MmwR3l0SYITwFdAuLAhORuH6q5u/tpU4T49GoJcc2or54I73hGJ1hSkHY2oWY",
|
||||
"CsDZAs3wHPQ3fGEJZq4dBpnmLo3sCBPoZR0cTa5ga+ie+SrtD8HNm+WNvzBDX6oRHGs3a0bkwFRormw9",
|
||||
"gLakPcsCwg5A3/ZjCCMGd76c7r//+S9EpCzB05CnnxrthCVUVO4It4XEbbOzBoV/krScfh6mVcuIaBjG",
|
||||
"B+dhvJuRdOY6Q5huEIl1q1myNYWMbfcF3+wAmaYPhoinZA4MKe9rNF5mhrwD2LR7MF47wqQCnCE+QVOi",
|
||||
"UFFSGiPSV6Ca7S5W7q3YFhBndOEWJ8PiiKzWZZsMP3/+/Pujlubqto/F1m2fPz6kiNKARIwruyYRGVCF",
|
||||
"D0Czr0A5MkjrMzuI4gqc2xJnspNUa1rIf/4YoWxZ9bCYri1rPTABJsY6aN6wzuTMF8G10/5OrnDsdsL0",
|
||||
"7TMeHO/+QxG8X3bqxHEg1dZDbl3Ljy9LDJlvCDKsFaqJSsKvQK10D3lAxK18K2Yl92OQX/z+eHrPYCB4",
|
||||
"ybKBEqQwIXVa8AmxQK7NOxKc5yYkCBV4Cnv4WOsFbVpVEB9gsomH/0CoAmHqI9R75rniWBLp0cAyzJRs",
|
||||
"Y967WthDxuC2L4YKqlu/6Uvjrn1x6biXY/vQ1erhzATmDF2QbOwrv+xpev9VaUnthc6+kHbk6uNRIg/G",
|
||||
"daE6PEHZqZWr2tmQeO6vt6/SktgotP+FTYmeitptiaaOWQa2csb5FZ62TemGDc0YN+FBbJAMrSgHG6ii",
|
||||
"aUHyg4dBZWxXg8+cZltTVWtqp+t2NA+6csqZ1Pq5ifu0VSm05pziAqdGBfYR30cJ8oYsN7t1N1aVYk1E",
|
||||
"S0RnbirKmg363QWXe0ynCFUIvm7aX6kD8oXpf7VGRTuncyVSkyZCfimhfNRjEraAMNKvlSqQbv/t/z1L",
|
||||
"0N/eJSjU+DhCZqAp2rHvefLG+zYpNJDeA5o/2tiXw5mrB/R42HkVahkshxjvdMkdwE2ynPbg+2XUuQ4W",
|
||||
"EOkAUuviUpXuz2Dy4poRSmGKaWMSG8uNvn3yvZZrzXSD6vnRMbqwUXxT/ZFrZhmi1koX1avPUd9zuQCX",
|
||||
"oyi/09vbldc9sL+n3sjli9sH2w6I8/hUd+tjnRDnMKrKW+gDUuvastTR5SBca2h6Hg+qnsdtLOxPetwH",
|
||||
"O6yTN8kk1DT0kACe56YUIsnLvHfyXSSV66G1ieUgwcImG7W0DO1clamtUJL9wNZlbbaI0JlMTL5uUBus",
|
||||
"XXsqcDFDGXE10g5h1PY5I/6DZGJNQY6xTzCh8ouy81WC9gFocvOFHAqtdKNoAXTnYL4qES56Inpjbg5L",
|
||||
"qORikvuMicE8+Xh40/M+Kvf6gus703F92kM4GV8aoCNRn9Z0wQ6etL6GLgrIkUePSLzWth1E6mGVy91G",
|
||||
"xcsF1h7w+lz+VAR7F00/JBSODbl9HEC+55TGi9gZU+ey0P+lUFkzTZv+r3J9/PD53DUE7cBxHjzvKmoV",
|
||||
"rZUN+U8A8CPbNufOZ/JYps25zeQ9YNDvayIVF8bZDf4o7OyImFtomdzTVnfgpU0NuASmkN3QMTrH6cx+",
|
||||
"/3cS3ZDsxmdF247ogt8hkqG+AFnmcM0MI7t5q0VlM8PgzcubowTdmNFL72qgJugmwwqHJ3++fP/Xa2Ze",
|
||||
"RRbax+g1YKHGgJXmW7mBsz55C/T0O3mM/gRSDWAy4cK4YYl58u9//uuambrSkKECxECWY73TMQg0LicT",
|
||||
"EAnKBC8GnGYglUuivvjD0QuTBv3q/Ao5mF0zxdEYp7cTEnfFXxqYtjGrVhdOgAAqBEzI/b4eG6tkVS82",
|
||||
"ULB2hs3HVsG9suAYVBTUPuGqH/byHLkXD2H2n3sCsnOi/uXl+dE+h6OKjlrrp6uG7ZoL+eAR8l9JRsqv",
|
||||
"6+oIWaKPeH1UtHUox1idWreOA0xanB1XM0AzzDIKYtk70Q8xfIYGjxIbwyudn2Loix8m1wyzDAFRMxAI",
|
||||
"mLGGu2shVGPu23BWlyh8hLioRRBes5A56GxvxgfiC0E0ZyIM3fhWbzch6u+USo7g3vzVB7nYiB7BKZgA",
|
||||
"NBuOZad7/9e3f0d3eGHHSL3F2FXgPBLn9bTjr9J9uNya7Uu7EKsTt+YoeO8J6ue2RKnLIA9OrEMIWR8C",
|
||||
"AdWpz5H2Av37//3/Kk3VJjboPzmq3SrEthZ/WI3d7BCp0dLDmXw74eMQdrEAYtfK7ffIdbPcjUftZU9o",
|
||||
"ImFo2zM+SNb3mZn68VF5FjpQHsDVbuZCGHnmOgx9MVG97sKO+cWaN4v2BONz8/gSINvbmLMkOpjCStzG",
|
||||
"yjWh9/fTd29RrfXWao1Wpjjl011etXfk1i8uyRthAUltH2HyLnKIhigal/qCPwhz9QkBSOqJ9W6krWmV",
|
||||
"uhYCL//ku+6//ICGyJUE8pF4jVSxhVSQdyIeY89fx1VNQ81NytqlwiJ4Yvt1P+zRC8Rzoowx7W6mBQbr",
|
||||
"QejbFkZt0XeC852E+jUOomcbHESJqVFKTaFFK7F2ttd3r00q1cLUdppwkfdWw/KWunf+zAnzfpCR+5sW",
|
||||
"3wpelNRIeKHd6bHrxZh02YT7THwPoSbjcueErrt6yPj1qr1r5ESah2hunu59IC/LsaVUTblzIktMyT9c",
|
||||
"LTfTlxT9Hpm+pDuY9/XBq/qOtp28HyiAeu3R+mAgbbZQjYDVDjhgbLHZmGtf66e1Zn3T0Q0RlumtcLGP",
|
||||
"GS8U2h5KwCJth/SleRx6c3YzWPzSW1YC9rMCfAW6faM76eH8b6+JOoSi/kNJ6cDkj1h02iKvAcmVl7rv",
|
||||
"RQCZINfxs3FEwytb0dCn4P3oEJNVp6XfGjrfmvazFeAPUbGD0iC3yaHHGbKNbpHi0RjVrmiMn2bTaDbq",
|
||||
"2ep+qI1mZIvSbvDOvXODOnGWzm62Dvd8KJl7EFElLqf5lpmR+ANcKl6LP2g2ULUtPnZLuN7FifeYrLXR",
|
||||
"o/ZwZ9FOiyQcqGaiIVaTkTewc6I8UO6u13Bd0Wvjmu8rXeyAKHIHSUNx27wP0yApciRCsFz3CRvtoGMd",
|
||||
"uWtxJFuudaWl0ybKakAk+unGFruQYEDdYW545MnFmJVruVv6Mq8t1/+tttoaifo1dadOGzGzGAQExE36",
|
||||
"9ZBahKVLYR7lpXKKQTC865ODB8EcejcDhqoI2xVreD2R5spql19xMo1e4WMm1FhiX1ug59mTZx3o0BrJ",
|
||||
"65U+9zbaKq3AqBlUlGwUG5uzXyPo7vTatNdEKXb4SV/HsVI/EWnHpfhtIei0hre/xiJDGVBQpgA84wrJ",
|
||||
"sii4MFXcZ6YuvOumKxHcE2nrFYR+ICGF38YUvHweORq10PPdTsYXCT/XS3vEEPS2E1ErPPRIJ6JWsChg",
|
||||
"vQqV3OckuKrE6wMRLvygbWTvPYpr7hhR0C3+4bcUSeDb6D9eHEEgjQNFERQVqXl6puA6yW2WRPzbB63Z",
|
||||
"Fu90giSegFqgOaZzcKz38tUfj47RaSjwrdl5UZd2VkSdy2/bmPVF6EX/5Tl1kyRbSy3/UmKBmTKNqqId",
|
||||
"eVZ7XlUN/2vNrmqdrYwfKYyJqbWPW2Y5HLiv8Za4qhKSsD9IqO/qzQMaogq2aFjdJEfdz9rS3eHibELy",
|
||||
"W0mhW3H/DyUF2fsK6tLrhRwyTcLs68B15pEom6pZ5V7dIaDqLChXjQYF4WsvjNytP4lcc0hbxAlPFIjV",
|
||||
"vG7P+Z636mMNUH+lGll9jdvoZI9yzC9s/IPThhpUgvpZiekg4sxeSzMdjvVDV0Hdj0oeWDv5lZKHoQh3",
|
||||
"vA9JGC6ucp0x8tSNuQSlCJs+Lq9vruWA7D7s7hAF1+0ikXRzov4toXQg74hKZwliMAcx8DVXTUWbox2u",
|
||||
"hLhM+wETaeIc/SKIRHV6oZCh/rMnz9Dvq1DIY/SW34EpfkSUTX9wS0c3U8rHmB7r6UY4VSfouscnk+ve",
|
||||
"jdZgcWZjKu2WRn4QugWXReGvHZLnkBGsgC70158cnZirqQYWW4rTzIPusIuPwWx90RHDbWLkuRvf0NvR",
|
||||
"jzC9aNBom2fo4eTWr/OMnBps2oQdJYgN1X5EITmwR0/rvjplg0O+aJDZ+x9+0EciEOR+/FMQeTswAb8b",
|
||||
"pOUPRN6euXGPmVLsl3FIQZnIW+RhcCB5WdTn3JI1avQ0kpEtk6RgVd/lgr1ZI826W3KOaSm5bcDL2mSd",
|
||||
"vWeqheFtZcheIsVvYxVsa24mYIfpOXTOMi3V1KfuS1DSloEZKW51FxPJQiS6hcLeCDOT17g42qUsR5sa",
|
||||
"de5aVlofWqQQzapfEPVnBAQW6WwxwHdYwNELlGKREYapbds34SKFrE2RWk9zX4ciVV/j4zi3mgUQvkj/",
|
||||
"iQZFuoilneq/+J4D6y6FSzemc0YgHCzTPPTUZhPeS3p3zlSU9FJBFEmjvcYfJA++SyOg35KZ3+L8Ea38",
|
||||
"nugOVbk40PB2rXhqZ8Sm0OD09kHyZ07TWwfzONbX79y+erh2JadpFaGJHfB27FTSgF5eWunm4OB7Vyqo",
|
||||
"we8QTgi91lHJFKFdK8C39ohc7ohfzbxHE8cvSxEawIEUXImVq6u3hyAKAZLT+cPQxQc794FJox3NK8j8",
|
||||
"KpDnoFDhL8esxJQudkWfVlU3CA12SLfWmNb8MjJRp//x3j/kta6x8pi3uqWKQ13qZjbUNylVKiTWFSDs",
|
||||
"o6PdXPp22od2P1hUfMW+9oIwBtnIQTVeE3HV3a4x0epr//q86+5AfPW+dUOTpsET0b96pOzoRq9R+NBN",
|
||||
"1YGb/82P/PoY2M4MKexpf3y5qbztx9QWtHjbng3tmfdv0vFk1+SnKzN6a1b0a0vmMNs8IOk4sB3ioAPL",
|
||||
"EGaYLiRx5Qsp9TkcpjJ5JJFqm4SOh0ym0luBtDSmGz31GLAAcVqqWe/kp48a47Ybu/1wKWjvpDfEBRnO",
|
||||
"nxp6cPtZbdvkkvtd3nnIKzAlZ00pnLrtvLkNm1azEodiO69BaP2WVL2tiLR1lglniW9zVCsc5XoZrc55",
|
||||
"vl2qg5uPV9kXn+J2D7NFV1yob1FtfECN7pzRBYUaFFVvBdepMQleSon6GaQkgyFOVW1aqJdo+tQSd2mW",
|
||||
"FkQvzc9qMwT+tvp+3QGTLIUaJcE5Vk3lvCirE4UMSUcaLk+4stXVchw/RVOvZGIzls13M6ISV30wQSEb",
|
||||
"32Oqccpi4C64UKvvuUoOnz9+/u8AAAD//1g0ISeJ7AAA",
|
||||
"H4sIAAAAAAAC/+x963IbudXgq6C4WzXUpCn6MpPsyLU/FFljO7FjraXJt6mRiwK7D0mM0EAPgKbEuFyV",
|
||||
"X/sAW3nCPMlXuHY3iSabF1nOVP7YkhqNBs4FOPfzqZfyvOAMmJK9k0+9GeAMhPnx/ApP9f8ZyFSQQhHO",
|
||||
"eie9DyB5KVJAcxCScIYmXKA3k8E7rNJZL+nJdAY51u+pRQG9k55UgrBp7/Pnz0mvwALnoNwHzkohuVj9",
|
||||
"xPsC/1oCSs1jNBE8RxgVAuaElxIJkAVnEr6RiMG9GtlhvaRH9Lu/liAWvaTHcK4/Hh62LyvpnTNF1OJN",
|
||||
"trqSn3568xJxgSQtp6gPx9NjdDPjUp3MyrEg8ubIf7bAalZ9lWS9pCfg15IIyHonSpTQZQWXtIwA3D5r",
|
||||
"LOFOnuQ4HeSEkZsE3dD79CTFWbZoW49+d8sV/Sh4fkX025+igNVYaYB1wkWOVe+kl2EFA6VfTSLzvskg",
|
||||
"L7gCli7+DIvV3Z5RAkwNpsBAYAUZuoXFCySgoHgh0R1RM8LQs+9mSIAqBUNqBogLMiUM00AaHgqWmKtF",
|
||||
"1z4+0F+vrz/H92+BTdWsd/L02f+KLn1iaXwVQ1d4WpEp4QK9Or96gb57+gxxFvgkJzJ3PBJfXMVD2yDq",
|
||||
"LcmJasMSNQ/rE2QwwSVVvZPvnyR6zyQv897Jsyf6N8Lsb0/D7glTMAVhPnTF19GD4ttTw2e9U4sxcx5c",
|
||||
"AMsIm54WheBzTPWfUs4UMLM/XBSUpFjDfPiL1ID/VPvg/xQw6Z30/sewOs6G9qkchgnNJ5fobYbZFJBU",
|
||||
"eArZC4RRDgoPsHsD3WGJUgGGEvtZielAr0hwetT7nPQuBB9TyNcstLAjfrfdgv28kfWeC8EF6n/48Qz9",
|
||||
"8N33fzDLuCRThulPhYZ1djCo2Vlja3BfQtKP8Jg3WDydAlOnqSJzogyDF4IXIBSxSMbuychSw6ceME1z",
|
||||
"P/cU53SUYkoNA2DJmaYS/e2UaAbqJb08LUae7kCmmJp99T6ukFbSw3oVI5JFmCbppVwIsC+7IaykFI8p",
|
||||
"eI5beSUrhR2fyzXjA78kPTDHdtfpGwutzUJYUaqRLPMci0WnmXiptn1FgpRbgEKWaQpyHRjGnFPATA9W",
|
||||
"/BbYKOWlJcfNcDNkYA+VDmuxQkvHu6c6Vn+2V7SSvRqlJEu0WZEVH/8CqdLfq59Nq3Rt2WuV3OwBMsKq",
|
||||
"62It1Wfr39lMs26OcTcygPuCCJBbLdOSTBhblhauy8NuCcvqvA73kJbKMnXBKUkXg9QcxPp3rBQINjDI",
|
||||
"aGfwAi8oxxGZzYhIGjdcQobs7Cgjk0k7yCr8CiJvRynFlrxXSd9JaKsPFFalrG+xsJeZpipDM5CZs4wR",
|
||||
"84OFtRUT5/wWsugeZWkXtlYm1BJQuK9SzlIQTG4mjxg/ODnRkXIDGg6HYacNcmmQ+Dq2+VBS2Ip1cKk4",
|
||||
"4/liRGEOtA5f/aS6Bgw/wBxEFI7uLPY3ThOW7/ACjQHhsVQCpwr1CZuBIEqijN+xoy6M1pELNhFXygsY",
|
||||
"2bWuR7lWuQoQAzsW8TkIQTKQXdbqxNHYdRMjiTgtLKGlmnUT8s8MnTw+CeyLm/XM1AloUVCVGVHnTInF",
|
||||
"diBKFRddr287eFn6MrdgL+npL2JlVeaFVOCVvKykLZDdRZgChQmtbaWCwGHEphzUjHebwmjKncQeY/YY",
|
||||
"kaLbaHNMjlKeQUe55wCSTIXZwLhxKrOEeAlK6RlXSO3WauZwj/NCr7k3pXyM6bGm4BFOVexwK61SsJX0",
|
||||
"MMe0jLNj91Pq1ujxdqb159DZDNLb1c2mnE1IxO7yV0yJ1XP0Uetuvwi9arTWybAm/Ha8F/TWxBzTkYxT",
|
||||
"87L0NFOq0POk+t+MyFt9AYNQA3Mla3Bkgkw0lgorgUg5G9itxcWMNqlGYTGFDeJHnzCpMEthYM7IrNOF",
|
||||
"aSduuZDd7Poh6ut/t5qZ5MC1/hMH5Rq6Snp/56yL1rFGcnJUUkNofUUVtXQg1LabsiLXdbQYzDxtelmT",
|
||||
"5sLw3z95knydFLiJhtZTQtjg0+j+PObXY7qO5Fa8XXgb4S5o24SnyLWxjt4/xxappREycTah3SSx1J+k",
|
||||
"K0PGFEs1EjgjVhsiCvK4ROX+gIXAi7gYcRA9uuMR7JTOkUFTBixddxCwMh9b6Fd2qhhiBaQ8z4EZPT6A",
|
||||
"dV8dVPBSwbIYPLC3ck0UnnHaolQaq11nY88toZ0HV9y6wxEaF5rtbpsWwSVS2ah9Wp/CGWcK7lWE4o0B",
|
||||
"aEIoyJG1QkSsChdYzSTiE2RGI33pidKsGJk3kZphhfzryRaEL4mjtuYHr0gOUuG8MNqeVvIZ3CtUcEqR",
|
||||
"hh1IjfBuTCB5IS1pT9t3eCVKQGSCjvXo4wXO9XdSUmjYydrOYjY+TruAzowbfnssQZXFsZztDrPaNb6k",
|
||||
"zHPGFWckRalFd3C/OKZNNsmTay9mQ0iXkAqw4vqK2CxXl/SGTYgkKaZImheRHoawsaGSMQWkOFIzIlFq",
|
||||
"Zt8CDquSsIwu+yWWszHHIrusjMJLLOBUdDnyZqvoZWOUMwJyNF6MtIJjyBZnGdFbxfSiMefq60t2Oivm",
|
||||
"GdOw1ECBDI0XyM6btGiG7uP+0j/wt53qtPrpuT4hhNvw0lT6mZ9pXKa3oOxk3w9ywkoFyF/hCcq5VJqp",
|
||||
"9Bv6oqzjuokQO1H3ay4Y2DdQt5vXvxCjlmVeC7baA6HdTxfHfGIAg559N4shYgaYqoh0lcFU4AxarAEZ",
|
||||
"v2uR+O18i/hDqTCNINysj4+lxqnZB6cZ6BMaW2+0EY6+kQjuC0g1LaTYChQxwbNkt6xldUuY80tNqs26",
|
||||
"nVWzxNBpr3uLO5iDcAasXdHHC2CozzgbCJCcziE7cn7AVXz6z62samlrK5wdO2kC8uNbSiJnWJx2G+wc",
|
||||
"g9g5E5zSD+6OXaG1GZfK+6easDlNVYkp8gPMlTcDBGY+wqYox+mMsCgD5yBnzrbUnPTSRtvo5+jNhREG",
|
||||
"tIBqzq+5NVFYsalVqdoUTnInTxjcDSguFC+ONtqbzLTr4OZiMGJy1qgQZI4VjG5jsR+nr84Hl+dnH86v",
|
||||
"Bn8+/9vg+PjYbJdyfXlmkIpF0bZXM3c5piSNT42n8FTPZ8doGjVTX76/uKxJOXHjjLu+R/Z+drJw2x3/",
|
||||
"EyNagsD0tFQzd6WjNy87zWzlg61nd6/FiMrS28gTzMh4Y9d9wL1RkZiVU5B9cRNpLGEhWUF5HJztoIiT",
|
||||
"mYoHFiglyLhUjXOsem0X5bG6blqvAMiQHZWguxkwx/AGdESinDOiuHX0BdtJh4PcXz4fD+d8Moq5uZsc",
|
||||
"CJpbMou3SzNBL40ttikaG9eWY32RMMxSGJVMWZv/blP5I7f1mKus1bUIvV6Lab5jdEGbbWknY/d2jjdn",
|
||||
"bHL3odl9NUeDnBvLaeeYVtdbg2/abOB4irXOa+hbf+AbicKLIxdQFPn0Rqy1Y6e5kpfWgCettgSIkgmk",
|
||||
"i5TqhTjj3mhJdVjF45KuWEplPL9IizPB/wuV+bnbXdhEUjsCLuLRg6cKUTDcxoLIMCFAM4lyt8JCgASm",
|
||||
"jnvJFrh7B8KEtM1XcNgFcV+Ec+OovjJWpArDVjFAfSUwk0ZorfYUF1daEHDlyKAFhqN6lGR9QX+6fP8X",
|
||||
"dOlBtdF+13g5su2Ma+BGHxE58nQYNwdTvABRN/7loLC9QAW2JqlSaKxM+RyEQZ9R9qaMtEbSBEB3NfO1",
|
||||
"IrTAQl/ent02GxcNTEdrnTKrkTUmMAjM/VkISE3Q48e9Dlx3ujrEeCg30RFW8nEtfW08ZUcrwbwPQTnB",
|
||||
"3zHBVEYdQCuUdEgS2plk1h+3cTytR0iLP+Yg+NiVNqNH1NxF7C77jbaPrsAKP2BsRd2KUKMd3kt6d1h4",
|
||||
"E70gSsvzcQ+EUWnjBk7ZXaAKISxB8LOGgWOBiYRsi8AJd3/XjAluiVHSCrGLW/nOavHIm0N0nCmj6/i0",
|
||||
"4dPr/BbXYFN7Bpk+kN9u63jrzo4+gfOItHR5SyhF9inqr8pM9ok7LI5iEpMAaU7c/T18D+ihs4NrN+Nm",
|
||||
"wK6T1MW+1BOJm3WRu83AWRt5tDaQ1tn6zPGj+XiyqP1sB08wsdEXennZiJfGEq5vOGr/Lrj+YTTG6a37",
|
||||
"Tf84cu993MflWVtIRLLbOho3hOFu6wwNx1ergbM6xaqTVYDB9nqOirAElm1XZ53Gl1jRmpONczDnJggc",
|
||||
"Musi6/PCGq2PesnW4Ur1ZL6Nl4Oba20c3SuBi9lfCdytwhCyKTQDINal2nxwCJQzUsRcMJUdqs1sv6Nx",
|
||||
"KRKZGXGTkQwNrssnT55DsHU5jdTavAhLaZnB/7Y0acxHzkMN0Zg5xrMtgOPsfRGwKFGy1Oc8xZ3Z+lMo",
|
||||
"xYVZ1IyoiP96WcbkNpDaYjCG9tcGBq2+VO+6aGxwyYUQ8HkovK1Y93Y8jlvjsAQ0I8BIrs9in6Gll+wC",
|
||||
"vZT5QMMntaudba2xxcGwiwdTtqHqP27DFrfhMuQdACu/XBTuf2b8jmq+eU0i10pXOzVht5CNoly0MS5E",
|
||||
"YHbbKXCrXahhpChil8hrMp1RMp1p1Jg8Xh9h0omvbOx451hzRRSFNTuu+DDjaZnbsBFRsjHnt8YaNAep",
|
||||
"yLQte2qzvdkuIIbkt17Vf6mP7FWOqptio4aKrN0WuCW2FYicaClip5eDNTGiDXyaCJ6foE+Kn6BPDlTy",
|
||||
"BP3McA7ZwLBqgo6Pjz9+/vx5o3ub+LQpc6+sGKtr64jB+x0oQdJLEA7Ckctm0aZ45ebdlihCSsuiTkkC",
|
||||
"3/WS3tOZ/qclctBIg+suNjyfdmK/LfJBc3zfacqcsE7jtrEwhPyEpRIY+A5ZWCCfd7Dhs8vCpex0b4VL",
|
||||
"d514dGUGRUMqFlYhcFQQcF4hMraICxsJu51xoygoAdkei90Mq21C878IlZwhyu9AoDEvWZZoia2wQSQw",
|
||||
"t+/ZFOLh970ISptj4reyVuFKsXbINo7UYCDYS9wqKlivPPu1xAIzRVhbZPgWuaizRcHVDCT5u8098Iv3",
|
||||
"Kc9LBktzgYQxH9tTwNdBczd3Z4OSarqvh1SDlFYwX1OL14Vx1so2LN9eS3lptfxBIbiI3BQ/EqDZwGT0",
|
||||
"1cJxkB2O+t89e3bUHuVn3Hzx47lNc16CnZ0hjO9yqvh8nQ20E0s12CSUBK9DD495qU7GVMtjteCBUpDN",
|
||||
"mrf5zFp3y4VWPeRaG8Y6p/a7Ny8TlHIBMkEC56N8nKCMyNvRdJwgUiRIQV5QE4yYm5i2BGmxnaQgo0GJ",
|
||||
"XEbkxUtaTr0790Lw+5zfm8gwF3RVi1GI2jLiEWavyxyzgQCc6XMFOX9Ix8gv8116n578ApQuJoRt7yqn",
|
||||
"92mC5nmCuEAZT29BmHoomLB6aHV3Z7kD3gYctwWUVem43ewHIRowanYKhjH05qWJMhA4vUWFXwZhU/3L",
|
||||
"VICxv224JqLXcW9pCWu3fRk4cWnT+mSJ1Myag8CU2oMHkUlz4cHwubsFoMVZf1YKASxETbSGYEgFxTrJ",
|
||||
"0ax7lIOUeNrNeTwhjMjZ/vbnh7BhhwBUUTLnETOKWcCDvNVqZsvlqqDoYA3Ro9aekmuTBRwzenxZ9MRm",
|
||||
"adgmN5yzGx0flecvfli60nKBXTqbeMNpa2NH1kzQJqKay3tkypptox+QbKT4rrSzjBMLnaQyPruzsra2",
|
||||
"TSjqFuXVGTHrbebt+Nj4XjeLXxwgm2AQj/NJscgIw3TJdc0ZDBQfcBOW7X7JMdPEo/+rnvnfzMONtvOY",
|
||||
"5YNpoRT2C7FxlqROpUx622Zeb3w/Ho9RX1PzC0kD6lG8EXl75j2h8aSkUfXJCm3MIawqV+F/tIluIm85",
|
||||
"XUMSK6Za5WxRrjbhsgU/cfisbiSyjBhwXAm2VVI2hv6OLl+vI3W8VIVUIwnAtvLWTygu1imDM06zUcbv",
|
||||
"2L6xhNvWm6pCQ6wAP3Cm77hWv/W+KbkFuhiluOzI13mp9o6n5GlqhK71Bo8DhOlsFAUr06ELuMHprfcB",
|
||||
"eOOC+Y7etlVTbc5QJQp93OWSr2LsbeLkIYpe+fJWtRggJxutwHuZTZaoJ8rJt4TSvWxqmwNxTB7tqLIc",
|
||||
"dHyjc324bexjpdxTqF4TMGgz9Um2pcG/EDyFrBQx8dNHPWbICMJDGz8y9AEgQx8XVFDM0Ifngx+OVkKx",
|
||||
"vd9uFCKX2qq5MH0OentkvuIL37yReihSPO7Crbuzj92Q56XWKGLu0xXVbeepLFwPMVdEG5Ih8CeeCNnd",
|
||||
"XpoJPHGxCz6IIdhJBUyMTbamzG2IQ/bGUrE+7aSyZ+9mOl1JEQnG02AUrVig9Yy6dMrnchRqnmMWMZq8",
|
||||
"4sFYZsrQuTi5Xry8pKtluMw4RIU6WTF3esZLPcCYmWRc6OqedyIgXlxN6W2o1hRYihfdS6xopb8ZWy3l",
|
||||
"rJf4qjkmH5y1XLptd98WgI4Xvvn9k42lD9y6k4DuGJVceafUEgAZzzFdtEjTREAVVLZDBMmqwMk1x0Xt",
|
||||
"rsQ45ihhgAUqBP/FfjpBf8iQjfjb7Ets95tKymOa01v7uQlRqACBMrzY2ikY3HQVtKKBGRLSUksoJiHF",
|
||||
"VQsALECclrFcxfdOLRrOCdyBOEF6mJaebtH7Ny/P0J/+66oe70rY4PTiDfrXP/6JznCWLa7ZhIs7LLIB",
|
||||
"LtUMEZNtBUzCgLBBBoWaJYhxmxfmrDdaQhOlmh0dXzNTDPrEmAVJiuw6bTKpLZheZZ72TZEvdGMCpW/0",
|
||||
"u76guCEm82ZF7YaVTGlqI9O6mtcu+cGVJM8UF3w1qO3MFvAe6LsckN6sL7DyntxyiWY8B4rH6P3lMbrS",
|
||||
"4uWEUNAb10O+/TZs8pqZXX77LeqbmuA4VQMjFx6doFfceAxAIKnKsURYAKpK2t8RNUMcF2Sgj70psOSa",
|
||||
"2bxXifr+82dv3yRoUmqpBP30Rh5ZeBkw4xyQLCA9vmbX7IyzuUYnZzX55PnRyTUboHPrhdJf9wXD0U1b",
|
||||
"efKbY/3KWyKVRKUEdPPJ3NFJvcvC5xu7eNeaocBTwqzDq+8OGmRKzqPvnyQox/fo2ZMnR2ben5jEE0AX",
|
||||
"7y+vbPGTQqGbpXr8N6hvK/sXFC/QHWEZv7NvvyvNoYCEaz4hUYqFWKAbd9vdvECvzq9cTwCJbs6v8PQm",
|
||||
"QRenV2evkY/fQDe+xP4N6rvi/L4ov/1MqLlTwez58+c/oJ+uzszzcxeUZJ7iLBMgpVnXuBldivrNLhEG",
|
||||
"UVczQO/OLmw5kAlOAfWlEoBzM8Prq6uLBPHJhKQEU01Aly//fGRziEtmItEVuhnmaXFzzTirCGFMGBYL",
|
||||
"hFmmB/NSGTuq4SVL15plXZDQC0RMBiWnEt0JXFyzip6sfoxMSg3ChtqlVrOyghOmpOVHSlJwrhjHZBc2",
|
||||
"u1sf14I6xpQnw6HTvI+dJ3nossBrfsSeZbfTizc1qeWk9/T4yfETo+YWwHBBeie958dPjp9bJ/DMnHdD",
|
||||
"c0gMcK3IvLs0rRGIcPYm6530/k8JYtGsR9/sQfJzvJlBrST4mtYLLe82aojvMEE9dGPtyzHJudrcMHTw",
|
||||
"6DDW9XboMNL1bOkw0jam+PxxqcnDsydPtmpRsBRE6NWGTvpDE/URfaTeQGb7qmVmCZE7euXK8UtAwJSJ",
|
||||
"4/qcVIJZfAsBZrVmELVIVttlAY1hhufE1MgwZnY8lcambcNMx8SbXe8HfuG2lmbvpGfFATPrMJROaeUk",
|
||||
"fS2chlGdmChoHRUuD1cbvY15vBVn5ZN7V5z/zfFGaIryeGwRCGp/ftAEinCNQj0vVCWAtmGE4SeSfR6G",
|
||||
"1iMa1k2K34Dg0FJK47hwASJNlnppujMENCRbfmGpkZKlJRMN80eeLfYgo/qmQ1qrZVPLpYvAmVHVjPGW",
|
||||
"yN+W4jOv352eVQ0MrG7Ql4RNKQxKCQnyeVNOpB5IksHmMkVhG3FKbLZY+rwnI+7afeilWyQSkHKRQXaI",
|
||||
"m8HiyoToAFvUYemg2wrQriwTvG6WacrMxvuvkcHMkG6yV72I+S7Sl62E/5CCV5vYx9kub64k8P1H6Nvv",
|
||||
"Yqt6KDzi1aYX4cU91FcSvTy/PDs6BHubmbeX95o8azzI68W9MzukE9OuiF0daT/EdezArL6S+sqrtcy+",
|
||||
"3xZl2wYGj0fUjiIOJKwZEkQZTAhz6S8VQbsKjxsktjbJysZAWWg9oli1EZUuVquTPPL0sJ+Ootcmjh8A",
|
||||
"v3YmhB2O+/a0GWA5yLDCCfJmyj8cdcZ57PgyQvq+srkvD9MkIVM1ZkcKcm1CH5R0bFWbLyzJtlKObz25",
|
||||
"P+XYmYzsSqxp1RHRroTSqIyy4cJbGtvNyhHqGHxpidNXxl+1dWzVEOA3d0k2+1s84m25RE4HOFbdjKA1",
|
||||
"O6s4atlyBshFE/JSDvwTZNQypAQm9GhXe4jzSg1tCWODmmiyi68TKX2d4iS4uyTCDOEpoFtYFJiIxPXT",
|
||||
"NX9vLzybGI9GLTm2EfXFG+kNx+gMUwrC1kvEVADOFmiG56C/4YtYMHPtMMj06dLIjjCBXtbB0TwVbEXj",
|
||||
"M1+X/yFO82ax6S98oC9VbI61GzYjcmAqNNe2HkDbxIBlAWEHoG/7MYQRgztf3Phf//gnIlKW4GnI00+N",
|
||||
"dsISKip3hNtC4rbZXYPCP0laTj8P06pJSDQM44PzMN7NSDpzvUBM/4/EutUs2Zqy0rbfhm9vgUybD0PE",
|
||||
"UzIHhpT3NRovM0PeAWwafBivHWFSAc4Qn6ApUagoKY0R6StQzQYnK/dWbAuIM7pwi5NhcURW67JNpp8/",
|
||||
"f/7DUUtzfdu5ZOu23x8fUkRpQCJ2Kru2IBlQhQ9As69AOTJI6zM7iOIKnNsSZ7KTVHtJy2nv88cIZcuq",
|
||||
"a8l0bZHxgQkwMdZB84Z1Jme+8K6d9hu5cmK3E6ZvmPLgePcfiuD9slPvlQOpth5y65q8fFliyHwLmGGt",
|
||||
"GE5UEn4FaqVfzAMibuVbMSu5H4P84vfH03sGA8FLlg2UIIUJqdOCT4gFcm3+keA8NyFBqMBT2MPHWi9o",
|
||||
"06qC+ACTTWf4j4QqEKY+Qr1ZoivEJZEeDSzDTMm2w3tXC3vIGNz2xVC1des3fTnetS8usXs5tg9drR7O",
|
||||
"TGDO0AXJxr7y656m938rLam9qNoX0o5cTT5K5MFOXaiYJyg7tXJVOxsSz/319lVaEhvF/b+wKdFTUbst",
|
||||
"0dRKy8BWzji/wtO2Kd2woRnjJjyIDZKhFeVgA1U0LUh+8DCojO1q8JnTbOtNQCq10/W3mgddOeVMav3c",
|
||||
"xH3aqhRac05xgVOjAvuI76MEeUOWm926G6vqtCaiJaIzNxVlfQz63QWXe0ynCFUIvm7aX6kD8oXpf7VG",
|
||||
"RftJ58qyJk2E/FpC+ahsEraAMNKvlSqQbv/t/z1L0F/fJSjU+DhCZqAp2rEvP3njfZsUGkjvAc0fbceX",
|
||||
"w5mrB/R42HkVahkshxjvdMkdwE2ynPbge3TUTx0sINJ1pNY5pmoXkMHkxTUjlMIU08YkNpYbfffkBy3X",
|
||||
"mukG1fOjY3Rho/im+iPXzB6IWitdVK8+R31/ygW4HEXPO729Xc+6B/b31JvHfHH7YBuDOI9Pdbc+Foc4",
|
||||
"h1FV3kIzSK1TzFIXmYOcWkPT5XpQdbluO8L+qMd9sMM6eZNMQk1DDwngeW5KIZK8zHsn30dSuR5am1gO",
|
||||
"EixsslFLk9jOVZnaCiXZD2xd1maLCJ3JxNbZ9ai1du2pwMUMZcTVSDuEUdvnjPgPkok1BbmDfYIJlV/0",
|
||||
"OF8laB+AJjdfyKHQSjeKFkB3DuarEuGiHNEbc8MsoZKLSe4zJgbz5OPhTc/7qNzri7zvTMf1aQ/hZHxp",
|
||||
"gI5EfVrT9zx40voauiggRx49IvFa23YQqYdVLncbFS8XWHvA63P5UxHsXTT9kFC4Y8jt4wDyPac0XsTO",
|
||||
"mDqXhf4vhcqaadp045Xr44fP5649a4cT58HzrqJW0VrZkP8EAD+ybXPufCaPZdqc20zeAwb9viZScWGc",
|
||||
"3eBZYWdHxNxCy+SetroDL21qwCUwheyGjtE5Tmf2+99IdEOyG58VbXvgC36HSIb6AmSZwzUzB9nNWy0q",
|
||||
"mxkGb17eHCXoxoxeelcDNUE3GVY4PPnT5fu/XDPzKrLQPkavAQs1Bqz0uZUbOGvOW6Cn38tj9EeQagCT",
|
||||
"CRfGDUvMk3/945/XzNSVhgwVIAayHOudjkGgcTmZgEhQJngx4DQDqVwS9cXvj16YNOhX51fIweyaKY7G",
|
||||
"OL2dkLgr/tLAtO2wanXhBAigQsCE3O/rsbFKVvViAwVrZ9jMtgrulQXHoKKg9glX/bCX58i9eAiz/9wT",
|
||||
"kJ0T9S8vz4/2YY4qOmqtn64atmsu5INHyH8lGSn/XldHyBJ9xOujoq1DOcbq1Lp1HGDS4uy4mgGaYZZR",
|
||||
"EMveiX6I4TM0eJTYGF7p/BRDX/wwuWaYZQiImoFAwIw13F0LoRpz34azukThI8RFLYLwmoXMQWd7Mz4Q",
|
||||
"XwiiORNh6Ma3l7sJUX+nVHIE9+avPsjFRvQITsEEoNlwLDvd+7+8/Ru6wws7Ruotxq4C55E4r6cdf5Xu",
|
||||
"w+V2cF/ahVhx3BpW8N4T1M9tiVKXQR6cWIcQsj4EAqpTnyPtBfrX//v/VZqqTWzQf3JUu1WIbS3+sBq7",
|
||||
"2SFSo6WHM/l2wsch7GIBxK5t3O+Q66C52xm1lz2hiYShbQn5IFnfZ2bqx0flWeh6eQBXu5kLYeQP12Ho",
|
||||
"xYnqdRd2zC/WZ7NoTzA+N48vAbK9jTlLooMprMRtrFwTen87ffcW1VpvrdZoZYpTPt3lVXtHbv3ikrwR",
|
||||
"FpDU9hEm7yKHaIiicakv+IMcrj4hAEk9sd6NtDWtUtdC4OUffaf/lx/QELmSQD4Sr5EqtpAK8k7EY+z5",
|
||||
"605V08Rzk7J2qbAInth+3Q979ALxnChjTLubaYHBehD6toVRW/Sd4HwnoX6Ng+jZBgdRYmqUUlNo0Uqs",
|
||||
"ne313WuTSrUwtZ0mXOS91bC8pU6hv3DCvB9k5P6mxbeCFyU1El5osXrs+j0mXTbhPhPfQ6jJuNw5oeuu",
|
||||
"HjJ+vWopG+FI8xDNzdO9GfKyHFtK1ZQ7J7LElPzd1XIzPVDR75DpgbqDeV8zXtXjtI3zfqQA6rVH64OB",
|
||||
"tNmuNQJWO+CAscVmY65Vrp/WmvVNRzdEWKa3wsU+ZrxQaHsoAYu0HdKX5nHozdnNYPFrb1kJ2M8K8BXo",
|
||||
"9o3upIfzv70m6hCK+o8lpQOTP2LRaYu8BiRXXuq+FwFkglzHzwaLhle2oqFPwfvRISarTku/NXS+Ne1n",
|
||||
"K8AfomIHpUFuk0OPM2Qb3SLFozGqXdEY52bTaDbq2erO1EYzskVpN3jn3rlBnU6Wzm62Dvd8KJl7EFEl",
|
||||
"Lqf5lpmR+ANcKl6LP2g2ULUtPnZLuN7FifeYR2ujR+3heNFOiyQcqGaiIVaTkTewc6I8UO6u13Bd0Ws7",
|
||||
"Nd9XutgBUeQYSUNx27wP0yApwhIhWK77hI120LGO3LU4ki3XutLSaRNlNSAS/XRji11IMKDuMDc88uRi",
|
||||
"zMq13C19mdeW6/9WW22NRP2aulOnjZhZDAIC4ib9ekgtwtKlMI/yUjnFIBjeNefgQTCH3s2AoSrCdsUa",
|
||||
"Xk+kubLa5VecTKNX+JgJNZbY1xboefbkWQc6tEbyeqXPvY22SiswagYVJRvFxubs1wi6O7027TVRih1+",
|
||||
"0tdxrNRPRNpxKX5bCDqt4e2vschQBhSUKQDPuEKyLAouTBX3makL77rpSgT3RNp6BaEfSEjhtzEFL59H",
|
||||
"WKMWer4bZ3yR8HO9tEcMQW/jiFrhoUfiiFrBooD1KlRyH05wVYnXByJc+EHbyN57FNfcMaKgW/zDbymS",
|
||||
"wLfRf7w4gkAaB4oiKCpS8/RMwXWS2yyJ+LcPWrMt3ukESTwBtUBzTOfgjt7LV384OkanocC3Ps6LurSz",
|
||||
"Iupcftd2WF+EXvRf/qRukmRrqeVfSywwU6ZRVbQjz2rPq6rhf63ZVa2zlfEjhTExtfZxyywHhvsab4mr",
|
||||
"KiEJe0ZCfVdvHtAQVbBFw+omOerOa0t3h4uzCclvJYVuxf0/lBRk7yuoS68Xcsg0CbOvA9eZR6JsqmaV",
|
||||
"e3WHgKqzoFw1GhSEr70wcrf+JHLNIW0RJzxRIFbzuv3J97xVH2uA+ivVyOpr3EYnexQ2v7DxD04balAJ",
|
||||
"6mclpoOIM3stzXRg64eugroflTywdvJvSh6GIhx7H5IwXFzlOmPkqRtzCUoRNn3cs765lgMe92F3hyi4",
|
||||
"bheJpJsT9W8JpQN5R1Q6SxCDOYiBr7lqKtoc7XAlxGXaD5hIE+foF0EkqtMLhQz1nz15hn5XhUIeo7f8",
|
||||
"DkzxI6Js+oNbOrqZUj7G9FhPN8KpOkHXPT6ZXPdutAaLMxtTabc08oPQLbgsCn/tkDyHjGAFdKG//uTo",
|
||||
"xFxNNbDYUpxmHnSHXXwMZuuLjpjTJkaeu50bejv6EaYXDRpt8ww9nNz6dfLIqcGmTdhRgthQ7UcUksPx",
|
||||
"6GndV6dsnJAvGmT2/scfNUsEgtzv/BRE3g5MwO8GafkDkbdnbtxjphT7ZRxSUCbyFnkYHEheFvU5tzwa",
|
||||
"NXoaycj2kKRgVd/lgr1ZI826W3KOaSm5bcDL2mSdvWeqheFtZcheIsXvYhVsa24mYIfpOXTOMi3V1Kfu",
|
||||
"S1DSloEZKW51FxPJQiS6hcLeCDOT17g42qUsR5sade5aVlofWqQQzapfEPVnBAQW6WwxwHdYwNELlGKR",
|
||||
"EYapbds34SKFrE2RWk9zX4ciVV/j4zi3mgUQvkj/iQZFuoilneq/+J4D6y6FSzemc0YgHCzTPPTUZhPe",
|
||||
"S3p3zlSU9FJBFEmjvcYfJA++SyOg35KZ3+L8Ea38nugOVbk40PB2rXhqPGJTaHB6+yD5M6fprYN5HOvr",
|
||||
"d25fPVy7ktO0itDEDng7dippQC8vrXRzcPC9KxXU4HcIJ4Re66hkitCuFeBbe0Qud8SvZt6jieOXpQgN",
|
||||
"4EAKrsTK1dXbQxCFAMnp/GHo4oOd+8Ck0Y7mFWR+FchzUKjwl2NWYkoXu6JPq6obhAY7pFtrTGt+GZmo",
|
||||
"0/947x/yWtdYecxb3VLFoS51Mxvqm5QqFRLrChD20dFuLn077UO7HywqvmJfe0EYg2zkoBqvibjqbteY",
|
||||
"aPW1f33edccQX71v3dCkafBE9K8eKTu60WsUPnRTdTjN/+pHfn0H2M4HUtjT/vhyU3nbj6ktaPG2/TG0",
|
||||
"Z96/SceTXZOfrszorY+if7dkDrPNA5KOA9shGB1YhjDDdCGJK19Iqc/hMJXJI4lU2yR0PGQyld4KpKUx",
|
||||
"3eipx4AFiNNSzXonP3/UGLfd2O2HS0F7J70hLshw/tTQg9vPatsml9zv8s5DXoEpOWtK4dRt581t2LSa",
|
||||
"lTgU23kNQuu3pOptRaSts0w4S3ybo1rhKNfLaHXO8+1SHdx8vMq++BS3e5gtuuJCfYtq4wNqdOeMLijU",
|
||||
"oKh6K7hOjUnwUkrUzyAlGQxxqmrTQr1E06eWuEuztCB66fOsNkM431bfrztgkqVQoyQ4x6qpnBdldaKQ",
|
||||
"IelIw+UJV7a6Wo7jp2jqlUxsxrL5bkZU4qoPJihk43tMNbgsBu6CC7X6nqvk8Pnj5/8OAAD//2XM3suJ",
|
||||
"7gAA",
|
||||
}
|
||||
|
||||
// GetSwagger returns the content of the embedded swagger specification file
|
||||
|
||||
@@ -75,22 +75,31 @@ func (s *Server) resolveEntityID(ctx context.Context, idOrSlug string) (uuid.UUI
|
||||
return id, err
|
||||
}
|
||||
|
||||
// entityCols requires the entities table to be aliased as `e`.
|
||||
// entityCols requires the entities table to be aliased as `e`, with
|
||||
// entity_status left-joined and aliased as `st` (see withEntityStatus).
|
||||
const entityCols = `e.id, e.slug, e.type, e.name, e.state, e.attributes,
|
||||
e.maintenance_until, e.version, e.created_at, e.updated_at`
|
||||
e.maintenance_until, e.version, e.created_at, e.updated_at,
|
||||
st.health, st.last_check_at`
|
||||
|
||||
func scanEntity(row pgx.Row) (gen.Entity, error) {
|
||||
var e gen.Entity
|
||||
var state *string
|
||||
var attrsJSON []byte
|
||||
var maint *time.Time
|
||||
var health *string
|
||||
var lastCheckAt *time.Time
|
||||
err := row.Scan(&e.Id, &e.Slug, &e.Type, &e.Name, &state, &attrsJSON,
|
||||
&maint, &e.Version, &e.CreatedAt, &e.UpdatedAt)
|
||||
&maint, &e.Version, &e.CreatedAt, &e.UpdatedAt, &health, &lastCheckAt)
|
||||
if err != nil {
|
||||
return e, err
|
||||
}
|
||||
e.State = state
|
||||
e.MaintenanceUntil = maint
|
||||
if health != nil {
|
||||
h := gen.EntityHealth(*health)
|
||||
e.Health = &h
|
||||
}
|
||||
e.LastCheckAt = lastCheckAt
|
||||
var attrs map[string]any
|
||||
if len(attrsJSON) > 0 && json.Unmarshal(attrsJSON, &attrs) == nil && len(attrs) > 0 {
|
||||
e.Attributes = &attrs
|
||||
@@ -113,6 +122,7 @@ func (s *Server) ListEntities(ctx context.Context, req gen.ListEntitiesRequestOb
|
||||
)
|
||||
SELECT ` + entityCols + ` FROM entities e
|
||||
JOIN entity_types et ON et.name = e.type
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.type IN (SELECT name FROM tt)
|
||||
AND ($2::text IS NULL OR e.state = $2)
|
||||
AND ($3::text IS NULL OR et.domain = $3)
|
||||
@@ -159,7 +169,7 @@ func (s *Server) GetEntity(ctx context.Context, req gen.GetEntityRequestObject)
|
||||
return nil, err
|
||||
}
|
||||
e, err := scanEntity(s.pool.QueryRow(ctx,
|
||||
"SELECT "+entityCols+" FROM entities e WHERE e.id = $1", id))
|
||||
"SELECT "+entityCols+" FROM entities e LEFT JOIN entity_status st ON st.entity_id = e.id WHERE e.id = $1", id))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -231,6 +241,7 @@ func (s *Server) GetBlastRadius(ctx context.Context, req gen.GetBlastRadiusReque
|
||||
SELECT `+entityCols+`, b.depth
|
||||
FROM blast_radius($1, $2) b
|
||||
JOIN entities e ON e.id = b.entity_id
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
ORDER BY b.depth, e.slug`, id, depth)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -246,13 +257,20 @@ func (s *Server) GetBlastRadius(ctx context.Context, req gen.GetBlastRadiusReque
|
||||
var state *string
|
||||
var attrsJSON []byte
|
||||
var maint *time.Time
|
||||
var health *string
|
||||
var lastCheckAt *time.Time
|
||||
var d int
|
||||
if err := rows.Scan(&e.Id, &e.Slug, &e.Type, &e.Name, &state, &attrsJSON,
|
||||
&maint, &e.Version, &e.CreatedAt, &e.UpdatedAt, &d); err != nil {
|
||||
&maint, &e.Version, &e.CreatedAt, &e.UpdatedAt, &health, &lastCheckAt, &d); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
e.State = state
|
||||
e.MaintenanceUntil = maint
|
||||
if health != nil {
|
||||
h := gen.EntityHealth(*health)
|
||||
e.Health = &h
|
||||
}
|
||||
e.LastCheckAt = lastCheckAt
|
||||
var attrs map[string]any
|
||||
if len(attrsJSON) > 0 && json.Unmarshal(attrsJSON, &attrs) == nil && len(attrs) > 0 {
|
||||
e.Attributes = &attrs
|
||||
@@ -283,10 +301,13 @@ func (s *Server) GetGraph(ctx context.Context, req gen.GetGraphRequestObject) (g
|
||||
nodes, err = s.queryEntities(ctx, `
|
||||
SELECT `+entityCols+`
|
||||
FROM blast_radius($1, $2, $3) b JOIN entities e ON e.id = b.entity_id
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
ORDER BY e.slug`, rootID, depth, req.Params.RelType)
|
||||
} else {
|
||||
nodes, err = s.queryEntities(ctx, `
|
||||
SELECT `+entityCols+` FROM entities e ORDER BY e.slug LIMIT $1`,
|
||||
SELECT `+entityCols+` FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
ORDER BY e.slug LIMIT $1`,
|
||||
graphNodeCap+1)
|
||||
if err == nil && len(nodes) > graphNodeCap {
|
||||
nodes = nodes[:graphNodeCap]
|
||||
@@ -523,14 +544,18 @@ func (s *Server) GetFleetHealth(ctx context.Context, req gen.GetFleetHealthReque
|
||||
Type string `json:"type"`
|
||||
}{}
|
||||
|
||||
// Exclude 'check' entities (internal probes) — only entities actually
|
||||
// being monitored should count toward fleet health.
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT e.slug, e.type, st.health, st.last_check_at
|
||||
FROM entity_status st JOIN entities e ON e.id = st.entity_id
|
||||
WHERE e.type <> 'check'
|
||||
ORDER BY e.slug`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
stale := 0
|
||||
for rows.Next() {
|
||||
var slug, typ, health string
|
||||
var lastCheck *time.Time
|
||||
@@ -544,6 +569,8 @@ func (s *Server) GetFleetHealth(ctx context.Context, req gen.GetFleetHealthReque
|
||||
resp.Summary.Degraded++
|
||||
case "down":
|
||||
resp.Summary.Down++
|
||||
case "stale":
|
||||
stale++
|
||||
default:
|
||||
resp.Summary.Unknown++
|
||||
}
|
||||
@@ -560,6 +587,9 @@ func (s *Server) GetFleetHealth(ctx context.Context, req gen.GetFleetHealthReque
|
||||
Type: typ,
|
||||
})
|
||||
}
|
||||
if stale > 0 {
|
||||
resp.Summary.Stale = &stale
|
||||
}
|
||||
return resp, rows.Err()
|
||||
}
|
||||
|
||||
|
||||
@@ -1974,14 +1974,14 @@ func (s *Server) GetTrends(ctx context.Context, req gen.GetTrendsRequestObject)
|
||||
f, _ := slopeNum.Float64Value()
|
||||
t.Slope = float32Ptr(float32(f.Float64))
|
||||
if f.Float64 > 0.01 {
|
||||
t.Direction = gen.TrendDirectionImproving
|
||||
t.Direction = gen.Improving
|
||||
} else if f.Float64 < -0.01 {
|
||||
t.Direction = gen.TrendDirectionDegrading
|
||||
t.Direction = gen.Degrading
|
||||
} else {
|
||||
t.Direction = gen.TrendDirectionStable
|
||||
t.Direction = gen.Stable
|
||||
}
|
||||
} else {
|
||||
t.Direction = gen.TrendDirectionUnknown
|
||||
t.Direction = gen.Unknown
|
||||
}
|
||||
items = append(items, t)
|
||||
}
|
||||
|
||||
@@ -125,6 +125,7 @@ func newServer(pool *db.Pool, agentID uuid.UUID) *mcp.Server {
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, st.health, st.last_check_at
|
||||
FROM entity_status st JOIN entities e ON e.id = st.entity_id
|
||||
WHERE e.type <> 'check'
|
||||
ORDER BY e.slug`), nil
|
||||
})
|
||||
|
||||
|
||||
@@ -94,6 +94,13 @@ func runCheckPass(ctx context.Context, pool *db.Pool) {
|
||||
}
|
||||
|
||||
// runCheck executes a single check and processes the result.
|
||||
//
|
||||
// check_defs.entity_id identifies the *check* (probe) entity itself;
|
||||
// check_defs.target_id identifies the entity actually being observed (the
|
||||
// host/service/etc). Health, metrics, and events must attach to the target
|
||||
// so the observed entity's own record reflects reality — not the internal
|
||||
// probe. Signals stay keyed by the check entity (cd.EntityID), matching how
|
||||
// they are created below and resolved elsewhere.
|
||||
func runCheck(ctx context.Context, pool *db.Pool, cd sqlcgen.ListEnabledCheckDefsRow) {
|
||||
q := sqlcgen.New(pool)
|
||||
start := time.Now()
|
||||
@@ -107,9 +114,14 @@ func runCheck(ctx context.Context, pool *db.Pool, cd sqlcgen.ListEnabledCheckDef
|
||||
}
|
||||
result.metrics["probe_latency_ms"] = float64(latency)
|
||||
|
||||
targetID := cd.EntityID
|
||||
if cd.TargetID != nil {
|
||||
targetID = *cd.TargetID
|
||||
}
|
||||
|
||||
for metric, value := range result.metrics {
|
||||
_ = q.InsertMetricSample(ctx, sqlcgen.InsertMetricSampleParams{
|
||||
EntityID: cd.EntityID,
|
||||
EntityID: targetID,
|
||||
Metric: metric,
|
||||
Value: value,
|
||||
Tags: []byte(`{}`),
|
||||
@@ -121,18 +133,18 @@ func runCheck(ctx context.Context, pool *db.Pool, cd sqlcgen.ListEnabledCheckDef
|
||||
"entity", cd.EntitySlug, "kind", cd.Kind, "error", result.err)
|
||||
}
|
||||
|
||||
prevHealth := currentHealth(ctx, pool, cd.EntityID)
|
||||
prevHealth := currentHealth(ctx, pool, targetID)
|
||||
|
||||
if result.signalKind == "" || result.health == "healthy" {
|
||||
resolveSignal(ctx, pool, cd.EntityID, cd.EntitySlug)
|
||||
resolveSignal(ctx, pool, cd.EntityID, targetID, cd.EntitySlug)
|
||||
_ = q.UpsertEntityStatus(ctx, sqlcgen.UpsertEntityStatusParams{
|
||||
EntityID: cd.EntityID,
|
||||
EntityID: targetID,
|
||||
Health: "healthy",
|
||||
LastCheckAt: &[]time.Time{time.Now()}[0],
|
||||
Details: []byte(`{}`),
|
||||
})
|
||||
if prevHealth != "" && prevHealth != "healthy" {
|
||||
emitSchedulerEvent(ctx, pool, "health.changed", cd.EntityID, "info",
|
||||
emitSchedulerEvent(ctx, pool, "health.changed", targetID, "info",
|
||||
map[string]any{"slug": cd.EntitySlug, "from": prevHealth, "to": "healthy"})
|
||||
}
|
||||
return
|
||||
@@ -156,7 +168,7 @@ func runCheck(ctx context.Context, pool *db.Pool, cd sqlcgen.ListEnabledCheckDef
|
||||
}
|
||||
|
||||
_ = q.UpsertEntityStatus(ctx, sqlcgen.UpsertEntityStatusParams{
|
||||
EntityID: cd.EntityID,
|
||||
EntityID: targetID,
|
||||
Health: result.health,
|
||||
LastCheckAt: &[]time.Time{time.Now()}[0],
|
||||
Details: []byte(`{}`),
|
||||
@@ -164,11 +176,11 @@ func runCheck(ctx context.Context, pool *db.Pool, cd sqlcgen.ListEnabledCheckDef
|
||||
_ = sig
|
||||
|
||||
if prevHealth == "" || prevHealth == "healthy" {
|
||||
emitSchedulerEvent(ctx, pool, "signal.raised", cd.EntityID, severity,
|
||||
emitSchedulerEvent(ctx, pool, "signal.raised", targetID, severity,
|
||||
map[string]any{"slug": cd.EntitySlug, "kind": result.signalKind, "evidence": result.evidence})
|
||||
}
|
||||
if prevHealth != result.health {
|
||||
emitSchedulerEvent(ctx, pool, "health.changed", cd.EntityID, severity,
|
||||
emitSchedulerEvent(ctx, pool, "health.changed", targetID, severity,
|
||||
map[string]any{"slug": cd.EntitySlug, "from": prevHealth, "to": result.health})
|
||||
}
|
||||
}
|
||||
@@ -188,21 +200,23 @@ func emitSchedulerEvent(ctx context.Context, pool *db.Pool, eventType string, en
|
||||
_ = observability.Event(ctx, sqlcgen.New(pool), eventType, &entityID, severity, "scheduler", "", data)
|
||||
}
|
||||
|
||||
// resolveSignal resolves any open signal for the given check entity.
|
||||
func resolveSignal(ctx context.Context, pool *db.Pool, entityID uuid.UUID, slug string) {
|
||||
// resolveSignal resolves any open signal raised by the given check entity.
|
||||
// checkID matches how signals are keyed (UpsertSignal uses the check's own
|
||||
// entity id); targetID is the observed entity whose status this affects.
|
||||
func resolveSignal(ctx context.Context, pool *db.Pool, checkID, targetID uuid.UUID, slug string) {
|
||||
q := sqlcgen.New(pool)
|
||||
// Check if there's an open signal on this entity
|
||||
tag, err := pool.Exec(ctx, `UPDATE signals SET state = 'resolved', updated_at = now()
|
||||
WHERE entity_id = $1 AND state = 'raised'`, entityID)
|
||||
WHERE entity_id = $1 AND state = 'raised'`, checkID)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
if tag.RowsAffected() > 0 {
|
||||
emitSchedulerEvent(ctx, pool, "signal.resolved", entityID, "info",
|
||||
emitSchedulerEvent(ctx, pool, "signal.resolved", targetID, "info",
|
||||
map[string]any{"slug": slug})
|
||||
}
|
||||
_ = q.UpsertEntityStatus(ctx, sqlcgen.UpsertEntityStatusParams{
|
||||
EntityID: entityID,
|
||||
EntityID: targetID,
|
||||
Health: "healthy",
|
||||
LastCheckAt: &[]time.Time{time.Now()}[0],
|
||||
Details: []byte(`{}`),
|
||||
@@ -250,10 +264,74 @@ func housekeeping(ctx context.Context, pool *db.Pool) {
|
||||
slog.Error("scheduler: prune idempotency keys", "error", err)
|
||||
}
|
||||
|
||||
staleSweep(ctx, pool)
|
||||
|
||||
// Log housekeeping completion
|
||||
slog.Debug("scheduler: housekeeping done", "pruned_idempotency_before", cutoff.Format(time.RFC3339))
|
||||
}
|
||||
|
||||
// staleMultiplier and staleFloor bound how long an entity can go unobserved
|
||||
// before its last-known health is no longer trusted. An entity is stale once
|
||||
// it has gone longer than staleMultiplier times its fastest enabled check's
|
||||
// interval (or staleFloor, whichever is larger) without a fresh observation —
|
||||
// covering both a stalled scheduler and a disabled/broken check_def.
|
||||
const (
|
||||
staleMultiplier = 3
|
||||
staleFloor = 5 * time.Minute
|
||||
)
|
||||
|
||||
// staleSweep marks entities whose last observation has aged past their
|
||||
// check's expected cadence as 'stale', so the system never reports an old
|
||||
// health value as if it were current. Runs once per housekeeping pass.
|
||||
func staleSweep(ctx context.Context, pool *db.Pool) {
|
||||
rows, err := pool.Query(ctx, `
|
||||
SELECT e.id, e.slug, st.health
|
||||
FROM entity_status st
|
||||
JOIN entities e ON e.id = st.entity_id
|
||||
JOIN (
|
||||
SELECT target_id, MIN(interval_s) AS min_interval
|
||||
FROM check_defs
|
||||
WHERE enabled AND target_id IS NOT NULL
|
||||
GROUP BY target_id
|
||||
) iv ON iv.target_id = st.entity_id
|
||||
WHERE st.health <> 'stale'
|
||||
AND (st.last_check_at IS NULL
|
||||
OR st.last_check_at < now() - make_interval(secs => GREATEST(iv.min_interval * $1, $2)))`,
|
||||
staleMultiplier, int(staleFloor.Seconds()))
|
||||
if err != nil {
|
||||
slog.Error("scheduler: stale sweep query", "error", err)
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
type staleEntity struct {
|
||||
id uuid.UUID
|
||||
slug string
|
||||
health string
|
||||
}
|
||||
var stale []staleEntity
|
||||
for rows.Next() {
|
||||
var se staleEntity
|
||||
if err := rows.Scan(&se.id, &se.slug, &se.health); err != nil {
|
||||
continue
|
||||
}
|
||||
stale = append(stale, se)
|
||||
}
|
||||
rows.Close()
|
||||
|
||||
for _, se := range stale {
|
||||
_, err := pool.Exec(ctx,
|
||||
`UPDATE entity_status SET health = 'stale', updated_at = now() WHERE entity_id = $1`, se.id)
|
||||
if err != nil {
|
||||
slog.Error("scheduler: mark stale", "entity", se.slug, "error", err)
|
||||
continue
|
||||
}
|
||||
slog.Warn("scheduler: entity stale", "entity", se.slug, "prev_health", se.health)
|
||||
emitSchedulerEvent(ctx, pool, "health.stale", se.id, "warning",
|
||||
map[string]any{"slug": se.slug, "from": se.health, "to": "stale"})
|
||||
}
|
||||
}
|
||||
|
||||
// checkHTTP performs an HTTP health check.
|
||||
func checkHTTP(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkResult {
|
||||
cfg := struct {
|
||||
|
||||
16
migrations/016_fix_check_status_misattribution.up.sql
Normal file
16
migrations/016_fix_check_status_misattribution.up.sql
Normal file
@@ -0,0 +1,16 @@
|
||||
-- 016_fix_check_status_misattribution.up.sql
|
||||
-- The scheduler historically wrote entity_status and metric_samples keyed by
|
||||
-- the *check* entity (check_defs.entity_id) instead of the entity being
|
||||
-- monitored (check_defs.target_id). Every real host/service/lxc/etc. entity
|
||||
-- was therefore permanently stuck at health='unknown' while all observed
|
||||
-- health and metrics accumulated on the internal probe entities instead.
|
||||
-- The Go fix (scheduler.go) starts writing to target_id going forward; this
|
||||
-- migration removes the now-orphaned check-entity rows so dashboard/fleet
|
||||
-- health rollups stop double-counting probes as if they were monitored
|
||||
-- entities. Historical metric_samples on check entities are left in place
|
||||
-- (time-series data, not safe to reattribute retroactively) but are no
|
||||
-- longer queried through any entity-scoped view once checks stop being
|
||||
-- written to.
|
||||
|
||||
DELETE FROM entity_status
|
||||
WHERE entity_id IN (SELECT id FROM entities WHERE type = 'check');
|
||||
253
plans/2026-07-08-liveness-drift-and-ux-cohesion.md
Normal file
253
plans/2026-07-08-liveness-drift-and-ux-cohesion.md
Normal file
@@ -0,0 +1,253 @@
|
||||
# 2026-07-08 — Liveness, drift, and UX cohesion
|
||||
|
||||
**Status:** Code complete for Phases 1–4 core scope; not yet deployed to the
|
||||
live containers (pending explicit go-ahead — see below). Phase 5 partially
|
||||
covered by pre-existing endpoints; full CRUD UI deferred.
|
||||
|
||||
- **Phase 1 (drift/staleness):** done. Health/metrics/events misattribution
|
||||
fix, staleness sweep, `/entities` health+freshness, dashboard/fleet-health
|
||||
`check`-entity exclusion, Entities table health column. Migration 016
|
||||
applied to the live dev DB (safe cleanup, additive-only).
|
||||
- **Phase 2 (detail sidebar + legibility):** done. `EntityDetailContent`
|
||||
extracted and shared between the full `#/entity/:slug` page and a new
|
||||
`EntitySheet` opened from the Entities table (master-detail, no
|
||||
navigation). Adds a **Monitoring** card (per-entity `check_defs`: kind,
|
||||
interval, enabled/disabled with click-to-toggle) and renders attributes as
|
||||
a key/value list instead of raw JSON.
|
||||
- **Phase 3 (sessions rejoin chat):** done. Fixed the click-does-nothing bug,
|
||||
added a session rail inside Chat, fixed the local dev proxy to match
|
||||
production's `/agent` prefix-stripping. **Also found and fixed a real
|
||||
latent bug**: persisted `tool_calls` store the `tool_use`/`tool_result` as
|
||||
two entries sharing one `id`; Chat.svelte's keyed `{#each tool (tool.id)}`
|
||||
threw on the duplicate key and silently blanked the entire message list.
|
||||
This had presumably never been noticed because sessions were never
|
||||
clickable before this fix. Fixed in `chat.ts` by merging tool_calls by id
|
||||
before rendering.
|
||||
- **Phase 4 (agent efficiency):** core piece done — prior turns' tool
|
||||
calls/results are now replayed into the conversation (previously dropped
|
||||
entirely), and a compact live fleet-health snapshot is injected into the
|
||||
system prompt each turn so the agent starts oriented. Prompt caching and
|
||||
reconsidering the default model are **not done** (lower priority, no
|
||||
measured regression without them).
|
||||
- **Phase 5 (CRUD):** `PatchEntity` and a full `/checks` CRUD API
|
||||
(list/create/patch, including enable/disable) already existed server-side;
|
||||
the new Monitoring card's toggle uses `PatchCheck`. **Not done**: a
|
||||
"run check now" endpoint (no scheduler on-demand entrypoint exists yet),
|
||||
relationship editing, and an entity attribute editor UI.
|
||||
|
||||
**Not yet deployed** — the live `oikos-api`/`oikos-scheduler`/nomos
|
||||
containers still run the pre-fix binaries; rebuilding and restarting them
|
||||
needs an explicit go-ahead since it touches the running homelab control
|
||||
plane.
|
||||
|
||||
Addresses five felt problems with the current system: (1) the agent reports
|
||||
stale machine state as if it were fresh, (2) sessions can't be opened and feel
|
||||
disconnected from chat, (3) the Nomos agent re-derives state every turn and
|
||||
wastes iterations, (4) the UI feels dead — tables with no context, no sense of
|
||||
what is monitored, (5) no way to inspect or customize entities and their checks.
|
||||
|
||||
The unifying UX principle for this plan: **master-detail with a detail
|
||||
sidebar**, not full-page navigation. Selecting an entity, session, or signal
|
||||
opens a right-hand detail panel over the current list, so the operator keeps
|
||||
context and drills in without losing their place. Full pages remain
|
||||
addressable (deep links) but are no longer the primary way to inspect a row.
|
||||
|
||||
Sequencing is driven by pain: **drift/staleness is Phase 1.**
|
||||
|
||||
---
|
||||
|
||||
## Root causes (verified in code)
|
||||
|
||||
### Drift / staleness — root cause was worse than a missing TTL
|
||||
Live-DB inspection (`oikos-postgres-1`) found the real cause: `check_defs` has
|
||||
two entity references — `entity_id` (the internal probe/"check" entity) and
|
||||
`target_id` (the host/service actually being observed). The scheduler wrote
|
||||
`entity_status`, `metric_samples`, and scheduler-sourced `events` keyed by
|
||||
`cd.EntityID` (the probe) instead of `cd.TargetID` (the target) —
|
||||
[scheduler.go:110-171](../internal/scheduler/scheduler.go) (pre-fix). Verified
|
||||
against the live database:
|
||||
|
||||
```
|
||||
entity_status by type: only type='check' rows ever had real health (24
|
||||
healthy, 1 down); every host/service/lxc/vm/proxmox-host was frozen at
|
||||
'unknown' since creation.
|
||||
metric_samples: 17,559 rows, 100% attached to type='check' entities — zero
|
||||
attached to any real host or service.
|
||||
events: 45 of 46 scheduler-sourced rows attached to type='check' entities.
|
||||
```
|
||||
|
||||
So this wasn't staleness in the TTL sense — the entities you actually care
|
||||
about (`host:hubris`, `service:authentik`, etc.) **never received an
|
||||
observation at all**. Every health check, metric, and event the scheduler
|
||||
produced was filed under an internal bookkeeping entity the UI doesn't even
|
||||
surface distinctly. This is the literal mechanism behind "the agent tells me
|
||||
stale/wrong state."
|
||||
|
||||
**Fixed** (this session): `runCheck`/`resolveSignal` now resolve
|
||||
`targetID := cd.TargetID` and write status/metrics/events there, falling back
|
||||
to the check's own id only if `target_id` is unset. Signals remain keyed by
|
||||
the check entity (unchanged, matches their existing resolution logic). A new
|
||||
migration ([016_fix_check_status_misattribution](../migrations/016_fix_check_status_misattribution.up.sql))
|
||||
deletes the orphaned check-entity `entity_status` rows so rollups stop
|
||||
double-counting probes as monitored entities; historical `metric_samples` on
|
||||
check entities are left as-is (time-series data, not safe to reattribute).
|
||||
|
||||
On top of the misattribution fix, a genuine staleness gap also existed and is
|
||||
now closed: `entity_status.health` was written only when a check ran, with no
|
||||
TTL — a stalled scheduler or disabled check_def would leave the last health
|
||||
value looking current forever.
|
||||
- `last_check_at` is recorded but was never surfaced. The Entities table
|
||||
showed `entity.updated_at` (row mutation time), not observation time
|
||||
([Entities.svelte:108](../web/src/pages/Entities.svelte), pre-fix).
|
||||
- The `/entities` list endpoint returned neither `health` nor `last_check_at`
|
||||
— only `/graph?include=status` and `/fleet/health` did
|
||||
([impl.go:344](../internal/httpapi/impl.go), pre-fix).
|
||||
|
||||
### Sessions
|
||||
- Clicking a session calls `loadSessionMessages()` but never navigates to the
|
||||
chat page ([Sessions.svelte:17](../web/src/pages/Sessions.svelte)); it mutates
|
||||
the chat store while the user stays on the session list, so nothing appears to
|
||||
happen. There is also no session switcher inside Chat.
|
||||
|
||||
### Agent efficiency
|
||||
- Multi-turn history replay **drops all `tool_use`/`tool_result` pairs**; only
|
||||
prior final text is replayed ([agent.go:108-127](../cmd/nomos/agent.go)). Each
|
||||
new turn re-discovers the fleet from scratch, re-calling tools already run.
|
||||
- Cold start: the system prompt injects no fleet snapshot
|
||||
([agent.go:81](../cmd/nomos/agent.go)); default model is
|
||||
`deepseek/deepseek-v4-flash` ([agent.go:34](../cmd/nomos/agent.go)); tool
|
||||
schema + system prompt are rebuilt each call with no prompt caching.
|
||||
|
||||
### Dead UI / no inspection
|
||||
- Entities table = slug/type/name/state/updated; no health, no last-seen, no
|
||||
signal count.
|
||||
- EntityDetail dumps `JSON.stringify(attributes)` raw
|
||||
([EntityDetail.svelte:123](../web/src/pages/EntityDetail.svelte)) and never
|
||||
shows the entity's `check_defs` — the operator cannot see *what is monitored*,
|
||||
when it last ran, or what it returned.
|
||||
- No CRUD anywhere: no entity editor, no check management (enable/disable/edit/
|
||||
run-now), no relationship editing. `check_defs` do not appear in the web app.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Kill the drift (highest priority)
|
||||
|
||||
Goal: the system never presents stale observations as fresh, and freshness is
|
||||
visible everywhere health is.
|
||||
|
||||
**Backend**
|
||||
- Add a staleness sweep to `housekeeping()`
|
||||
([scheduler.go:243](../internal/scheduler/scheduler.go)): for each
|
||||
`entity_status` where `now() - last_check_at > staleAfter` (default
|
||||
`max(3 × check interval, 5m)`), transition health to a new `stale` value and
|
||||
emit a `health.stale` event once (not every pass).
|
||||
- Treat `stale` as a first-class health in dashboard rollups
|
||||
([dashboard.go:57](../internal/httpapi/dashboard.go)) and fleet health
|
||||
([impl.go:516](../internal/httpapi/impl.go)) — do not fold it into `unknown`.
|
||||
- Extend the `/entities` list response with `health` and `last_check_at`
|
||||
(join `entity_status`), so the table can show freshness without N graph calls.
|
||||
- Nomos: when answering about state, tool results should carry `last_check_at`
|
||||
and a stale flag so the agent can hedge ("healthy as of 4m ago") instead of
|
||||
asserting stale data. (Verify the MCP topology/health tools include it.)
|
||||
|
||||
**Frontend**
|
||||
- Entities table: replace the `Updated` column with **health dot + relative
|
||||
"checked 2m ago"**, and add an **open-signal count** badge per row. Stale rows
|
||||
get a distinct muted/amber treatment, not a green dot.
|
||||
- Global header: add an "as of {time}" and make the SSE connection dot a real
|
||||
liveness indicator (last event received, reconnect state).
|
||||
|
||||
**Acceptance:** disable a check or stop the scheduler → within one stale window
|
||||
the affected entity shows `stale` in the table and dashboard, an event fires,
|
||||
and asking Nomos "is X healthy?" yields a freshness-qualified answer.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Detail sidebar + entity legibility (less navigation)
|
||||
|
||||
Goal: inspect any row in place; make an entity's monitoring self-evident.
|
||||
|
||||
- Introduce a reusable **DetailSheet** (right-side panel) used across Entities,
|
||||
Signals, Sessions, Executions. Row click opens the sheet; URL hash updates for
|
||||
deep-linking; Esc / click-away closes. Full `#/entity/:slug` page remains for
|
||||
direct links but reuses the same detail component.
|
||||
- Entity detail content (in the sheet):
|
||||
- Header: slug, type, **health + freshness** ("checked 2m ago" / "stale
|
||||
18m").
|
||||
- **Monitoring card**: the entity's `check_defs` — kind, schedule, enabled,
|
||||
last result + evidence, next run. This is the missing "what is watched."
|
||||
- Attributes rendered as a key/value panel, not raw JSON.
|
||||
- Relations, open signals, recent executions, metrics sparklines (reuse
|
||||
existing EntityDetail sections).
|
||||
- Backend: endpoint to list `check_defs` for an entity with last-result join
|
||||
(currently checks are only visible to the scheduler).
|
||||
|
||||
**Acceptance:** from the Entities list, one click reveals what an entity is,
|
||||
what's monitoring it, when it was last seen, and its open signals — without a
|
||||
full page load or losing the list.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Sessions rejoin chat
|
||||
|
||||
Goal: sessions are openable and live next to the conversation.
|
||||
|
||||
- Fix: clicking a session navigates to `#/chat` and loads it
|
||||
([Sessions.svelte:17](../web/src/pages/Sessions.svelte)).
|
||||
- Add a **session rail inside Chat** (collapsible left list: title, last-active,
|
||||
active highlight) so switching sessions never leaves the chat surface. The
|
||||
standalone Sessions page becomes a thin wrapper / can be retired from nav.
|
||||
- Show session metadata (message count, last actor) and allow rename/delete.
|
||||
|
||||
**Acceptance:** clicking any past session opens its transcript in the chat view;
|
||||
starting a new chat and switching back and forth works without navigation.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — Agent efficiency
|
||||
|
||||
Goal: stop re-deriving state; start each turn already oriented.
|
||||
|
||||
- Persist and replay tool evidence across turns
|
||||
([agent.go:108-127](../cmd/nomos/agent.go)): either replay `tool_use`/
|
||||
`tool_result` pairs with consistent ids, or persist a compacted per-turn
|
||||
"evidence summary" and replay that. Removes redundant re-querying.
|
||||
- Inject a compact fleet snapshot (counts by health, open signals, stale set)
|
||||
into the system prompt ([agent.go:81](../cmd/nomos/agent.go)) so the agent
|
||||
starts oriented instead of spending iterations on discovery.
|
||||
- Add prompt caching for the system prompt + tool schema (rebuilt every call
|
||||
today); revisit the default model
|
||||
([agent.go:34](../cmd/nomos/agent.go)) — evaluate a stronger default for
|
||||
fewer, better tool calls.
|
||||
- Surface per-turn iteration/token/cost in the chat UI (data already logged to
|
||||
`agent_activity`) so inefficiency is visible and measurable.
|
||||
|
||||
**Acceptance:** a 3-turn conversation about the same entity does not re-call the
|
||||
same read tools each turn; median iterations-per-answer drops.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — Customize & inspect (CRUD)
|
||||
|
||||
Goal: manage the system from the UI, not just observe it.
|
||||
|
||||
- Entity editor (attributes, state) via existing mutation endpoints.
|
||||
- Check management from the entity detail sheet: enable/disable, edit config/
|
||||
thresholds, and **run-now** (trigger a single check pass on demand — new
|
||||
scheduler entrypoint).
|
||||
- Relationship add/remove.
|
||||
- Raw DB-row view toggle in the detail sheet for power inspection.
|
||||
|
||||
---
|
||||
|
||||
## Suggested order of work
|
||||
|
||||
1. Phase 1 backend (staleness sweep + `/entities` health/freshness) →
|
||||
Phase 1 frontend (table freshness + liveness header).
|
||||
2. Phase 2 DetailSheet + entity monitoring card.
|
||||
3. Phase 3 sessions fix (small; can slot in earlier if desired).
|
||||
4. Phase 4 agent efficiency.
|
||||
5. Phase 5 CRUD.
|
||||
|
||||
Phases 1–3 are the ones that most directly turn "the system feels dead and I
|
||||
don't trust it" into "it's alive and I can see and act on it."
|
||||
Reference in New Issue
Block a user