jakemon is the observability plane of the Simple Agility stack: the one place where the fleet's metrics, logs, alerts, and externally-captured signals are collected and retained, and the data backbone every product's optimization loop joins against. This page is the contract; how jakemon is built and operated lives in JakemonHub. Unbuilt pieces named here are sequenced in SimpleAgilityRoadmap.
Today it is Prometheus + Loki + Grafana + Alertmanager on docker2, one Grafana Alloy agent per host pushing to them, two custom exporters (Cloudflare analytics, search visibility), and a nightly shipper that Hypertext Transfer Protocol (HTTP) POSTs retained search-visibility history into Wikantik. It is config, dashboards, Python-standard-library exporters and bash — there is no application code in jakemon and that is a rule, which is why the agent runtime is a separate stack-level component.
| Job | Examples today | Owner |
|---|---|---|
| 1. Fleet telemetry — host, container, database, edge and application metrics; log aggregation; alert rules and routing; dashboards | node/cAdvisor/postgres exporters, 33 alert rules → Telegram, 17 provisioned dashboards | plane |
| 2. External data capture and retention — polling third-party sources and keeping the history | Google Search Console (GSC), Bing and Yandex webmaster Application Programming Interfaces (APIs) → webmaster_* gauges + snapshot files; Cloudflare zone analytics; IndexNow submissions; the URL Inspection sweep | plane |
| 3a. Source-generic signal detection — anything derivable from a single external source with no product knowledge | visibility/opportunities.py (content_gap, decay, …), the measured Click-Through Rate (CTR)-by-position curve | plane, shipped as evidence |
| 3b. Domain decision-making — ranking, gating, calibration, backlog, actuation, effect measurement | Wikantik OpportunityEngine, traffic gate, WeightCalibrator, EffectEvaluator | product |
The rule that decides 3a from 3b: if computing it needs the product's own tables or vocabulary, it belongs in the product. Every product in the suite has its own optimization needs and joins its own data set against what the plane makes visible — whether that is gathered from external sources like search engines or constructed from the product's own logged errors and user behavior. As the stack evolves, opportunities.py-like logic shifts left into the products; the plane keeps the capture and the retention.
Read this before the "do not build this" table below, because the table is meaningless without it and reading the table alone will lead you to delete things you should keep.
Telemetry emitted so somebody can observe the system goes to the plane. Data that is a functional input to the product's own behavior — its loop, its ranking, a user-facing feature, a compliance obligation — is a product table.
The same event can legitimately produce both. A retrieval query is logged to stdout (plane: "how much traffic is the retriever taking, is it erroring") and written to retrieval_query_log (product: the agent_gap rule reads it to decide what content is missing). That is not duplication to be cleaned up; the two records answer different questions, have different retention needs, and have different owners.
Product tables that are correct and must not be "consolidated" into Loki: Wikantik's retrieval_query_log, briefing_log, content_change_log, search_visibility_snapshot, content_opportunity_seen, and the /admin/audit log. Each is a functional input or a compliance record, not observability.
The test to apply: if this table vanished, would the product behave differently, or would only a dashboard go blank? Behave differently → product table. Dashboard goes blank → it belonged in the plane.
Two mechanisms, chosen by a rule, not by taste.
If a product needs to join the data with its own tables, retain it longer than the plane's retention, or show it to its own users, the plane ships it and the product owns a table. Prometheus is a time-series store, not a join engine; Loki retention is the plane's decision, not the product's.
Every push feed has the same shape (the search-visibility feed, ship_visibility.py → POST /admin/insights/ingest → search_visibility_snapshot, is the reference implementation):
(as_of, natural key).--since window, so a new product instance can be loaded with history and a gap can be healed.as_of, source/engine and, where the plane computed something, enough to reproduce it (e.g. the CTR curve rows carry their as_of so a consumer can report ctrCurveSource: imported:<date> versus builtin).admin:insights only, verified 403 everywhere else), never a general admin credential. The credential lives in the plane's gitignored .env; without it the shipper idles rather than crash-loops.The ingest body is a JavaScript Object Notation (JSON) envelope: a feed name, a feed_version, and rows. Everything else is feed-specific.
{
"feed": "search_visibility",
"feed_version": 1,
"as_of": "2026-08-17",
"source": "jakemon",
"rows": [
{ "engine": "google", "site_host": "wiki.wikantik.com",
"page_path": "/wiki/HybridRetrieval", "query_text": "hybrid retrieval",
"impressions": 41, "clicks": 2, "position": 12.4 }
]
}
The envelope evolves like the log envelope, with one extra rule because feeds cross a version boundary between two independently-deployed repositories:
feed_version increments only on a breaking change — a removed field, a renamed key, a changed unit or meaning. Additive changes do not bump it.feed_version it does not know, and should say so in the error body rather than ingesting rows it will misread.skippedNonConformantCount.The stack is instanced per tenant, but one tenant may run several instances of one product (wiki-prod and wiki-support, say) against a single plane. That makes "which instance receives this feed?" a real question, and the answer is deliberately simple:
(instance name → base URL + scoped credential) entry in the plane's configuration. A new product instance is a new shipper target; nothing is inferred and nothing is broadcast.instance column. Each instance ingests into its own database; search_visibility_snapshot is keyed (snapshot_date, engine, site_host, page_path, query_text) and needs no more. Adding instance would be redundant and would invite a shared-store design nobody has agreed to.site_host is the web-property key, and it is what decides what a given instance is sent: a shipper target declares the hosts it is authoritative for. Two wikis covering different properties get different slices of the same capture.instance IS required on metrics and logs, because there the plane is genuinely shared — one Prometheus and one Loki hold every instance at once, and without the label two deployments' series are indistinguishable. This is the reverse of the feed case and the distinction is easy to get backwards.A product may not poll a third-party source to get around a missing feed. The process:
Known gap: there is no generic aggregate-and-ship mechanism today. ship_visibility.py is bespoke to search visibility, and a second feed (Roller's engagement data, for example) currently means writing a second shipper by hand. Generalizing it is a plane work item tracked in SimpleAgilityRoadmap; until it lands, budget for a bespoke shipper per feed and do not assume a feed can be conjured from an existing metric.
A one-off question ("error rate on /api/bundle since the deploy?", "is the alert still firing?") is a read-only Prometheus Query Language (PromQL) / Loki Query Language (LogQL) query against the plane's HTTP APIs. No table, no feed. Two hard limits:
Today pull is raw HTTP on the trusted Local Area Network (LAN); an agent-facing query surface on the plane (Model Context Protocol (MCP)) is planned — see SimpleAgilityOnCallAutomation.
The plane relies today on the default stores behind Prometheus, Loki and the visibility snapshot files. In scope for the plane's evolution — and therefore not for any product to build — is a more accessible, queryable Operational Data Store (ODS) carrying a deeper layer of Key Performance Indicators (KPIs) on Service Level Objectives (SLOs) and Service Level Agreements (SLAs): availability, latency and error budgets per service and per tenant, computed once by the plane and shipped or queried like any other feed. A product that needs an SLO number reads it from the plane; it does not compute one from its own request log.
Apply The rule that decides where data lives first. This table is about observability infrastructure, and every row assumes the data in question exists only so somebody can watch the system. An agent adding a capability to any Simple Agility product should stop if it finds itself writing any of the following:
| You are about to write… | Use instead | Not covered by this row |
|---|---|---|
A metrics table, a counter store, a /stats cache of request rates | /metrics scraped by the plane (telemetry contract); PromQL for reads | a counter the product acts on (rate-limit state, quota) |
| A log table, or an events table whose only purpose is "user did X" reporting | structured log lines with an event field → Loki; LogQL for reads | a table the product's own loop reads (retrieval_query_log), or a compliance audit log (/admin/audit) |
| A poller for Search Console / Bing / Yandex / Cloudflare | the plane's exporters + a push feed | — |
| Alert evaluation, paging, escalation timers | Prometheus rules + Alertmanager; the product exposes health and metrics only | in-product user notifications, which are a feature |
| An SLO / uptime / error-budget calculator | the plane's ODS/KPI layer (roadmap) | — |
| A cross-service trace store | correlation_id in the log envelope; the plane correlates | — |
| A per-instance dashboard | a provisioned Grafana dashboard in the jakemon repository | a product admin screen showing product state |
What does belong in the product: the tables that hold shipped feeds, the joins against product data, ranking/gating/calibration, the backlog, actuation surfaces, effect measurement, and any user-facing presentation of the result.