Content Intelligence Design

The content loop is open: the wiki-content agent writes pages, adjusts titles and summaries, and curates the Knowledge Graph — and nothing measures whether any of it worked. This design closes that loop.

It answers three questions in order: does anyone arrive? (acquisition), when someone asks, do we answer? (demand), and did our change help? (effect).

Status

Phases 0 and 1 are shipped (v2.4.4 and v2.4.5 on production). The fact store, ingest endpoint, snapshot shipper, ingest metrics, and demand-signal capture are live; 99,132 rows of real search history spanning 2026-06-04 to 2026-08-14 are loaded and queryable. Phase 2 — the loop itself — is in progress.

The full specification lives in the repository at docs/superpowers/specs/2026-08-16-content-intelligence-design.md.

The measured baseline

Design choices here follow from what the traffic actually is. Measured against production:

Reader-behaviour analytics therefore has no denominator, and the audience that exists today is machines. Acquisition and agent-side gaps come first; reader behaviour is gated behind a measured traffic threshold.

Engine asymmetry is the headline finding

Page-rollup rows for wiki.wikantik.com, most recent snapshot in the fact store:

EngineClicksImpressionsCTRAvg position
Google82,5180.32%36.0
Bing21081.85%4.7
Yandexno page dimension

Google wins on raw volume by accumulating 2,518 impressions. Bing converts roughly six times better per impression, from a first-page rank against Google's page four. Bing is by a wide margin the healthier channel; most of Google's volume is impressions nobody acts on.

Three months of trend from the same store:

MonthGoogleBing
2026-0670
2026-0782
2026-0882

Both engines are flat. That is a useful null result: nothing drifts on its own, so a deliberate change should be attributable rather than lost in noise.

Two properties of this wiki predict the asymmetry. Submission is already lopsided — the wiki pushes new and changed URLs via IndexNow, which Bing and Yandex consume and Google does not participate in. And Bing's index feeds the retrieval layer behind several AI answer surfaces, so for a wiki whose live consumers are agents, ranking in Bing is ranking in the layer those agents read.

A caution recorded deliberately. An earlier draft of this page read Google's clicks as zero, taken from a single snapshot. The trend shows that window was unrepresentative. Retained history is what turns an anecdote into a measurement — demonstrated on the very first question the fact store was asked.

The traffic gate — why most rules ship dormant

The fact store's first real job was to say whether its own rules could work. The answer, measured on 2026-08-17, was mostly no.

QuantityValue
Pages with any impressions479
Total impressions / 28 days2,626
Total clicks / 28 days10
Median page impressions2
p90 / p99 / max12 / 53 / 154
Pages clearing the specified 200-impression threshold0
Pages clearing the specified 100-impression threshold1

Three of the four native rules are driven by search volume, and at this level none can produce a meaningful per-page verdict. A page reported at "Bing position 2.0" was seen once.

Lowering thresholds until the backlog looks healthy does not fix that — it manufactures work out of sampling noise and hands it to an autonomous consumer, which is the primary risk the design names. So the three volume-driven rules now sit behind a site-level traffic gate and report themselves as suppressed, with the measured and required values, rather than returning a quietly empty list. Agent gap is not gated: its denominator is retrieval traffic, not impressions.

The search-driven half of this loop is not yet a working feedback signal. The agent-driven half is.

A grain correction worth recording

The rules were specified over a shared query set per page. That data does not exist in the store: the ingest payload carries per-page and per-query rows as two disjoint projections, so every query row arrives with no page attached. The collector does compute the cross product for its own gauges but never writes it to the snapshot files.

Three things depend on it and are dormant until it ships: the shared-query restriction on engine divergence, the search-vocabulary half of the vocabulary-gap rule, and effect measurement's query-intersection control. Each now degrades explicitly and records which mode it ran in, rather than silently comparing something weaker than advertised.

The opportunity backlog

Wikantik does not recompute what the collector already detects. Its rule engine imports those five detectors unchanged and adds four that need data the collector cannot see:

Every rule states a minimum support and stays silent below it. Silence is a valid output.

Effect measurement

Every content change records a 28-day baseline at write time and is evaluated 28 days later, with a difference-in-differences adjustment against the site's own trend.

Two details decide whether the numbers mean anything. Rows in the fact store are trailing 28-day aggregates stamped with the window's end date, so the after-window is the single snapshot nearest the change date plus 28 — summing across snapshot dates would count overlapping windows repeatedly and inflate the result several-fold. And a relative CTR threshold is undefined when the baseline had no clicks, which is the common case on a site drawing ten clicks a month, so that case resolves explicitly instead of dividing by zero.

This is a weak quasi-experiment and the design says so: one site, no control group, no randomisation. A single verdict is suggestive, not conclusive; the value is in aggregate across many changes of the same type. Insufficient data is an expected and honest verdict.

The ledger is also the mechanism by which the engine's guessed priority weights stop being guesses. Until a rule type accumulates twenty evaluated changes, its weight stays at the configured default and both read surfaces report it as uncalibrated.

Scope decision

Rules, joins, and read surfaces are built for the two Wikantik sites only. The fact store retains rows for every site the collector polls, because history cannot be created retroactively and the schema is site-agnostic by construction. Going deep on the stack with the richest metadata is what produces transferable knowledge about which rules actually work.

Implementation notes worth carrying forward

Defects surfaced during implementation that no unit test would have caught, all living in seams between components: