Search Visibility Exporter (webmaster_* metrics)

The visibility-exporter container in jakemon's central stack on inference polls each configured search-engine webmaster API for every tracked site and exposes the results as Prometheus gauges. It is stdlib-only Python living in jakemon/visibility/.

Sites and providers

visibility/sites.json maps each site to its provider identifiers — Google Search Console (sc-domain: properties), Bing, and Yandex. As of 2026-10-11 it tracks seven sites: wiki.wikantik.com, wikantik.com, jakefear.com, maiiavorobiova.com, thelocalwiki.com, berlin.thelocalwiki.com and sandiego.thelocalwiki.com. A site's optional origin field says where to fetch its sitemap when that differs from the property name: thelocalwiki.com lives at www.. Provider adapters live in visibility/providers/{google,bing,yandex}.py; IndexNow keys live in visibility/indexnow-keys/.

The data window: read this before interpreting numbers

Metrics are rolling-window snapshots, not counters: each poll covers a trailing 28-day lookback ending 2 days ago (search engines lag ~2–3 days). So a Grafana range of "90d" shows how the 28-day-window gauge evolved — it does not sum to 90 days of traffic. Site totals aggregate clicks/impressions with an impression-weighted average position.

Metric families

MetricLabelsMeaning
webmaster_clicks / webmaster_impressionssite, engineSite totals for the window
webmaster_avg_positionsite, engineImpression-weighted average position
webmaster_query_impressions / _clicks / _positionsite, engine, queryTop queries
webmaster_page_impressions / _clicks / _positionsite, engine, pageTop pages
webmaster_query_pageengine, query, pageGoogle query→page pairs (value = impressions)
webmaster_indexed_pagessite, engineIndexed pages. Bing and Yandex report their own counts. Google's comes from a rotating URL Inspection sweep, because the Sitemaps API's indexed field is a hardcoded 0. The number appears only after a full cycle, about 2.6 days for wiki.wikantik.com.
webmaster_url_coverage / webmaster_url_sweep_age_seconds / webmaster_url_sweep_upsite, stateURL Inspection coverage by state, the age of the oldest inspection result, and whether the sweep ran
webmaster_sitemap_submitted_pages / webmaster_sitemap_urlssite, engineSitemap size as submitted, and as discovered
seo_opportunity_score / seo_opportunities_count / seo_addressable_clickssite, type, querySource-generic SEO opportunities: striking distance, CTR gap, content gap, cannibalization, decay
webmaster_exporter_up / _last_success_timestamp_secondsengineProvider health

Querying it directly (the agent path)

Grafana needs auth; Prometheus on inference.lan:9090 does not. The fastest analysis loop is PromQL over the API, e.g. top pages for a site:

curl -s 'http://inference.lan:9090/api/v1/query' \
  --data-urlencode 'query=topk(25, webmaster_page_impressions{site="wiki.wikantik.com", engine="google"})'

The search-visibility Grafana dashboard (uid search-visibility, in dashboards/services/) renders these plus AI-referral log panels from Loki. The 2026-08-08 SEO pass on the wiki was driven entirely from these metrics — the workflow is documented from the consumption side in the wikantik SEO notes.

Alerts on outcomes, not just the pipeline

The seo-outcomes alert group (all severity warning) fires on what happened to the sites, not only on whether the exporter ran:

This group exists because, until 2026-10-01, only the pipeline was alerted on. When Google dropped wiki.wikantik.com after a 20-day outage, every panel showed it and nothing fired. Each alert links to its Incident<AlertName> runbook.

The feed to Wikantik

visibility-shipper (ship_visibility.py --loop) is the plane's reference push feed. Nightly, it POSTs the retained snapshots, the opportunities and the measured CTR-by-position curve to Wikantik's /admin/insights/ingest. It uses a dedicated jakemon-shipper account that holds only the admin:insights grant. The rules every feed follows are on SimpleAgilityObservabilityPlane. VisibilityShipperDown and VisibilityShipperStale alert when the feed goes quiet.

Supporting modules

history.py (snapshot history), opportunities.py (under-performing query/page detection), report.py (report rendering), actions.py (IndexNow submissions and similar) — all unit-tested beside the code (test_*.py).

See Also