Roadmap: Agentic Retrieval and Query Decomposition

Gap statement: single-pass bundle vs the managed multi-step retrieval loop

assemble_bundle is a single-pass pipeline: one query in, one fused, reranked, cited section set out. The managed platforms now demo agentic retrieval: decompose a complex question into sub-queries, retrieve per sub-query (possibly across sources), iterate, and evaluate sufficiency before returning. Wikantik's philosophical answer — "the agent drives the loop through MCP tools, informed by the coverage signal" — is real, but the project's own behavioral looping evaluation came back flat: client agents did not actually loop better given the signal. That makes server-side decomposition worth a measured attempt.

Market grounding

Current Wikantik state

Proposed direction

  1. Coverage-conditional decomposition (not always-on): if the first pass returns coverage=weak|partial, decompose the query into 2–4 sub-queries via the local LLM, run per-sub-query retrieval in parallel, fuse and dedupe into one bundle. Single-pass behavior preserved when coverage is strong — bounds latency and cost to the queries that need it. (Superseded by the Phase 1 measurement — see below: coverage does NOT discriminate the addressable class, so the built prototype triggers on query STRUCTURE, not coverage.)
  2. Fail-closed: any decomposition failure (Ollama down, malformed output, timeout) degrades silently to the single-pass result — same posture as the BM25 fallback.
  3. Sub-query provenance in the bundle: mark which sections each sub-query contributed, so client agents (and the eval harness) can see the decomposition working.

Phase 1 scope and acceptance criteria

First session scope: the multi-hop eval set ONLY — 15–25 multi-part/comparative questions with agreed gold sections, added to eval/bundle-corpus. No pipeline code. (The KG-rerank lesson: measure the ceiling before building. If single-pass retrieval already scores well on the multi-hop set, this brief may be closable without code.)

The decomposition feature is accepted only when all four hold:

  1. Multi-hop lift: recall@12 on the multi-hop set improves by a meaningful margin (target: ≥ +0.10 absolute) over single-pass.
  2. No single-intent regression: the existing eval set stays ≥ 0.74 — decomposition must never fire (or must be harmless) on well-covered queries.
  3. Fail-closed proven by test: Ollama down, malformed decomposition output, and timeout each degrade to the single-pass result — one test per failure mode, none returning an error to the caller.
  4. Latency bounded: p95 on decomposed (weak-coverage) queries stays within the envelope the owner sets ([OWNER] — a product parameter; propose ≤ 2 s against the 1.25 s single-pass baseline), with strong-coverage queries unchanged.

Phase 1 measure-first finding (2026-07-11, dense-chunk measurement complete)

The measure-first step ran without building pipeline code (the LLM-on-query-path decision stays [OWNER]-gated). The [MEASURE] gate — run the full RELATIONAL set through the dense-chunk bundle — is now complete (a working local qwen3-embedding:0.6b embedder was stood up to unblock the dense path; see eval/bundle-corpus/baseline-notes.md).

The multi-hop eval set already exists. eval/bundle-corpus/queries.csv has a RELATIONAL category = 9 multi-part/comparative questions (r01–r10), most with 2 gold sections (two "hops") — the "author a multi-hop eval set" first-session scope is largely already met; extend rather than author from scratch.

Rigorous single-pass result (n=9, dense-chunk path, per-hop). Each RELATIONAL question ran through the dense-chunk bundle (real cosines), scoring each of its 2 gold sections in top-12:

Trigger insight (invalidates coverage-conditional). All nine RELATIONAL queries return strong coverage (topSim 0.65–0.86) — including every cross-page miss, because one side matches strongly and pins topSimilarity high while the other hop is crowded out. So the roadmap's coverage-conditional trigger would never fire for the addressable class. The built prototype therefore triggers on query structure (a lexical comparative/conjunctive heuristic gating the LLM planner), not coverage.

Projected ceiling (pre-build). If decomposition recovered all four cross-page comparative questions, full multi-hop success rises 3/9 → 7/9 (78%) — the addressable ceiling. The two intra-page misses are out of scope for decomposition.

Phase 2 build + acceptance A/B (2026-07-11) — REJECTED under BOTH fusion strategies

The scoped prototype was built and measured (structure-conditional; lexical heuristic → fail-closed LlmQueryPlanner gemma4-assist:latest think:false → per-sub-query retrieval → fusion → existing rerank/cut; default-off, byte-identical off, fail-closed; 6 SDD tasks + final-review fix, all reviewed clean, on main). Two fusion strategies were built and A/B'd over the 9 RELATIONAL questions (local qwen3-embedding dense + remote gemma4-assist planner, warm):

Armhop-recall@12full multi-hop (both hops)
Control (decomposition off)0.6113/9
Treatment — N-ary RRF fusion0.5002/9
Treatment — round-robin fusion0.5562/9

Verdict: REJECTED — decomposition ships behind a flag, default OFF. Neither fusion clears the gate; both are net-negative vs single-pass. RRF is worse because it score-sums 1/(k+rank) across sub-queries, so a topic co-mentioned by several sub-queries outranks a gold section present in only one list — amplifying the majority side. Round-robin interleave (reserves per-sub-query position) fixes that amplification and recovers r07, but is still below control.

The cost is structural, not a fusion-tuning problem (this is the load-bearing finding):

  1. The bundle is always full (12 sections), so surfacing a minority-side section MUST displace an existing one — and the displaced section is sometimes a co-located gold. r05 (both hops on one page) regressed 2/2→1/2 under both fusions. A "backfill/union" fusion that never displaces would be a no-op on a full bundle (it can't lift), so there is no free lunch.
  2. The planner doesn't reliably retrieve the specific minority-side gold. r08 stayed 1/2 under both — the blue-green gold ("When canary wins") was retrieved by no sub-query, so no fusion can surface it.

Both fusion modes ship behind the default-off flag (wikantik.bundle.decomposition.fusion = rrf | roundrobin) as measured, documented dead-ends. Full write-up: eval/bundle-corpus/baseline-notes.md.

Honest prerequisite before more tuning: the remaining levers (a planner prompt that forces one sub-query per named comparison entity; a trigger that skips single-page conjunctive questions) are ≥2 coupled knobs — tuning them against n=9 risks overfitting. Do the [AGENT] corpus-expansion step (15–25 multi-hop questions) FIRST, then tune against a set large enough to trust.

Verify before designing on these

Non-goals

Investigation starting points

See Also