Search Index and Knowledge Graph Rebuild

This page documents the operator runbook for rebuilding every derived index Wikantik maintains over its page corpus. Use it when:

The rebuild is implemented as a four-phase pipeline orchestrated by bin/kg-rebuild.sh. Each phase is independently skippable, so a typical experiment iteration only re-runs the cheap phases — you don't re-chunk every time you swap an extractor model.

bin/kg-rebuild.sh and bin/kg-extract.sh are local-development tools. They log in with the testbot admin account from test.properties, read the database credentials from the local tomcat/tomcat-11/conf/Catalina/localhost/ROOT.xml, connect with psql to localhost, and need curl, jq, java and psql on PATH plus a running Tomcat (TOMCAT_URL, default http://localhost:8080). Against a remote host, call the underlying admin endpoints yourself (POST /admin/content/rebuild-indexes, POST /admin/content/reindex-embeddings, POST /admin/knowledge-graph/extract-mentions). For a Lucene-only rebuild see bin/trigger-rebuild-indexes.sh and docs/admin/IndexRebuild.md in the repository; the operator handbook is Wikantik Operations.

What gets rebuilt

The pipeline touches four derived data layers, in order:

  1. Chunks (kg_content_chunks) and Lucene full-text index — wiped and rebuilt from page markdown on disk by ContentIndexRebuildService.
  2. Chunk embeddings (content_chunk_embeddings) — wiped (via ON DELETE CASCADE from phase 1) and repopulated by BootstrapEmbeddingIndexer against the configured embedder model.
  3. Knowledge graph (kg_nodes, kg_edges, kg_proposals) — optional, opt-in only. Removes pending proposals and machine-tier AI-inferred nodes; keeps everything a person wrote or approved (tier human). Skipped by default.
  4. Proposals and entity mentions (kg_proposals, chunk_entity_mentions) — repopulated by bin/kg-extract.sh, which runs the per-page entity-extraction pipeline: one LLM extraction call per page, consolidated into proposals, then mention attribution for accepted names that already exist as Knowledge Graph nodes.

Source-of-truth markdown in docs/wikantik-pages/ is never touched. Page metadata (frontmatter, canonical IDs, structural spine) is preserved.

Cascade behaviour worth knowing

Because chunk_entity_mentions.chunk_id and content_chunk_embeddings.chunk_id both reference kg_content_chunks(id) with ON DELETE CASCADE, phase 1 implicitly wipes both downstream tables. Phase 2 fully repopulates embeddings, and phase 4 re-attributes mentions, so the cascade is a feature: you can't end up with stale rows pointing at chunks that no longer exist. (If you run phase 1 with --skip-extract, mentions stay empty until a later phase 4 run.)

Sample commands

A starter set covering the workflows you'll use most. Anything after -- (or any unrecognised arg) is forwarded verbatim to bin/kg-extract.sh; an argument that script does not know makes it exit with unknown argument (see the extractor CLI flag reference).

Plan and preview before committing

# Print the four-phase plan, exit 0. Touches nothing.
bin/kg-rebuild.sh --dry-run -- \
    --ollama-model qwen2.5:1.5b-instruct --concurrency 4

# Walk every page on disk, run the chunker in memory at the given config,
# print the size distribution. A couple of seconds. No DB, no Tomcat, no LLM.
bin/kg-chunker-stats.sh --chunker-merge-forward-tokens 300

End-to-end rebuilds

# Vanilla full rebuild using whatever's in wikantik-custom.properties
# for chunker and embedder, and the extractor CLI's own defaults.
# Useful as a "reset to defaults".
bin/kg-rebuild.sh

# Experiment iteration: small fast model, concurrency 6.
bin/kg-rebuild.sh -- \
    --ollama-model qwen2.5:1.5b-instruct \
    --concurrency 6

# Tabula rasa: also wipes pending proposals and machine-tier ai-inferred KG nodes
# before re-extracting. Use when comparing two models on the same pages
# and you want a guaranteed clean baseline.
bin/kg-rebuild.sh --reset-kg -- \
    --ollama-model qwen2.5:1.5b-instruct \
    --concurrency 6

# Nuclear option: PURGE every KG and hub table including human-authored
# content. Prompts you to type "PURGE" exactly to confirm. Use when
# starting fresh for a schema-design experiment or comparing extractor
# models without ANY prior bias.
bin/kg-rebuild.sh --purge-kg -- \
    --ollama-model qwen2.5:1.5b-instruct \
    --concurrency 6

Inner-loop iteration (after the first full rebuild)

# Just re-extract: the chunks and embeddings from the previous run are
# still good. Cheapest way to A/B two extractor models.
bin/kg-rebuild.sh --skip-chunks --skip-embeddings -- \
    --ollama-model qwen2.5:1.5b-instruct \
    --concurrency 6

# Same, but with a clean KG between trials so mentions/proposals from
# the prior run don't muddy the comparison.
bin/kg-rebuild.sh --reset-kg --skip-chunks --skip-embeddings -- \
    --ollama-model qwen3:8b \
    --concurrency 4

Smoke tests

# 5 pages end-to-end. Use after any structural change to confirm the
# whole pipeline still wires up before committing to a long run.
bin/kg-rebuild.sh --skip-chunks --skip-embeddings -- \
    --ollama-model qwen2.5:1.5b-instruct \
    --concurrency 4 \
    --max-pages 5

# Same but with a dry-run extraction: pages are extracted and consolidated,
# but nothing is upserted into kg_proposals.
bin/kg-rebuild.sh --skip-chunks --skip-embeddings -- \
    --ollama-model qwen2.5:1.5b-instruct \
    --concurrency 4 --dry-run \
    --max-pages 5

Targeted rebuilds

# I changed wikantik.chunker.* and restarted Tomcat — re-chunk and
# re-embed but defer extraction until I've checked chunk shape.
bin/kg-rebuild.sh --skip-extract

# I changed the embedding model — re-embed only. Chunks stay; extraction
# does not depend on vectors, so it is unaffected.
bin/kg-rebuild.sh --skip-chunks --skip-extract

# I want to wipe the KG without re-extracting, e.g. before manual
# curation work. Phases 1, 2, 4 skipped; phase 3 runs alone.
bin/kg-rebuild.sh --reset-kg --skip-chunks --skip-embeddings --skip-extract

Backwards-compatible (extractor only, no orchestration)

bin/kg-extract.sh is still callable directly when you don't need the chunker/embedder rebuild — for example, to extend an extraction run to pages that have no proposals yet.

# Run with the CLI defaults
bin/kg-extract.sh

# Override at the command line, no Tomcat dance needed; only pages with no
# kg_proposals rows yet
bin/kg-extract.sh --ollama-model qwen2.5:1.5b-instruct \
    --concurrency 6 --only-missing

The four phases in detail

Phase 1 — Chunk and Lucene rebuild

Triggered by POST /admin/content/rebuild-indexes. Polls GET /admin/content/index-status for the rebuild.state field, returning to IDLE when complete.

What it does:

Typical wall time: ~1-2 minutes for ~1000 pages.

To change chunker behaviour: edit tomcat/tomcat-11/lib/wikantik-custom.properties, restart Tomcat, then run phase 1. The rebuild service uses whatever chunker was constructed at engine startup.

Chunker tuning notes

The chunker has four tuning knobs (max_tokens, merge_forward_tokens, fragment_floor_tokens, overlap_tokens) plus a wikantik.chunker.enabled kill-switch for save-time chunking, and they're not equally useful in practice. fragment_floor_tokens merges a sub-floor section forward so it adopts the next section's heading, and overlap_tokens prepends each chunk's tail to the next chunk under the same heading; changing any of them invalidates chunk content, so run a full rebuild afterwards. The sweep below is an April 2026 measurement of merge_forward_tokens against the 979-page corpus of that time (measured with the chunker-stats tool, now bin/kg-chunker-stats.sh); the two newer knobs shift the absolute numbers:

merge_forward_tokensTotal chunksMean tokensp90
150 (default)21,406180276
25018,843204338
35017,535219416

max_tokens (the ceiling) barely moves the needle on this corpus — sweeping 512 → 1024 → 2048 gives nearly identical numbers because no prose blocks are large enough to bump the ceiling. The lever that works is merge_forward_tokens: each +100 of merge floor cuts chunk count by ~12%, which directly translates to ~12% fewer chunks to embed in phase 2.

Atomic blocks bypass the size knobs. Fenced code, lists, and tables emit as their own chunks regardless of size — even a 30-token code fence becomes its own chunk. That's why the save-time extraction prefilter has a pure_code rule: it skips the small atomic chunks the chunker can't merge away.

Use bin/kg-chunker-stats.sh --chunker-merge-forward-tokens N to sweep candidate values in seconds before committing to a phase 1 rebuild.

Phase 2 — Chunk embedding reindex

Triggered by POST /admin/content/reindex-embeddings. Polls the same status endpoint for embeddings.bootstrap.state. Terminal states: COMPLETED, SKIPPED_ALREADY_POPULATED, SKIPPED_NO_CHUNKS (treated as success), FAILED, DISABLED (treated as fatal).

What it does:

Duration depends almost entirely on the embedder host: minutes against a local GPU, much longer on a CPU-only embedder (batch size and timeout are coupled; see the embedding backfill section of docs/admin/IndexRebuild.md).

If hybrid retrieval is disabled (wikantik.search.hybrid.enabled=false), the script logs a warning and skips this phase rather than failing. That makes it safe to use the same script in a deploy where hybrid is intentionally off.

Phase 3 — Knowledge graph reset or purge (opt-in)

Skipped by default. Enable with either --reset-kg (selective prune) or --purge-kg (full destructive wipe). The two are mutually exclusive — if both are passed, purge wins and the script logs the override.

--reset-kg — selective prune

Prompts for [y/N] confirmation unless --yes is also passed.

What it does: runs bin/db/kg-reset-machine-tier.sql as one transaction (psql --single-transaction, so both statements apply together or not at all):

DELETE FROM kg_proposals
 WHERE status = 'pending';

DELETE FROM kg_nodes n
 WHERE n.provenance = 'ai-inferred'
   AND n.tier = 'machine'
   AND NOT EXISTS ( SELECT 1
                      FROM kg_edges e
                     WHERE e.tier = 'human'
                       AND ( e.source_id = n.id OR e.target_id = n.id ) );

Tier, not provenance, says who vouches for a row. kg_nodes.provenance records where a row came from (human-authored, human-curated, ai-inferred, ai-reviewed); kg_nodes.tier is machine or human. When a person approves a proposal its nodes and edges move to tier human, but they keep provenance ai-inferred, so the reset selects on both:

DeletedKept
every pending proposal (and its machine reviews)approved and rejected proposals (human verdicts)
nodes with provenance ai-inferred and tier machine, with the machine-tier edges, chunk mentions and node embeddings that cascade from themevery tier human node — human-authored, human-curated and human-approved (ai-inferred provenance, tier human) — and every tier human edge
a machine-tier ai-inferred node that is an endpoint of a tier human edge (deleting it would cascade-delete that human-approved edge)

The prompt, the "Before"/"After" counts (tier, provenance, COUNT(*)) and the SQL all use the same rule. Run SELECT tier, provenance, COUNT(*) FROM kg_nodes GROUP BY 1, 2 first to see exactly what a reset would remove: the machine / ai-inferred row is the deletable set. Releases up to 2.4.56 selected on provenance alone and so also deleted human-approved nodes (and their edges); --purge-kg is unchanged and still wipes everything.

Re-extraction after a reset (phase 4) re-proposes the deleted entities. Approving a new-node proposal is first-writer-wins: it creates the node only when no node of that name exists, and leaves an existing node (including every row the reset kept) completely unchanged, logging the skipped write at INFO.

Use the reset variant when you want to compare two extraction runs against a clean automation baseline while keeping your manual curation work intact.

--purge-kg — full destructive wipe

Prompts you to type PURGE (uppercase, exact) before proceeding. --yes bypasses the prompt for scripted workflows; use carefully.

What it does — single TRUNCATE across the entire KG layer:

TRUNCATE TABLE
    kg_nodes, kg_edges,
    kg_proposals, kg_rejections,
    chunk_entity_mentions,
    hub_centroids, hub_proposals, hub_discovery_proposals
RESTART IDENTITY CASCADE;

This destroys human-authored content too. Use it when you genuinely want a from-scratch KG — fresh schema design experiments, comparing extractor models without any prior bias from earlier runs, or recovering from a contaminated graph state where it's faster to start over than to clean up.

Both reset and purge print row counts before and after so you can see exactly what got removed.

What survives a --purge-kg:

After a purge, phase 4 will re-populate kg_nodes, kg_proposals, and chunk_entity_mentions from extraction. Hub clustering (hub_centroids) needs to be rebuilt by its subsystem separately — typically the next page-save cycle or an explicit hub-discovery run.

Phase 4 — Entity extraction

Forwards to bin/kg-extract.sh with whatever args you passed to bin/kg-rebuild.sh. kg-extract.sh is a thin launcher for the standalone wikantik-extract-cli jar (built with Maven on demand when the jar is missing or stale). It takes the JDBC URL, user and password from the local ROOT.xml (falling back to PG_JDBC_URL, PG_USER and PG_PASSWORD) and runs the same per-page pipeline as the admin batch POST /admin/knowledge-graph/extract-mentions:

  1. Warm the kg_node_embeddings cache.
  2. For each page (concurrently across a worker pool), stitch the body from its chunks, look up a top-K dictionary of existing nodes, and make one LLM extraction call.
  3. Consolidate the per-page results into one proposal per logical claim and, if a judge is enabled (--judge, default none), review each one.
  4. Upsert the accepted proposals into kg_proposals (skipped with --dry-run).
  5. For every accepted name that already maps to a kg_nodes row, attribute it to the chunks that mention it and write chunk_entity_mentions.

This is the only phase that takes hours rather than minutes (depending on model size, concurrency, and host load). See bin/kg-extract.sh --help for the launcher's header and bin/kg-extract.sh --jar-help for the CLI's complete flag list (reproduced in the extractor CLI flag reference below). The most common knobs:

FlagPurpose
--ollama-model <tag>Extractor model (default gemma4-assist:latest)
--concurrency <N>1-6, default 2 (the CLI clamps at 6; the admin batch clamps at 10). Raise for small models on a fast GPU
--max-pages <N>Cap to the first N pages — for smoke tests
--page-pattern <glob>Limit to page names matching a glob (* and ?)
--only-missingSkip pages that already have kg_proposals rows
--dry-runRun extraction but skip the kg_proposals upsert
--judge none\|ollama\|claudeOpt-in proposal review (default none: accept everything verbatim)
--report <path>Write the final status as JSON when the run completes

The save-time prefilter (not used by this phase)

Earlier, chunk-level extraction ran through a chunk prefilter controlled by --prefilter, --force, --stats-only and --chunker-stats-only on kg-extract.sh. Those flags are gone: kg-extract.sh rejects them as unknown arguments, and the per-page pipeline does not read the prefilter. The wikantik.knowledge.extractor.prefilter.* properties still exist, but only the save-time chunk extractor (the listener that runs when a page is saved) consults them. Three predicates, evaluated in this order so the most-specific reason for skipping is the one reported:

  1. pure_code — chunk text is a single fenced code block with no surrounding prose.
  2. no_proper_noun — chunk text contains no token matching \b\w*[A-Z]\w{2,}\b (the leading \w* lets mixed-case names like iPad, gRPC, eBay match; pure-lowercase identifiers like kubectl and short caps like Of do not).
  3. too_short — chunk's estimated token count (chars / 4) is below wikantik.knowledge.extractor.prefilter.min_tokens (default 20).

It is off by default (wikantik.knowledge.extractor.prefilter.enabled = false); prefilter.dry_run logs what it would skip without skipping.

Model selection and lineage

The extractor column on chunk_entity_mentions records the extractor's code. For the per-page Ollama extractor that is ollama: plus the configured model tag with :latest stripped (for example ollama:gemma4-assist, ollama:qwen2.5:1.5b-instruct); the save-time chunk extractor writes the bare model tag instead (for example gemma4-graph:12b). So you can A/B compare two models against overlapping page sets and tell their mentions apart at SQL inspection time. See Inspection queries below.

Pre-existing rows tagged with the literal 'ollama' (from before the lineage change) are left as-is. To backfill them to a known historical model, run a one-shot UPDATE — but per project convention, that goes in psql by hand, not in a Vxxx migration.

Non-destructive stats mode

bin/kg-chunker-stats.sh (split out of kg-extract.sh when extraction moved to the per-page pipeline) walks data and reports without writing anything. It exits 0 on completion. Use it in tight tuning loops where a real run would be wasteful.

It reads markdown directly from --pages-dir (default docs/wikantik-pages), runs ContentChunker in memory at the supplied overrides, and prints the chunk count, the tokens-per-chunk distribution (min, mean, p50, p90, p99, max), a size histogram, and what the save-time prefilter would skip among the re-chunked chunks. It does not need Tomcat, the database or an LLM.

# Default config
bin/kg-chunker-stats.sh

# Sweep merge_forward_tokens to find the right floor
bin/kg-chunker-stats.sh --chunker-merge-forward-tokens 300
bin/kg-chunker-stats.sh --chunker-merge-forward-tokens 500

Options: --pages-dir, --chunker-max-tokens, --chunker-merge-forward-tokens (must be <= max tokens), --chunker-fragment-floor-tokens (must be <= merge-forward tokens) and --chunker-overlap-tokens. Exit codes: 0 stats printed, 1 bad --pages-dir, 2 bad arguments. The run takes a couple of seconds for a thousand pages.

Skipping phases

Each phase has a --skip-* flag, useful when you've already done the expensive work and want to iterate on a downstream phase:

# I just changed the extractor model — re-extract only, keep chunks/embeddings.
bin/kg-rebuild.sh --skip-chunks --skip-embeddings -- \
    --ollama-model qwen2.5:1.5b-instruct --concurrency 6

# I want to inspect chunks first; defer extraction.
bin/kg-rebuild.sh --skip-extract

# I'm testing a new chunker config; embeddings still match for vector quality
# but the per-page chunk count will change. Re-do chunks + embeddings, defer
# extraction until I'm happy with chunk shape.
bin/kg-rebuild.sh --skip-extract

Combined with --dry-run, the planner output makes it easy to confirm what will happen before committing to a long run.

Experiment workflow

A typical tuning loop looks like this:

  1. Survey the chunker without writing anything:

    bin/kg-chunker-stats.sh --chunker-merge-forward-tokens 300

    Reads markdown from disk, runs the chunker in memory, prints the size distribution. Takes a couple of seconds. Does not require Tomcat or a DB.

  2. Decide on a chunker config, edit wikantik-custom.properties, restart Tomcat, then commit to phase 1 + phase 2:

    bin/kg-rebuild.sh --skip-extract

  3. Sweep extractor models by re-running phase 4 only:

    bin/kg-rebuild.sh --skip-chunks --skip-embeddings --reset-kg -- \
    --ollama-model qwen2.5:1.5b-instruct \
    --concurrency 6 --max-pages 20

    --reset-kg between trials gives you a clean slate; --max-pages 20 keeps each trial short.

  4. When satisfied, drop --max-pages and run for real.

Inspection queries

After a run, these psql snippets confirm the pipeline produced sane data. Pull credentials the same way the rebuild script does (the database and role names below are the defaults; use the ones in your ROOT.xml):

PGPASS=$(grep -oE 'password="[^"]+"' tomcat/tomcat-11/conf/Catalina/localhost/ROOT.xml \
         | head -1 | sed -E 's/password="([^"]+)"/\1/')
PSQL="env PGPASSWORD=$PGPASS psql -h localhost -U wikantik wikantik"

-- 1. Chunks per page (post-phase-1 sanity)
SELECT page_name, COUNT(*) FROM kg_content_chunks GROUP BY page_name
 ORDER BY COUNT(*) DESC LIMIT 20;

-- 2. Embedding coverage (post-phase-2 sanity)
SELECT COUNT(*) AS chunks, (SELECT COUNT(*) FROM content_chunk_embeddings) AS embeddings
  FROM kg_content_chunks;

-- 3. Mentions per extractor (post-phase-4: distinguishes models because
--    the extractor code carries the model tag: ollama:<model> for the per-page pipeline)
SELECT extractor, COUNT(*) AS n_mentions FROM chunk_entity_mentions
 GROUP BY extractor ORDER BY n_mentions DESC;

-- 3a. Model-by-model A/B comparison: how many distinct nodes did each
--     model touch on the same chunks? (Higher recall ≠ better — could
--     just mean the model is more verbose.)
SELECT extractor, COUNT(DISTINCT chunk_id) AS chunks_with_mentions,
       COUNT(DISTINCT node_id) AS distinct_nodes, COUNT(*) AS total_mentions
  FROM chunk_entity_mentions
 GROUP BY extractor ORDER BY extractor;

-- 3b. Mentions emitted by one model but not another for the same chunk —
--     useful for spot-checking quality differences. Replace the literals
--     with two model tags actually present in your data.
SELECT chunk_id,
       array_agg(DISTINCT extractor ORDER BY extractor) AS extractors,
       COUNT(DISTINCT node_id) FILTER (WHERE extractor='gemma4-assist') AS gemma_nodes,
       COUNT(DISTINCT node_id) FILTER (WHERE extractor='qwen2.5:1.5b-instruct') AS qwen_nodes
  FROM chunk_entity_mentions
 WHERE extractor IN ('gemma4-assist','qwen2.5:1.5b-instruct')
 GROUP BY chunk_id
HAVING COUNT(DISTINCT extractor) > 1
   AND COUNT(DISTINCT node_id) FILTER (WHERE extractor='gemma4-assist')
       <> COUNT(DISTINCT node_id) FILTER (WHERE extractor='qwen2.5:1.5b-instruct')
 ORDER BY ABS(COUNT(DISTINCT node_id) FILTER (WHERE extractor='gemma4-assist')
            - COUNT(DISTINCT node_id) FILTER (WHERE extractor='qwen2.5:1.5b-instruct')) DESC
 LIMIT 20;

-- 3c. Extraction lineage timeline — when did each model get used?
SELECT extractor, MIN(extracted_at) AS first_seen, MAX(extracted_at) AS last_seen,
       COUNT(*) AS mentions
  FROM chunk_entity_mentions GROUP BY extractor ORDER BY first_seen;

-- 4. Proposals queue depth and lineage
SELECT proposal_type, status, COUNT(*) FROM kg_proposals
 GROUP BY proposal_type, status ORDER BY 1, 2;

-- 5. Pages with zero mentions (no accepted name mapped to an existing node)
SELECT cc.page_name, COUNT(cc.id) AS chunks,
       COUNT(m.chunk_id) AS mentions
  FROM kg_content_chunks cc
  LEFT JOIN chunk_entity_mentions m ON m.chunk_id = cc.id
 GROUP BY cc.page_name HAVING COUNT(m.chunk_id) = 0
 ORDER BY 2 DESC LIMIT 20;

Configuration reference

Properties read at engine startup, set in tomcat/tomcat-11/lib/wikantik-custom.properties. Restart Tomcat after editing. Defaults are those of wikantik-main/src/main/resources/ini/wikantik.properties (the generated Wikantik Configuration Reference is authoritative).

PropertyDefaultAffects
Chunker
wikantik.chunker.max_tokens512Phase 1: hard ceiling on a non-atomic chunk
wikantik.chunker.merge_forward_tokens150Phase 1: floor below which a chunk merges into the next section
wikantik.chunker.fragment_floor_tokens24Phase 1: a whole section below this merges forward and adopts the next heading (must be <= merge_forward_tokens; 0 disables)
wikantik.chunker.overlap_tokens40Phase 1: tail of each chunk is prepended to the next chunk under the same heading (0 disables)
wikantik.chunker.enabledtrueKill-switch for save-time chunking
Embedding
wikantik.search.hybrid.enabledtruePhase 2: false skips embedding reindex with a warning
wikantik.search.embedding.modelqwen3-embedding-0.6bPhase 2: embedder model (the dense index is fixed at its dimension)
Extractor backend (server side: save-time extractor and the admin batch)
wikantik.knowledge.extractor.backendollamaollama / claude / disabled
wikantik.knowledge.extractor.ollama.modelgemma4-graph:12bOllama model tag (:latest is stripped before being recorded in the extractor column); bin/kg-extract.sh does not read it and defaults to gemma4-assist:latest
wikantik.knowledge.extractor.ollama.base_urlhttp://inference.jakefear.com:11434Ollama HTTP endpoint (the bundled CPU embedder in the container image has no chat model)
wikantik.knowledge.extractor.claude.modelclaude-haiku-4-5Anthropic model id when backend=claude
wikantik.knowledge.extractor.timeout_ms120000Per-call LLM timeout
wikantik.knowledge.extractor.confidence_threshold0.6Proposals below this are dropped, not filed
wikantik.knowledge.extractor.max_existing_nodes200Cap on the existing-nodes dictionary included in the prompt
wikantik.knowledge.extractor.concurrency2In-flight LLM calls for the admin batch. Hard-clamped to [1, 10]
wikantik.knowledge.extractor.per_page_min_interval_ms5000Save-time path only: rate limit between extractions of the same page on rapid edits
Save-time prefilter (the batch does not use it)
wikantik.knowledge.extractor.prefilter.enabledfalseMaster switch for the chunk prefilter
wikantik.knowledge.extractor.prefilter.dry_runfalseLog decisions only — no chunks actually skipped
wikantik.knowledge.extractor.prefilter.skip_pure_codetrueEnable the pure-code-block predicate
wikantik.knowledge.extractor.prefilter.skip_no_proper_nountrueEnable the proper-noun-absence predicate
wikantik.knowledge.extractor.prefilter.skip_too_shorttrueEnable the too-short predicate
wikantik.knowledge.extractor.prefilter.min_tokens20Too-short threshold (chars/4 estimate)

CLI flags on bin/kg-extract.sh override these settings for a single run, which is what makes the experiment loop fast — you don't have to restart Tomcat between trials of model changes.

Script options reference

FlagEffect
--reset-kgEnable phase 3 in selective-prune mode — deletes pending proposals and machine-tier ai-inferred nodes (bin/db/kg-reset-machine-tier.sql); every tier human row, including human-approved nodes, is kept. [y/N] prompt unless --yes.
--purge-kgEnable phase 3 in full-purge mode — TRUNCATEs every kg_/hub_ table including human-authored content. Requires typing PURGE exactly to confirm; --yes bypasses. Wins over --reset-kg if both passed.
--skip-chunksSkip phase 1
--skip-embeddingsSkip phase 2
--skip-extractSkip phase 4
--dry-runPrint the plan, exit 0
--yes / -ySkip the --reset-kg [y/N] prompt and the --purge-kg "type PURGE" prompt. Use only in scripted contexts you trust.
--help / -hPrint this script's header comment and exit
--Everything after this is forwarded to bin/kg-extract.sh (so every forwarded flag must be one the CLI knows)

Environment variables:

VariableDefaultPurpose
TOMCAT_URLhttp://localhost:8080Where the script POSTs the rebuild triggers
POLL_SECONDS10Status-poll cadence during phases 1 and 2
PROGRESS_SECONDS30How often a periodic progress line (count + rate + ETA) is emitted while a phase is in RUNNING state. Decoupled from POLL_SECONDS so state transitions still surface promptly.

Periodic progress feedback

While phases 1 and 2 are in their RUNNING states, the script emits a periodic progress line every PROGRESS_SECONDS (30 by default) so a long-running phase doesn't go silent between state transitions:

[kg-rebuild] embedding reindex: state=RUNNING (chunks 0 / 21367)
[kg-rebuild] embedding reindex: 1247 / 21367 (5.8%) — rate=41 chunks/s, ETA 0h08m
[kg-rebuild] embedding reindex: 2533 / 21367 (11.9%) — rate=42 chunks/s, ETA 0h07m
…
[kg-rebuild] embedding reindex: state=COMPLETED (chunks 21367 / 21367)

The rate / ETA tail is suppressed early in a run (when no work has happened yet) and at completion (when there's nothing left to estimate), so you won't see misleading 0/s, ETA 0h00m noise. To get more frequent updates: PROGRESS_SECONDS=10 bin/kg-rebuild.sh ….

Phase 4's progress comes from bin/kg-extract.sh's own log4j2 lines, not the orchestrator. The CLI logs an Extract-CLI progress: summary every --poll-seconds (default 30) and an Extract-CLI final: line at the end.

Troubleshooting

"unreachable — is Tomcat running and credentials valid?" — confirm Tomcat is up (tomcat/tomcat-11/bin/startup.sh) and that test.properties carries the testbot admin credentials. The script can't proceed without a valid admin session for the /admin/content/* endpoints.

Phase 1 returns 409 — a previous rebuild is still running. The script will wait for it to finish and proceed; if it doesn't, check the snapshot for an ERROR state and look at errors[] in the JSON for per-page failures.

Phase 2 returns 503 — hybrid retrieval is disabled. The script skips phase 2 with a warning and continues. To re-enable, set wikantik.search.hybrid.enabled=true and restart Tomcat.

Phase 4 exits immediately with unknown argument: --prefilter (or --force, --stats-only, ...) — those flags belonged to the retired chunk-level extractor. Drop them; see the extractor CLI flag reference.

Phase 4 progresses but proposals/mentions stay at zero — confirm the Ollama model you named is loaded on the inference host and that --ollama-url (default http://inference.jakefear.com:11434) is reachable from where the script runs. Try bin/kg-extract.sh --max-pages 3 --dry-run and read the Extract-CLI progress: lines. Mentions are only written for accepted names that already map to a kg_nodes row, so a purged KG shows proposals first and mentions only after approvals.

The run aborts mid-phase-4 and you want to resume — re-run with --skip-chunks --skip-embeddings, and add --only-missing to skip pages that already have kg_proposals rows (pages whose extraction filed nothing are retried).

Phase 3 --purge-kg confirmation rejected even though I typed yes — the purge specifically requires PURGE (uppercase, exact). It's deliberately stricter than the [y/N] of --reset-kg so you don't fat-finger your way into a destructive run. Use --yes to bypass entirely in scripted contexts.

Embedding phase shows state=DISABLED — wikantik.search.hybrid.enabled=false in your config. The rebuild script logs a warning and skips the phase rather than failing. To re-enable, flip the property and restart Tomcat.

Hub clustering empty after --purge-kg — by design. The hub-clustering subsystem writes its own data; phase 4's extraction doesn't repopulate it. It will fill back up on the next page-save event or when the hub-discovery service runs.

Extractor CLI flag reference

bin/kg-extract.sh has its own --help (the launcher header) and --jar-help (the CLI's usage). Every flag is also forwarded by bin/kg-rebuild.sh after the optional -- separator (or as the first unrecognised arg). An unknown flag aborts the run with unknown argument and exit code 2. Grouped by purpose:

Database connection (the launcher fills these in from the local ROOT.xml)

Extractor backend selection

Judge (opt-in proposal review)

Pipeline tuning

Misc

Chunk-size statistics moved to a separate tool, bin/kg-chunker-stats.sh (see above).

Cluster Hub: Wikantik Platform Development Hub: Architecture, MCP Protocols, and Agent Briefings (wikantik-development)