The Wikantik Knowledge Graph: Extraction, Curation, SHACL Gates, and Ontology Projection

The Wikantik Knowledge Graph (KG) is a human-curated, LLM-accelerated semantic property graph stored in PostgreSQL 18 (kg_* tables) and projected into an Apache Jena TDB2 RDF/OWL ontology. It transforms unstructured Markdown prose into a strongly typed network of 9 canonical entity classes and 21 closed relationship predicates, equipped with two-dimensional provenance tracking, mention-centroid vector embeddings, and 14 dedicated Model Context Protocol (MCP) tools.

Page Graph vs. Knowledge Graph: The Page Graph tracks editorial wikilinks between documents (page_links). The Knowledge Graph tracks domain entities and typed semantic predicates (kg_nodes, kg_edges). Never conflate them—see Page Graph vs. Knowledge Graph for the full architectural boundary.

1. End-to-End Knowledge Graph Lifecycle

Entities and relationships never enter the production graph unchecked from raw LLM output without provenance recording and shape validation. Every extraction, proposal, review, and ontology projection follows a strict five-stage pipeline:

flowchart LR
    subgraph Stage1["1. Inclusion & Extraction"]
        Save["Page Saved / Batch CLI<br/>Check KgInclusionPolicy<br/>(cluster policy + kg_include)"]
        LLM["LLM Extractor<br/>(Thinking disabled, strict JSON)<br/>Chunk & Page Level"]
        Save --> LLM
    end

    subgraph Stage2["2. Proposal & Judge Queue"]
        Props[("kg_proposals Table<br/>Confidence Score + Source Excerpt")]
        Rej[("kg_rejections Blocklist<br/>Prevents Re-Proposing Rejected Edges")]
        Judge["LLM Judge Experiment /<br/>Human Reviewer (/admin/knowledge-graph<br/>or review_proposals MCP)"]
        LLM --> Props
        Rej -. "Suppresses" .-> Props
        Props --> Judge
    end

    subgraph Stage3["3. Write-Time SHACL Gate"]
        SHACL["OntologyShaclValidator<br/>Validates Domain/Range Shapes<br/>at Write Chokepoints"]
        Curate["Direct Agent/Admin Curation<br/>(curate_nodes / curate_edges)"]
        Judge -- "Approved" --> SHACL
        Curate --> SHACL
    end

    subgraph Stage4["4. PostgreSQL 18 Property Graph"]
        Nodes[("kg_nodes (9 Classes)<br/>+ kg_edges (21 Predicates)<br/>Provenance + Tier (machine/human)")]
        Centroids[("kg_entity_embeddings<br/>pgvector Mention-Centroids")]
        SHACL --> Nodes --> Centroids
    end

    subgraph Stage5["5. Public RDF/OWL Projection"]
        ACL["Guest-Session ACL Gate<br/>(Fail-Closed on Restricted/Sourceless)"]
        Jena[("Apache Jena TDB2 A-Box<br/>/sparql · /id/* · JSON-LD @type")]
        Nodes --> ACL --> Jena
    end

2. Closed Schema: The 9 Entity Classes and 21 Predicates

To prevent predicate explosion (where an unconstrained LLM invents hundreds of near-synonymous edge labels like utilizes, leverages, is_built_with, relies_upon), Wikantik enforces a closed schema via PostgreSQL CHECK constraints (bin/db/migrations/V*.sql) and the formal wikantik.ttl OWL T-Box.

2.1 The 9 Canonical Entity Classes (kg_nodes.node_type)

2.2 The 21 Closed Relationship Predicates (kg_edges.relationship_type)

3. Mention-Centroid Entity Embeddings & Cross-Cluster Discovery

Every entity node v \in V_{\text{KG}} is projected into the dense embedding space (kg_entity_embeddings) without requiring a separate dictionary definition prompt. Instead, Wikantik computes the mention-centroid vector \vec{e}_v \in \mathbb{R}^d over the context-prefixed chunk embeddings \vec{u}_c of all wiki sections M(v) in which entity v is mentioned:

Why Mention-Centroids Work: Because each chunk vector \vec{u}_c encodes its page title, primary cluster, section heading, and technical body, the normalized centroid \vec{e}_v captures the distributional operational role of entity v across the entire wiki. When an agent invokes find_similar(node="Reciprocal Rank Fusion", topK=5), pgvector computes cosine similarity over kg_entity_embeddings and surfaces structurally analogous fusion/ranking concepts across machine-learning, quantitative-finance, and wikantik-development.

4. Provenance, Trust Tiers, and Inclusion Governance

4.1 Cluster-Inherited Inclusion Policy (KgInclusionPolicy)

Not every page should populate the Knowledge Graph (for example, scratch pages, legal terms of service, or personal hobby notes). As documented in KG Inclusion Policy, inclusion is governed by two rules:

1. Cluster Hierarchy Policy: Configured at /admin/knowledge-graph (Policy tab). Sub-clusters (generative-ai/ai-infrastructure) inherit their parent cluster's policy unless overridden, and multi-cluster pages resolve fail-closed (an explicit EXCLUDE on any membership wins).
2. Page-Level Frontmatter Override (kg_include: true | false): Explicitly includes or excludes a page. All 91 Hub pages across Wikantik carry kg_include: true.

4.2 Provenance & Two-Tier Trust (ProvenanceModel)

Every row in kg_nodes and kg_edges records two orthogonal trust columns (see Provenance Model):

• provenance: human_authored (entered via UI or curate_nodes/curate_edges), ai_inferred (auto-promoted above confidence threshold), ai_reviewed (LLM-extracted and approved via review_proposals), or auto_projected.
• tier: machine (default unreviewed extraction) vs. human (vetted by an administrator or trusted curation agent). Read tools (query_nodes, traverse, search_knowledge) accept min_tier="human" when an agent requires zero-noise, human-verified graph edges.

5. RDF/OWL Ontology & Write-Time SHACL Gate (wikantik-ontology)

Above the PostgreSQL kg_* tables sits the wikantik-ontology module powered by Apache Jena TDB2 (see Wikantik Ontology Management):

1. Write-Time SHACL Validation (OntologyShaclValidator): Before an edge is committed (either via proposal approval or curate_edges), the validator checks the edge against SHACL shapes defined in wikantik.ttl, rejecting domain/range violations before they can corrupt the graph.
2. Incremental Postgres \to RDF Projectors: Four projectors (EntityProjector, EdgeProjector, PageProjector, ConceptProjector) materialize RDF triples into TDB2 on page save, rename, or KG curation.
3. Fail-Closed Public/Restricted ACL Split: The public ontology endpoints (GET /sparql, GET /id/{type}/{id}, GET /export/ontology.ttl, and /knowledge-mcp sparql_query) only materialize entities whose source page is readable by an anonymous guest session. Any machine-extracted entity lacking a valid source_page is withheld fail-closed so restricted knowledge can never leak through SPARQL.

6. Agent MCP Curation & Traversal Examples

// 1. Traverse 2 hops from 'Wikantik' on /knowledge-mcp
{
  "name": "traverse",
  "arguments": {
    "startNode": "Wikantik",
    "direction": "OUTBOUND",
    "relationshipTypes": ["uses", "implements", "contains"],
    "maxDepth": 2,
    "minTier": "machine"
  }
}

// 2. Curate a verified node and edge directly on /wikantik-admin-mcp
{
  "name": "curate_edges",
  "arguments": {
    "op": "add",
    "edges": [
      {
        "sourceName": "Wikantik",
        "targetName": "Apache Jena TDB2",
        "relationshipType": "uses",
        "sourcePage": "WikantikKnowledgeGraph"
      }
    ]
  }
}

Cluster Hub: Wikantik Platform Hub: Knowledge Systems, Hybrid Search, and AI Agent Integrations (wikantik-platform)