Roadmap: External ACL Inheritance and Permission-Aware Retrieval

Gap statement: Wikantik enforces its own ACLs well, but cannot inherit anyone else's

Wikantik's page ACLs are enforced on every retrieval surface — the bundle's candidate retrieval, the format filter, and the ontology's public/restricted materialization split. What it lacks is the enterprise-defining capability: inheriting permissions from connected source systems so that a document synced from Confluence or Drive is only retrievable by users who could see it at the source. Without connectors this gap is moot; with them (RoadmapConnectorFramework) it becomes the enterprise gate.

Market grounding: permission sync is the monetized enterprise feature

Glean ingests permissions/ACL data alongside content from each source so retrieval respects source-system visibility, and enforces that a user who cannot access an item at the source never sees it in results (Glean connector docs).
Onyx gates document-level permission sync to its paid Enterprise Edition — a strong signal this is the monetizable layer (Onyx docs).
Bedrock Knowledge Bases applies metadata pre-filtering before vector search, so unauthorized documents are excluded from the search space rather than filtered after retrieval (AWS multi-tenant RAG architecture). The same reference architecture externalizes authorization into runtime-updatable Cedar policies (Amazon Verified Permissions).
The scaling trap, per Oso's analysis: post-filtering after vector search is slow (cycling through thousands of results to find enough authorized ones), and pre-filtering by passing permitted-ID lists into the query does not scale (vector-store $in operators cap around 10,000 values). The recommended pattern is in-database authorization — permission logic pushed into the database storing the vectors, expressed in the query itself — demonstrated over PostgreSQL + pgvector (Oso on RAG authorization).

That last point is the strategic opening: Wikantik already runs the exact substrate (PostgreSQL + pgvector) the recommended architecture calls for, while vector-DB-based competitors have to bolt authorization on. This can be done better here, not just at parity.

Current Wikantik state

Page ACLs: inline [{ALLOW view ...}]() markup; database-backed policy_grants and groups; JAAS permission model with fine-grained page/wiki permissions.
Retrieval enforcement: RestServletBase.checkPagePermission() on /api/*; the bundle enforces view ACLs via candidate retrieval; PublicProjectionFilter + guest-session view checks keep restricted pages out of the materialized RDF dataset entirely (a materialization-time pre-filter — the right instinct, already shipped).
Identity plumbing: SCIM 2.0 (/scim/v2/*) for IdP-driven users/groups; SSO via pac4j with identity-claim binding.
Audit: tamper-evident /admin/audit* log; checkPermission calls are the audited enforcement points.

Proposed direction

Principal mapping: map external principals (source-system users/groups) onto wiki principals, keyed by the identities SCIM/SSO already provision — a source_principals mapping table plus group expansion.
ACL sidecar per derived page: connectors write source ACL references at sync time; a resolver translates them into wiki-visible allow-lists on the derived page.
In-database enforcement: push the permission predicate into the pgvector query (WHERE on a permissions join) and into a Lucene filter clause for the BM25/HNSW paths — pre-filtering, never post-filtering.
Evaluate externalized policy (Cedar/Oso-style policy-as-code) as a later layer over the existing JAAS + policy_grants model, not a replacement.

Phase 1 scope and acceptance criteria

First session scope: the source_principals migration + ACL-sidecar schema + a resolver mapping ONE connector type's source ACL refs to wiki principals. Enforcement wiring comes after the resolver's semantics survive design review.

Phase 1 (enforcement) is done when these tests are green — all run under a restricted PostgreSQL role, never a superuser (established project finding: permission bugs escape superuser CI):

A user without source-side access to a restricted derived page gets zero sections/results from it on all three surfaces — /api/bundle, search, and the knowledge MCP tools — one test per surface, plus proof the exclusion happened by index-level pre-filter (inspect the query, not just the output).
Fail-closed: an unresolvable external principal, or a missing/stale ACL sidecar, denies access — with a test asserting deny, not a fallback to public.
A permission change at the source is reflected after the next sync, and the sync lag is observable (surfaced as a freshness/drift metric, not silent).
A user with legitimate access still retrieves the page (no over-blocking regression), and overall recall@12 on the eval corpus does not regress for unrestricted content.

Verify before designing on these

The $in ~10,000-value cap is this brief's load-bearing scaling claim, sourced from one vendor blog. Verify against current Pinecone/pgvector documentation before citing it in the design doc — though note the in-database pushdown pattern is independently justified (single round-trip, planner-optimized) even if the cap number is wrong.
Onyx gating permission sync to its Enterprise Edition is a packaging signal for RoadmapPackagingAndPricing, not a design input here.

Non-goals

No externalized policy engine (Cedar/Oso) in Phase 1 — evaluation only, as a possible later layer.
No admin UI for editing external ACLs — the source system remains the authority; Wikantik only mirrors.
No cross-tenant anything — tenancy is a deployment boundary (RoadmapMultiTenancyPosture).

Investigation starting points

[AGENT] Can the lucene-hnsw dense backend apply filters during HNSW traversal (filtered HNSW), or only pre/post? Spike it — this decides whether Lucene or pgvector is the enforcing index for dense retrieval.
[MEASURE] Group-expansion cost at query time vs materialized ACL flattening at sync time — benchmark both; each implies a different staleness window.
[OWNER] What permission-staleness window is acceptable (how long may a revoked-at-source document remain retrievable)? This is a security posture, not an engineering preference — the agent proposes options with measured sync-lag data; the owner sets the bar.
[AGENT] Permission-change propagation mechanics once the window is set: webhook-triggered re-sync vs crawl cadence; and whether the citation-staleness machinery can model "permission drift" too.
[AGENT] Threat-model the failure mode (permission-sync lag exposing a revoked document is a security incident, not a freshness bug) — produce the fail-closed design; [OWNER] signs off on the threat model before implementation.
[AGENT] Restricted-PG-role test rig: extend the existing convention to cover the new tables and the pushdown queries.