Data Minimization: deterministic redaction for ETL, RAG, and ChatClient (no extra LLM calls)

[LordKay-sudo]

Related issues (searched; not duplicates)

I searched the issue tracker before opening this:

• #1081 / #1011 — broad behavioral guardrails; no shipped deterministic redaction SPI
• #1184 — Bedrock provider guardrails, not framework-agnostic minimization
• #5615 — guardrails for tool calls, not ingest/RAG prompt content
• #5671, #5903, #5776 — DocumentPostProcessor for reorder/rerank, not PII/secret redaction

None of the above provide a built-in, dependency-free way to minimize sensitive/proprietary data before it reaches an LLM across ingest, retrieval, and chat.

---

Expected Behavior

Spring AI provides a small, optional Data Minimization building block (module or package) with:

1. A pluggable DataMinimizationPolicy (detectors + replacement strategy) that performs no LLM calls
2. SanitizingDocumentTransformer — implements DocumentTransformer for ETL (redact before embed/index)
3. SanitizingDocumentPostProcessor — implements DocumentPostProcessor (redact retrieved chunks before QueryAugmenter)
4. DataMinimizationAdvisor — implements CallAdvisor / StreamAdvisor (final gate on the assembled Prompt)

Built-in v1: RegexSensitiveDataDetector + ReplacementStyle (MASK, REMOVE). TOKENIZE (especially reversible tokenization) deferred to a follow-up. Optional fail-closed mode.

Example (same policy bean wired in all three places):

var policy = DataMinimizationPolicy.builder()
    .detector(RegexSensitiveDataDetector.defaults())
    .replacement(ReplacementStyle.MASK)
    .build();

// ETL — before vector store
vectorStore.write(new SanitizingDocumentTransformer(policy).transform(reader.read()));

// RAG — after retrieval, before generation
var ragAdvisor = RetrievalAugmentationAdvisor.builder()
    .documentRetriever(retriever)
    .documentPostProcessors(new SanitizingDocumentPostProcessor(policy))
    .build();

// Chat — non-RAG paths and final prompt gate
var client = ChatClient.builder(model)
    .defaultAdvisors(DataMinimizationAdvisor.builder(policy).build())
    .build();

[islam3iat]

This is an incredibly well thought out design. Data minimization without relying on external LLM calls is a massive production need right now.

I’d love to help bring this to life. Once the team triages this and gives the green light, I’d be happy to take on the first slice implementing the core DataMinimizationPolicy and the RegexSensitiveDataDetector.

[jewoodev]

Two questions that might help with triage:

1. Module placement — should this live in an existing module/package, or as a separate starter? Since the same policy bean is meant to be wired across ETL, RAG, and ChatClient, the boundary choice seems to shape the SPI surface.

2. ReplacementStyle.TOKENIZE in v1 — should tokenization be reversible, stateless/deterministic, or possibly deferred from the first slice? The reversible path would likely require a pluggable mapping store SPI and lifecycle, which noticeably widens the v1 surface.

Just flagging these for triage inputs — deferring to maintainers on the direction.

[LordKay-sudo]

Thanks both of you for the thoughtful feedback.

@islam3iat — Appreciate the offer. Let’s wait for maintainer direction on module placement (see below), then we can split v1 work to avoid overlapping PRs. A reasonable split could be:

• core SPI + RegexSensitiveDataDetector + DataMinimizationPolicy
• integrations: SanitizingDocumentTransformer, SanitizingDocumentPostProcessor, DataMinimizationAdvisor
• tests + example

Happy to coordinate here once triage lands.

@jewoodev — Good questions; my current preference for v1 (open to maintainer input):

1. Module placement: Start as a single optional artifact (e.g. spring-ai-data-minimization in spring-ai-community, or in-tree if preferred) that depends on spring-ai-model, spring-ai-rag, and spring-ai-client-chat — keeps one DataMinimizationPolicy bean usable from ETL, RAG, and advisors without pulling heavy deps into core. A Boot starter can follow in v2.

2. ReplacementStyle in v1: Ship MASK and REMOVE only in the first slice. Defer TOKENIZE (especially reversible tokenization + vault/mapping store) to a follow-up issue — it widens SPI and lifecycle more than regex masking for most regulated workloads.

Still deferring final placement and API to the Spring AI team.

[Jaiswal-Devpriya]

This is an interesting proposal. I’ve recently worked on content moderation and data-processing pipelines where deterministic redaction was required before downstream AI processing, so the ETL/RAG/ChatClient integration points resonate with real-world use cases.

I’d be interested in contributing once the maintainers provide guidance on module placement and the initial scope for v1.

[jewoodev]

@LordKay-sudo The split you outlined works for me. If maintainers green-light this, I'd like to take the integrations slice — SanitizingDocumentTransformer, SanitizingDocumentPostProcessor, and DataMinimizationAdvisor (CallAdvisor/StreamAdvisor). My recent work here has mostly been around the streaming chat paths (#6373, #6381), and I've also spent time tracing the ChatClient advisor flow, so the integration side feels like a good fit.

@islam3iat since the integrations consume the core SPI, if your PR goes up I'd be happy to look at it from the perspective of the three extension points (DocumentTransformer, DocumentPostProcessor, CallAdvisor/StreamAdvisor). Their contracts put some constraints on the detector/policy signatures, and catching mismatches early should save rework on both sides if this split is accepted.

As before, deferring final placement and scope to the Spring AI team.

[LordKay-sudo]

Small note: I'd narrow v1 to MASK + REMOVE and punt TOKENIZE to a follow-up. Reversibility/vault stuff feels like a bigger SPI than this issue needs. I have that working locally with tests, just need a call on whether this belongs in-tree or in spring-ai-community before I push anything.