Evaluate chunk boundaries around policy exceptions

Evaluate chunk boundaries with questions whose correct answer depends on an exception, definition, or scope statement outside the general rule. A retrieved passage can be relevant yet incomplete enough to produce the wrong policy answer.

Design a question that exposes the split

The synthetic policy permits ordinary returns but excludes personalized products. One chunk contains the general rule and another contains the exception. A question about a personalized item should not be answered from the general chunk alone. The needed fact is known before the retrieval run, so the test can distinguish relevant evidence from sufficient evidence.

Inspect the evidence unit

1. Keep document identity, section hierarchy, scope, and source offsets with each chunk.
2. Write review questions covering the general rule, the exception, and a case where clarification is needed.
3. Inspect the final selected passages after ranking and context-budget truncation.
4. Test whether parent-section expansion, overlapping context, or another retrieval strategy preserves the required relationship without flooding the prompt.

Do not solve every miss with larger chunks

Larger passages may preserve exceptions while increasing irrelevant content, latency, or truncation pressure. Compare the same questions across candidate strategies and evaluate sufficient evidence and final answers separately. No single chunk size is a universal requirement for useful RAG.

Read the coverage check

The runnable example names the two required passage identifiers and identifies the missing exception. Adding that passage satisfies the fixture’s coverage check, but human review is still needed to establish whether the source and resulting answer are correct for the real policy scope.

Run the worked example

Save the synthetic Node.js example and run node evaluate-rag-policy-exception-chunks.mjs with Node.js 22 or newer. The assertions check the illustrated records and calculations; they do not test a deployed provider, SDK, or access control. No credentials or network connection are needed.

// Synthetic teaching fixture. No credentials, network calls, or production verification.
import assert from 'node:assert/strict';

const required = new Set(['general-return-rule', 'personalized-item-exception']);
const incomplete = new Set(['general-return-rule']);
const missing = [...required].filter(id => !incomplete.has(id));
assert.deepEqual(missing, ['personalized-item-exception']);
const expanded = new Set([...incomplete, 'personalized-item-exception']);
assert.equal([...required].every(id => expanded.has(id)), true);
console.log('General policy was present; the decisive exception was missing.');

References and further reading

• DriftRail integration documentation

Maintained by DriftRail. AI assisted the October 2026 content and template revision. Examples are illustrative; they are not measured product benchmarks. Suggest a correction.