ADR-009: Repetition Canonicalization Model

• Status: Accepted
• Date: 2026-08-18
• Decision owners: Corpus, annotation, reference and database maintainers
• Scope: Exact and variant passage repetition over immutable text releases

Context

Risale-i Nur contains passages that recur in different books and chapters. Some are byte-for-byte repetitions, some preserve the same passage with punctuation or wording changes, and some quote only a smaller passage inside a larger context. These facts are valuable for navigation, study, search and the future reference graph, but none permits the corpus to store one occurrence and point every other location at its text.

A repeated paragraph still belongs to its own book, chapter and source position. Its sentence and token occurrences may carry different headings, notes, compounds, translations and references. Even identical visible wording can have different source provenance. Deduplicating core rows would therefore destroy reconstruction and identity facts that ADR-001 through ADR-006 require the project to preserve.

Similarity creates a second risk. A normalized hash, token n-gram score or edit distance can find useful candidates, but cannot decide that two passages have the same semantic role. Common introductory formulas, embedded quotations and one repeated sentence inside otherwise different paragraphs can produce high scores. A threshold is discovery evidence, not curation authority.

ADR-002 separates definition, occurrence and supersession identity. ADR-005 provides exact release-bound selections. ADR-006 provides source and transformation provenance. ADR-007 makes accepted annotation changes reviewable and immutable. ADR-008 shows how policy-qualified propagation must preserve its route and ambiguity. This ADR defines the repetition domain that uses those contracts.

Decision

Annotation boundary

Repetition is an annotation over accepted core selections:

text release ---> passage selections ---> repetition evidence
                                      +--> accepted repeat structures
                                      +--> canonical-choice decisions

Deleting every repetition record leaves every text release exactly reconstructible. Repetition import, review or canonicalization cannot insert, update, suppress, reorder, merge or redirect any core book, chapter, paragraph, sentence, token or exact byte.

Each repeated location remains independently addressable. A canonical occurrence is a recommended navigation or reference target within one accepted repetition structure. It is not the source of another occurrence’s text and does not make non-canonical occurrences aliases, children or copies.

Three semantic layers

The model separates three relation layers because their algebra and authority differ.

Layer	Accepted meaning	Transitive	Acceptance authority
Exact class	Every member satisfies one declared exact predicate	Yes, inside one profile	Deterministic certified build
Repeat family	Reviewed occurrences represent the same passage with tracked variation	Only as one reviewed member set	ADR-007 curation
Quotation relation	A target passage quotes or reuses a source passage	No; direction and extent matter	ADR-007 curation

near_exact is a candidate/evidence classification until review promotes an exact member set into a repeat family. variant is an accepted family kind with reviewed diff evidence. quotation is a directed relation, not an equivalence class. uncertain is a candidate disposition and never an accepted semantic relation.

This preserves the user-facing concepts exact, near_exact, variant, quotation and uncertain without pretending they all have cluster semantics. An implementation may expose a combined repeat-results view, but the stored authority remains separated.

Record classes

The logical model has these identity classes:

Record	Meaning
repetition_profile	Versioned scope, comparison, granularity and exclusion rules
passage_occurrence	One repetition-domain annotation anchored by one ADR-005 selection
exact_class	Deterministic equivalence class under one exact predicate
repeat_candidate	Generated/imported suggestion with no acceptance authority
repeat_evidence	Immutable fingerprints, metrics and source facts for a candidate or decision
passage_diff	Reproducible exact-byte/token alignment between ordered occurrences
repeat_family	Reviewed near-exact or variant member set
quotation_relation	Reviewed directed source-to-quotation relation with exact extents
canonical_choice	Reviewed recommendation of at most one occurrence for one structure
repetition_review	ADR-007 proposal/review binding exact candidate and decision bytes
repetition_release	Immutable published inventory, relation and evidence roots
repeat_propagation_policy	Versioned rules for canonical/related reference traversal

Physical table names are deferred. Membership and evidence may use normalized child tables, but no implementation may collapse passage occurrence, exact class, reviewed family, quotation relation and canonical choice into one overloaded row.

Repetition profiles and scopes

A repetition_profile declares all facts that can alter discovery meaning:

• profile identity and semantic version;
• coordinate profile and accepted text release set;
• permitted passage granularity and minimum/maximum extent;
• a finite deterministic occurrence-enumerator and its ordering rules;
• exact comparison predicate;
• candidate normalization and feature versions;
• heading, note and structural-separator inclusion policy;
• handling of whitespace, punctuation and sentence/member boundaries;
• maximal-run and contained-candidate policy; and
• validation and publication requirements.

A profile binds an exact ordered release manifest. The normal corpus profile covers one Turkish corpus release, but a deliberately reviewed cross-edition profile may cover several independently reconstructible releases. Equal bytes in undeclared releases are not members of that scope.

The occurrence-enumerator makes coverage finite and auditable. For example, it may emit every paragraph plus every contiguous sequence of two through twelve sentences inside a chapter, but it cannot claim completeness over unspecified arbitrary spans. Its output is deduplicated by canonical selection identity before comparison.

Headings, notes and separators are never silently stripped. The profile chooses one of these explicit treatments for each annotation kind:

• included: selected bytes participate in comparison;
• excluded_from_extent: candidates cannot cross the annotated boundary;
• masked_with_boundary: bytes do not participate, but a typed boundary marker remains in the comparison preimage; or
• separate_analysis: a distinct profile and release is produced.

Every excluded or masked range references its exact source selection and policy reason. Concatenating text from opposite sides of an exclusion without a boundary marker is invalid because it can manufacture a repeat that never exists contiguously.

Initial F6 scope policy

F6 starts with the immutable repetition-scope-policy-v1. A concrete profile identity binds this policy to exactly one analysis projection, one occurrence enumerator, one exact predicate, one coordinate profile and one accepted Turkish core release manifest root. Projection or enumerator choice is therefore part of identity, not a runtime filter over a shared result set.

The initial matrix contains four concrete profiles:

Analysis projection	Occurrence enumerator	Exact predicate
full_context_v1	whole_paragraph_v1	exact_passage_bytes
full_context_v1	sentence_run_2_12_v1	exact_sentence_sequence
prose_only_v1	whole_paragraph_v1	exact_passage_bytes
prose_only_v1	sentence_run_2_12_v1	exact_sentence_sequence

Each profile covers the complete ordered book and chapter inventory of its bound core release. Comparison can find occurrences in the same chapter, across chapters or across books. It cannot silently add another edition, language or core release. A later cross-edition analysis requires a successor policy with an explicit ordered release set. Ottoman renderings, lexical roots, translations and normalized strings do not enter the exact preimage.

The enumerators are finite and deterministic:

• whole_paragraph_v1 visits every paragraph in canonical document order, emits each non-empty whole paragraph exactly once and records an empty paragraph as an explicit enumeration exclusion. It emits no paragraph fragment and never joins adjacent paragraphs.
• sentence_run_2_12_v1 emits every contiguous run of 2 through 12 sentence occurrences inside one chapter, ordered first by start sentence and then by run length. A run may cross a paragraph boundary only when its exact inter-sentence and inter-paragraph separators remain in the witness. It never crosses a chapter boundary.

The two enumerators remain separate profiles even when a sentence run happens to cover the same visible text as one paragraph. They make different structural claims and use different exact predicates. Repeated one-sentence paragraphs remain discoverable through the paragraph profile; the initial sentence-run profile does not create single-sentence candidates.

Annotation projections

Both projections bind the exact accepted heading and note dataset roots used during the build. Missing, stale or geometrically invalid annotation evidence is an integrity failure; the builder cannot silently continue as if no annotations existed.

Annotation or boundary	full_context_v1	prose_only_v1
Accepted heading marker span	included	excluded_from_extent
Accepted note body selection	included	excluded_from_extent
Accepted note marker span	included	masked_with_boundary
Sentence/paragraph structural separator	included	included

excluded_from_extent rejects an enumerated occurrence that intersects the excluded selection; it does not trim the occurrence. masked_with_boundary retains a typed, length-delimited marker in the comparison preimage. Consequently the prose before and after a note marker can never become adjacent bytes. An unmatched note marker recorded as source_gap is masked in the same way and never causes an invented note body.

The initial closed exclusion reason set is:

• empty_core_extent for a paragraph with no selectable core byte;
• heading_marker_intersection for a prose-only occurrence touching an accepted heading marker; and
• note_body_intersection for a prose-only occurrence touching an accepted note body.

Chapter boundaries and the 2-through-12 sentence bound define the enumerator universe; they are not post-enumeration exclusions. Invalid geometry, unavailable release roots, unreconstructible separators and missing witness bytes are typed failures, not exclusions or unique results.

Exactness, dominance and coverage

exact_passage_bytes compares length-delimited, unnormalized UTF-8 core bytes plus any typed mask markers. exact_sentence_sequence compares the ordered exact sentence selection digests and every declared separator. Neither predicate performs NFC, case, apostrophe or punctuation folding. Such transforms may later generate near-duplicate candidates but cannot enter an exact class.

Paragraph profiles use structural_units: every qualifying exact paragraph class is published independently. Sentence-run profiles use maximal_only inside the bounded 2-through-12 universe. One sentence-run class dominates another only when its complete member inventory has a deterministic one-to-one document-position mapping to the shorter class and every mapped member is the same strict left/right sentence extension. A class with an additional unextended occurrence is therefore not dominated. The longest qualifying dominator wins; equal-length ties use the exact class ID. A contained exact run remains rooted as dominated evidence and is not rejected or deleted.

Class frequency does not change exact truth, so the initial policy has no frequency cap or stop-text exclusion. Presentation may page or de-emphasize large classes without changing release membership.

Coverage accounts separately for the deterministic enumerator inventory, evaluated exact work, profile exclusions, dominated matches and typed failures. A profile is exact complete only under the partition rules in this ADR. Exact acceptance does not create a repeat family, quotation relation or canonical choice.

Changing any projection treatment, sentence bound, enumerator order, exact predicate, dominance rule or release scope issues a successor profile. Published profile and occurrence identities are never rewritten in place. F6-002 and F6-003 implement the two enumerators against this contract; F6-004 materializes both annotation projections.

Passage occurrence

A passage_occurrence is the repetition-domain use of one ADR-005 selection. It binds:

• one repetition profile;
• one coordinate profile, text version and immutable text release;
• one ordered, coalesced selection ID;
• its exact document-order and containing hierarchy anchors;
• a declared granularity such as paragraph, sentence sequence or bounded span; and
• the exact comparison-mask witness used by the profile.

The selected bytes stay in core. The occurrence stores no replacement text. One canonical (profile_id, selection_id, granularity, mask_witness_root) has one occurrence ID. A wider passage containing a repeated sentence and that sentence alone are different occurrences because their selections differ.

Selections may be nested or overlap. This is necessary for a long repeated passage and its repeated subpassage. Geometry alone creates no family membership, quotation or canonical relationship. Cross-release numeric offsets are non-comparable under ADR-005.

Exact comparison

An exact predicate is named and versioned. Initial profiles may use both of these distinct predicates:

• exact_passage_bytes: the ordered selected UTF-8 bytes and typed mask boundaries are identical; and
• exact_sentence_sequence: the ordered array of exact sentence-selection digests, including declared inter-sentence separators, is identical.

These predicates are not interchangeable. Two selections can reconstruct the same visible characters but have different sentence structure, or can have equal token text while differing in exact punctuation and whitespace.

The fingerprint preimage contains at least the predicate/profile ID, release-bound comparison-mask format, ordered member count, member lengths, exact bytes or exact member digests and typed boundary markers. Length-prefixing or canonical structured encoding prevents ambiguous concatenation. NFC, case folding, apostrophe folding, stemming and punctuation removal are forbidden in an exact preimage.

A cryptographic digest indexes equality but does not prove it alone. Exact-class construction verifies preimage length and byte/member equality after hash grouping. Hash collision, unavailable witness bytes or mismatched structure returns integrity_error; it never creates a class.

An exact_class has at least two occurrences. Its members are sorted by stable document order and occurrence ID. Under one profile and repetition release, every passage occurrence belongs to at most one exact class for each exact predicate. Class membership is a derived equivalence relation and must be reflexive, symmetric and transitive over its declared member set.

Exact discovery may be accepted automatically only when the profile, source releases, occurrence inventory and comparison implementation are certified and the complete preimages were verified. Automatic exact acceptance does not automatically select a canonical occurrence.

Granularity and dominance

Repeated material can generate a long run, all of its constituent sentences and many overlapping n-grams. A profile therefore declares a deterministic dominance policy:

• maximal_only publishes maximal exact runs and records contained matches as derived support;
• structural_units publishes declared paragraph/sentence-sequence units independently; or
• all_qualified publishes every candidate meeting explicit bounded extent rules.

Suppression by dominance is not rejection and does not assert that a contained repeat is false. Release metrics retain counts and evidence roots for discovered, dominated, published and rejected candidates. A later profile can publish a different granularity without changing the earlier release.

Generic formulas and very short passages can form large, low-value classes. Minimum extent, stop-pattern and frequency-cap rules belong to the versioned profile. They can control discovery/publication volume but cannot turn a non-exact candidate into exact or discard already published identity without a successor release.

Candidate generation and evidence

repeat_candidate is immutable evidence, not an accepted relation. It binds the exact profile, source release set, occurrence IDs, generator artifact/version, parameters, feature/fingerprint roots and generated result bytes.

Candidate generation may use normalized text, token n-grams, MinHash/LSH, sequence alignment, lexical roots, Ottoman renderings or existing source metadata. Each feature is labeled by authority class. Core exact bytes and stable selections remain the witness; normalized values and lexical annotations are reproducible features only.

Scores are stored as named metrics with algorithm version and scale. There is no generic confidence field whose meaning changes by importer. Changing an algorithm, threshold, normalizer or feature set creates a new profile or generator artifact and new candidates. It does not mutate old scores or accepted decisions.

Candidate dispositions include:

• pending_review;
• probable_near_exact;
• probable_variant;
• probable_quotation;
• uncertain;
• false_positive; and
• superseded_candidate.

No candidate disposition creates an effective family, quotation relation or canonical choice. false_positive is retained as benchmark evidence and can suppress the same candidate under the same generator/profile version without becoming a global text rule.

Passage diffs

A passage_diff is evidence between two ordered passage occurrences. It binds exact left/right selection IDs and witness roots, a diff algorithm/version and a canonical ordered edit script. Operations identify exact source and target spans and use typed equal, insert, delete, replace, move or boundary_change semantics.

The diff must replay from the exact left witness to the exact right witness. A display summary or edit count cannot replace the replayable script. Token-level assistance may improve readability, but every operation remains anchored to exact ADR-005 coordinates.

A diff is directional evidence. Reversing its display does not reuse its identity unless the identity contract explicitly stores both orientations in canonical order and proves lossless inversion. Diff similarity does not imply family membership.

Reviewed repeat families

A repeat_family asserts that its complete reviewed member set represents one repeated passage despite tracked differences. Its accepted kind is:

• near_exact: differences are bounded and judged not to change passage identity under the profile; or
• variant: members are recognized versions of the same passage with substantive, explicitly reviewed variation.

Family acceptance binds the exact ordered member inventory, every member selection witness, supporting candidate/evidence IDs, required diffs, relation kind, rationale and ADR-007 decision. Pairwise similarity edges cannot be transitively closed into a family. The reviewer accepts the full set as one assertion.

Family members are tagged references to either one passage occurrence or one exact class. Expanding exact-class members yields the family’s complete occurrence set. No occurrence may appear both directly and through an exact class, and an expanded occurrence belongs to at most one effective family for the same profile and repetition release. Exact duplicates therefore remain a known subset inside one broader variant family without creating competing family assertions.

Membership is unordered semantically but has deterministic presentation order by corpus document order and occurrence ID. Reordering a list for UI does not change identity. Changing the accepted member set or family kind changes what the family denotes and creates a successor family under ADR-002/ADR-007.

Quotation relations

Quotation is directional and non-transitive. A quotation_relation binds:

• one exact source occurrence or source sub-selection;
• one exact quotation occurrence or quotation sub-selection;
• direction and relation kind;
• extent mapping and optional passage diff;
• source attribution/provenance evidence; and
• one accepted ADR-007 decision.

Relation kinds may distinguish verbatim_quotation, adapted_quotation, formulaic_reuse and source_uncertain. The last remains non-effective until review selects an accepted kind. If several quotations share one source, normalized membership rows may avoid repeated source facts, but no Cartesian equivalence is inferred among the quotations.

A quotation source is not called canonical unless it is independently chosen by a canonical_choice for an exact class or repeat family. Attribution and canonical navigation are separate decisions.

Canonical occurrence choice

An exact class or reviewed repeat family may have zero or one effective canonical_choice under one policy. Absence is valid and returns unselected, not the first member. A choice binds:

• the exact class/family identity and release;
• one member occurrence ID;
• a versioned choice policy;
• rationale and evidence;
• ADR-007 proposal/review/decision IDs; and
• issuance and optional supersession facts.

Choice policies may prefer a declared source edition, author’s primary placement, fullest context or editorially designated occurrence. Corpus order, shortest UUID, highest similarity or earliest discovery cannot silently choose authority.

Changing a canonical occurrence issues a new choice identity and supersedes the old choice. It normally preserves the class/family identity because membership did not change. Historical references that explicitly named the old occurrence or old choice continue to resolve exactly. APIs only resolve the current recommendation when given an explicit policy and repetition release.

Identity contracts

Every identity-bearing class publishes immutable defining facts and exactly one ADR-002 issuance channel:

Kind	Issuance channel	Identity-bearing facts	Identity-preserving additions
Profile	package_deterministic	Rule document digest and semantic version	Usage statistics
Passage occurrence	package_deterministic	Profile, release, selection, granularity, mask root	Review notes
Exact class	package_deterministic	Profile/predicate, scope release root, sorted occurrence IDs, witness root	Display label
Candidate	package_deterministic	Profile, exact participant set, generator and result digest	Triage note
Evidence	package_deterministic	Evidence kind, participant IDs, producer, payload digest	Retention metadata
Diff	package_deterministic	Ordered endpoints, algorithm/version, exact edit-script digest	Rendered summary
Repeat family	curation_generated	Profile, kind, canonical tagged members, expanded occurrence root, decision bytes	Display order
Quotation relation	curation_generated	Profile, directed exact extents, kind, decision bytes	Citation note
Canonical choice	curation_generated	Structure, member, policy, decision bytes	Presentation label
Review	curation_generated	ADR-007 proposal bytes, reviewers and decision	Derived effective cache
Release	package_deterministic	Scope/profile and all inventory/decision roots	Build location
Propagation policy	package_deterministic	Rule document digest and semantic version	Usage counters

IDs use ADR-002/ADR-003 canonical UUID rules. Hashes, normalized strings, paragraph IDs or discovery-run row numbers are not public IDs by themselves. Two occurrences with the same bytes have different IDs; equality belongs the exact class.

Package-issued kinds use the repetition package’s immutable identity release key and versioned path grammar. Curation-issued kinds receive UUIDv7 once at accepted issuance; later packages preserve that UUID and cannot remint it as UUIDv5. An imported external identifier remains a namespaced crosswalk and never switches an entity’s issuance channel.

Published records are immutable. A source-release correction creates new selections and passage occurrences even when visible text appears unchanged. A new repetition release may crosswalk or supersede earlier structures, but cannot retarget their member IDs.

Review and correction

Certified exact-class construction can be a deterministic release action because its predicate is fully reproducible. All other semantic promotion requires ADR-007:

• accepting or changing a near-exact/variant family;
• accepting or changing a quotation relation;
• choosing or changing a canonical occurrence;
• overriding a false-positive decision; and
• retiring a published repetition structure.

A proposal freezes exact IDs and action bytes. Review binds those bytes. Apply verifies the expected parent state and participant witness roots atomically. Direct DML and threshold-based promotion are forbidden.

Correction never edits published membership. A mistaken family is replaced, split, merged, restructured or retired through typed supersession. A mistaken canonical choice is compensated and superseded without changing its family. Invalid exact results require a corrected source/profile/compiler and successor repetition release; they are an integrity incident, not ordinary semantic review.

Publication and rebuild closure

A repetition_release binds at least:

• repetition profile and exact source release manifest roots;
• complete passage occurrence inventory and mask root;
• exact fingerprint/preimage evidence root;
• accepted exact-class inventory and membership root;
• classified candidate, dominated and false-positive evidence roots;
• accepted family, quotation and canonical-choice roots;
• unresolved semantic-review inventory root;
• provenance, transformation, validation and curation heads; and
• compiler/build artifact versions.

Exact discovery coverage and semantic curation coverage are separate statuses. An exact release is complete only when every in-scope qualified occurrence was evaluated under the declared exact predicates and all exact failure sets are empty. Semantic curation may remain open while pending candidates are published explicitly. complete never means that every similar passage has been semantically classified.

The release first commits a finite exact_work_root over every applicable (passage_occurrence_id, exact_predicate_id) work item. That universe is partitioned by five disjoint, exhaustive and individually rooted manifests:

• matched, with an exact-class/equality witness;
• evaluated_unique, with a certified no-peer result inside the complete work universe;
• not_evaluated, with a controlled pending reason;
• excluded, with a profile-qualified exclusion reason; and
• failed, with a typed integrity/processing failure.

Every work item occurs exactly once. matched results additionally partition into published and dominated presentation manifests without changing their evaluated state. Exact status is complete only when not_evaluated and failed roots are empty and all partition/member roots verify. A missing row can never mean unique, excluded, rejected or no repeat.

Private audit closure and public disclosure are separate. The restricted release control record may bind classified provenance/evidence roots under ADR-006. A public manifest or API exposes only disclosure-approved projection roots and opaque signed attestation IDs; it never exposes private raw roots, hashes/commitments guessable from private data, absolute paths, source selectors, diagnostics or secret-bearing metadata. Authorization is re-applied when evidence or propagation routes are read.

Immediately before any release becomes visible, one transaction locks and revalidates the expected repetition/curation head, authority grant and revocation state, freeze epoch, profile and propagation-policy IDs, source/core release roots, exact work partition, accepted family/quotation/canonical inventories and public projection roots. Any mismatch publishes nothing. Release rows, effective indexes and visibility pointer change atomically, so a concurrent canonical decision, freeze or revocation cannot publish stale state.

Runtime does not read legacy SQLite, ad hoc reports or untracked candidate files.

Two clean builds from the same source releases, profiles, evidence and curation heads must reproduce the same passage occurrence IDs, exact classes, family/relation IDs, canonical choices and release roots. Changing only worker count, batch size, insertion order or cache state cannot change output.

Reference propagation

References always retain the originally authored target. Repetition can derive related routes under an explicit repeat_propagation_policy; it never silently rewrites a reference to the current canonical occurrence.

A propagation result reports orthogonal facts:

• route_status: resolved, unselected, unavailable, ambiguous, stale_evidence, forbidden_relation or integrity_error;
• relation_layer: exact_class, repeat_family, quotation or not_applicable;
• relation_kind: exact predicate, near_exact, variant or quotation kind;
• target_mode: canonical, all_members, quotation_source, quotations or not_applicable;
• target_cardinality: zero, one, many or not_applicable;
• source and target occurrence/release IDs;
• repetition release, policy ID and complete traversed decision/supersession route; and
• whether each target is exact, reviewed variant or directional quotation evidence.

An exact class can support all_members without claiming one preferred location. A canonical route requires an effective reviewed choice. A variant family route must expose that wording differs. Quotation traversal is direction-specific and does not make source and quotation interchangeable.

Composition with ADR-008 multilingual alignment is explicit and ordered. For example, Turkish occurrence -> canonical Turkish repeat -> aligned English unit is a two-hop derived route. The reverse order can yield a different target set. Implementations retain every hop and never materialize an accepted direct reference from the composition.

Composed route status uses this closed precedence, highest first:

integrity_error > stale_evidence > forbidden_relation > ambiguous > unavailable
  > unselected > resolved

All branches are evaluated under the same bound policies/releases before aggregation. If every branch is resolved, destinations are the deduplicated union and no branch may be silently narrowed. If any branch is non-resolved, the aggregate returns the highest status and an empty destination set while retaining every traversed hop and branch result allowed by RLS. A resolved branch therefore cannot mask a stale, forbidden, unavailable or ambiguous branch.

If a bound repetition/alignment release, evidence root or policy is unavailable or stale, propagation fails closed. It cannot fall back to normalized text search, a newer canonical choice or the first result.

Query behavior

The logical model supports separate operations to:

• get one exact passage occurrence and reconstruct it from core;
• list exact duplicates under a named profile/repetition release;
• inspect a reviewed family with exact member diffs;
• follow a directed quotation relation;
• retrieve the effective canonical choice under an explicit policy;
• list unresolved/false-positive candidate evidence for curation;
• report exact and semantic coverage separately; and
• derive a policy-bound repetition or composed multilingual route.

Responses distinguish unique, not_evaluated, excluded, pending, unselected, rejected, retired, ambiguous and integrity_error. An empty member list cannot choose among these states. F2-010 and F2-020 define concrete API/RPC envelopes and RLS.

Validation Requirements

Implementations must prove at least:

1. Every core text release reconstructs to the same certified digest with all repetition records present or absent.
2. Passage occurrences round-trip exact ADR-005 selections; masks account for every included/excluded byte and cannot concatenate across an unmarked boundary.
3. Exact fixtures verify full structured preimages after hash grouping and reject collision, normalization-only, punctuation-only and sentence-boundary mismatches unless a profile explicitly classifies them as candidate features.
4. Two clean builds reproduce occurrence IDs, exact classes, evidence roots and dominance results independently of batch and insertion order.
5. Exact classes have at least two unique members, satisfy one predicate and reject competing same-profile membership.
6. Nested and overlapping passage fixtures remain valid while geometry alone creates no membership or quotation relation.
7. Near/variant thresholds can create candidates but cannot publish families or canonical choices without exact ADR-007 proposal/review bytes.
8. Family fixtures reject pairwise transitive closure, incomplete member inventories, direct/exact-class double membership, stale diffs and competing effective membership.
9. Quotation fixtures preserve direction and extents and prove that A quotes B plus B quotes C does not infer A quotes C.
10. Canonical fixtures permit zero or one effective choice, preserve historical choices and never change member/core identities when a recommendation changes.
11. Partial releases root the complete (occurrence, predicate) work universe and its disjoint matched, evaluated_unique, not_evaluated, excluded and failed partition; only empty pending/failure roots permit exact complete status.
12. Publication fixtures recheck locked head, authority/revocation, freeze, policy, source and inventory roots and prove any concurrent mismatch publishes no rows.
13. Public projection fixtures reject private roots, guessable commitments, paths, selectors and diagnostics while restricted audit closure remains verifiable.
14. Propagation fixtures retain original references, relation precision, target sets and every multilingual/repetition hop; any non-resolved branch suppresses all destinations under the declared status precedence.
15. Correction fixtures exercise replace, split, merge, restructure and retirement without mutating published structures or losing historical resolution.
16. Release rebuild succeeds without legacy paths and contains no duplicate core text tree or replacement passage bytes.

Consequences

Benefits

• Repeated passages become searchable and referenceable without weakening exact corpus reconstruction or source provenance.
• Exact duplicates can be discovered at scale deterministically, while semantic variants remain reviewable instead of threshold-driven.
• Canonical navigation can change safely without changing passage or reference identity.
• Nested repeats, larger passage variants and directional quotations remain expressible.
• Translation and repetition routes can compose explicitly without invented direct edges.
• False positives become reusable benchmark evidence rather than discarded manual work.

Costs

• Exact classes, reviewed families and quotations require separate storage and queries.
• Scope/mask profiles must be versioned when headings and notes become available.
• Near/variant family review binds complete member sets and replayable diffs, not only a convenient score.
• Large exact classes and nested candidates require deterministic dominance and paging.
• Canonical recommendations require curation history even for exact repeats.

Alternatives Considered

Deduplicate core paragraphs and reuse one row

This loses independent hierarchy, provenance, annotations and permanent references and makes reconstruction depend on repetition. It is rejected.

Put every similarity edge in one cluster

Similarity and quotation are not transitive. Connected components can join passages that were never reviewed as one family. It is rejected.

Use normalized text hash as exact authority

Normalization can erase punctuation, whitespace, apostrophe, case and diacritic evidence. It is useful for candidate discovery but cannot prove exact equality. It is rejected.

Automatically choose the first or longest occurrence as canonical

Corpus order and extent are useful review evidence but do not establish editorial authority. Canonical choice remains explicit curation.

Rebuild and replace all clusters after every algorithm change

Disposable candidates may be rebuilt, but published IDs, manual choices and references must survive. New profiles/releases and typed supersession are required.

Treat quotation as a variant family

This makes source and quotation interchangeable and creates invalid transitive relations. A directed quotation model is required.

Deferred Decisions

• F2-010 and F2-020 define API/RPC envelopes, paging, RLS and error representation.
• F2-018 defines compatibility classes and the final approval matrix.
• F2-019 defines shared physical provenance, evidence, run and curation serialization.
• F3/F4 define physical selection, annotation and stable-ID schemas used by occurrences.
• F6 implements this initial scope policy, exact pipelines, MinHash/LSH candidates, sequence diffs, benchmark fixtures, review UI and canonical lookup RPCs.
• F7/F8 define concrete composition with multilingual alignment and reader presentation.
• The reference-graph phase decides which derived routes may become reviewed reference assertions; this ADR grants no automatic promotion.

Revisit Conditions

Revisit this ADR if:

• a legitimate accepted repetition cannot be represented by exact classes, reviewed families or directed quotation relations;
• a required comparison cannot retain a complete exact selection/mask witness;
• one occurrence legitimately needs competing effective family membership under the same profile and repetition release;
• production scale makes exact preimage verification or immutable member-set review infeasible without weakening authority; or
• reference propagation needs an equivalence claim that cannot expose wording variation, route or target multiplicity.