ADR-001: Core Versus Annotation Boundaries

• Status: Conditional; accepted only when F2-001-W01 is accepted and landed on main
• Date: 2026-08-18
• Roadmap task: F2-001
• Decision owners: corpus architecture and data governance maintainers

Context

Munderecat must preserve the accepted source text exactly while allowing headings, notes, compounds, Ottoman renderings, lexical analyses, repetitions, translations and future references to improve independently. The legacy systems proved that these layers are useful, but also showed the cost of letting a reconstruction path depend on mutable interpretation tables: an annotation change can then alter text, spacing, punctuation or order.

The boundary cannot be inferred from whether data is generated or manually curated. A deterministic compound match is still an interpretation of a span. A manually verified punctuation mark is still part of the source-faithful text. Likewise, data needed by the reader is not automatically core merely because it is important.

This ADR fixes ownership and mutation rules before physical schemas are designed. ADR-002 through ADR-006 and ADR-019 will separately decide stable identity, supersession, package format, token form versus occurrence, span coordinates and the exact provenance primitives.

Decision

Terms

Canonical core is the minimum accepted representation from which one specific text release can be reconstructed exactly and in source order without consulting an annotation table.

Annotation layer is separately versioned non-core domain authority and its assertions. It can be deterministic or curated, required or optional, and visible or internal. Those properties do not make it core. It contains two distinct kinds:

• Occurrence annotation is a classification, relationship, alignment or rendering anchored to entities or spans in one or more exact core releases.
• Domain authority is a reusable semantic or orthographic entity such as a lexeme, dictionary sense, compound identity or mapping pattern. It is not tied to a corpus occurrence until an occurrence annotation links it to core.

Domain authority records have their own version, provenance and curation lifecycle. They cannot supply missing core text, and their mutation cannot move or rewrite an occurrence anchor.

Control evidence identifies source releases and files and records import, transformation, validation and curation events. It proves how a core release or annotation-layer fact was obtained but is not itself corpus content or domain authority. Its physical contract belongs to ADR-006 and F2-019.

Cache is a disposable projection or index that can be rebuilt from accepted core, accepted annotations and their versioned contracts. A cache is never an authority for either layer.

Canonical core ownership

For each accepted text release, core owns:

• the work and text-version membership needed to identify that release;
• ordered book and chapter membership;
• paragraph and sentence boundaries and their order;
• every token occurrence in sentence order;
• exact source characters, including punctuation, whitespace, separators, apostrophes, diacritics and Arabic passages;
• the parent and position facts required to reconstruct sentence, paragraph, chapter and complete release text exactly; and
• the stable text-release identity and content digest that select and bind those facts.

The text-release identity and digest are part of the core release envelope because they select exactly which reconstructible fact set is addressed. Source locators, input checksums, importer versions, run records, signatures and validation results that justify that envelope are control evidence. This ownership distinction does not decide whether the later physical schema stores them in adjacent tables.

An implementation may store exact characters as token surfaces, separators, lossless fragments or another representation selected by later ADRs. The invariant is behavioral: reconstruction uses only the selected core release and returns the accepted source bytes, or an explicitly versioned encoding normalization when a byte-exact source contract cannot apply.

Token occurrences and their exact surface values are therefore core. Whether equal surface values share a token_form record, and whether that shared record is core, reference data or a derived index, is reserved for ADR-004. Search normalization, stems, roots, lemmas and senses are not needed for exact reconstruction and are not core.

Annotation-layer ownership

The following facts belong to the annotation layer even when their literal characters occur in core text. The two right-hand columns distinguish reusable domain authority from release-anchored occurrence assertions:

Domain	Core fact	Domain authority	Occurrence annotation
Heading	The source line, punctuation and position	Heading kinds or controlled labels	That the span is a heading, its level, parent and coverage range
Note or hasiye	Marker and body characters in source positions	Note kinds	Marker classification, marker-to-body link and note range
Segment	All selected token occurrences	Segment kinds	Segment boundary, parent and relations
Compound	Token occurrences such as lisan-i haliyle	Compound identity and accepted variants	Boundary, selected variant, child relation and acceptance state
Ottoman	The accepted Latin text and order	Orthographic forms, mapping patterns and display rules	Pattern application and span rendering
Lexical or dictionary	Exact token surface	Lexeme, lemma, root, dictionary entry and sense	Morphological analysis, homograph and selected-sense links
Repetition	Every occurrence’s complete source text	Similarity and relation kinds	Similarity edge, cluster and canonical-occurrence choice
Translation	Each accepted language version’s exact text	Alignment and propagation kinds	Cross-version alignment, omission, split, merge and propagation decision
Reference	Referenced core entities remain unchanged	Citation roles and relation kinds	Citation edge, scope and editorial state

A translation accepted as a text version has its own canonical core and reconstructs without the Turkish release. Its alignment to Turkish is annotation. Likewise, a future independently sourced Ottoman edition could have its own core; the current Latin-to-Ottoman mapping remains a rendering annotation and does not become a second hidden core.

Literal text is never removed from core because an annotation classifies it. A heading marker remains in source order. A note body remains reconstructible. A repeated passage is not deduplicated out of any book. An Arabic phrase may be not_applicable to Ottoman mapping while remaining ordinary core text.

Direction of dependency

The dependency direction is one way:

source release -> canonical core -> occurrence annotation -> cache/API projection
                         |                 ^
                         |          domain authority
                         +------- control evidence -------+

• Every occurrence annotation references all exact core releases that it spans or relates and uses typed core entities or spans. A heading normally names one; cross-version alignment names at least two.
• Domain authority is independently addressable and does not pretend to be a corpus occurrence. Its use in corpus is explicit through an occurrence annotation.
• Core reconstruction cannot join annotation, cache, search or dictionary tables.
• Core rows do not point to optional domain annotations to determine their text, boundaries, parentage or order.
• Specialized annotation tables keep their domain semantics. An unconstrained table_name + record_id relation is not an acceptable substitute for typed, validated ownership.
• Annotation-layer relationships are allowed only through explicit typed contracts, for example a compound occurrence containing a child compound.

The shared span layer is annotation infrastructure, not core. It anchors assertions to immutable occurrence coordinates but cannot own or rewrite the selected text. Its exact coordinate and containment contract is reserved for ADR-005 and F4-001.

Mutation and release lifecycle

An accepted core release is append-only from the perspective of product and annotation roles. Its text, hierarchy, boundaries and positions cannot be updated, deleted, reparented or renumbered in place. A genuine source correction produces an explicit successor core release under ADR-002; it is not applied as an annotation and does not rewrite the historical release.

Domain authority and occurrence annotations have independent lifecycles. They may be generated, reviewed, accepted, rejected, retired or superseded without changing core. Before an annotation-layer identity has stable public references, a release pipeline may rebuild its derived materialization deterministically. Once a published identity is referenceable, correction must preserve history through the lifecycle and supersession contract selected by ADR-002 and the production curation ADR; truncate and recreate is no longer valid.

Deterministic derivation does not grant permission to overwrite curated decisions. Generated candidates and accepted curation remain distinguishable, and a rebuild must replay declared curation inputs rather than infer them from the current table contents.

Caches and denormalized reader/search projections may be replaced at any time. They must declare the exact core and annotation release inputs from which they were built, and stale caches cannot be used as curation evidence.

Required enforcement properties

Later schemas, roles and pipelines must make the boundary enforceable:

1. Annotation writers and curation executors have no direct insert, update or delete capability over accepted core rows.
2. Every occurrence anchor resolves within each declared text release and passes the coordinate, parent-containment and order checks owned by ADR-005.
3. Hard deletion of accepted core is forbidden. Referential actions cannot cascade from core deletion into independently published annotations or evidence.
4. Annotation deletion, retirement or rebuild cannot alter a core row or its reconstruction result.
5. Every annotation import records before and after digests for every core release it references; any unequal or missing value fails the import and publishes no annotation release.
6. Core release digests cover source-faithful hierarchy, boundaries, exact content and order. Mutable timestamps, caches and annotation values are excluded.
7. A reader can always reconstruct core when every annotation subsystem is empty, unavailable or pending. Annotation joins may enrich units but cannot reorder, merge, split, suppress or manufacture core text.
8. Repair tools report core defects and require an explicit successor-release flow; they cannot silently convert an annotation import into core maintenance.

The exact triggers, grants, digest serialization and foreign keys are implementation decisions owned by later schema and migration tasks. Those implementations may be stricter but cannot weaken these properties.

Classification test

When a new datum is proposed, maintainers apply these questions in order:

1. Is the value itself required to reconstruct one accepted text release exactly and in order, or to select that exact fact set by stable release identity and digest? If yes, it belongs to the core release envelope.
2. Does it prove which source, file, input, transformation, validation or curation event justified a fact? If yes, it is control evidence.
3. Is it a reusable semantic or orthographic concept that remains meaningful before any corpus occurrence is selected? If yes, it is domain authority.
4. Does it state what one or more core entities or spans mean, how they relate, how they render or how they align? If yes, it is an occurrence annotation and must bind every referenced core release.
5. Can it be reproduced entirely from accepted core and accepted annotation-layer data without losing authority? If yes, it is a cache or derived materialization.

Importance, UI visibility, query frequency, manual authorship and deterministic generation do not change the answer.

Consequences

Benefits

• Corpus fidelity has one small authority surface and remains testable without Ottoman, dictionary, heading, repetition or translation systems.
• Annotation pipelines can improve independently and fail without damaging reader reconstruction.
• The same span and provenance contracts can support many domains without erasing their semantic differences.
• A Turkish citation can later propagate through reviewed translation alignment while each language keeps an independently reconstructible text version.
• Production curation can retain stable history instead of relying on destructive rebuilds.

Costs

• Some source characters and their interpretation appear in different layers; for example heading text is core while heading hierarchy is annotation.
• Reader queries need explicit joins or projections to combine core with accepted annotations.
• Source correction is deliberately more expensive than annotation correction because it creates a successor release and revalidation work.
• Annotation releases must retain provenance and core-digest bindings even when the algorithm appears completely deterministic.

Alternatives Considered

Put every reader-visible structure in core

This would make headings, compounds and Ottoman improvements core migrations and would couple exact reconstruction to evolving interpretation. It was rejected.

Keep only raw paragraph text in core

Sentence and token occurrence identities are required for durable spans, search and future references. Re-deriving them with changing tokenizers would move anchors and break public identities, so this was rejected.

Treat generated annotations as disposable caches

Deterministic generation does not capture manual acceptance, provenance or external references. Published annotation identities need a governed lifecycle, so only unpublished materializations and query projections are disposable.

Maintain separate unrelated models per annotation domain

This avoids shared constraints but duplicates span, provenance, lifecycle and release logic and makes cross-domain references ambiguous. Domains will share infrastructure contracts while retaining typed semantic tables.

Deferred Decisions

• ADR-002: stable identity, supersession and historical addressability.
• ADR-003: corpus package serialization and release artifact layout.
• ADR-004: token form versus occurrence representation.
• ADR-005: span endpoints, coordinate space and containment.
• ADR-006 and F2-019: shared source, provenance, run and evidence primitives.
• F2-014: table-by-table permanence classification.
• Production curation ADR: concrete moderation states and mutation procedures.

Revisit Conditions

• A source cannot be represented exactly without a fact currently classified as annotation.
• A planned domain requires core reconstruction to depend on an optional layer.
• A new primary text edition needs a different release boundary rather than an annotation over an existing edition.
• PostgreSQL enforcement cannot satisfy the eight required properties without changing the ownership model.