ADR-005: Text Span Coordinate Contract

• Status: Conditional; accepted only when F2-005-W01 is accepted and landed on main
• Date: 2026-08-18
• Roadmap task: F2-005
• Decision owners: schema, corpus and annotation maintainers

Context

Compounds, segments, headings, notes, repetitions, translations and references all select exact portions of accepted text. Their semantic tables differ, but allowing each domain to invent coordinates would make overlap, correction, hover and future citations disagree about what was selected.

ADR-001 classifies shared spans as annotation infrastructure: they anchor assertions to immutable core but cannot own or rewrite source text. ADR-002 requires boundary changes to issue new occurrence-annotation identities. ADR-004 supplies stable sentence and token occurrences while preserving punctuation, whitespace and literal content that may not have a token form.

A token-ID-only range is insufficient for punctuation, note markers and subtoken selection. A raw byte or UTF-16 offset is not portable across PostgreSQL, Python and JavaScript. One range stretched across sentences would also hide hierarchy joins and make source-order validation ambiguous.

Decision

Primitive span

A text span is one immutable, non-empty, contiguous interval in one reconstructed core sentence occurrence. Its canonical identity key consists of:

• coordinate profile ID;
• core release ID;
• sentence occurrence ID;
• zero-based inclusive start_scalar; and
• zero-based exclusive end_scalar.

Every span also carries the core release digest and text version ID as required, immutable consistency bindings. One accepted core release ID resolves to exactly one digest, and a sentence occurrence in that release resolves to exactly one text version. A disagreement is integrity_error; it cannot create or reuse another span ID. Digest and text version are therefore not additional uniqueness dimensions.

The interval is written [start_scalar, end_scalar). It is valid only when:

0 <= start_scalar < end_scalar <= sentence_scalar_count

A primitive span never crosses a sentence, text version or core release. Whole sentence, paragraph and chapter entities remain directly referenceable core targets; a domain may also use spans when it specifically needs selected text.

Zero-length insertion points are not spans. A future point-anchor kind may represent editorial insertion positions without weakening the non-empty span invariant.

Coordinate profile

The v1 profile is unicode-scalar-half-open-v1. Offsets count Unicode scalar values in the exact unnormalized sentence sequence reconstructed under ADR-004. They do not count UTF-8 bytes, UTF-16 code units, locale characters, grapheme clusters or display columns.

The accepted corpus and package use strict UTF-8 and reject U+0000 as well as unpaired surrogates. This is the exact string repertoire supported by PostgreSQL UTF-8 text; invalid byte sequences are also rejected. A supported runtime converts its native string indexing to scalar offsets before reading or writing coordinates. It must not normalize, case-fold, strip diacritics or replace apostrophes while converting.

Normative v1 slicing reconstructs the exact sentence first and then uses:

• PostgreSQL substring(text FROM start_scalar + 1 FOR end_scalar - start_scalar), with char_length for scalar count;
• Python text[start_scalar:end_scalar]; and
• JavaScript Array.from(text).slice(start_scalar, end_scalar).join("").

All three results must encode to identical UTF-8 bytes. Database collation does not participate in counting, slicing, equality or hashing.

A scalar boundary may fall between a base character and combining mark. The shared contract remains exact and lossless; a domain or UI may impose a content-addressed grapheme-boundary policy for user-authored selections. Such a policy is an additional validation rule, not a new coordinate space and not permission to move stored boundaries silently.

Changing offset unit, normalization behavior or half-open semantics requires a new coordinate profile and new span identities.

Exact excerpt witness

Every span records three independently recomputable witnesses:

• selected scalar count, equal to end_scalar - start_scalar;
• exact UTF-8 byte count of the selected scalar sequence; and
• lowercase SHA-256 of those exact UTF-8 bytes.

The excerpt itself is reconstructed from core and is not duplicated as authoritative annotation text. A stored display excerpt is a replaceable cache. Any offset, count or hash disagreement means integrity_error; normalized or visually similar text cannot repair the span.

The core release digest is an explicit binding even when the sentence ID was preserved into a successor release. A span cannot resolve against whichever release happens to be current.

Every external or imported offset request supplies its expected core digest and all three source-side excerpt witnesses. The authority recomputes them from accepted core and rejects a missing or mismatched value before ID lookup, reuse or issuance. It never fills absent expected witnesses from current core, because that would validate the wrong fragment selected against normalized or stale authoring text. Span issuance holds one immutable core snapshot and verifies its digest both before and after the atomic operation.

ADR-004 body coordinates

For one reconstructed sentence, token occurrence body intervals are derived without heuristics. Let cursor start at the scalar length of sentence initial_text. For each occurrence in position order:

body_start = cursor
body_end = body_start + scalar_length(body)
unit_end = body_end + scalar_length(trailing_text)
cursor = unit_end

cursor must equal the sentence scalar count after the final occurrence. These intervals cover form and literal bodies; initial and trailing text remain addressable by scalar spans even though they are not synthetic token forms.

Alignment classes

Every span declares one alignment class:

• scalar: arbitrary valid scalar boundaries;
• occurrence_body_range: starts at one form-or-literal body start and ends at a same-sentence body end, covering contiguous occurrence positions; or
• token_body_range: the same rule, with every covered occurrence required to be a form occurrence.

A scalar span carries no occurrence-boundary witnesses. Every aligned span requires both first and last occurrence IDs as validation witnesses, with first_position <= last_position. The witnesses must belong to the bound sentence/release; every integer position in their inclusive interval must exist exactly once; and the first body start and last body end must equal the span’s exact start/end scalars. A mismatch fails; an importer never snaps a scalar boundary to the nearest token.

For token_body_range, the selected text includes exact punctuation/whitespace in the trailing_text of intermediate occurrences, but excludes trailing content after the last body. This gives compounds their complete internal spelling without capturing the next separator.

Domain contracts may require a stricter class. Compound occurrences normally require token_body_range; a punctuation marker may require scalar; a structural literal may use occurrence_body_range. A looser domain cannot relabel a misaligned span as token-aligned.

Token membership projection

Ordered span membership is derived from the validated coordinates and ADR-004 body intervals. For an aligned range, members are exactly the contiguous occurrence positions from the first witness through the last witness.

A physical span_occurrence_members table or API array is a reproducible projection. It includes form and literal occurrences when the alignment class permits them. A span_token_members projection contains only form occurrences and is complete for a token_body_range. Neither projection has independent public identity or may disagree with the span. Rebuilds may replace them after recomputing the same ordered occurrence IDs. Domain-specific member roles, such as a compound connector or child term, live in typed domain tables and do not alter geometric membership.

For a scalar span, APIs may report body relationships as contained, partial or disjoint; they cannot manufacture token membership by rounding partial boundaries.

Shared anchor identity and reuse

Within one coordinate profile, core release and sentence, one exact coordinate pair has one text span ID. All annotation domains reuse that anchor rather than creating coordinate duplicates. The unique coordinate key is:

(coordinate_profile_id, core_release_id, sentence_id, start_scalar, end_scalar)

The alignment class and occurrence witnesses are deterministic properties of those coordinates under the bound core release. The strength order is token_body_range > occurrence_body_range > scalar. A span exposes the strongest class it satisfies; a caller cannot create a second ID merely by requesting a weaker class.

shared-anchor-registry-v1 is the sole span and selection issuance authority. Domain packages and runtime matchers submit canonical keys to it but never mint shared-anchor IDs. The registry serializes atomic create-or-reuse against the unique key, so two writers cannot publish different IDs for one anchor.

For each core release, the registry may open exactly one deterministic bootstrap window before domain publication. One registry-owned anchor package declares one immutable identity_release_key and uses these ADR-002 UUIDv5 path grammars:

kind = "text_span"
path = [identity_release_key, coordinate_profile_id, core_release_id,
        sentence_id, start_scalar, end_scalar]

kind = "text_selection"
path = [identity_release_key, coordinate_profile_id, core_release_id,
        "members", member_span_id...]

The member IDs in a selection path are the exact canonical ordered sequence. The UUID strings use lowercase canonical RFC text, scalar offsets use ADR-002 safe JSON integers, and no placeholder or omitted path element exists in encoded input. Core digest and text version do not appear because the canonical key determines both as consistency bindings. The registry closes and records the bootstrap before accepting curation. Domain package builders must consume this anchor package; separate packages cannot declare their own anchor identity release key or independently derive shared IDs.

The first anchor created after bootstrap closure uses generated-once UUIDv7 through the same registry. If a submitted canonical key already exists, every writer reuses its stored ID and issuance channel. A proposed ID or channel that conflicts with the stored row aborts the whole publication; it is never remapped silently. Later packages preserve every issued ID instead of reminting according to the current import path.

After issuance, profile, release, sentence and boundaries are immutable. Correcting any of them creates or reuses the correct new span and creates a successor domain annotation. Existing annotations and references remain bound to the historical span. A domain correction does not globally supersede a shared span that remains a valid geometric anchor for other annotations; anchor lifecycle changes require a defect in the anchor’s own identity-bearing facts and an explicit registry event.

Text selections

A text selection composes one or more primitive spans when selected content is disconnected or crosses sentence boundaries. It owns:

• stable selection ID;
• one coordinate profile, core release and text version;
• one or more ordered member span IDs; and
• contiguous zero-based member positions.

Every member must bind the same profile, release and text version. Members are strictly ordered by canonical document order and cannot duplicate or overlap. Two touching spans in the same sentence are coalesced into one primitive span, preventing multiple coordinate encodings of the same selection. Gaps and sentence boundaries remain explicit.

Within one profile and core release, one canonical ordered member sequence has one selection ID. The unique selection key is the exact ordered array of member span IDs after validation and coalescing. A writer reuses an existing selection with that key; it cannot issue a duplicate by changing a label, domain or presentation separator.

A selection never combines languages, text versions or core releases. Translation alignment, repetition and cross-release projection relate two or more independently valid selections through typed domain edges. Those relations bind every involved core release under ADR-001.

Changing member identity, order or cardinality creates a new selection identity. Members and historical selections are never rewritten in place.

Overlap, nesting and containment

Primitive geometric predicates are defined only for spans sharing coordinate profile, core release, text version and sentence. Any pair outside that frame is non_comparable; equal numeric offsets across different frames do not overlap, contain or touch.

Primitive spans may be equal only through shared-anchor reuse. Distinct spans may be disjoint, touching, partially overlapping or nested. No global exclusion constraint forbids overlap because valid cases include a long compound and its child compound, a note marker inside a heading and intersecting editorial analyses.

Geometry does not create semantic parentage. For example, both hamele-i ars and hamele-i ars ve yer may be valid nested compound occurrences, while the larger range does not automatically become the smaller compound’s semantic parent.

An explicit typed containment relation must:

• state the owning domain and relation kind;
• connect compatible entity kinds;
• bind all involved releases;
• use proper geometric containment where its kind claims containment; and
• remain acyclic.

Shared v1 validation applies geometric containment only to primitive spans in one comparable frame. Generic containment between selections is forbidden: a domain that needs it must later define typed member-union semantics, gap behavior and a dedicated validator rather than comparing first and last numeric offsets.

Equal-coordinate semantic assertions are separate annotations over one shared span, not parent/child spans. Partial overlap is reported for review only when a domain’s own policy forbids it; shared span validation preserves it.

Examples

Given the reconstructed sentence fragment:

lisan-i hal ile

and form bodies lisan-i, hal, ile, the compound lisan-i hal is a token_body_range from the first body’s start through the second body’s end. It includes the internal space owned by the first occurrence and excludes the space after hal.

For:

(asm)

the asm token body can be token-aligned while the opening and closing parentheses are independently selectable scalar intervals. Selecting the full (asm) text uses a scalar span; no punctuation token form is invented.

A punctuation-only sentence such as *** has an ADR-004 literal occurrence and can be selected as an occurrence_body_range. A nested pair such as hamele-i ars and hamele-i ars ve yer produces two valid spans. Whether both are accepted compounds is a compound-domain decision, not a coordinate decision.

Annotation ownership

A text span or selection states only where. It does not state:

• that the text is a compound, heading, segment, note, repetition or translation;
• which lexeme, root, sense or Ottoman rendering applies;
• whether an overlapping candidate is canonical;
• what a selected passage means; or
• which selected entity should be preferred or current.

Specialized annotation tables own those assertions and reference span/selection IDs through typed foreign keys. An unconstrained table_name + record_id target remains forbidden. Annotation writers cannot update core text, token occurrences or core positions while creating a span.

Generated candidate spans and accepted annotations remain distinct. The existence of a geometrically valid anchor is not evidence that any semantic interpretation was accepted.

Validation and publication

Before an annotation release is publishable, shared validation proves:

1. Coordinate profile, core release, text version and sentence all resolve and agree.
2. Scalar bounds are non-empty and within the exact reconstructed sentence.
3. Scalar count, UTF-8 byte count and excerpt hash match independently reconstructed core text.
4. A scalar span has no occurrence witnesses; an aligned span has both required witnesses and exactly matches ADR-004 body intervals.
5. The sole registry atomically reuses coordinate/selection keys, never aliases two keys to one ID and aborts any ID or issuance-channel conflict.
6. Selection members use one release/version/profile, have contiguous member positions, are source-ordered, non-duplicated, non-overlapping and canonically coalesced.
7. Explicit containment edges satisfy typed compatibility, comparable-frame geometry and acyclicity; generic selection containment is absent.
8. Span issuance and every occurrence annotation record the bound core digest before and after the atomic operation; any difference publishes nothing.
9. Superseded spans and selections remain directly resolvable, and no current-view resolver silently changes authored historical targets.

Validation occurs against one immutable core snapshot in the same staging/import boundary as the annotation release. A current-release alias, search index, cached excerpt or frontend string is not an anchor authority.

Query and API behavior

Span APIs return at least:

• span ID, coordinate profile and core release ID;
• text version and sentence ID;
• half-open scalar boundaries;
• exact reconstructed excerpt plus scalar/byte/hash witnesses;
• strongest verified alignment class;
• first/last occurrence witnesses when aligned; and
• lifecycle/supersession status without automatic retargeting.

Selection APIs return ordered member spans without joining their excerpts with an implicit space or newline. A domain renderer chooses presentation separators; exact selected text remains the ordered list of exact fragments.

Clients persist span or selection IDs, not excerpts or normalized keys. APIs that accept ad hoc offsets require an explicit coordinate profile, core release, expected core digest, expected excerpt scalar count, expected UTF-8 byte count and expected excerpt SHA-256. Missing or stale values fail before lookup or issuance; the server does not synthesize source-side expectations from current core.

Consequences

Benefits

• Every annotation domain shares one exact, cross-runtime coordinate space.
• Token compounds remain efficient while punctuation, subtoken and literal content stay addressable.
• Nested compounds and other overlaps are preserved without weakening domains that require non-overlapping canonical selections.
• Cross-sentence and multilingual work becomes explicit composition rather than an ambiguous range across hierarchy joins.
• Boundary corrections preserve historical citations through immutable IDs and supersession.

Costs

• JavaScript and other UTF-16 runtimes must convert code-unit indexes to scalar offsets at API boundaries.
• Exact excerpt witnesses and redundant token anchors add validation/storage cost.
• Multi-sentence annotations require selection/member records rather than one range.
• Domain schemas still need their own overlap, containment and acceptance rules.

Alternatives Considered

Inclusive token start and end IDs only

This is convenient for compounds but cannot address initial punctuation, trailing content or subtoken markers and inherits token-boundary changes. Token IDs remain validated witnesses for aligned scalar spans instead.

UTF-8 byte offsets

Byte slicing is deterministic but awkward and unsafe in normal database/browser text APIs. Exact byte count and hash remain witnesses while scalar indexes are coordinates.

UTF-16 offsets

This favors browser internals but disagrees with PostgreSQL and Python for non-BMP characters. Runtime adapters must convert instead.

Grapheme-cluster offsets

Grapheme segmentation is versioned and presentation-oriented. It is useful as an optional authoring policy but too mutable to be the canonical storage coordinate.

Global document offsets

One number across a chapter makes sentence insertion or hierarchy correction move unrelated anchors and conceals parent containment. Sentence-local intervals keep the blast radius explicit.

One range crossing sentences

Such a range depends on implicit join separators and cannot express disconnected selection. Ordered primitive members make every boundary explicit.

Reject all overlap

This would discard valid nested compounds and intersecting annotations. Overlap rules belong to each typed semantic domain.

Deferred Decisions

• F2-007: concrete production curation and review state transitions.
• F2-008/F2-009: translation and repetition relations between selections.
• F2-012/F2-013: physical annotation ER model and referenceable entity catalog.
• F3/F4: SQL tables, constraints, scalar helpers, selection members and migration of the 16,986 legacy segments.
• F5: compound/member roles, nested-compound policy and Ottoman rendering spans.
• F8: browser scalar/code-unit adapters and visual highlight behavior.

Revisit Conditions

• A supported runtime cannot reproduce scalar slicing and excerpt hashes for golden non-BMP/combining-mark fixtures.
• Required source content cannot be addressed by sentence-local primitive spans plus ordered selections.
• ADR-004 body intervals cannot uniquely validate token-aligned boundaries.
• A domain demonstrates that canonical member coalescing destroys identity-bearing semantics that cannot live in its typed relation.