İçeriğe geç

ADR-008: Multilingual Alignment Model

Geliştirme · 0.0.0-dev

Yayın

Doküman
0.0.0-dev
Uygulama
0.0.0

Bu sayfa

Uygulama
0.0.0
  • Status: Accepted
  • Date: 2026-08-18
  • Decision owners: Corpus, translation, annotation and database maintainers
  • Scope: Alignment between independently reconstructible text versions

Munderecat must add English and later languages without changing the accepted Turkish corpus or making one language’s sentence IDs stand in for another language. A translation may split one source sentence, merge several sentences, move a passage, omit content or add translator/editorial material. The source and target hierarchies may also divide books and chapters differently.

The existing English work exposes four different facts that must not be conflated:

  • exact published English source wording;
  • generated output redistributed to resemble Turkish sentence structure;
  • omission/drop and order reports; and
  • human semantic review findings and dispositions.

A generated aligned line may contain a copied Turkish fallback when no English text was found, or it may differ from the exact English source despite passing structural checks. Conversely, an English preface may be valid target text even when Turkish has no counterpart. Empty strings and mirrored line counts cannot express these cases safely.

ADR-001 makes each accepted language version independent core and makes cross-version alignment an annotation. ADR-002 preserves stable identity, ADR-005 supplies exact single-version selections, ADR-006 supplies provenance/closure, and ADR-007 supplies the production review and correction lifecycle. This ADR defines the alignment domain that uses those contracts.

Every accepted language, translation, edition or revision is represented by its own text_version and immutable text_release. A release reconstructs its complete exact text, hierarchy and occurrences without consulting another language or any alignment.

Language metadata uses a controlled registry and a canonical BCP 47 language tag, with script and direction declared where needed. Those labels describe a text version; they do not replace its stable identity. Two English translations of the same work are different text versions, and two releases of one translation are different immutable release envelopes.

A translation release owns its own paragraph, sentence and token occurrence IDs. It never reuses Turkish IDs merely because an aligned output has the same line count. Source/target roles are explicit for one alignment release and do not imply that either text is globally primary. The same model can align two editions in one language.

Alignment is downstream of both cores:

source text release ----> source selections ---+
+--> alignment release
target text release ----> target selections ---+

Deleting every alignment record leaves both releases exactly reconstructible. An alignment import cannot insert, alter, suppress, reorder or fill core text.

The logical model has these identity classes:

Record Meaning
alignment_profile Versioned unitization, scope, disposition and validation rules
alignment_unit One stable alignable selection in exactly one text release
alignment_scope Exact source/target unit universes to which a coverage claim applies
alignment_group One ordered bipartite correspondence or empty-side disposition
alignment_order One immutable release-candidate order manifest for one scope side
alignment_candidate Generated/imported suggestion that has no acceptance authority
alignment_review ADR-007 review of exact candidate/proposal bytes
alignment_release Immutable published graph, coverage roots and policy bindings
propagation_policy Versioned rules for derived cross-version reference traversal

Physical table names are deferred. An implementation may normalize membership rows, but it must preserve these identities and cannot replace a group with unrelated pairwise edges.

An alignment_unit binds:

  • exactly one text version and immutable text release;
  • exactly one canonical ADR-005 text selection;
  • one alignment profile and unit kind; and
  • exact selection witness and provenance.

The selection can be a complete sentence, a clause span, multiple consecutive sentences or another ordered selection permitted by the profile. Its members remain in canonical document order and never combine text versions. The unit does not copy or own the selected text.

The default Turkish profile uses accepted sentence selections. A target translation is not forced into Turkish sentence boundaries. When a semantic alignment requires a clause split or multi-sentence unit, the target gets its own exact selections over its own release. Artificial newlines in generated aligned output are boundary candidates, not target core facts.

Within one side of one scope, accepted units form a canonical ordered non-overlapping inventory under that profile and exactly cover the profile’s unitizable ranges. A finer unit may replace a coarse unit through ADR-002 supersession, but both cannot be effective in the same scope release. Nested spans owned by other annotation domains remain valid and do not create alignment membership.

Changing a unit’s release, selection, profile or unit kind changes its referent and issues a successor ID. Adding confidence, evidence or review notes preserves its ID and appends curation/provenance facts.

An alignment_scope binds exact source/target text releases, one profile and one exact scope extent on each side. It may cover a whole work, one book, one chapter pair or another explicit structural selection. It never assumes that source and target chapter IDs or boundaries correspond.

For each side, the profile expands the extent into an ordered exact core-fragment inventory and applies a versioned coverage mask. Within sentence bodies, mask ranges use ADR-005 coordinates. Core separators or structural fragments outside that coordinate frame retain exact core IDs and witnesses. The mask partitions every scalar/fragment in the extent exactly once into:

  • unitizable, which must be covered gap-free and without overlap by the alignment-unit selections; or
  • structural_excluded, with a controlled profile rule and exact witness.

This scope-to-unit/exclusion partition has counts and roots and is checked before any alignment coverage is evaluated. Omitting a clause, punctuation-bearing range or other unitizable source text from the candidate universe is therefore a closure failure.

The exhaustive partition produces one immutable candidate universe on each side. It classifies every exact candidate atom exactly once as:

  • included as one alignment unit in the scope; or
  • structural_excluded with a controlled reason, evidence and policy rule.

Every unitizable range must become exactly one included unit; it cannot be dropped as a candidate rejection. Structural exclusion is for exact accepted-core material outside the declared alignment layer, such as a separator or structural class excluded by the profile. Extraction headers that are not part of the accepted target text are upstream ADR-006 source-intake exclusions and never enter the scope. Content inside the accepted scope that lacks a counterpart is not excluded; it receives a source-only or target-only alignment disposition.

The scope stores exact extents, mask/profile, ordered unit IDs, controlled exclusions, counts and roots for both sides. A publication claim is only as broad as its declared scope. A chapter-level release may be complete for that chapter while the rest of the work remains unavailable; it cannot advertise whole-work coverage.

An alignment_group has:

  • one stable group ID and owning alignment scope;
  • one non-empty ordered source member list, one non-empty ordered target member list, or exactly one permitted empty side;
  • contiguous zero-based member positions on each non-empty side;
  • one disposition and derived cardinality shape;
  • exact evidence, review state and confidence facts; and
  • lifecycle/supersession state under ADR-007.

A correspondence is one group, not a Cartesian expansion. If source units [S1, S2] jointly correspond to target units [T1, T2, T3], the authority is one 2:3 group. Five or six pairwise edges would falsely claim independent equivalence and could not reconstruct the reviewed grouping.

Members are unique within a group. On each non-empty side they occupy one contiguous, non-interleaving interval of that side’s unit inventory: a unit between the first and last member cannot belong to another group. In a complete release each unit participates in exactly one effective group on its side. Cross-scope reuse requires a separate group bound to that scope/release; it does not mutate the historical group.

Exactly three accepted dispositions exist at the shared layer:

Disposition Source members Target members Meaning
correspondence 1 or more 1 or more The selected contents correspond under review
source_only 1 or more 0 Source content has no target counterpart in scope
target_only 0 1 or more Target content has no source counterpart in scope

For correspondence, shape is derived and validated rather than independently edited:

Source count Target count Shape
1 1 one_to_one
1 2 or more split
2 or more 1 merge
2 or more 2 or more complex

source_only and target_only carry a reviewed reason code such as translation_omission, translator_addition, editorial_matter, version_scope_difference or not_applicable, plus explanatory evidence. Domain profiles can extend reason codes without inventing a non-empty counterpart.

unresolved, missing_unreported, needs_review and low-confidence states belong to candidate/review workflow. They are not accepted dispositions and block complete publication for the affected scope. A rejected candidate likewise does not remove the underlying unit from coverage; another accepted group or explicit disposition is needed.

A target member must resolve to exact text in the declared target release and carry an exact target-source witness. A placeholder, empty string, generated paraphrase, normalized surrogate or text copied only from the source side without that independent target witness cannot satisfy it. Textual equality alone is not rejection: Arabic, proper names, quotations or other source-script wording legitimately retained in the target source remains valid target text when it round-trips that release and provenance.

When Turkish content has no English counterpart, the accepted result is source_only with no target members. The reader may deliberately display Turkish as a UI fallback, but the response labels it as source fallback and does not return it as English text or English coverage.

When English source content has no Turkish counterpart, it remains exact English core and receives target_only when inside the alignment scope. It is not discarded merely to force mirrored structure. Material determined not to belong to the accepted English text release is handled earlier by source intake exclusion, not by alignment omission.

Generated redistributed output can propose selections only when each proposed target selection round-trips exact target source characters. Any substitution, duplication, untraceable wording or unsupported fallback fails candidate promotion. The raw candidate remains provenance/negative evidence if required.

One immutable alignment_order record per side is the sole order authority for an alignment release candidate. It binds the scope, side and complete ordered array of all effective group IDs having members on that side. Array position is the only group rank; groups store no second rank field. The array is unique, gap-free and validated against the groups’ non-interleaving unit intervals and canonical document order.

reordered is not guessed from filenames or stored as an unverified label. It is a derived relation when the relative target order of two or more correspondence groups in the two order manifests differs. A release records the exact inversion/reorder evidence and reviewed explanation required by its profile. Source-only and target-only groups occur only in the manifest for their non-empty side.

Order coordinates do not change either core. They explain traversal and permit the reader to show that the aligned passage occurs elsewhere. Inserting another group or correcting order creates successor order records and an alignment release while preserving unchanged group IDs. Correcting group members still creates a successor group. Neither operation reorders text.

Every alignment release proves two-sided coverage over its scope. On each side the included unit inventory is partitioned exactly once into rooted, disjoint accepted_membership and unresolved inventories. The release validates:

  1. accepted source units appear exactly once as source members;
  2. accepted target units appear exactly once as target members;
  3. unresolved units appear in no effective group and retain exact candidate/review reason and evidence;
  4. accepted plus unresolved is exactly the immutable included inventory on each side;
  5. every effective group has a valid disposition/cardinality, unique members and non-interleaving intervals;
  6. each side has exactly one complete order manifest consistent with its intervals and explicit reviewed reorder evidence;
  7. scope-to-unitizable/exclusion closure is gap-free over both exact scope extents;
  8. excluded candidate-universe records have controlled reasons and evidence;
  9. no group references a unit outside the exact scope/releases; and
  10. no competing candidate is counted as accepted coverage.

The release emits exact sets and roots for accepted membership, unresolved units, missing/gapped extent ranges, duplicate memberships, invalid empty sides, order conflicts, stale selections and orphan bundled evidence. A release is complete only when both unresolved sets and every failure set are empty. A deliberately partial release publishes both total partitions, its exact scope boundary and status; no unit is silently uncovered and no unresolved unit is misreported as omission or addition.

Coverage counts are not semantic quality scores. A fully covered release may contain reviewed omissions and additions. Conversely, high similarity does not close one missing disposition.

The following are separate immutable facts:

  • exact source and target text artifacts/releases;
  • generated/imported alignment candidate and tool/config/run;
  • aligned-output artifact used as candidate evidence;
  • drop, missing, order or validation report;
  • human review finding and its disposition;
  • accepted ADR-007 curation decision and transaction; and
  • published alignment release and closure attestations.

Confidence ranks candidates for review. It never changes review state or grants publication. A structurally valid line-count match can still be semantically wrong; a human-approved mapping can still have modest machine confidence. Both values remain queryable without one overwriting the other.

Only committed, content-addressed and review-disposed legacy artifacts can support an import proposal. Untracked failed reports, ignored prompt caches and local working-tree state are not release inputs. Legacy APPROVED or MINOR_ISSUES labels are evidence, not automatic Munderecat acceptance; the import freezes exact artifacts and uses the ADR-007 lifecycle.

ADR-006 output and used-source closure apply to alignment releases. Bundled translation evidence must support an active group, required decision, reproducibility artifact or fixture. Raw duplicate Turkish trees, full prompt caches and unused reports are not retained merely because they existed in the legacy workspace.

Every addressable class has an explicit identity contract:

Class Identity-bearing facts Identity-changing correction
alignment_profile Semantic rule payload/version and digest New profile ID
alignment_unit Profile, text release, exact selection and unit kind New unit ID
alignment_scope Profile, relation/direction, source/target releases, exact extents and both candidate-universe/exclusion roots New scope ID
alignment_group Scope, disposition and ordered source/target member IDs New group ID
alignment_order Scope, side and complete ordered group-ID array New order ID
alignment_candidate Producing run/output occurrence and exact proposal payload digest New candidate ID
alignment_review Exact proposal revision/digest, reviewer actor and decision payload New review ID
alignment_release Parent, scope/profile, group/order/coverage roots and accepted transaction inventory New release ID
propagation_policy Semantic rule payload/version and digest New policy ID

Confidence, reviewer commentary, provenance display labels and lifecycle/publication status do not redefine units or groups. Additional evidence appends immutable control facts. A correction to reviewed bytes is a new review, not an edit to the old decision.

Bootstrap identities accepted in a canonical package use ADR-002 deterministic UUIDv5 paths declared by their entity-kind contract and are stored in that package. Public records created by interactive curation use generated-once UUIDv7. Persisted pipeline candidates use their declared package-deterministic channel or generated-once channel; the channel is immutable after issuance. Once issued, no ID is recomputed from normalized or translated text.

Changing a member, boundary or disposition creates a new group and an ADR-002 replacement/split/merge/restructure event as appropriate. A source/target role, release, extent, profile or universe-root change creates a new scope first; groups under the old scope remain historical. Order-only change creates a successor order and release while preserving unchanged groups. Correcting only evidence or confidence appends a curation event under the same subject ID. Retiring an alignment does not retire either text unit or release.

If either core text release is superseded, existing alignment IDs remain bound to the historical releases. Alignment to the successor release requires new selections, units, groups and a reviewed alignment release. Similar text or a successful crosswalk cannot silently retarget historical IDs; it can generate migration candidates with provenance.

An immutable alignment_release binds:

  • exact source/target text version and release IDs/digests;
  • direction/relationship kind and alignment profile version;
  • source/target exact extents, coverage masks, candidate-universe, exclusion and unit inventory roots;
  • both accepted-membership and unresolved partition roots;
  • ordered effective group inventory and exact source/target order-record IDs/roots;
  • review/curation policy and transaction inventory;
  • coverage report, graph root and provenance closure attestations; and
  • parent alignment release when publishing a correction lineage.

Publication uses ADR-007 atomic compare-and-publish and authority/freeze guards. The graph, both coverage sides, review decisions and manifest become visible together or not at all. A target text release cannot be published as translated merely because its alignment is pending; text availability and alignment availability are separate states.

A clean rebuild reconstructs both cores independently, recreates units from exact selection IDs, replays the ADR-007 curation release chain and regenerates groups/order/ coverage from the immutable inventories. It must reproduce the declared extent/mask, unit, accepted/unresolved partition, graph, order and coverage roots before replacing a projection. It never reads a duplicate Turkish source tree or a legacy runtime path.

References always retain the exact entity/selection originally cited. Alignment can derive a candidate presentation in another text version; it does not rewrite, clone or silently redirect the authored reference.

Every propagation request names:

  • source reference and exact source text/alignment release;
  • target text version/release;
  • alignment release and versioned propagation policy;
  • allowed review state, disposition, shape and confidence classes; and
  • whether multi-hop traversal is permitted.

The result includes the complete traversed group/unit route and orthogonal fields:

Field Closed values
route_status resolved, no_counterpart, unavailable, ambiguous, integrity_error
anchor_precision exact_unit, covering_unit, not_applicable
destination_cardinality zero, one, many, not_applicable
group_disposition correspondence, source_only, target_only, not_applicable
traversal_direction source_to_target, target_to_source

A sub-unit citation through a split can therefore be resolved, covering_unit, many, correspondence without losing either precision or cardinality. A traversal starting from the non-empty side of an empty-side group returns no_counterpart, zero and the group’s original disposition/direction. There is no reverse lookup from an empty side because no unit exists there; starting from the English member of a target_only group and traversing toward Turkish is the valid reverse no_counterpart case.

One-to-one alignment does not imply word-level equivalence. A citation smaller than its aligned unit cannot be reported as an exact target span unless a separately reviewed finer alignment supports it.

Direct pairwise alignment is authority. A Turkish-English and Turkish-Arabic path does not automatically establish an English-Arabic alignment. Multi-hop composition, when a policy permits it, is a derived route that exposes every hop, release, precision loss and ambiguity. It never becomes a new accepted edge without ADR-007 review.

Multi-hop composition is fail-closed and deterministic:

  1. every hop must be a directly accepted group in the exact declared releases;
  2. any invalid hash, graph, membership or coverage fact yields integrity_error, no destination set and precedence over every other status;
  3. a missing/unpublished required release yields unavailable when integrity is valid;
  4. reaching an empty-side disposition yields no_counterpart and no destination;
  5. incompatible accepted routes or target sets yield ambiguous; the policy cannot silently choose one;
  6. destination sets are composed by exact IDs and never narrowed from many to one;
  7. anchor_precision is exact_unit only when every hop is exact, otherwise covering_unit; and
  8. repeated release/group nodes or an unbounded path yield integrity_error.

Navigation works in both directions for correspondence groups, while original disposition and traversal direction preserve empty-side meaning. Reversal never changes a source-only omission into a target-only addition.

Storage and APIs must keep these questions distinct:

  • reconstruct one exact text release;
  • list accepted alignment units for one release/scope;
  • fetch one historical alignment group and its supersession summary;
  • resolve current alignment under an explicit release/policy;
  • report two-sided coverage, omissions, additions and reorder evidence;
  • inspect candidate confidence and human review separately; and
  • derive a policy-bound reference propagation route.

pending, partial, unavailable, no_counterpart, ambiguous, rejected and integrity_error remain distinct from original source_only/target_only group dispositions. An empty response does not choose among them. F2-010 and F2-020 will define concrete envelopes and authorization.

Implementations must prove at least:

  1. Each source and target text release reconstructs to its own certified exact digest with every alignment table empty.
  2. Every scope extent has gap-free exact mask/unit/exclusion closure; every unit selection resolves in one bound text release and round-trips its exact witness. Unsupported fallback is rejected while independently sourced identical/Arabic text remains valid.
  3. Group fixtures validate 1:1, 1:n, n:1, n:m, source-only and target-only cardinalities without pairwise expansion.
  4. Empty/empty groups, duplicate members, cross-release members, repeated unit membership and contradictory dispositions fail closed.
  5. Both included inventories partition exactly once into accepted membership or rooted unresolved sets; only empty unresolved/failure sets permit complete status.
  6. Sole order records round-trip monotonic and reordered fixtures and reject missing, duplicate or interleaving group intervals and unexplained inversions.
  7. Confidence changes cannot accept a candidate; review/curation fixtures enforce exact proposal bytes, quorum, provenance and historical supersession.
  8. Updating either text version leaves historical alignment IDs resolvable and requires new reviewed units/groups for the successor release.
  9. Clean rebuild reproduces unit, group, side-order, coverage and provenance roots without a legacy path or duplicate Turkish corpus.
  10. Propagation fixtures combine status, precision, cardinality, disposition and direction without loss and always retain the original reference.
  11. Multi-hop propagation obeys status precedence, never narrows target sets, exposes every hop and never materializes a direct accepted edge without review.
  12. Source/target core reconstruction hashes are identical before and after alignment import, correction, retirement and rebuild.
  • English and later languages can be added without destabilizing Turkish identity or reconstructors.
  • Split, merge, reorder, omission and addition remain exact, reviewable facts instead of being hidden by mirrored line counts.
  • Existing English alignment work can contribute high-value candidates and evidence without importing its duplicate Turkish tree or trusting altered output as source.
  • Sentence/span-level references can later appear across languages with explicit precision and policy rather than guessed redirects.
  • Coverage and semantic confidence become measurable independently.
  • Every translation needs its own certified text release and stable unitization.
  • Many-to-many groups and two side-order projections are more complex than one foreign key from a Turkish sentence to an English line.
  • Omission/addition and partial coverage require explicit reviewed records.
  • A successor translation release requires new alignment identities even where wording appears unchanged.

This cannot represent split, merge, reorder or independently corrected translation text and makes English reconstruction depend on Turkish structure. It is rejected.

This loses exact target provenance, duplicates text for merges, invents values for omissions and cannot reconstruct the published translation. It is rejected.

A Cartesian edge set asserts equivalences that were never reviewed and loses grouping and order. One bipartite group is required.

UI fallback may be useful, but persisting it as target text falsely claims translation coverage. A source-only disposition plus labelled presentation fallback is required.

Infer acceptance from confidence or legacy status labels

Bölüm başlığı “Infer acceptance from confidence or legacy status labels”

Similarity, line-count validation and legacy review reports are evidence, not current Munderecat authority. ADR-007 review is required.

Infer alignment transitively through a pivot language

Bölüm başlığı “Infer alignment transitively through a pivot language”

Translation choices and granularity do not compose losslessly. Multi-hop routes may be derived under policy but cannot become accepted direct alignment implicitly.

  • F2-010 and F2-020 define public API/RPC envelopes, pagination, authorization and empty/pending/unavailable behavior.
  • F2-018 defines compatibility classes and final approval matrix.
  • F2-019 defines physical provenance/curation serialization shared by alignment.
  • F7 defines source extraction, exact English text packaging, unitization, alignment candidate import, review migration and concrete propagation policies.
  • Reference graph phases decide when a derived propagation route may be presented or promoted into a reviewed cross-language reference assertion.

Revisit this ADR if:

  • a reviewed translation relation cannot be represented by ordered selections and one bipartite group;
  • a target source cannot be reconstructed independently while preserving its accepted publication wording;
  • a required annotation legitimately needs one unit in multiple effective groups within the same scope and release; or
  • practical propagation needs a precision class that cannot be expressed without claiming unsupported sub-unit equivalence.