RESOLUTION.md

Observation resolution in k.LAB

This document describes the observation-resolution path implemented by ResolverService, starting at ResolverService.resolve(Observation, ContextScope). It records the current behavior, the contracts at the resolver/runtime boundary, the state that survives between calls, and known gaps. It is intended as a baseline for changes to resolution logic, dataflow encoding, inter-resolution state, and reentrancy.

The existing resolution API returns a contextual Dataflow describing work within the current knowledge graph. All runtime mutations must result from executing such a resolved plan; new contextualization kinds extend Dataflow/Actuator contracts, not a separate mutation endpoint. This differs from the graph-reproduction dataflow extracted from provenance to reconstruct contents from scratch. See the two contracts.

The companion Observation strategies defines strategy syntax, matching, setup and graph composition. Observable expressions defines the semantic activities that strategies serve. This guide describes their Resolver and Runtime contracts; explicitly identified limitations are not implied capabilities.

1. Executive summary

Resolution is a planning phase between runtime observation registration and runtime contextualization:

A root substantial submission clears inherited client observation focus before reuse lookup, registration, resolution and execution. The observer and resolution constraints remain in scope. Nested instantiated-member submissions retain their collective focus for membership registration; their independent resolution subsequently clears it. Quality submissions retain their bearer.

Registered provisional observations are retrievable by their negative IDs within the owning root transaction, including callbacks from the Resolver that carry a context-observation header. Registration retains them in a temporary lookup registry, not the commit graph: unsuccessful model candidates must not become persisted observations. Transaction graph assets take precedence once compilation adds the selected candidates. The temporary registry is discarded when the root transaction commits or fails, and is not visible to unrelated transactions.

Dataflow carries an explicit ResolutionOutcome: RESOLVED, NO_MODEL, or FAILED. Acknowledgment accepts an error-free empty resolution as NO_MODEL: a substantial exists even when no strategy or candidate contributes an executable explanation. Runtime-owned characterization requires successful discovery with no matching model to accept NO_MODEL; a matching but unresolvable characterization model still fails. Reported resolution errors and execution failures are not suppressed. A no-model plan contains no computation; its isEmpty() is false, including after interface-based JSON transport. Individual substantial semantics also select acknowledgment when a transported observable carries a stale collective description type. ConceptImpl.singular() changes INSTANTIATION/CONNECTION to ACKNOWLEDGEMENT; the inverse collective conversion restores the corresponding creation type without modifying the original concept. This prevents newly instantiated members from being treated as requests to instantiate another collective.

The no-model plan's successful Resolution Activity records the decision. Actual characterization executes a member-bound UPDATE plan before classification/root commit and records CHARACTERIZED only after success.

1. RuntimeService registers an unresolved observation and opens submission/resolution transactions.
2. It calls Resolver.resolve(...), locally or through ResolverClient.
3. ResolverService asynchronously creates a fresh ResolutionCompiler.
4. ResolutionCompiler queries existing runtime knowledge, asks the reasoner for observation strategies, asks resource services for candidate models, validates contextualizers with the runtime, and builds a ResolutionGraph.
5. DataflowCompiler turns the successful graph into nested Actuator objects and ServiceCall computations.
6. The runtime compiles those actuators into executors, commits the resolution transaction, runs contextualization, and commits the submission transaction.

Two different graphs are involved and must not be confused:

• ResolutionCompiler.resolutionCache is a per-call graph of runtime assets used to record parent/child registration during one compilation. It currently has no read side.
• ResolutionGraph is the resolver's semantic proof/plan: observations, strategies, models, and references connected by coverage-bearing edges. A root instance is stored in the context scope, but its intended cross-call catalog is not implemented.

The dataflow produced by one resolution is an incremental plan against the knowledge graph as it exists at that moment. A separate runtime workflow must eventually assemble those fragments, explicitly encode submitted inputs, close internal references, and export a reconstruction dataflow that can recreate an entire selected knowledge graph without resolving each observation again. The extraction and external-execution APIs for that workflow currently exist only as stubs.

The runtime HTTP exchange currently uses Jackson polymorphic JSON for the executable Dataflow. The architectural persistence format is instead the observation language defined by ../klab-languages/org.integratedmodelling.languages.observation/src/org/integratedmodelling/languages/Observation.xtext. The text currently saved in activity metadata is only a demonstrational, incomplete encoding of that language; the runtime does not yet rebuild and re-execute a dataflow from it.

2. Main classes and responsibilities

Class	Current responsibility
ResolverService	Service lifecycle, asynchronous root entry point, context instrumentation, model ingestion, submitted-resource handling, and an incomplete observation-language encoder
ResolutionCompiler	Recursive resolution policy: runtime query, strategies, models, dependencies, coverage, and runtime requirement checks
ResolutionGraph	Mutable intermediate graph and shared context-level resolver state
PrioritizerImpl	Orders models using service defaults overridden by the scope namespace
DataflowCompiler	Converts a successful ResolutionGraph into DataflowImpl and nested actuators
ResolverClient / ResolverController	Remote request construction and asynchronous job transport
RuntimeService	Owns observation registration, transactions, resolver invocation, executable compilation, contextualization, and commit/failure handling
CompiledDataflow	Runtime-side validation, observation binding, storage creation, dependency ordering, and executor construction
DataflowGraph	Planned server-side extraction and adaptation of a cumulative provenance/dataflow graph; currently a stub
DataflowEncoder	Produces the current demonstrational observation-language representation stored for diagnostics/provenance

Important contracts are in Resolver, Dataflow, Actuator, ContextScope, ResolutionConstraint, and Coverage.

3. Entry conditions established by the runtime

The normal caller is RuntimeService.submit(...), not application code calling the resolver directly.

Before calling the resolver, the runtime:

• validates or registers the submitted observation;
• establishes submission and resolution activities and transactions;
• links the observation, context, observer, cohort, and activities in the pending knowledge-graph transaction;
• creates a resolution scope with executing(...).contextualizeFor(observation);
• serializes the active resolution constraints into resolution-activity metadata;
• handles predefined Observation.ContextualizationData itself when the adapter is available, bypassing ResolverService.resolve(...).

Observation IDs are semantically significant:

• id > 0: an existing, resolved runtime observation;
• id == Observation.QUERY_ID (0): a transient query or query result, never a stored graph asset;
• id < 0: a registered but unresolved observation;
• id == Observation.UNASSIGNED_ID (-1): not registered and illegal for normal submission.

The resolver assumes that declareContextScope(...) has already instrumented the context with a root ResolutionGraph. Calling resolve(...), submitResource(...), or getSubmittedResources(...) without that declaration currently leads to a null dereference.

4. Root method: ResolverService.resolve

ResolverService.resolve(observation, contextScope) returns a CompletableFuture<Dataflow> built with CompletableFuture.supplyAsync(...).

No executor is supplied, so work runs on Java's default asynchronous executor (normally the common fork-join pool). Each call constructs:

new ResolutionCompiler(service)
    -> resolve(observation, scope)
    -> ResolutionGraph
    -> new DataflowCompiler(observation, graph, scope).compile()
    -> Dataflow

If the returned graph is empty, the fallback is exact:

• an error-free ACKNOWLEDGEMENT receives a non-empty, computation-free Dataflow with NO_MODEL, meaning its unexplained existence is accepted;
• characterization receives NO_MODEL only after successful discovery finds no model;
• other unresolved mandatory requests receive Dataflow.empty(...), meaning resolution failed.

This is an acknowledgement lifecycle rule for substantials, not a SUBJECT-only exemption.

Exceptions are not converted to resolver notifications here. They complete the future exceptionally. In the normal runtime path, RuntimeService catches that exceptional completion, fails the resolution scope, and substitutes an empty dataflow with an error notification.

Reentrancy consequence

The compiler and mutable resolution graph are now per attempt. The context-level graph acts only as a thread-safe catalog from which each attempt snapshots submitted resources. Calls still run concurrently, but do not share their JGraphT graph, dependencies, service prototypes, observations, or synthetic IDs. Section 11 describes the remaining catalog-commit limitation.

5. ResolutionCompiler: root observation path

5.1 Per-call initialization

A fresh compiler creates a directed resolutionCache, inserts RuntimeAsset.CONTEXT_ASSET, and keeps a reference to the resolver service. This graph is populated by getObservationGeometry(...) and requireObservation(...), but no code consults it to detect recursion, duplicate work, or cycles. It is currently write-only bookkeeping.

The compiler also declares a local minimum worthwhile contribution of 0.15. A second static field with the same value exists in ResolverService but is unused.

5.2 Obtaining the parent graph

The public compiler entry point calls:

resolve(observation, scope, ResolverService.getResolutionGraph(scope))

The retrieved object is the root graph installed in declareContextScope(...). Recursive work usually happens in child ResolutionGraph instances, which share selected maps/lists with that root.

5.3 Positive-ID short circuit

The first recursive observation check is:

if observation.id > 0: return parentGraph

At the root this produces a valid non-empty dataflow with no computation. Inside other branches, positive observations are normally handled earlier as references, so this short circuit is mostly a root “nothing to resolve” path.

5.4 Geometry selection

getObservationGeometry(...):

1. records a parent-to-observation edge in the per-call resolutionCache;
2. starts from observation.getGeometry();
3. if geometry is null and the semantics are dependent and a context observation exists, inherits the context observation's geometry;
4. returns null otherwise.

A null or empty geometry makes resolution return an empty graph without adding an explanatory notification.

The geometry becomes a worldview-bound Scale through GeometryRepository.scale(geometry, scope).

Persisted observations take the inverse path in KnowledgeGraphNeo4j.adapt(...): their encoded geometry definition is decoded through GeometryRepository.get(...). The repository uses Caffeine caches for canonical geometry/scale pairs and merged scales. Scale construction is intentionally performed outside a cache mapping function because Scale.create(...) calls the configured geometry promoter, which reentrantly publishes the same pair through GeometryRepository.put(...). Loading through Cache.get(key, mappingFunction) would therefore attempt a same-key recursive cache update and abort observation adaptation before identity lookup. After construction, putIfAbsent converges concurrent creators on the first published canonical pair, and alternate geometry keys are registered only after the primary load completes.

5.5 Querying knowledge that already exists

Before model resolution, query(...) asks the runtime for existing knowledge only for:

• contextual qualities, using direct positive-ID lookup; and
• enumerable collective substantials: subjects, agents, events, and relationships.

Other semantics receive a synthetic zero-coverage QueryMatch without a runtime call.

For a quality, the compiler builds a probe with the requested observable and scale, then calls scope.getObservation(probe). A match is treated as complete by definition and becomes the positive-ID reference. Quality presence is not estimated by intersecting geometries.

For an enumerable collective, the detached query is:

scope.observation(observable)
    .geometry(requestedScale)
    .query()
    .submit()
    .join()

Although the outer resolver API is asynchronous, this step blocks the resolver worker until the runtime query future completes.

The result is normalized into:

• the returned query observation;
• the observation to reference in the dataflow;
• requested and covered scales;
• a Coverage measured by unioning the covered scale into a zero-initialized requested scale.

A collective query view may remain an ID-0 reference; ResolutionGraph assigns it a synthetic negative key for the duration of graph compilation.

Complete existing coverage creates a reference-only child graph and stops semantic resolution.

Partial coverage is retained as a reference and resolution continues for missingScale(...). Scale exclusion is not universally representable because a Scale is a Cartesian product of extents and not all extents implement exclusion. The compiler checks the proportions of the computed complement. If they do not add up, it deliberately resolves the full requested scale again instead of risking under-resolution.

5.6 Reuse from previous resolutions

After the runtime query, the compiler calls:

parentGraph.getResolving(observable, scale)

It would accept cached resolvables whose gain is at least 0.15, unioning contributions until coverage is complete.

Incomplete: ResolutionGraph.getResolving(...) always returns an empty list and ResolutionGraph.accept(...) does nothing. No previous semantic plan is reused.

5.7 Root strategy loop

The compiler adds a k.LAB provenance constraint to a derived scope, then asks:

Reasoner.computeObservationStrategies(observation, scope)

Strategies are tried in returned order. Each is resolved into its own child graph. Irrelevant coverage is discarded. Accepted results are merged before checking cumulative observation coverage. The loop stops when that coverage is complete. If the request remains incomplete, it returns an empty graph, discarding the partially built graph and its query reference from the result.

Observation.ContextualizationData branches remain in this method but are TODOs. In the standard runtime submission path, direct predefined contextualization is intercepted before calling the resolver, so these TODOs primarily affect alternative/direct resolver use.

6. Strategy operations

For strategy selection, functor behavior, matching gaps, and the replacement composition design, see OBSERVATION.md, especially the current S3c record (Section 2 is historical). In particular, current ResolutionGraph.merge assembles graph structure and coverage; it is not the proposed typed binary merge language operation. Operational id fields now identify producer graphs; inputs maps consumer ports to earlier producer names. Legacy transformation-target strings are retained for old consumers but the new Reasoner does not lower merges to that convention.

The running strategy comparison preserves the currently used eight-strategy document and translates its intended behavior into the proposed contracts. Comma-separated match alternatives are disjunctions in the revised adapter and matcher. The comparison also makes endpoint prerequisites, no-model acknowledgement, and boolean/categorical composition explicit review gates.

Each ObservationStrategy creates a child graph initialized at zero coverage. Operations are processed in declaration order.

6.1 RESOLVE

RESOLVE means resolve another observable:

1. contextualizeScope(...) validates that a dependent has a context observation, reporting a scope error if it does not.
2. It adds a Geometry resolution constraint containing the requested scale.
3. The observable is queried against runtime knowledge.
4. If needed, an unresolved observation is registered for the missing geometry.
5. That observation is recursively resolved.
6. Complete prerequisite results are kept under the operation ID in a strategy-local graph table. They contribute output coverage only if selected as the final producer. Legacy unnamed plans retain their earlier immediate-merge behavior.

contextualizeScope(...) currently does not alter the scale for collective semantics or use the resolutionSoFar argument.

6.2 OBSERVE

OBSERVE searches for models that can produce the operation's observable:

1. contextualize the scope;
2. call ResourcesService.resolveModels(observable, scope);
3. ingest returned namespace documents into runtime Model objects;
4. rank models with PrioritizerImpl;
5. try models in sorted order;
6. attach named prerequisite graphs by input port, intersecting model coverage;
7. merge each relevant model before testing cumulative output completeness.

The initial named-plan subset requires observe to be the final producer. Endpoint coverage cannot complete its output in the absence of a model. Input edges retain port names through DataflowCompiler, including parallel ports referencing the same observation. Full model-input type validation and candidate rollback remain pending; see S3c's continuation prompt.

The notifications carried by the ResourceSet are currently not copied into resolver output.

Context propagation correction (2026-09-09): strategy selection and model lookup retain the requesting scope. When the selected model explains the requested substantial, its computation and dependencies run in a derived scope within(explainedObservation). This includes collective substantials as contexts for their shared dependency plan; it does not implement per-member plans. Quality/process models retain their existing context. Model resolution receives the producer's geometry-constrained scope and then adds the model's namespace/project constraints. A collective dependency's registration clears context only in its derived scope, so subsequent qualities still refer to the original substantial and its explanatory model's lexical constraints.

This fixes the case where a Region model's Elevation/Slope dependencies failed the new context.exists() guard. The guard remains necessary; the Resolver must supply the context.

6.3 APPLY

APPLY sends all operation contextualizables to RuntimeService.resolveContextualizables(...). An empty requirement set rejects the strategy. Otherwise, service prototypes are fetched as needed, and the external requirements are accumulated in the root resolution graph.

Incomplete: the accepted APPLY contextualizables are not emitted by DataflowCompiler. compileStrategy(...) ends with a TODO to add APPLY work to the observation actuator. A strategy can therefore pass capability validation and contribute dependencies without producing the corresponding computation.

7. Model resolution and ranking

7.1 Ingesting model documents

ResolverService.loadModel(...) adapts KimModel into ModelImpl:

• resolves output observables and dependencies with the reasoner;
• copies annotations and metadata;
• records namespace, project, scope, and scenario status;
• converts model resource URNs into a contextualizable;
• appends declared contextualization;
• assigns Coverage.universal().

Actual namespace/model coverage, learners, annotation processing, and processed symbol metadata are TODOs. Assigning universal coverage means model coverage filtering cannot currently express the source model's true spatial or temporal limits unless other code reconstructs them.

7.2 Ranking: the Prioritizer contract

Prioritization orders model candidates for one observe operation. It does not rank observation strategies, change their tiers, establish coverage or execute a model. Resources performs candidate selection; Resolver ingests the returned models, excludes those without a compatible output, prepares their ranking and tries them in order. A highly ranked model can still fail to resolve its dependencies; subsequent candidates may supply all or part of the required coverage.

Inputs, direction and lifetime

PrioritizerImpl receives the request Observable, context observable, requested Scale, context scope and service ranking configuration. It captures lexical constraints and maintains request-local caches for semantic distance, spatial/temporal calculations and criterion maps. Inputs must remain stable during that ranking session. The object is not shared between resolution attempts and is not a service-transport bean.

Semantic compatibility always uses Reasoner.semanticDistance(candidateOutput, request, context). For multi-output models the lowest nonnegative output distance is used. No compatible output gives Integer.MAX_VALUE; Resolver excludes that model even if the semantic-distance ranking criterion is disabled. Reasoner/service errors propagate rather than becoming a poor but usable score. Reasoner.resolves(request, candidate, context) expresses the same capability with request-first arguments. Semantic distance applies to all observables, including inherency and other clauses; predicate heads may admit subsumption while non-predicate heads retain their equality requirement.

One comparison policy

Configuration maps criterion property names to integer priorities. Positive values enable criteria, smaller values run first, and nonpositive values disable criteria. These integers are ordering positions, not weights. Equal priorities use alphabetical property-name order; no weighted aggregation is implied. Unknown keys and null priorities are configuration errors.

The effective ResolutionNamespace constraint in the submission/resolution scope selects the namespace whose KimNamespace.getResolutionCriteria() overrides service defaults. That single policy applies to every candidate, irrespective of its namespace. Candidate Model.ResolutionInfo criteria are descriptive metadata and never choose the comparator. The namespace is retrieved through the existing Resources API even when none of its models are candidates. No namespace constraint or an empty namespace policy means service defaults; an unavailable namespace produces a warning and uses defaults. Retrieval errors propagate.

The policy is captured when the ranking session is created. ResolutionNamespace is a replacing, not accumulating, constraint: when resolution enters a model's dependencies, the model namespace becomes the effective namespace in that child scope. A new ranking session therefore uses that namespace's policy without changing the parent's session. For example, a request in namespace A uses A's order to compare models from A and B; dependencies of a selected B model use B's order. Different namespace policies do not conflict because they govern different ranking sessions.

Members produced by INSTANTIATION are a further explicit handoff: their individual submissions adopt the producing instantiation model's namespace and project, rather than the submission's original namespace. The compiled Actuator carries computationConstraints, keyed by computation index, through interface-based JSON transport. Runtime binds each executor's new outcomes to those lexical constraints and applies them to each member's submission scope. Collective registration, transaction/provenance and other constraints remain inherited; the parent scope is unchanged. When several instantiators contribute members, each member uses its own producer's policy. Absent producer bindings inherit the execution scope; unrelated model metadata cannot select it.

The namespace policy map is a portable bean property using the existing Jackson interface mapping. Current source adaptation does not yet populate it from a namespace ranking declaration; the property defaults to empty. Defining a source spelling and adapting it is separate from this policy-selection contract.

Comparison is lexicographic. Semantic distance is ascending (zero is exact; distances are not capped). All implemented benefit scores descend. Equal score vectors use model URN in lexical order, with missing URNs last. Candidates sharing the same scores and URN compare equal; choosing between different revisions under the same identity requires a version-selection policy outside Prioritizer. An empty criterion order still uses this deterministic identity tie-break. The selected policy is immutable for the session.

Service defaults place lexical scope first and semantic distance second. The remaining configured order is trait concordance, evidence, time specificity, time coverage, space specificity, space coverage, subjective concordance and inherency. Some are vocabulary placeholders, as below; their configuration does not make them implemented. Disabled criteria are absent from ranking maps.

Criteria and score semantics

Criterion	Calculation and direction
im:lexical-scope	Descending: generated model without namespace or active scenario 100; resolution namespace 75; same non-null project 50; other visible model 0. Scope ranking does not grant access.
im:semantic-concordance	Ascending minimum compatible semantic distance over model outputs; exact 0, broader compatible meanings positive.
im:space-coverage	Descending 100 × area(intersection) / area(request) in square metres.
im:space-specificity	Descending 100 × area(intersection) / area(model). A more narrowly applicable model scores higher when it covers the request.
im:time-coverage	Descending 100 × duration(intersection) / duration(request).
im:time-specificity	Descending 100 × duration(intersection) / duration(model).
Trait concordance, inherency, evidence, network remoteness, subjective concordance, reliability	Not implemented as independent criteria; score -1 and listed by unsupportedCriteria(). Inherency already contributes to semantic distance.
Combined scale coverage/specificity/coherency; spatial and temporal coherency	Not implemented; score -1. Aggregation across dimensions and resolution-fidelity policy remain to be defined.

Benefit scores are finite and bounded to 0–100 when available. -1 explicitly means unavailable, not zero coverage. All candidates receive the same unavailable value for an unsupported criterion, so it cannot discriminate. listCriteria() returns active property names in order; unsupportedCriteria() exposes the active unimplemented subset. getRanking(model) computes on demand and returns an immutable criterion map, as does computeCriteria(model). Preparation also computes the single-candidate case, which ordinary sorting would not visit. These are local inspection methods; they do not automatically persist ranking metadata in Activities or Dataflows.

Geometry and time boundaries

Portable model geometry is decoded through GeometryRepository before reading extents. A missing model coverage or absent model/request dimension yields unavailable scores for that dimension. Explicit universal model coverage yields coverage 100 and specificity 0 in a requested dimension; it is distinct from missing coverage. Spatial intersections use the shape's projection-aware operations and square-metre areas. Disjoint or empty shapes score zero. Degenerate/nonfinite area denominators yield unavailable ratios rather than NaN or infinity. Point/line resolution-fidelity ranking requires a separate policy; polygon area ratios do not invent one. Geometry errors propagate; there is no arbitrary positive fallback score after a topology failure.

Temporal intervals use overlap duration, so disjoint intervals score zero rather than receiving a positive distance-based bonus. A bounded request is needed; missing request endpoints yield unavailable scores. Open model endpoints act as unbounded limits: a covering open interval has full coverage and zero duration-specificity. The public computeTemporalCriteria helper retains -1 as the open-model-endpoint sentinel; the model-based path uses nullable bounds so an actual timestamp of -1 is not confused with absence. Reversed bounds are errors. A point request scores coverage 100 when contained by the model interval and specificity 100 only for the same model point; otherwise these point scores are zero. Coherency remains unavailable because sampling resolution is not the same as interval overlap.

Model ingestion currently assigns universal coverage in paths described in section 7.1. Ranking cannot recover a model's true spatial or temporal limits from that placeholder. Coverage scores therefore describe the supplied model contract, not a proof of executable resource availability.

Remaining policy decisions

Define evidence, reliability, locality and subjective-ranking sources and scales before activating those criteria. Specify cross-dimension aggregation and spatial/temporal resolution fidelity before implementing coherency or combined scale scores. Neither deterministic ordering nor a high score substitutes for eligibility constraints such as blacklists, whitelists or UsingModel; their enforcement belongs to discovery/resolution.

7.3 Resolving one model

For each model:

1. add ResolutionNamespace and ResolutionProject constraints;
2. ask the runtime to resolve/validate all model contextualizers;
3. reject the model if requirements are empty;
4. cache runtime ServiceInfo prototypes used later to bind tagged inputs;
5. merge external requirements into the root graph;
6. recursively resolve every model dependency;
7. ignore unresolved optional dependencies and reject the model on unresolved mandatory ones;
8. merge successful dependencies with their stated names.

Geometry/semantic constraints returned with runtime requirements are not filtered here.

8. ResolutionGraph semantics

8.1 Nodes and edges

Nodes are Resolvable objects:

• Observation;
• Observable reference targets;
• ObservationStrategy;
• Model.

Edges point from a target to what resolves it. Each ResolutionEdge contains:

• coverage;
• a local name used to bind model/strategy inputs;
• an observation lookup ID for reference edges.

The graph uses JGraphT DefaultDirectedGraph, which does not allow parallel edges.

Risk: the same node cannot be connected twice to the same parent with different local names or coverage records. Repeated use of one semantic/model object can silently lose edge-specific binding information unless distinct node identities are guaranteed.

8.2 Coverage algebra

A child graph starts:

• at coverage 1.0 for a Model, because all mandatory dependencies intersect its validity;
• at coverage 0.0 for observations, observables, and strategies, because alternatives union their contributions.

The target's native model/observation coverage is then intersected into the initial coverage when available.

Merging a child:

1. copies all child vertices and edges;
2. adds an edge from the parent target to the child target;
3. assigns either the child's resolved reference ID or a synthetic internal ID;
4. intersects coverage for a model parent or unions it otherwise.

The graph reports proportional coverage through getResolvedCoverage() and the complete Coverage object through getCoverage().

8.3 Shared versus child-local state

Child graphs share these objects with their parent:

• observations, used to recover referenced observations by edge ID;
• localResources;
• serviceInfos;
• rootScope.

Dependencies are held only on the root graph and accessed through rootGraph().

Each child has its own JGraphT graph, target, coverage, notification list, and parent pointer.

8.4 What actually survives between calls

ResolverService.declareContextScope(...) stores one root ResolutionGraph under the private __RESOLUTION_GRAPH__ scope-data key. It also reloads persistent submitted resources into that graph's localResources.

What survives between attempts is the context catalog of submitted local resources. Each createAttempt() snapshots that catalog and owns fresh observation/service-info maps, dependencies, graph structure, and synthetic IDs. Child graphs share metadata within that attempt; they do not publish it back to the context catalog.

What does not currently become reusable resolution state:

• successful child graph vertices and edges are never merged back into the stored root graph;
• getResolving(...) is a stub;
• accept(...) is a stub;
• the per-call resolutionCache is discarded;
• no completed dataflow catalog is supplied to DataflowCompiler;
• local submitted resources are stored but not converted into immediate models.

Consequently, the current implementation effectively resolves afresh on each call, apart from runtime knowledge queries and the submitted-resource catalog. Candidate branches within one attempt still share metadata and requirements; candidate-level acceptance/rollback is a separate requirement in the strategy proposal.

9. Dataflow compilation

DataflowCompiler.compile() now copies the following resolution output into DataflowImpl:

• a context-specific name;
• a transport-safe Geometry projection of the requested coverage;
• proportional resolvedCoverage;
• accumulated ResourceSet requirements;
• root actuator computation;
• resolver notifications.

Each actual root node returned by ResolutionGraph.rootNodes() is compiled. A non-observation root is rejected as an illegal state.

9.1 Observation compilation

An observation becomes a reference actuator when:

• its ID is positive; or
• its ID is already in the compiler's per-dataflow catalog.

Otherwise the compiler walks outgoing resolution edges:

• ObservationStrategy creates an OBSERVE actuator and recursively compiles strategy content;
• Observable creates a REFERENCE actuator from the graph's observation lookup.

When strategies and existing references both contribute, references become children of every strategy actuator. When there are only references, they become the returned root actuators.

9.2 Strategy and model compilation

A strategy compiles:

• model children through compileModel(...);
• observation children through recursive compileObservation(...).

A model:

• compiles dependent observations/references as children of the observation actuator;
• adapts its contextualizers into runtime ServiceCalls;
• uses cached ServiceInfo input tags/local names to override a call parameter with an Identifier;
• builds quality sharding hints from @type, @split, @maxSize, @minSplitSize, and @fillCurve annotations.

Contextualizable adaptation supports:

• direct service calls;
• resource URNs;
• according-to/classification/lookup tables;
• expressions;
• literals;
• target metadata (_target, _targetId).

Risk: an unsupported or structurally empty contextualizable produces null, which is added to the actuator computation. Failure then occurs later and less clearly during encoding or runtime compilation.

Incomplete: strategy APPLY computations, transformation-target internal IDs, explicit actuator dependency links, inherited parent dataflow catalogs, and several contextualizable metadata fields remain TODO.

9.3 Fixed while producing this document

Three circumscribed issues were corrected:

1. compilation used to iterate each graph root but repeatedly compile the original requested observation; it now compiles the actual root node and its geometry;
2. resolver requirements and coverage were dropped when constructing DataflowImpl; requirements, a plain Geometry projection, and proportional coverage are now copied;
3. unconditional debug/profane System.out output was removed from the graph/compiler.

Coverage is resolver-local state and must not cross a service boundary. An initial version of fix 2 assigned the live CoverageImpl to DataflowImpl. Remote JSON decoding then failed because the polymorphic decoder attempted to instantiate that runtime implementation. DataflowImpl and EmptyDataflow now type their coverage property as Geometry, matching the Dataflow interface, and DataflowCompiler explicitly calls coverage.as(Geometry.class). The numeric fraction remains available separately as resolvedCoverage.

This also establishes a broader DTO rule for resolution work: transported DTOs and their nested values should be ordinary mutable POJOs with no live resolver/runtime implementation state. Records are deliberately avoided in this implementation.

These fixes are covered by DataflowCompilerTest and DataflowCoverageSerializationTest.

9.4 Incremental resolver output versus context reconstruction

The dataflow returned by one call to the resolver is incremental. It is compiled for the current runtime context and may contain REFERENCE actuators pointing to observations that already exist in the runtime knowledge graph. Re-executing that fragment in an empty digital twin is not expected to reconstruct its prerequisites.

This is also reflected in the ContextScope.getDataflow() contract: a scope-level view may be a subgraph focused on one observation and may reuse information available in upstream scopes. The same contract also anticipates a root context dataflow assembled from all incremental resolutions, capable of recreating the context when run again. That assembly path is not implemented yet.

Future code must keep these two artifacts explicit:

• incremental resolution dataflow: the minimal delta needed to resolve one request against a particular knowledge-graph state;
• reconstruction dataflow: an assembled, self-sufficient plan (plus declared external prerequisites) that can reconstruct an entire selected knowledge graph without repeating the original sequence of incremental resolution decisions.

Conflating them would either make ordinary resolutions unnecessarily large or produce persisted dataflows with dangling references that only work in the original runtime.

10. Transport, encoding, and decoding

10.1 Executable service transport

For a remote resolver:

1. ResolverClient creates a ResolutionRequest containing the observation and all scope resolution constraints.
2. If the context observation is unresolved, it adds an UnresolvedContextObservation constraint because the remote resolver cannot retrieve it from the runtime knowledge graph.
3. The controller restores constraints into the authorized service context and submits the resolver future to the context job manager.
4. The client job protocol ultimately decodes the result as Dataflow.class.

JacksonConfiguration registers polymorphic serializers/deserializers for Dataflow, Actuator, Observation, ServiceCall, Geometry, and their nested assets. This JSON representation is the current service wire format. Resolver Coverage objects are not DTOs and cannot cross this boundary; only their plain geometry projection and scalar resolved fraction can. The JSON form is not the intended durable, editable, or resource-level representation of a dataflow.

DataflowSerializationTest pins name, requirements, and scalar resolver coverage through a JSON round-trip. DataflowCoverageSerializationTest additionally pins the non-null geometry projection and verifies that no Coverage implementation leaks onto the wire. These are only seed contracts; actuator trees, calls, geometries, identifiers, notifications, and all subtype variants need broader coverage.

Scope propagation also depends on one runtime invariant: all child ServiceContextScope instances share one observation cache, but a cache miss must load through the child that made the request. The cache therefore stores values only and does not retain a loader bound to the scope constructor. This matters when identity adjustment creates a copy before the digital twin is instrumented. The runtime now creates and assigns the digital twin before publishing the root scope in ScopeManager; remote creation still declares the server-side scope first, applies the returned configuration to the client peer, and then instruments/registers that peer in its owning service.

10.2 Observation language: architectural role

References to k.DL as the dataflow language are obsolete. Dataflows are represented in the observation language, whose grammar is:

grammar org.integratedmodelling.languages.Observation

The top-level grammar deliberately hosts two closely related document forms:

strategies <preamble> <observation-strategy definitions>
dataflow   <preamble> <definitions and actuators>

This is not merely a shared parser implementation. Observation strategies are a key architectural element: they express how semantic observations and identifications are obtained, and their syntactic definitions use the same observation language that represents the resolved executable plan. Strategy knowledge can be extended together with semantic ontologies, allowing a worldview or project to add new resolution behavior without hard-coding it into the resolver.

The coordinated proposal in OBSERVATION.md, Section 7.1 retains these two forms with distinct ASTs and validators. Strategy-local patterns and model search belong to strategies; selected executable computations and submitted-object definitions belong to dataflow. Replacing implicit strategy apply does not remove dataflow apply: the latter must retain explicitly bound contextualizer calls. The proposed dedicated pattern language in Section 4.5 is owned by Observation, preserving ordinary observable matches and simplifying the shared grammar.

The grammar already gives a dataflow document explicit slots for:

• name, documentation, imports, and version;
• worldview, resource, component, and namespace requirements;
• metadata and coverage;
• definitions;
• nested reference, resolve, and observe actuators;
• the observation strategy used by an actuator;
• child actuators and applied computations.

The intended end state is a lossless bidirectional path:

resolution graph
    -> compiled Dataflow/Actuator model
    -> observation-language source
    -> parsed and validated dataflow
    -> rebuilt executable dataflow
    -> runtime compilation and re-execution

Compiled dataflows will be persistable as k.LAB Resource objects. Once resource-backed, they can be catalogued, versioned, transported with their requirements, and annotated or otherwise reused from k.IM like other k.LAB resources. This closes an important architectural loop: a dynamic resolution can become curated knowledge, and k.IM can add semantics and annotations to the persisted executable plan.

The observation-language source must therefore preserve semantics, references, local names, strategy identity, coverage, requirements, computation order, and every value needed to rebuild the same executable behavior. Human readability is useful, but reliable reconstruction and re-execution are the governing contract.

10.3 Reconstruction export and external execution

The runtime must eventually be able to extract the provenance/dataflow subgraph for a context, assemble its incremental resolution fragments, close references over the selected knowledge graph, and serialize the result in the observation language. This workflow must bypass semantic resolution when replayed: it reconstructs the already chosen plan and graph rather than asking the resolver to discover those choices again. This is the proposed strict replay mode. A reusable adaptive plan may instead retain explicitly declared context-bound continuations that invoke resolution later; that is a different contract and must never be selected silently during replay. See OBSERVATION.md, Section 7.4.

The proposed implementation uses a snapshot-scoped builder: select knowledge-graph roots, scan committed observation/activity provenance, assemble incremental fragments, close references, validate executable bindings and submitted definitions, and build portable Dataflow beans before serializing source. It must fail explicitly when provenance cannot supply a required computation or input. The existing DataflowGraph stub is an access point, not an implemented builder. The full contract and Resource packaging path are in OBSERVATION.md, Sections 7.2–7.3.

Reference closure needs an explicit rule:

• a reference to an observation inside the exported graph becomes part of the reconstruction document;
• a reference intentionally supplied by the target knowledge graph remains external, but must be declared as a prerequisite with a stable identity and validity requirement;
• any unresolved reference makes the exported dataflow invalid.

Explicitly submitted values

Computed observations can be recreated by re-running their actuators. Objects explicitly submitted to the knowledge graph have no producing actuator, so their values are inputs to reconstruction and must be encoded as such. This includes root observations and any other submitted objects, preserving at least:

• semantic observable;
• geometry;
• metadata;
• identity and relationships required by downstream references;
• the submitted value or a lossless resource-backed representation of it.

The proposed reconstruction source uses dataflow define statements for these explicitly submitted objects, including roots. The observation grammar already has a DefinitionBody/define ... as ... facility, making it the natural integration point. Keep required definitions in the dataflow package, with explicit versioned imports where needed; a sibling k.IM document must not be an implicit prerequisite. k.IM may annotate or expose the persisted Resource. S1 must define the supported typed object records. The encoder/adaptor contract must ensure that the definition carries enough information to rebuild the runtime object. Large or externally stored values may be represented through resource references, but the exported artifact must still be complete: it must carry or declare every value needed for reconstruction.

Validity envelope

A replayable dataflow must also say where and when it is valid. These constraints may live in the dataflow preamble or in a tightly versioned sibling manifest/document, but they must be machine-readable and validated before execution. At minimum the envelope must be able to constrain:

• worldview and semantic ontology versions;
• required namespaces, components, services, adapters, and resources;
• spatial, temporal, and other coverage;
• required existing observations or knowledge-graph identities;
• contextual assumptions, parameters, and resolution constraints that affect behavior;
• compatible observation-language and runtime versions.

Validity is different from availability: loading all named resources is not sufficient if the target graph, geometry, worldview, or versions do not satisfy the conditions under which the plan was compiled.

Current stubs

The API already sketches this workflow:

• ContextScope.getDataflow() promises a scope-focused view of the cumulative context dataflow;
• DigitalTwin.getDataflowGraph(context) promises extraction from provenance;
• DigitalTwinImpl.getDataflowGraph(...) constructs a server-side DataflowGraph;
• DataflowGraph.adapt() is intended to produce a serializable DataflowImpl;
• ClientDigitalTwin.getDataflowGraph(...) is the client-side access point;
• RuntimeService.runDataflow(...) and the dataflow-taking CompiledDataflow constructor sketch independent/external execution.

The implementation is presently empty or demonstrational:

• DataflowGraph returns null requirements and coverage, an empty computation, and null from adapt();
• DigitalTwinImpl.getDataflowGraph(context) currently ignores the requested context and always passes the root scope;
• ClientDigitalTwin.getDataflowGraph(...) returns null;
• the independent CompiledDataflow constructor is marked unused;
• RuntimeService.runDataflow(...) compiles root actuators but does not execute them and returns null.

These are parts of one architectural feature: extract, close, assemble, encode, persist, validate, load, bind to an existing or empty knowledge graph, and execute.

10.4 Current demonstrational encoding

The runtime stores:

Utils.Dataflows.encode(dataflow, resolutionScope)

in resolution activity metadata. That uses org.integratedmodelling.common.services.client.resolver.DataflowEncoder, which writes:

• a dataflow <sanitized-name>; preamble;
• nested actuator type, target/observable, alias, strategy, children, and apply calls.

This output resembles the dataflow branch of Observation.xtext, but it is incomplete. Definitions and most preamble fields are TODO, and computations are currently asked to encode with KlabLanguage.KIM rather than consistently using the observation-language context.

Although the sibling language repository defines and parses the grammar, the current resolver and runtime do not provide an end-to-end source-to-Dataflow adaptation, validation, reference relinking, and execution path. Current output must not yet be treated as a durable replay artifact.

10.5 Competing incomplete APIs

There are currently multiple divergent encoding surfaces:

• ResolverService.encodeDataflow(...) is an older incomplete encoder intended for this purpose but not aligned with the current observation-language grammar;
• ResolverClient.encodeDataflow(...) returns null;
• ResolverService.retrieveAsset(...) returns null;
• DataflowEncoder is the encoder actually used by the runtime for metadata;
• Jackson JSON is the format actually used for service execution transport.

These should converge on one canonical observation-language codec backed by the grammar in klab-languages. A safe decoder must reject unsupported versions or required constructs, validate requirements and references before execution, preserve observation/reference identity, and never rely on Java object identity surviving a round-trip. JSON may remain an internal service transport, but it must not become a second, semantically divergent persistence contract.

11. Reentrancy and concurrency risks

The public API promises asynchronous work, and independent calls can run concurrently. Resolution attempts now use isolated graphs created from a snapshot of the resolver-side context resource catalog. The remaining shared context catalog is limited to thread-safe resource publication; successful semantic-plan reuse is still unimplemented.

11.1 Resolution-attempt isolation

Each call creates an attempt graph with its own:

• observation and service-prototype maps;
• dependency set;
• atomic synthetic-ID sequence;
• mutable JGraphT graph.

Child graphs within that attempt share these objects because they are one compilation workspace. Concurrent root attempts do not. Context resources are maintained in a copy-on-write catalog and snapshotted when an attempt begins, so a resource submitted concurrently is visible either to the whole attempt or to the next one, never partially.

The runtime also coalesces concurrent submissions of the same singular substantial identity within one runtime/context. With the current default strategy the key is context + cohort semantics + URN. Only the owner performs resolution and commit; other callers receive independent future views of the same result, so cancellation by one waiter does not cancel shared work. This is process-local; multi-runtime deployments will require a knowledge-graph uniqueness constraint or distributed admission protocol when they permit concurrent writes to the same context.

For persisted identity checks, the submitted URN is the logical namespace:name identity (for example, test.tanzania:ruaha). Neo4j stores that observation under the context-local catalog URN <context-id>:individuals:<namespace:name>. The default identification strategy compares the logical URNs after removing only that catalog wrapper. A match returned by register(...) is a terminal submission result: its already resolved positive-ID observation is returned without creating resolution, provenance, or knowledge-graph state again.

Cohort eligibility is based on the fundamental enumerable substantial types—subject, agent, event, and relationship, which normally also carry COUNTABLE. Reasoner.baseSubstantialType(...) removes non-identifying traits, roles, and modifiers to produce the common cohort observable. A null, owl:Nothing, or non-substantial reasoner result is invalid and falls back to the original singular semantics. Registration always requests creation of a missing cohort, including the first root submission where no observation transaction exists yet. Cohorts are durable context-catalog assets: creation uses a short independent knowledge-graph transaction that atomically stores the cohort and its Context -HAS_CHILD-> Cohort link. Therefore a later failed observation submission legitimately leaves an empty cohort in the graph. Creation is serialized and rechecked per local knowledge-graph instance; cross-runtime uniqueness still requires the graph-level constraint mentioned above. Later submissions must find the cohort before applying the identification strategy. If an older Reasoner stored a cohort using decorated rather than canonical semantics, lookup re-normalizes existing cohort observables and reuses the semantic match instead of creating a parallel cohort.

Instantiators return individual observations while contextualizing a collective. The runtime must submit each outcome in executionScope.within(collective): registration then records both the durable cohort membership and the structural collective -HAS_CHILD-> individual relationship, while contextualizeFor(individual) removes the collective focus before resolving the independent countable observation. The random generator currently assigns identities of the form random:<KSUID>; these are valid, distinct logical URNs. They are intentionally ephemeral, so a reproducible generator should eventually derive stable identities from identifying source data, but the current URNs do not prevent persistence or commit membership.

All nested runtime transactions form one atomic transition. They share not only the transaction graph, ID allocation, cohort geometry, and contextualizers, but also the added, modified, and failures collections used to assemble the root commit. This is essential after cohorts became durable independently of observation submission: a generated individual is an added observation, its pre-existing cohort is a modified asset, and the HAS_MEMBER/HAS_CHILD edges are added links in the same root commit. The cohort must not be mislabeled as an added cohort. The final commit ID is attached to every newly stored observation, including secondary submissions that returned an intermediate commit result before the root commit existed.

The cohort's durable ownership link is context-local. KnowledgeGraphNeo4j translates the CONTEXT_ASSET sentinel to the contextualized graph's rootContextId, so both cohort lookup and the independent Context -HAS_CHILD-> Cohort write address one specific digital twin even when the same database hosts several contexts. Because that link is created before the observation transaction, the root commit also asserts it whenever the cohort is modified. The assertion does not write a duplicate Neo4j relationship; it closes the synchronization gap for a client that had already loaded the Context adjacency before the cohort existed. ClientKnowledgeGraph applies the asserted link idempotently, making the cohort visible as a Context child without a full graph reload.

Knowledge-graph transaction operations are fail-fast. A failed node creation, invalid allocated ID, update, or link rolls back the graph transaction and makes the enclosing contextualization fail; it must never be logged and swallowed while activities and a commit are reported as successful.

11.2 Service boundary and blocking

The resolver still blocks on runtime query .join(), but now uses a virtual-thread-per-task executor so that wait no longer occupies the common fork-join pool. The runtime/resolver exchange remains one serialized ResolutionRequest followed by one serialized Dataflow; duplicate in-flight runtime submissions are coalesced before issuing that request.

Scale and Coverage are service-local runtime objects and must never cross this boundary. Request observations, geometry constraints, unresolved context observations, dataflow coverage, actuator observations, actuator coverage, and resolved actuator geometry are projected to plain Geometry instances at the producing boundary. Projection also recursively sanitizes geometry values in metadata, adapter parameters, resolution constraints, and contextualizer service-call parameters.

11.3 No committed resolution catalog

The runtime has graph transactions and each resolver attempt now owns all speculative mutable state, so rejected strategies and failed attempts cannot contaminate another call. The resolver still has no atomic commit step for publishing a successful semantic plan into a reusable context catalog; currently only explicitly submitted resources survive between attempts.

11.4 Recommended state model

A reentrant design should distinguish:

• immutable request snapshot: observation, effective constraints, context/observer IDs, requested geometry, service catalog/version;
• per-attempt workspace: recursion stack, query results, candidate graphs, notifications, requirements, synthetic reference IDs;
• committed context catalog: immutable or copy-on-write entries from successful resolutions, keyed by semantic identity plus geometry, constraints, worldview, and service/resource versions;
• runtime references: stable observation IDs/URNs, never live object identity;
• commit protocol: atomically publish a successful catalog delta or discard the entire attempt.

A future cache key must include every input capable of changing model eligibility or ranking. Observable equality alone is insufficient.

12. Known weaknesses and vulnerable spots

Priority	Area	Current weakness	Consequence
Critical	Strategy compilation	APPLY operations are validated but not emitted	A “successful” plan may omit intended computation
Critical	Inter-resolution state	Cache lookup/accept are stubs and successful graphs are not committed	No semantic reuse; future partial cache changes could be inconsistent
High	Distributed admission	Submission coalescing is process-local	Two runtime instances writing one context still need graph-level uniqueness or distributed admission
Critical	Reconstruction export	DataflowGraph extraction/adaptation is empty	No whole-context dataflow can be assembled from incremental resolutions
High	Encoding	Observation-language output is not grammar-complete or round-trippable	Persisted source cannot yet rebuild and replay a resolution
High	External execution	runDataflow(...) compiles partially but never executes and returns null	Persisted or external plans cannot run against a knowledge graph
High	Submitted inputs	Explicitly submitted objects have no encoded define values	Whole-graph replay loses root/input observations and their data
High	Validity	No executable validity envelope is enforced	A plan may run with incompatible graph state, geometry, semantics, or resources
High	Failure reporting	Many empty-graph exits carry no notification	Runtime reports only generic “empty dataflow”
High	Graph representation	No parallel edges	Repeated bindings to the same node can lose local-name/coverage information
High	Cycle handling	Per-call resolution cache is never consulted	Recursive semantic/model dependencies can recurse indefinitely
High	Model fidelity	Ingested models receive universal coverage	Ranking/coverage can accept geographically or temporally invalid models
High	Constraint enforcement	Many constraint types are not enforced	Caller intent may be serialized but ignored
Extension	Ranking configuration	Namespace bean policies are supported; source declaration adaptation is pending	Service defaults apply until namespace overrides are supplied
Medium	Async execution	A virtual thread blocks on runtime query .join()	No carrier starvation, but reciprocal service latency and cancellation still need end-to-end deadlines
Medium	Notifications	ResourceSet notifications are discarded	Missing diagnostics for model/resource failures
Medium	Contextualizables	Unsupported form compiles to null	Delayed null failure rather than resolver diagnostic
Medium	Geometry	Dependent-without-context logs an error but continues	Work may continue with an invalid semantic context
Medium	Thresholds	Duplicate hard-coded 0.15; service copy unused	Scope configuration cannot tune contribution policy
Medium	Scope lifecycle	Resolver graph presence is assumed	Direct/misordered calls fail with null dereference
Low	Compiler catalog	Unused local Map<Observable,String> shadows the ID catalog concept	Confusing maintenance surface
Low	Observation-language encoders	Service/client encoders diverge or return null	Public encoding behavior depends on implementation

The first implementation phase should add diagnostics and tests before filling cache or APPLY behavior, because both affect the semantic contract of generated plans.

13. Testing strategy

13.1 Layer 1: deterministic unit contracts

Test ResolutionGraph without services:

• model coverage intersects while alternatives union;
• complete/empty/relevant thresholds;
• reference lookup for positive IDs and multiple ID-0 query views;
• synthetic IDs are unique within one attempt;
• child merge preserves local name and coverage;
• repeated source/target pairs have a defined policy;
• notifications and requirements follow accepted graphs only;
• failed candidate graphs leave no committed state.

Test DataflowCompiler with small hand-built graphs:

• actual root nodes are compiled exactly once;
• existing references are attached to each contributing strategy;
• model dependencies retain their stated local names;
• transformation targets bind the intended inputs;
• every contextualizable variant becomes the correct runtime functor;
• unsupported contextualizables fail with a specific notification;
• APPLY operations appear in the final actuator;
• sharding annotations are validated and propagated;
• requirements, coverage, proportional coverage, and notifications survive compilation.

Test PrioritizerImpl:

• comparator antisymmetry and transitivity over models with different custom criteria;
• deterministic tie handling;
• scenario/namespace/project precedence;
• every enabled criterion has a non-placeholder score;
• constraints blacklist, whitelist, and force models as specified.

13.2 Layer 2: resolver orchestration with service fakes

Use controlled fake Reasoner, ResourcesService, and RuntimeService implementations:

• no geometry;
• dependent without context;
• complete direct contextual quality reference;
• absent contextual quality proceeding to semantic resolution;
• collective ID-0 query view with multiple contributors;
• rejection of detached quality and non-enumerable ID-0 queries;
• temporal union across collective events and functional relationships;
• absence of temporal cohort bounds for continuant substantials;
• one complete strategy;
• several incomplete model contributions whose union completes;
• optional versus mandatory dependencies;
• unsupported service requirements;
• recursive/cyclic model dependencies;
• predefined contextualization and direct resolver invocation;
• exception and cancellation propagation.

Assertions should cover both graph/dataflow shape and notifications, not only isEmpty().

13.3 Layer 3: codec compatibility

Create a versioned golden corpus of dataflows containing:

• every actuator type;
• nested children and repeated references;
• every ServiceCall parameter value supported by Jackson;
• coverage and resolved geometry;
• requirements with all resource classes and services;
• annotations, identifiers, expressions, lookup tables, and classifications;
• info/warning/error notifications;
• unknown optional and required fields.

For JSON transport, assert semantic equality after encode/decode and compatibility with at least the previous released schema.

For observation-language persistence, use grammar-valid golden sources covering both the strategies and dataflow document forms. Require:

compile Dataflow
    -> encode observation-language source
    -> parse
    -> adapt and validate
    -> rebuild Dataflow
    -> compile for runtime execution

The rebuilt dataflow must preserve actuator/reference identity, strategy URNs, local bindings, execution order, requirements, metadata, and coverage. Re-encoding the rebuilt form should be semantically idempotent. Persisting it as a Resource, retrieving it, and annotating/referencing it from k.IM should be covered by integration tests.

Maintain separate golden cases for:

• one incremental resolution fragment whose references are deliberately external;
• one whole-context reconstruction export with every internal reference closed;
• submitted root and non-root objects encoded through define, including values, semantics, geometry, and metadata;
• a reconstruction artifact with external prerequisites and a validity envelope;
• rejection caused by an unresolved reference or invalid worldview/resource/coverage constraint.

13.4 Layer 4: runtime integration

Run RuntimeService.submit(...) against an in-memory/test digital twin:

• resolution and submission transactions commit in order;
• failure rolls back observations, executors, storage, provenance, and resolver catalog deltas;
• query ID-0 observations are never persisted;
• positive references retain the correct knowledge-graph links;
• child submissions complete before parent transaction commit;
• contextualization uses the intended geometry and storage;
• activity metadata records a decodable dataflow or is explicitly diagnostic-only.
• extracting a root-context dataflow, clearing the target digital twin, and replaying it recreates the same selected observations, links, metadata, and computable values;
• a reconstruction export does not call semantic resolution during replay;
• an external dataflow binds declared prerequisites in an existing graph and rejects missing or incompatible bindings without partially mutating the graph;
• explicitly submitted values survive export, resource persistence, reload, and replay.

Existing RuntimeServiceQueryTest is the starting point for query identity, coverage, and contributor geometry. ResolutionCompilerQueryTest protects direct quality reuse and CohortGeometryTest protects the occurrent/continuant temporal-boundary distinction.

13.5 Layer 5: concurrency and reentrancy

Use barriers rather than sleeps:

• resolve two unrelated observations in one context concurrently;
• resolve the same observable/geometry concurrently and define deduplication behavior;
• resolve overlapping geometries concurrently;
• submit a local resource while resolution reads the scenario catalog;
• cancel one waiter while another shares work;
• inject failure immediately before catalog commit;
• reconnect a remote resolver during a request;
• stress synthetic reference IDs and requirement merges.

Verify deterministic results, no collection exceptions, no cross-request edges, and an atomic committed catalog.

13.6 Property and mutation testing

Coverage and graph composition are suitable for generative tests:

• union is monotonic;
• intersection is non-increasing;
• coverage remains in [0, 1];
• accepted contributions never reduce non-model coverage;
• reordering independent candidates does not change the final semantic coverage;
• encode/decode is idempotent modulo transient IDs;
• a completed dataflow has no dangling reference ID.

Mutation tests should target branch conditions around completeness/relevance, optional dependencies, query IDs, and graph merge direction.

14. Tests added with this trace

• klab.services.resolver/.../DataflowCompilerTest
  • proves the actual graph root is compiled rather than the original request;
  • proves projected geometry, proportional coverage, and requirements reach DataflowImpl.
• klab.services.resolver/.../DataflowCoverageSerializationTest
  • reproduces the remote-boundary failure caused by transporting CoverageImpl;
  • proves the plain geometry projection survives a polymorphic JSON round-trip and is not a Coverage implementation.
• klab.services.resolver/.../ResolverTransportSerializationTest
  • proves request observations, direct constraints, and nested parameter values project both Scale and Coverage to plain geometry before and after JSON serialization.
• klab.services.resolver/.../ResolutionGraphConcurrencyTest
  • proves concurrent attempts receive independent mutable graphs and consistent resource snapshots.
• klab.services.runtime/.../RuntimeServiceQueryTest
  • proves same-identity submissions execute once, expose cancellation-independent future views, and release their in-flight admission entry.
• klab.core.common/.../DataflowSerializationTest
  • proves dataflow name, requirements, and proportional resolver coverage survive polymorphic Jackson serialization through the Dataflow interface.

Focused verification commands:

.\mvnw.cmd -q -pl klab.services.resolver -am "-Dtest=DataflowCompilerTest,DataflowCoverageSerializationTest" "-Dsurefire.failIfNoSpecifiedTests=false" test
.\mvnw.cmd -q -pl klab.core.common -am "-Dtest=DataflowSerializationTest" "-Dsurefire.failIfNoSpecifiedTests=false" test

15. Suggested improvement sequence

The strategy-specific sequence is now maintained in OBSERVATION.md, Section 9, including review, matching/setup fixes, typed graph plans, coverage and candidate isolation, Dataflow lowering, lifecycle follow-ups, scoped member resolution, logical operations, and contextual aggregation. Use that ledger for strategy implementation; the broader recommendations below also cover resolver state and transport work.

1. Define invariant-rich test fixtures for graphs, coverage, and actuator trees.
2. Make all failure exits produce structured resolver notifications.
3. Complete strategy APPLY and transformation/local-name compilation.
4. Add explicit recursion/cycle detection.
5. Define an atomic context-catalog commit for successful isolated attempts.
6. Implement cache keys and invalidation before implementing getResolving(...).
7. Add propagated deadlines and cancellation to the explicit resolver executor/runtime query.
8. Implement provenance-to-DataflowGraph extraction, incremental-fragment assembly, reference closure, submitted-value definitions, and validity-envelope generation.
9. Complete the observation-language codec defined by Observation.xtext, including strict source-to-dataflow validation, resource persistence, re-execution, and compatibility tests.
10. Complete transactional external dataflow execution against empty and existing knowledge graphs.
11. Enforce all advertised resolution constraints and define source declaration and adaptation for namespace ranking policies.
12. Add multi-service integration tests and graph-level uniqueness before enabling distributed shared-work deduplication.

16. Trace map

Start future investigations at these methods:

• ResolverService.resolve
• ResolverService.declareContextScope
• ResolutionCompiler.resolve(Observation, ContextScope)
• ResolutionCompiler.query
• ResolutionCompiler.resolve(ObservationStrategy, ...)
• ResolutionCompiler.resolve(Model, ...)
• ResolutionCompiler.resolve(Observable, ...)
• ResolutionGraph.merge, addReference, getResolving, and accept
• DataflowCompiler.compile, compileObservation, compileStrategy, and compileModel
• PrioritizerImpl.compare and computeCriteria
• ResolverClient.resolve and ResolverController.resolveObservation
• RuntimeService.submit, compile, and createPredefinedDataflow
• RuntimeService.runDataflow
• CompiledDataflow.compile, requireObservations, store, and executor ordering
• ContextScope.getDataflow
• DigitalTwin.getDataflowGraph and DigitalTwinImpl.getDataflowGraph
• DataflowGraph.adapt
• ClientDigitalTwin.getDataflowGraph
• DataflowEncoder.encode
• JacksonConfiguration.configureObjectMapperForKlabTypes
• ../klab-languages/org.integratedmodelling.languages.observation/src/org/integratedmodelling/languages/Observation.xtext
• observation-strategy parsing/adaptation in WorkspaceManager and LanguageAdapter

When changing one layer, trace the change through all subsequent layers. In particular, a new graph edge or actuator field is incomplete until its JSON service representation, observation-language syntax and adaptation, runtime compiler, Resource persistence, k.IM reuse, provenance behavior, and failure rollback are all defined and tested.

17. Semantic-update contextualizations

An observation request can describe an operation on existing observations. For predicate X and substantial Y, X of each Y requests classification of Y members; it does not request a new observation whose observable is the classification directive. Z of Y requests characterization within Y regardless of abstraction. Collective inherence alone chooses classification and its member acquisition. Neither follows merely from an executor's Java return type.

Planning and member support

The Tier-0 classifier strategy resolves each Y, then observes the classifier directive with that graph bound as its members input. Member acquisition is an intrinsic prerequisite, so this is a direct strategy even though it includes recursive resolution. The classifier model is selected in the requesting scope with its lexical/model constraints retained. Recursive member work must retain those constraints while focusing context on the appropriate observation.

The Resolver represents the directive as an operation target before observation allocation. It returns the existing portable Dataflow, with UPDATE actuators carrying effect = SEMANTIC_UPDATE, operationObservable, contextualization, requestedSupport, coverage and typed target bindings. The node has no result observation; its transient node identity distinguishes it from other UPDATE nodes. COHORT_MEMBERS bindings name prerequisite producers; OBSERVATION bindings identify the existing member for individual characterization. These are execution bindings, not scalar contextualizer arguments. Interface mappings in JacksonConfiguration preserve the plan across services without annotations or Jackson dependencies in semantic beans.

Complete cached cohort support is reused. Partial support requires resolution of missing support; if subtraction cannot represent the remainder, planning conservatively resolves full support. Requested support, producer coverage and completed attribution are separate facts. Runtime awaits producer execution and each new substantial's acknowledgement, then enumerates durable and transaction-local members within the requested/producer support intersection. Bindings deduplicate members. A completed empty cohort succeeds; a missing prerequisite does not.

Predicate model discovery and precedence

ModelKbox's observable index expands predicate heads through the Reasoner's resolving closure, including concrete and abstract ancestors. The full candidate observable then passes directional semantic-distance validation, including its bearer and contextualization. A model's predicate may subsume the requested predicate; the reverse does not suffice. Other observable heads must remain equal. PrioritizerImpl evaluates semantic distance for every observable, including inherence and other clauses for non-predicates. SEMANTIC_DISTANCE (im:semantic-concordance) uses the minimum nonnegative candidate-output-to-request distance; smaller is better, with no score cap that would collapse distinct distances. Incompatible candidates are excluded. The configured criterion order applies (by default lexical scope precedes semantic distance); within equal higher-priority criteria, exact and nearer explanations precede broader ones. Classification and characterization remain distinct operations. Reasoner.resolves(request, candidate, context) uses this same direction.

Semantic identity across service boundaries

Abstraction and collectivity are operational semantics, not display hints. Adapted observable URNs must agree with the semantic concept tree, including each inside an inherence restriction. Projecting the inherent of X of each Y must return each Y; silently returning singular Y changes instantiation into acknowledgement and changes which strategies match. Ontological ancestry can use Y's singular OWL class, but model-discovery reconstruction must preserve the requested arity.

Concept and observable builders cross the Reasoner boundary as portable semantic operations. Removing inherence for validation or building concrete Z-of-Y for characterization must work with a remote Reasoner as well as a local instance. These operations construct semantics only; they do not mutate runtime observations. Scope, context focus and lexical resolution constraints remain part of the resolution request through recursive and remote calls.

Invocation, validation and follow-up

A classifier receives the full operation Observable unchanged, the member when requested by its signature, and a scope contextualized to that member. Component code chooses any semantic projection it needs: a classifier selecting descendants of X removes inherence itself before requesting X's closure. Runtime independently extracts X to validate the result. Semantic closure excludes the entire OWL bottom-equivalence node, including named unsatisfiable classes; filtering only literal owl:Nothing would admit invalid candidates.

A result must be a satisfiable, concrete predicate equal to or specializing X. A concrete X is a valid result even if it has no descendants. NOTHING is always an error. A null result is permitted only when the operation originates in a model dependency and that original dependency is optional, after a classifier has been found and linked. Optionality does not excuse invalid concepts or execution failures. A member already bearing a valid concrete specialization of X (including X itself) satisfies the request. Runtime skips classifier invocation and produces no pending attribution or repeated characterization for that member, regardless of how the predicate was acquired. Invalid same-family attributions remain errors; this reuse does not authorize replacement of an existing classification. Valid results become pending attributions, then detached semantic replacements; they do not become new observations.

Runtime resolves concrete Z of singular Y in the staged member's scope after attribution. It awaits this characterization before completing classification and the root transaction, without resubmitting the entire member or invoking classification again. Executor/lifecycle bookkeeping prevents repeated scheduler visits from repeating the same work for a root transaction, event and support. Successful discovery with no characterization model yields NO_MODEL; actual resolution or execution failure propagates and rolls back the enclosing semantic updates. No-model success creates no CHARACTERIZED effect. Successful executed characterization links to the existing member.

The individual characterizer executor supports local public contextualizers returning void or primitive boolean (false fails), and dependency-only models. Remote/adaptor characterizers and other unsupported semantic-update forms fail explicitly. Collective classification dependencies and runtime-owned individual characterization do not imply support for arbitrary root directives, additional independent operation-submission entry points. Singular predicate inherence is characterization; distributed predicate inherence is classification.

Outcomes and durable effects

Resolution outcome	Meaning	Execution convention
RESOLVED	A contextual plan was obtained	Execute the plan and its prerequisites
NO_MODEL	Successful absence of executable explanation for an eligible lifecycle request	No effective actuators; isEmpty() is false
FAILED	Resolution failed	isEmpty() marks failure; do not treat it as acknowledgement

For ACKNOWLEDGEMENT, an error-free resolution with no significant explanation preserves a substantial's existence, including when no strategy applies. Characterization's no-model outcome requires successful model discovery with no candidate. Neither path suppresses service errors or failed contextualizers. Resolution completion is distinct from execution completion, and a child ActivityFinished is distinct from durable root commit.

The knowledge-graph contract defines atomic semantic updates, before/after provenance, commit propagation and cache invalidation. Execution activities carry their actual contextualization type and link to affected observations through its typed effect. FlowCharts carry the accepted resolution graph under Metadata.IM_RESOLUTION_GRAPH on the completed Resolution Activity. Execution activities carry the contextual plan under Metadata.IM_DATAFLOW_GRAPH from creation through completion, including failures. Descriptions are optional human-readable text, not serialized plans; clients use activity type, identity and triggering hierarchy for cataloguing.

These diagrams audit contextual planning and execution. Neither is the separate provenance-extracted graph-reproduction dataflow.