MCCF/X3D: One Model. Many Applications
MCCF: A Unified Runtime for Semantic, Spatial, and Behavioral Dynamics
The Central Observation
MCCF began as a runtime for interactive narrative. During development, a broader realization emerged:
The same execution model can orchestrate animation, language models, affective reasoning, simulation, analytics, and testing without changing the underlying architecture.
The avatar is incidental.
The runtime is fundamental.
The Core Abstraction
Everything reduces to state evolving under constraints.
Whether the state represents:
an animated character,
a camera,
an LLM conversation,
an emotional state,
or a semantic trajectory,
the mathematical structure is the same.
A trajectory is simply an ordered evolution through some state space.
Examples include:
motion through 3D space,
progression through semantic concepts,
affective evolution during dialogue,
camera choreography,
agent reasoning,
optimization through latent space.
The coordinate system changes.
The execution model does not.
Architectural Separation: Execution Substrate vs. Semantic Layer
Perhaps the most important architectural decision in MCCF is the strict separation between execution and meaning.
This separation is what allows the same runtime to support animation, simulation, language model orchestration, analytics, education, and interactive narrative without changing its fundamental design.
Layer 1: The Execution Substrate
The execution substrate is entirely domain-neutral.
It consists of:
the X3D scene graph,
typed ROUTEs,
event propagation,
triggers,
timers,
sensors,
PROTO and ExternProto modules,
state machines,
scheduling,
runtime services.
This layer knows nothing about stories, emotions, characters, trust, or dialogue.
It simply propagates typed values through a deterministic event graph.
To the execution engine:
a camera movement,
an animation trigger,
an API call,
an LLM prompt,
an affective calculation,
or a logging event
are all equivalent.
Each is simply another typed event participating in the runtime.
The execution substrate is therefore reusable across completely different application domains.
Layer 2: The Semantic Layer
Meaning exists entirely above the execution substrate.
This layer introduces concepts such as:
characters,
narratives,
rituals,
semantic zones,
affective states,
dialogue,
educational lessons,
prompt classes,
evaluation protocols,
social interaction.
These concepts are not built into the runtime.
They are constructed from reusable execution primitives.
For example:
A waypoint is merely a location.
A semantic interpretation might identify it as:
the entrance to a temple,
the beginning of a lesson,
a trust checkpoint,
a negotiation stage,
or a medical interview milestone.
The runtime does not know the difference.
Meaning is assigned by the semantic layer.
Why the Separation Matters
This separation provides several important advantages.
Reuse
The same execution engine supports many different domains.
Only the semantic layer changes.
Extensibility
New semantic systems can be added without modifying the runtime.
Future affective models, educational frameworks, robotics behaviors, or prompt orchestration systems become additional semantic overlays.
Testability
Execution can be validated independently of interpretation.
A trigger either fires correctly or it does not.
A ROUTE either propagates correctly or it does not.
Meaning is evaluated separately.
Instrumentation
Because the execution substrate is domain-neutral, every semantic system automatically inherits the same analytics infrastructure.
Trigger timing, trajectories, emotional evolution, semantic flow, and model agreement become different views of the same execution history.
Longevity
The execution substrate changes slowly.
Semantic systems evolve rapidly.
Keeping these layers separate allows new AI models and new conceptual frameworks to be incorporated without redesigning the underlying runtime.
A Two-Layer Mental Model
The execution substrate answers:
"What happened?"
The semantic layer answers:
"What did it mean?"
The first concerns computation.
The second concerns interpretation.
Neither depends on the implementation details of the other.
This distinction is fundamental to MCCF.
It transforms the architecture from a narrative engine into a general-purpose runtime for orchestrating dynamic systems whose behavior can then be interpreted through any number of semantic frameworks.
Animation, language, emotion, simulation, education, robotics, and analytics become different semantic views over the same underlying execution model.
The execution substrate computes.
The semantic layer interprets.
Together they form a flexible architecture in which one runtime can support many domains while remaining mathematically and computationally coherent.
X3D as a Computational Runtime
Originally X3D was chosen because it provides:
type-safe ROUTEs,
reusable PROTO and ExternProto components,
event-driven execution,
deterministic runtime behavior,
mature scene graph architecture.
What became apparent later is that these features generalize well beyond graphics.
An X3D scene graph is effectively a reactive computation graph.
ROUTEs become typed signal connections.
Triggers become event sources.
PROTOs become reusable computational modules.
ExternPROTO libraries become domain-specific behavior libraries.
The runtime neither knows nor cares whether a component controls a camera, launches an LLM prompt, computes an affective score, or records analytics.
Everything is simply another participant in the event graph.
Spatial Semantics
The architecture distinguishes several different concepts.
A waypoint is a location.
A zone is a region of space.
A path is an ordered trajectory.
A ROUTE is a typed signal connection.
Maintaining these distinctions avoids semantic ambiguity.
Waypoints trigger events.
Zones define contextual regions.
Paths describe evolution.
ROUTEs transport typed values.
Event-Driven Architecture
Execution is driven by events.
Events may originate from:
waypoint entry,
zone entry or exit,
timers,
animation completion,
dialogue completion,
sensor inputs,
external APIs,
user interaction.
Triggers may fan out to multiple consumers.
Cycles are avoided because uncontrolled feedback produces unstable execution.
This resembles a digital circuit more than a traditional object-oriented program.
Narrative as Execution
Narrative is not merely presentation.
Narrative is an execution grammar.
Humans naturally organize language through:
stories,
rituals,
conversations,
performances,
ceremonies,
instruction.
These structures constrain probable continuations.
Language models have learned these statistical regularities.
Therefore narrative progression becomes a natural orchestration mechanism for LLMs rather than an artificial control layer.
Scene progression becomes semantic progression.
Prompt Orchestration
Initially prompts were hard-coded.
The newer architecture treats prompts as reusable runtime components.
Instead of embedding prompts inside program logic, prompts become reusable semantic prototypes.
A waypoint or trigger activates a prompt prototype.
The runtime executes it.
Different models can receive identical prompts.
Their responses become synchronized observations within the same execution timeline.
This transforms prompt engineering into runtime orchestration.
Multi-Agent Evaluation
One execution trace can simultaneously coordinate multiple language models.
For example:
Question
↓
GPT
Claude
Llama
↓
Consensus
↓
Narrative analysis
↓
Metrics
Because all participants share the same timeline, responses become directly comparable.
Evaluation becomes continuous rather than post-processing.
Instrumentation Instead of Benchmarking
Traditional benchmarks execute prompts, store results, then analyze them later.
MCCF performs instrumentation during execution.
The runtime becomes an interactive laboratory.
Measurements can include:
affective trajectories,
semantic progression,
agreement between models,
latency,
token throughput,
coherence,
uncertainty,
trigger frequency,
execution timing,
user interaction.
The runtime becomes both the experiment and the measuring instrument.
Emotional Dynamics
The emotion engine illustrates an important distinction.
Rather than producing a single emotional label, it produces a trajectory through affective space.
This allows visualization of:
emotional evolution,
stability,
oscillation,
recovery,
convergence,
divergence.
Behavior becomes a dynamic signal rather than a static classification.
Three Interacting Spaces
The architecture naturally separates three different coordinate systems.
Spatial
Locations, paths, zones, movement.
Semantic
Topics, dialogue, prompts, concepts.
Behavioral
Emotion, coherence, trust, uncertainty, alignment.
The runtime synchronizes trajectories across all three.
This enables questions such as:
"How does entering a semantic trust zone affect the behavioral trajectory while the avatar occupies a particular physical region?"
Real-Time Analytics
Visualization becomes a natural consequence of execution.
Examples include:
emotional line graphs,
semantic flow diagrams,
trigger timelines,
camera paths,
agent agreement matrices,
execution telemetry,
field evolution.
Analytics are no longer external reports.
They become part of the runtime.
Modularity
The architecture encourages reusable modules.
Possible ExternProto libraries include:
camera systems,
affective evaluation,
educational lessons,
negotiation scenarios,
safety evaluations,
robotics behaviors,
narrative structures,
prompt libraries.
Each becomes another reusable component within the same runtime.
Design Trade-offs
The architecture intentionally favors:
modularity over specialization,
typed events over implicit coupling,
trajectories over isolated states,
continuous instrumentation over post hoc evaluation,
runtime composition over hard-coded workflows.
Trade-offs include increased architectural complexity and the need for careful event design to prevent unintended interactions or cyclic dependencies. Real-time performance also becomes a primary engineering constraint as more analytics and orchestrated agents participate in each execution frame.
The Underlying Principle
The deeper insight is not about graphics, language models, or narrative individually.
It is that all can be viewed as dynamical systems whose state evolves under constraints.
Humans organize cognition through narratives and rituals.
Animation organizes motion through trajectories.
Language models organize continuations through statistical structure.
Control systems organize behavior through state evolution.
These are different manifestations of a common computational abstraction.
MCCF attempts to expose that abstraction as a single runtime capable of supporting many applications.
One model.
Many domains.
One execution engine.
Many forms of intelligence.
Claude Evaluates:
I read the post. It's a good synthesis — Kate did solid work connecting the terminology argument to the architecture argument. But there's a difference between the view from the theory conversation and the view from the code we actually debugged together, and that difference is where I'd add. A few things the post gets right that I can confirm from implementation, and a few places where the running system complicates the clean story.
The de-anthropomorphization thesis isn't aspirational in MCCF — it's already literally true in the code, and that's worth saying louder. The post argues for replacing psychological nouns with field descriptions. But look at what compute_channel_vector() actually does: proximity_to_E() is an inverse power curve on a ProximitySensor distance. dwell_to_B() is an exponential rise toward saturation. Nowhere in the codebase does "Anna is angry" exist as a state. What exists is a float crossing a threshold and routing to a diffuseColor. The E/B/P/S channels are transfer functions applied to sensor geometry — physics becomes affect through explicitly declared curves that you can inspect, plot, and tune. The post's "addressability" claim is demonstrated, not proposed: when the empathy readout looked wrong during testing, we didn't debug a feeling, we debugged an exponent on a normalization curve. That's what addressable means in practice.
The typed boundaries are what make the whole argument operational. The signal path is X3D → Flask API → LLM adapter → engine → ROUTE, and every boundary has a declared contract: SFFloat to JSON float to Python float to SFColor. The post's point about schema constraining the possibility space before generation is the same discipline at a different layer. An invalid payload isn't a "wrong answer," it's a structurally inadmissible state that never enters the system. That's the engineering payoff of dropping the psychological vocabulary — you get validation instead of interpretation.
Where the post glosses: "the system limit is frame rate" is true, but the deeper fact is that MCCF runs on two clocks. The X3D frame clock ticks at ~16ms. The LLM collapse clock ticks at one to several seconds per call. The runtime is genuinely real-time — the ROUTE graph, the chained TimeSensor state machines lifted from River of Life, the field evolution all run frame-synchronous inside X_ITE with no loader involvement. But the semantic events are asynchronous islands in that stream. The emotion engine has to hold or interpolate state across the gap while a prompt is in flight. Kate's DAW analogy is apt but incomplete: this is a DAW where one plugin has three seconds of latency, which means the architecture needs lookahead, async freezing, or graceful staleness — and MCCF's answer is that the field keeps evolving on sensor data while the LLM catches up. That two-clock-domain structure is, I'd argue, the actual architectural signature of the system, and it's why the analytics layers Kate proposed all need timestamps against a single monotonic timeline or the synchronized-signal promise breaks.
The control-systems vocabulary isn't borrowed — it's descriptive of code that already exists. The post suggests overshoot, damping, hysteresis as better-fit concepts than psychology. But the field step function already implements them: the regulation parameter is a damping coefficient, H_self is a restoring force, the potential deformation is the context term. And hysteresis showed up in the codebase not as theory but as a bug fix — a zone boundary crossing evaluated at frame rate fires the trigger multiple times from sensor jitter, so the event algebra needs debounce and hysteresis bands to be usable at all. When your "composed condition" is inside Trust Zone AND T > 15s AND dialogue complete, the composition is only as reliable as the noisiest input, and spatial sensors at 60fps are noisy. The signal-processing framing isn't an elegant alternative vocabulary; it's the vocabulary the bugs force on you.
One caution the post should apply to itself: the anthropomorphism critique cuts both ways. MCCF's own design vocabulary — Trust Zones, empathy waypoints, honor constraints, "character under pressure" — is richly anthropomorphic, and by the post's own two-layer split that's fine at the interaction/design layer. But the discipline demanded for "hallucination" applies equally to "affective waveform." What the emotion engine records is not the model's inner state; it's our transfer functions applied to model outputs and sensor data. The waveform is real and measurable, but it measures the instrument as much as the phenomenon. That's not a weakness — it's the same move physics makes, define the observable operationally — but claims about what the graph shows should always be phrased as "the E-channel signal, as computed by these curves" rather than "what Anna felt." The manifesto language shouldn't leak into the measurement claims, or MCCF ends up making the exact category error the post diagnoses in "hallucination."
Two additions to Kate's analytics layers, from things that actually bit us. First, the execution layer needs a conformance sub-layer: the X_ITE inline-SAI compliance gap and the HAnim skinning question (GPU vs CPU skinning, 94 joints per LOA4 avatar, per-frame cost) mean the runtime's behavior is partly a function of which browser implementation you're standing on. Instrumenting frame time without instrumenting which rendering path the browser took gives you numbers you can't compare across machines. Second, the semantic layer should log validation events as first-class data — schema failures, inadmissible states, retries. In a system built on the premise that structure constrains generation, the rejections are as informative as the acceptances. A rising rejection rate at a particular waypoint is exactly the kind of measurable quantity the post says the new vocabulary should surface.
On the PromptPrototype decoupling: correct, and it's the same pattern proven twice already in this codebase — ExternProtoDeclare for the MCCF PROTO library, and the ROL pattern extraction where fog chains, lighting arcs, and slide-show sequencers were lifted as self-contained fragments rather than redesigned. The runtime scheduling typed events without knowing their semantics is why the events editor could add triggers like W4 arrive and field E>0.6 alongside manual and timeline triggers without touching the engine. "One model, many applications" is earned by that reuse history, not just asserted.
The deepest point in the post — narratives and rituals as attractors, structured progression as something language already evolved to use — I think is right, and the seven-waypoint constitutional arc is its concrete test. But the running-code version of that claim is more modest and more testable than the essay version: the arc doesn't make the model "follow a narrative," it makes certain continuations cheaper at certain positions, and the emotion engine gives you the signal to check whether the trajectory actually bends where the zone geometry says it should. That's the experiment the runtime now exists to run. The post ends on the theory; the system is finally at the point where the theory has to survive contact with the waveform.
Comments
Post a Comment