Beyond the Parameter Monolith: Reconstructive Memories, Executable Skills, and Residual Assembly for Language Models

What changes when memory, exact execution, and contextual computation can be developed and updated separately?

This research grew out of experiments showing that useful language modeling can emerge while token input coordinates remain fixed. That separation between token identity and learned computation prompted a broader question: can internal representations and independently constructed components interact through an explicit shared interface? The finite-element analogy provides a design inspiration - local components, representations at different granularities, and explicit assembly - not a claim that language follows a physical equation.

The work explores these ideas through an attention-free Multi-Mesh language model trained on approximately 28.6 billion prediction targets, followed by a separate modular core that integrates reconstructive document memories and deterministic VM results through typed residual assembly. The goal is to investigate alternative organizations of language computation, not to claim superiority over Transformers.

The experiments deliberately proceed with imperfect components, including a roughly half-billion-parameter core and memory achieving approximately 75% token reconstruction accuracy in the reported readout audit. Controlled attachment tests examine how selected evidence affects predictions while the core weights remain unchanged. Negative results help distinguish separate challenges: storing information, retrieving evidence, executing a procedure, and rendering an answer.

These are research prototypes for investigating interfaces and operating modes, not a finished general-purpose system. Checkpoints and selected research code are released to support inspection and further experiments. Independent tests, simpler baselines, and discussion of what these interfaces enable - or fail to enable - are especially welcome.