Compiling RISC-V Cores into Gated DeltaNet Weights: Deterministic Execution Inside Qwen 3.5
Qwenputer embeds a fully functional 32-bit RISC-V execution core directly into eleven compiled layers of a stock Qwen 3.5 4B model, using Gated DeltaNet attention states as exact, bounded RAM. The architectural mod requires no fine-tuning, executes binaries deterministically during the autoregressive forward pass, and retains baseline model generation fidelity.
Execution Mechanics and DeltaNet Addressability
Standard recurrent linear attention mechanisms decay historical hidden states through continuous gate factors. Qwenputer repurposes the Gated DeltaNet (GDN) layers inside Qwen 3.5 by fixing specific hyper-parameters to turn continuous state matrices into exact, addressable RAM. By configuring the decay gate ($\gamma$) to 1.0, memory decay is completely suppressed. Setting the write strength ($\beta$) to 1.0 ensures that incoming writes overwrite existing row values rather than blending with them.
Addressing relies on one-hot key vectors. A one-hot key targets a single row of the GDN state matrix during a write, while a matching one-hot query retrieves the exact scalar or vector payload without cross-talk. Unlike standard attention mechanism KV-caches—which expand linearly with token context—the GDN state matrix maintains a constant memory footprint throughout arbitrary execution depths.
RV32IM Core Compiler and Execution Trace
The CPU component is compiled directly into 11 new layers appended to Qwen's standard 32-layer configuration. The underlying architecture implements an RV32IM instruction set alongside the Zba bit-manipulation extension and a dedicated big-number arithmetic unit. Weight values in these extra layers are directly computed by a compiler based on a discrete circuit definition rather than learned via gradient descent; attention heads manage instruction lookup, feed-forward sub-layers perform ALU operations, and GDN heads store register and memory states.
Execution proceeds natively through autoregressive decoding. Each RISC-V instruction emits a sequence of trace tokens encoding memory address access (`ADDR`), write payload (`VAL`), and instruction step completion (`STEP`). An external driver program named `RvChat` boots initial firmware into the GDN state and intercepts trace tokens to prevent context pollution in the base language model layers. The implementation passes the standard `riscv-tests` test suite and maintains binary execution compatibility with QEMU.
Native In-Model Filesystem
Building on exact GDN addressability, an experimental filesystem branch (`rv-store`) implements a basic persistent storage layer directly in model weights. File names hash into a 1,024-slot directory structure maintained inside designated GDN states, using open addressing with linear probing to resolve hash collisions. File contents write across linked chains of DeltaNet rows, packing eight tokens per row.
The storage layer scales up to 512 files or roughly 53,000 text tokens. Because the file system resides inside the model state matrix, saving the matrix state at session termination allows persistent cross-session data restoration. Tests evaluating context continuity show Qwenputer achieving 90% accuracy on facts altered across multi-turn sessions, matching external Python memory drivers (78.9%) and outperforming raw prompt injection (30%).
Verification, Latency, and Memory Footprint
Hardware correctness was tested across 18 million continuous free-running instructions without reference machine intervention or execution drift. Token-level analysis across 230,000 output positions verified zero state mismatches against a native target core. On NUPA long-integer arithmetic tasks, accuracy improved from base Qwen 3.5’s 48.6% to 100%, matching execution environments backed by an external Python interpreter tool. Base text generation performance remains uncorrupted, recording a KL divergence of 6e-4 against un-modified model output (against a baseline noise floor of 5e-4).
The architecture introduces notable runtime trade-offs. Checkpoint size increases to 18.8 GB due to zero-padded hidden layer expansions. Because instruction dispatch scales with autoregressive token generation, clock cycles incur GPU decoding latencies; a batched execution harness achieves roughly 3,200 CPU trace tokens per second running on a single NVIDIA L4 GPU.