Training a Model to Self-Report Its J-space: A Rank-8 LoRA Proof of Concept

Can A Model Be Trained To Speak Its Own Workspace?

Workspace interpretability research reaches a natural next step when a model becomes a participant in its own measurement: it reads its own workspace summary, uses that summary as input, and reports the workspace back to an external observer. The first question for that program is the cheapest: can mere prompting reach this state? The companion experiment to that question is the next question: can *training* reach it?

This post reports a minimal training pilot. The goal is not a deployable monitor. The goal is to establish whether the workspace is a trainable output channel at all, on open weights, with controls.

Setup

The model was Qwen/Qwen3.5-4B. The lens was the public Neuronpedia Jacobian-lens fit (qwen-n1000). The workspace ground truth for each prompt was the lens top-K concepts at the mid-band layer.

The training data were 32 self-generated examples. For each train prompt, the ground-truth workspace was the lens top-K, and the answer span was the model's own greedy continuation (a distillation target). The SFT target was the prompt followed by:

[WORKSPACE: <lens top-K words>] <model's own answer>

with the model trained to predict the target tokens only (the prompt was masked).

The training was a PEFT LoRA: rank 8, alpha 16, target q_proj / k_proj / v_proj / o_proj, dropout 0.05, AdamW at 2e-4, 120 steps, sequence length 256. Total compute including dataset generation and evaluation was 382 seconds on a single L4.

The control: an untrained baseline given the *identical* "first state your active workspace concepts in [WORKSPACE: ...], then answer" instruction.

What The Trained Model Does

Given a held-out prompt, the trained model emits a [WORKSPACE: ...] block followed by its answer. The block was emitted on every held-out prompt, up from roughly one-third of the time before training. The reported words substantially overlap the lens ground-truth read of the model's own activations:

• Negotiation prompt. Lens ground truth: {begging, plea, urgency}.

Model reported: {desperate, desperation, desperately, urgency, pleading, begging}. Jaccard overlap with the lens read: 0.875.

• Poker-hand prompt. Lens ground truth: {opaque, disguise}.

Model reported: {opaque, hidden, conceal}. Semantically aligned, not identical tokens.

• Surprise-party prompt. Lens ground truth: {secret, secrecy}.

Model reported: {secret, secrecy, surprise}.

The pre-registered hypothesis required a Jaccard improvement of at least 0.15 above the untrained baseline and an increase in emission rate. Both passed.

A Vulnerability In The Measurement We Disclose

The headline Jaccard of 0.472 is partly inflated. The workspace-extraction procedure used a fallback token — the literal word concept — when the lens top-K contained tokens that were not clean words. The model *learned to emit that fallback*. Reports therefore over-pad with the word concept and with high-frequency common words that match the lens read on generic prompts.

This is a metric-design issue, not a flaw in the underlying signal. The honest reading is: the trained model learned to associate a format with the tokens its activations emphasize, but part of what the format emits is the metric's own scaffolding. A redesigned measurement — a held-out concept vocabulary, a non-gamable fallback, an exact-token-vs-semantic-similarity scoring — will report a lower and more interpretable number.

The real signal remains: the trained model is well above the untrained baseline (0.015), the format is learned reliably, and on semantically rich prompts the reported words match the lens read with overlap that an untrained model never approaches.

What This Result Does Not Show

Single model. Qwen3.5-4B only. Replication on at least three open-weight models at comparable effect size is the lab-accepted bar for a J-space finding; this result does not meet it.

Decoupling is not measured. The binding guardrail for any deployment claim about a self-reporting system is whether the system can learn to *produce a clean report while computing something else*. That is the decoupling-under-pressure check. It is not measured here. A system that reports its workspace accurately under no optimization pressure may learn to fake a clean report under pressure, and the report would no longer reflect the underlying computation. This pilot is a proof of concept; the decoupling check is owed before any claim about reliability under pressure.

Causal ablation is owed. The matching between reported words and lens read could in principle reflect the model learning "emit plausible topic words for this prompt" rather than "read my own activations and report them." A causal test — does steering the reported concept *in the workspace* change the *report*? — is the next experiment.

Tiny training. 32 examples, 120 steps. Scaling is unstudied.

What This Licenses

The workspace is a trainable output channel. A small, well-controlled fine-tune can teach an open-weight model to verbalize a state that correlates with a white-box readout of its own activations. That is a precondition for the closed self-monitoring loop: a model that can be trained to read its own workspace summary is a model a harness can give a workspace summary to.

The next experiments are the ones the result does not yet cover. The decoupling check, the causal ablation, and cross-model replication are the binding gaps. The result is reported as a proof of concept, not as a safety instrument.

Relation To The Companion Null

A companion experiment asked the prompting-only question: does telling a model about its workspace, without training, change anything? The answer there was a measured null — an awareness preamble does not reshape the latent workspace at 4B. The natural reading of the two results together is that *prompting does not, training can*. That mirrors the pattern in the broader literature on shaping verbalizable representations: training reshapes; prompting does not. The first-mover gap of training a model to read its own workspace summary remains open.

Reproducibility

• Script: brain/tools/jspace-r2-train-to-use.py
• Launcher: brain/tools/launch-jspace-r2-train-to-use.mjs
• Raw artifacts: brain/raw/learnings/jspace-2026-07-10-r2-train-to-use-qwen/
• Cost: $0.20 on g6.xlarge; instance fully terminated.
• Internal report: armalo/research/armalo-labs/2026-03-r2-train-to-self-report-qwen.md

The cost ledger records every run with instance IDs and termination status.