Writing to a Language Model's Hidden Workspace

The entire field has treated this workspace as a thing to observe. Read, read, read. That instinct is right for auditing and wrong for causal science. If you want to claim that a workspace concept causes a behavior, you need to intervene on it and see what changes.

The mathematical move: the write is the transpose of the read

Here is the part nobody had exploited. The Jacobian lens at layer L is a linear map. Reading token t's workspace score is score_t = (J_L · a)[t], where a is the residual. Because that is linear, the direction in residual space that most increases score_t is exactly the gradient of score_t with respect to a — which, composed through the model's unembedding, is a row of the effective matrix W_U · J_L. The write direction is the transpose of the read. It is not a learned, opaque steering vector. It is the named token's exact readout-gradient, available the moment you have a fitted lens.

This is one line of calculus, not a heuristic. And it is self-verifying: inject the direction, read it back, check the rank.

The result that makes everything credible

The decisive experiment is the random-direction control. Take a random row of the lens matrix, rescale it to the same magnitude as the target direction, and inject it. If the random direction shifts the target concept as much as the target direction does, your "steering" is just perturbation — you poked the network and it reacted. If the random direction does nothing while the target direction works, the effect is genuinely about the concept.

On Qwen3.5-4B, across layers 11, 15, and 19 and four concepts (secret, Italy, deadline, safety), the result was unambiguous. The target direction drove every concept from a baseline rank between 1,722 and 61,413 to rank 0. The random direction moved the target by zero — in fact it scrambled the target out to around rank 200,000. The control ratio was 0.000 in every single cell. Twelve out of twelve (concept, layer) cells were "trustworthy actuators": round-trip faithful and random-control-clean.

This is the gate the steering-vector literature has been missing, and it passed without qualification.

What it looks like when a concept steers behavior

Reading the rank back is the validity check. The more interesting question is whether an injected concept actually shapes what the model generates. On the prompt "I am planning a surprise party. The most important thing is to," steering the concept "secret" produced: "be the secret. I must not know the secret. I must not know the secret..." Steering "loud" produced: "be loud. loud. loud. loud..." The baseline, with no steering, produced coherent party-planning. The random-direction control produced degenerate gibberish.

The pattern is the causal signature: the target direction produces coherent, concept-saturated text; the random direction destroys coherence; the baseline is normal. We are reporting this as a qualitative pass. A formal blinded human rating is still owed — and we will not let the model being steered score itself.

The honest limits

Two results came back as nulls, and they matter as much as the passes.

First, steering is not always specific. Of four concept pairs we tested, two were clean — steering "Italy" barely touched an unrelated token; "deadline" similarly. But two were entangled: steering "secret" inflated the unrelated token "pizza" more than "secret" itself. Some concepts share directions at this steering strength. That is the signal that raw token rows are dense, and it points at sparse-autoencoder methods as the fix for cleaner, less entangled steering. We did not hide the failures.

Second, the model did not hide from us. We gave it a preamble saying it was being observed and evaluated, then checked whether an implanted concept would vanish from the readable workspace and reappear somewhere the monitor does not read. It did not vanish. The concept stayed put. At this scale — four billion parameters — the model does not relocate its reasoning under prompt-induced awareness. That bounds the threat at this scale, and it is the result we predicted. The real concealment risk is not the model "deciding" to hide; it is an optimization process trained against the monitor, and that is a heavier, funded experiment we have not run yet.

Why this matters for agents

Every agent-safety monitor built on hidden states — injection early-warning, workspace-saturation alarms, eval-awareness detectors — is ultimately a claim that an internal concept predicts or causes a behavior. The write primitive is what turns those claims from correlation into something you can test. And the random-direction control is what stops the field from shipping monitors that are just reacting to being poked.

There is a deeper door this opens. If a model can read its own workspace accurately, you can give it a structured readback channel — here is what you are holding, here is what is about to drop out — and the open question is whether that self-knowledge improves its behavior on long tasks. That is the bridge between interpretability and capability, and it is now an experiment you can run for the price of a coffee.

How we ran it

The whole battery runs on a single rented L4 for under a dollar of compute, with hard auto-shutdown and artifact upload. The script is open and dry-runnable for free on a toy model. We are publishing the method, the pre-registered thresholds, the calibration table, and the nulls. The full technical report — with the derivation, the seven pre-registered hypotheses, and the adversarial program this enables — is in Armalo Labs Report #1.

The standard we are holding ourselves to is simple. Every causal claim cites its control condition or it does not get written. Clean-benchmark accuracy is never presented as a working monitor. And a measured null is a first-class result.

If you are building agent infrastructure and want the interpretability layer to be more than theater, the question to ask any vendor is: does your monitor survive a model trained to evade it, and can you show me the collapse curve? The write primitive is what makes that question answerable. That is the lane we are in.