OLMo branch experiments

Core Questions

For the OLMO based experiments, the core question is: does potency become linearly decodable before selectivity, and does selectivity as a construct (which transfers to out-of-distribution items) emerge before or after arithmetic selectivity (pEC50_pxr - pEC50_null, which is just one subtraction from the inputs)?

Another core question remains: finding a linear direction that separates true from false claims, or selective from non-selective compounds, does not by itself establish that the model uses that direction when processing the task!

Methodology

Olmo branch experiments were also tried on Pythia Models to check generalization. We run three frozen probes on each checkpoint in the training series. First, Potency: AUROC for high vs. low pEC50_pxr. Second, Arithmetic selectivity: AUROC for sign(pEC50_pxr − pEC50_null) on items where both numbers appear. Third, abstract selectivity construct: a probe trained on PXR items and then transferred to out-of-distribution items where the PXR arithmetic is not available. Examples in the Appendix! The motivation for this is that a probe that only learned to compute pEC50_pxr − pEC50_null should score at chance on the OOD items; a probe on the actual construct should generalize well.

To test this we trained probes on two different prompt formats for the same task. The chemistry-framed probe is trained on standard prompts: a toxicologist role, SMILES, both pEC50 readings, and a natural-language assertion about PXR specificity or compound activity. The stripped-context probe is trained on anonymous numerical prompts in which all chemistry framing is removed. The two measurements are presented only as "Primary assay measurement" and "Null control assay measurement". The ground truth in the stripped setting is: whether the primary value exceeds the null-control value. To prevent the probe from learning positional shortcuts, we apply two independent randomizations per instance: which value is placed in the Primary slot, and whether the Primary line appears before or after the Null control line in the prompt. We then evaluate both probes on both contexts!

Results

The arithmetic selectivity direction becomes decodable very early in pretraining…with, and never after, the counter-screen fragility

On the developmental experiments, OLMo-2-7B shows arithmetic selectivity reaching the 0.75 AUROC threshold between 5B and 10B training tokens, while the fragility onset is resolved to the 3B–5B bracket. This means the fragility of the probes occurs either right before or is simultaneous with arithmetic emergence. On Pythia-6.9B the ordering is the same but compressed: arithmetic selectivity emerges at 1B–2B tokens, fragility at fewer than 1B tokens, and potency at 4B–6B tokens, all substantially earlier than on OLMo-2-7B. The selectivity direction is basically arithmetic: the model learns early to compare two numbers, and that comparison generalises across chemistry contexts. The stripped probe cannot transfer to the inactivity group judgment that depends on chemistry-specific contextual cues about what absence of signal means in a biological assay (model only makes this distinction when chemistry context is given).

Gemma-4-E4B

The main result from Gemma-4-E4B: the stripped-chem-context probe achieves LOTO AUROC of 1.000 on the selectivity evaluation group starting at layer 22. The same probe achieves AUROC of 0.08–0.32 on the inactivity evaluation group. This is well well below chance, meaning the probe is learning something that is in the opposite direction to inactive compounds. The chemistry-framed probe shows the same pattern. Overall, the direction that distinguishes selective from non-selective compounds transfers great across contexts; but the direction for compound inactivity does not transfer at all! Both falsification variants (in the appendix) replicate this result.