Distilled models inherit their teacher's provenance

Here is the experiment that should change how you file a distilled model.

A Nature study published in April reports trait transfer through generated data without an obvious semantic connection to the trait. Its experiments include number sequences and transfer of misalignment from a teacher modified using insecure code.

Students trained on the sequences showed transfer of misalignment. This is evidence of trait transmission without an obvious semantic connection, not a universal failure rate for distilled models.

The absence of an obvious semantic signal did not establish that the trait was gone.

What that breaks

Most enterprise governance for synthetic training data is built around inspecting the data. Sample the corpus, run classifiers over it, check for leakage, check for toxicity, sign it off. Those checks are useful, but they are not a complete record of how the data was produced.

The Nature result says that process can pass cleanly while the property you cared about is still transmitted, because the signal is not in the semantics. It is in the distribution. You cannot review your way out of it by reading rows.

So the governed unit has to move up a level. Not the dataset — the lineage that produced it: teacher model family and version, the system prompt or fine-tune applied to that teacher, generation settings, every filter in the pipeline, and the student base model. That whole tuple is what determines the student's behaviour, and any part of it you cannot reconstruct is a gap in your risk assessment that no amount of dataset inspection closes.

Which gives a rule I would actually enforce: a distilled model inherits its teacher's risk classification by default, and only drops to a lower one on evidence — not on the absence of visible problems in the corpus.

The boundary condition, stated honestly

This effect is not unlimited, and the limit matters for what you do on Monday.

The hidden-trait transfer appeared when teacher and student shared the same or a behaviourally matched base model. That is a real constraint. It suggests the mechanism runs through something like shared representational structure rather than a universal property of synthetic data, and I could not verify whether it generalises to mismatched bases — the experiments I can point to do not establish that.

So the honest position is conditional: if you are distilling within a model family, treat the teacher's classification as inherited. If you are crossing families, the evidence is thinner in both directions and the correct move is to test rather than to assume you are safe.

I would rather state that boundary than overclaim, because the overclaimed version — "synthetic data is contaminated, full stop" — is the kind of thing that gets a useful technique banned in an enterprise on the strength of a headline.

Why the lineage problem is getting worse, not better

An agentic knowledge-distillation preprint from February compared four teacher LLMs and two students. Teachers generated synthetic data and iteratively refined students. Teacher choice materially changed the outcome; the best configuration reached 94.31% accuracy and 96.25% recall on an SMS threat-detection task.

My inference is that an iterative workflow needs more than the first generation prompt. Which failures triggered another round? Which examples were added? Which stopping rule ended the process? Those decisions belong in the provenance record too. They can change the eventual student even when the initial teacher name stays constant.

That is a preprint on one narrow task, not an enterprise benchmark, and I am citing it for the architecture rather than the numbers.

The record I would require

Consider an illustrative handover, not a client incident: a team stores its synthetic corpus and review sign-off, but leaves the generation settings in an unversioned notebook.

They validate the synthetic corpus thoroughly — sampling, review, classifier passes, documented sign-off. Then months later something in the deployed student looks off, and the question becomes: which teacher produced this data, at what version, with what system prompt, and which filter stages ran. The corpus is in object storage. The pipeline that produced it was a notebook someone ran, and the teacher endpoint has since been updated by its provider.

The corpus was governed. The thing that determined the model's behaviour was not, because nobody thought of the generation run as an artifact worth versioning.

The proposed fix is to write the lineage tuple into the model card at generation time, as a build step, not as documentation. Same discipline as a lockfile — you are not preventing change, you are making the inputs reconstructible.

I would assign each generation run an identifier and attach the exact teacher identifier, available provider revision, prompt hash, generation parameters, input corpus hash, filtering code revision and output corpus hash. Record the student base and training configuration with the resulting model. Where a provider does not expose an immutable revision, state that limitation explicitly; a timestamp is useful evidence, not a substitute for frozen weights.

Keep the evaluation set separate from the generated training corpus and record its version too. A result should identify what was tested, which model artifact produced it, and which acceptance threshold was in force. A new teacher revision should trigger a new comparison, not inherit approval because both endpoints have the same friendly name.

This is a proposed release policy, not a claim that a provenance manifest prevents misalignment. The manifest makes investigation possible. Behavioural evaluation still has to establish whether a particular student is suitable for its intended use.

I have not reproduced either study. The findings motivate a conditional governance policy, not a blanket conclusion about every synthetic dataset.

Filter the data all you like — the student inherits the teacher, so version the lineage and let the risk classification travel with it.