Pose landmarks are data minimisation, not anonymity

Google's new sign-language-to-text system makes a good privacy move. SL2T runs MediaPipe Holistic on the phone, discards the camera video immediately and sends pose-landmark coordinates to a server for translation. Pixels stay on the device.

That is real data minimisation. It is not the same as anonymity.

The coordinate stream is designed to preserve the motion of hands, arms, torso, head and face closely enough to translate American Sign Language into English. A representation that removes appearance can still retain proportions, timing and movement patterns. The privacy review has to follow that transformed data to the server instead of stopping where the video was deleted.

The product needs the trace to remember enough

Google DeepMind's August 12 launch says SL2T was trained on more than 100,000 hours across over 50 sign languages, with roughly a quarter of the data in ASL. On Pixel 11, it powers ASL-to-English input in Gboard and Live Transcribe.

Its architecture solves a hard product problem. Sending raw video would expose background, clothing, bystanders and a recognisable face. Extracting landmarks on-device narrows the server's input to geometry. It also reduces bandwidth and lets the original feed disappear immediately.

But the server cannot receive meaningless geometry. Sign languages encode meaning through simultaneous whole-body movement, including non-manual markers and spatial constructions. SL2T bypasses intermediate gloss labels and translates the landmark sequence directly. The useful signal and the residual privacy risk live in the same time series.

This is the uncomfortable rule for every representation change: removing one class of information does not prove that the remainder has lost identity.

rendering diagram…

Skeleton research supplies the counterexample

A 2023 AAAI paper tested whether skeleton trajectories could leak information that was no longer obvious to a human looking at them. Across three action-recognition architectures, its preliminary classifiers achieved an average 87% accuracy on the dataset's gender label and 80% top-1 person re-identification accuracy before a dedicated anonymiser was applied.

The experimental detail matters. The paper used ETRI-activity3D for gender-label classification and NTU RGB+D 60 for identity, both action-recognition datasets rather than sign-language input. ETRI used 3D positions for 25 body joints; its gender experiment used subject-disjoint splits. The NTU re-identification setup put the same people in train and test under different camera views. These are not SL2T's public coordinate schema, users, device or task.

So the percentages do not transfer to Google's product, and I would not quote “80% identifiable” as an SL2T finding. The paper establishes the narrower counterexample we need: removing RGB pixels did not, by itself, remove identity and attribute signal from a motion representation. The researchers had to train an adversarial anonymisation layer to suppress it while preserving action recognition.

Google's launch post does not disclose the production landmark fields, the server-side retention period, whether the coordinates receive an additional privacy transform, or whether re-identification was tested. Absence from a launch post is not evidence that the controls are absent. It means an external reviewer cannot treat immediate video deletion as evidence that the server input is anonymous.

Move the control boundary with the data

In a privacy design review, I would give the landmark sequence its own row—not tuck it under “video: deleted.” That row needs purpose, access, retention, deletion and secondary-use limits, plus the exact representation and version. If the coordinates are kept for model improvement, debugging or abuse detection, each retention purpose needs an owner and an expiry.

Then attack the representation you actually retain. Can a held-out classifier link two sessions to the same consenting test participant? Can it infer a protected or sensitive dataset label better than the agreed baseline? How much does normalising body scale, reducing temporal precision or adding a learned anonymiser change translation quality? Record both utility and attack accuracy; “looks abstract” is not a metric.

The translated text is a separate boundary again. Deleting a pose sequence after translation does not govern the message, search or prompt it produced. A useful data-flow record therefore names all three lifetimes: ephemeral camera frames, server-side motion coordinates and the resulting text wherever the user chooses to send it.

This extends an earlier AndyMental rule: location is not a security control. Here, format is not a privacy control. On-device extraction and raw-video deletion are architectural properties worth keeping, but the transformed stream still needs named controls of its own. The same logic behind a behavioural AI inventory applies: inventory what the system can retain and reveal, not only the component or file type it no longer stores.

SL2T matters because it makes a neglected language interface genuinely usable, and its raw-video boundary is better than shipping every frame to a server. The next step is not to dismiss that work. It is to describe the residual data honestly enough that teams can protect it.

Delete the pixels, keep the benefit—and make purpose, retention and identity testing follow the motion trace that remains.