Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success

The Platonic Representation Hypothesis (PRH)The Platonic Representation Hypothesis (Huh et al, 2024) argues that as vision and language models scale, their representation spaces converge toward a shared statistical model of reality. The idea is that images and text are different projections of the same underlying world. As models recover that world more faithfully, they begin to agree on which inputs are similar. For example, if a vision model places two images close together in the representation space, a language model tends to place their captions close together too.

In this work, we bring PRH to robotics and ask: can physical robot motion be viewed as another projection of that same world, such that more capable embodied models better align their vision and language representations with the structure of motion trajectories?

Figure 1: a robot reaching for a mug connects physical reality to sensory observations and motion execution. A scatter plot relates Sensorimotor Alignment to success rate across twelve VLAs. — Figure 1: **Left:** We test the Platonic Representation Hypothesis (PRH) in robotics, using image–text features as the sensory representation and end-effector trajectories as the motor representation: a more capable embodied model should have sensory representations that align more closely with this motor representation. To validate this, we define **Sensorimotor Alignment** (S²). **Right:** We measure S² across 12 vision-language-action models (VLAs) and find that S² correlates positively with SimplerEnv task success rate (Pearson r = +0.717), providing empirical support for this PRH-inspired view of embodiment.

Our evidence suggests the answer is yes. To test this, we use Sensorimotor Alignment (S²), which measures the overlap between neighborhoods of a model's sensory representations and neighborhoods of end-effector trajectories. Higher alignment means episodes nearby in sensory representation space are also nearby in motor representation space.

Our first finding is that S² is positively related to downstream robot task success. We test a range of Vision-Language-Action models (VLAs) and World Action Models (WAMs) in simulation, as well as policies on real robots. Across these settings, models with higher S² tend to achieve higher task success rates. This pattern provides empirical support for extending PRH to robotics.

S² also explains, at the representation level, several phenomena that prior work has observed only through closed-loop results. For example, VLAs often ignore the language instruction and act mainly on what they see. We find that their task success correlates with the S² of their vision features, not their text features.

In practice, S² can serve as a low-cost offline proxy for identifying which models are likely to achieve higher task success before closed-loop evaluation. For instance, it can identify promising vision-language model (VLM) backbones even before costly VLA fine-tuning.

Measuring sensorimotor alignment#

For each recorded episode, the model's image and text features form its sensory representation, and the end-effector trajectory forms its motor representation. We compute S² by constructing these representations and comparing their neighborhoods, as follows.

Reference dataset. To measure cross-modal alignment, we use the DROIDDROID: A Large-Scale In-the-Wild Robot Manipulation Dataset (Khazatsky et al, 2024) and BridgeDataV2 datasets, which pair sensory inputs with physical motions. From a dataset, we sample $N$ episodes and compute both representations from each episode. For episode $i$ , we randomly sample one image frame and pair it with the task instruction. This image-text pair forms the sensory input $\mathbf{x}_i$ .

We measure sensorimotor alignment on DROID, a large dataset of real robot demonstrations. For each episode, we encode an image frame and the text instruction with the vision and text encoders of a robot foundation model, which forms the sensory representation. The recorded end-effector trajectory forms the motor representation. We use dynamic time warping (DTW) to measure distances between trajectories.

Sensory representation. For each episode $i$ , we encode the image frame and text instruction with the vision and text encoders of a robot foundation model, obtaining a sensory embedding $\mathbf{z}_i = f(\mathbf{x}_i) \in \mathbb{R}^{d}$ . Here, $f$ applies the vision encoder to the image and the text encoder to the instruction, and we average-pool the output token features into a single $d$ -dimensional vector.

Motor representation. For the sampled image frame in $\mathbf{x}_i$ , we extract end-effector positions, orientations, and gripper states over the next $h$ steps, forming the motor representation $\boldsymbol{\tau}_i \in \mathbb{R}^{h \times 7}$ . Each row contains the position coordinates $x,y,z$ , roll, pitch, and yaw angles $\phi,\theta,\psi$ , and the recorded gripper coordinate $g$ . We encode positions as vectors in $\mathbb{R}^3$ and orientation angles as rotations in SO(3), so that distances reflect the geometry of translations and rotations.

Sensorimotor alignment. We measure alignment with a mutual nearest-neighbor metric comparing the $k$ -nearest-neighbor sets in sensory and motor spaces. Greater overlap indicates stronger alignment between an episode's sensory and motor representations. To identify these neighbors, we use cosine distance in sensory space and dynamic time warping (DTW)Dynamic Programming Algorithm Optimization for Spoken Word Recognition (Sakoe and Chiba, 1978) in motor space to compare trajectories executed at different speeds. Smaller distances mean closer neighbors:

$d_{\mathrm{sensory}}(i,j) = 1 - \cos(\mathbf{z}_i, \mathbf{z}_j),$ $d_{\mathrm{motor}}(i,j) = \mathrm{DTW}(\boldsymbol{\tau}_i, \boldsymbol{\tau}_j)$

For each episode $i$ , we use $d_{\mathrm{sensory}}$ to form the sensory $k$ -nearest-neighbor set $\mathcal{N}_{\mathrm{sensory}}(i)$ and $d_{\mathrm{motor}}$ to form the motor $k$ -nearest-neighbor set $\mathcal{N}_{\mathrm{motor}}(i)$ . We then take the intersection of the two sets, divide its size by $k$ , and average over all $N$ episodes:The Platonic Representation Hypothesis (Huh et al, 2024)

$m_{\mathrm{NN}} = \frac{1}{Nk}\sum_{i=1}^{N}\left|\mathcal{N}_{\mathrm{sensory}}(i)\cap\mathcal{N}_{\mathrm{motor}}(i)\right|$

For each episode, we find its ten closest neighbors in the sensory representation space and its ten closest neighbors in the motor representation space. Higher agreement between the two neighbor sets means higher sensorimotor alignment. We repeat this for every episode to illustrate the neighborhood agreement underlying S².

Greater overlap indicates that episodes with similar sensory representations also have similar motor trajectories, as the figure below shows for a single query episode.

A query episode, "place cans into cabinet", with its four nearest neighbors by trajectory similarity (motor neighbors, top row) and by vision-text representation similarity (sensory neighbors, bottom row). Two sensory neighbors also appear among the motor neighbors and are marked with green checks; the other two are marked with red crosses. — Figure 2: **Measuring Sensorimotor Alignment (S²) by nearest-neighbor overlap.** For the query episode (left), sensory neighbors (bottom right) have similar vision-text features, and motor neighbors (top right) have similar trajectories. Green checks mark shared neighbors; red crosses mark neighbors found only in the sensory set. A larger overlap means that episodes with similar sensory representations also follow similar motor trajectories.

Of the four sensory neighbors shown, two are shared with the motor neighbor set (green checks). The other two are found only in the sensory set (red crosses). One of them even shares the query's instruction, "place cans into cabinet", yet calls for a different motion.

The overlap above illustrates the idea behind S². Our reported score uses a centered, normalized variant of Centered Kernel Nearest-Neighbor Alignment (CKNNA) from PRH,The Platonic Representation Hypothesis (Huh et al, 2024) with $k = 10$ neighbors; Appendix A.3 of our paper gives the full definition.

Conclusion#

Our results support a PRH-inspired view of embodiment: more capable embodied models tend to align their sensory representations more closely with the structure of motion trajectories. By making sensorimotor alignment measurable offline, S² helps identify promising models and VLM backbones before costly fine-tuning or closed-loop evaluation, offering a low-cost, representation-level view of robot foundation models that closed-loop success rates alone cannot reveal.

Limitations. These findings are correlations across a limited set of model families, benchmarks, and training recipes, with few models per comparison and one real-robot setting. Models with similar S² can still differ in task success, so closed-loop evaluation remains the final step in model selection. Whether training directly for higher S² improves robot task success remains an open question.

Code and reproduction#

The code repository includes released results and reproduction guides, with the available coverage documented for each experiment.

Citation#

Please cite this work as:

Xie, Yunfei, Howard H. Qian, Shiyi Lan, Kaiyu Hang, and Chen Wei. "Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success." arXiv preprint arXiv:2610.31670, 2026.

Or use the BibTeX citation:

@misc{xie2026sensorimotoralignment,
  title = {Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success},
  author = {Xie, Yunfei and Qian, Howard H. and Lan, Shiyi and Hang, Kaiyu and Wei, Chen},
  year = {2026},
  eprint = {2610.31670},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  url = {https://arxiv.org/abs/2610.31670}
}

Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success

Measuring sensorimotor alignment#

S² Correlates with Robot Task Success Rate#

S² Correlates with Task Success Across Model Families#

S² Correlates with Task Success More Closely Than Alternative Metrics#

S² Is Robust to Evaluation Choices#

Representation-Level Explanations with S²#

What Carries the Correlation: Vision, Text, or Both?#

Freezing Vision Encoders Limits S² Gain#

Features at Early Denoising Steps Already Carry Strong Sensorimotor Alignment#

Mid-Depth Blocks Carry the Strongest Sensorimotor Alignment#

Conclusion#

Code and reproduction#

Citation#