Attractor Labs

How Well do Machines Understand Space from Listening?

Direction, Distance, and Motion in Audio Language Models

Figures move as you read. Sound is optional; headphones recommended.

In our first post on audio world models, we discussed how our ears help us make sense of the world around us. Here, we look more closely at the cues we use to judge direction, distance, and movement, and how models’ spatial understanding compares with ours. Understanding space through sound means knowing where sound sources are relative to us, how far away they are, and how they move.

Key findings

We tested how well frontier models make sense of space through sound. The best-performing model scored about 35 percentage points below the published human score on MMAU-Pro.

In the best-performing model in our evaluation, direction, distance, and motion are recoverable from earlier audio representations, but become harder to recover with linear probes as those representations reach the language model. This highlights the need to build models that make sense of space through sound.

How we make sense of space

How does this come so naturally to us? Sound interacts with our head and outer ears, creating spatial cues that help us locate its source. Differences in arrival time and sound level between the ears help us judge left and right. The outer ears filter sound differently depending on its direction, providing spectral cues that help us distinguish above from below and front from back. Sound level, reverberation, and frequency content help us judge distance, while changes in these clues help us follow movement. Research also found that listeners become more accurate at estimating distance after repeatedly hearing unfamiliar sounds. Familiarity with speech can also help listeners judge how far away a voice is.1

Together, these cues and our experience help us build a sense of the space around us.

How models make sense of space

For a model, the question is how to extract these spatial cues from the signal produced by a microphone. A representation of that signal commonly used in perception models is the mel spectrogram, which shows how energy across frequencies changes over time. Subtle differences in these representations can reveal where a sound comes from. In stereo audio, for example, small timing differences between the left and right channels help reveal whether a sound source moves from left to right or from right to left. The model also needs to identify each sound in the first place. This means connecting where a sound comes from with what produces it.

Take a recording of several people having a conversation. A question might be: “Where is the person asking the puzzle situated in this room?”2 The choices are left or right.

One conversation, two channels

Preparing figure…

The playhead moves through the real recording. Click “Turn sound on” to listen, or drag the timeline to inspect the waveforms and mel spectrograms together.

The waveform shows apparent differences in amplitude, which encodes to loudness. The spectrogram shows a slighly darker left channel representation. However, answering this question requires identifying the people talking, knowing what a puzzle is, understanding which one of them is asking it, and locating them.

How well do models use these cues in sound?

We tested Gemini 4 Argon (October 2026), Qwen3.8-Omni-Flash (September 2026), MiDashengLM-Spatial (October 2026), and Spatial-Omni3 on the spatial reasoning subset of MMAU-Pro4, a benchmark for understanding speech, environmental sounds, and music. We evaluate 321 multiple-choice questions5 asking where sounds come from, how they move, and what they tell us about what is happening.

We score the answers using deterministic answer matching, with shuffled options and lowercase answer choices.6

How much of the benchmark can each model answer?

Preparing figure…

Each arc shows accuracy on the MMAU-Pro spatial subset. Click a model name to follow its arc and compare it with the published human score.

MiDashengLM-Spatial scores highest at 53.3%, followed by Qwen at 48.0%, Spatial-Omni at 42.4%, and Gemini at 39.6%. Qwen and Gemini use high reasoning. The best model7 thus scores 34.9 percentage points below the published human score of 88.2%8.

We further test whether the models use the differences between the two audio channels, and how well they answer without hearing the recording.9

Condition MiDashengLM-Spatial Qwen3.8-Omni-Flash Spatial-Omni Gemini 4 Argon
Stereo 53.3% 48.0% 42.4% 39.6%
Dual-mono 45.5% 48.9% 42.7% 36.4%
Question-only 41.4% 38.9% 34.6% 43.9%
Random baseline 34.4% 34.4% 34.4% 34.4%

Gemini 4 Argon appears to misuse the sound, scoring lower with audio than with the question alone. For the best-performing model, MiDashengLM-Spatial, adding sound improves accuracy by only about 12 percentage points.

All four struggle to connect sounds to their positions. On our example question, all four models chose “left,” while the benchmark’s reference answer is “right.”

Looking inside the model

To understand where this difficulty arises, we look inside MiDasheng and examine how spatial information reaches its language model. We follow that information through its layers, compare what we can recover with what the model uses in its answers, and test whether making spatial information more apparent helps it answer correctly.

From sound to an answer

Preparing figure…

Follow the moving signal through the two encoders, fusion, projector and language model. This is an illustration of the architecture, not a recording of model activations.

Direction

Where does direction remain recoverable?

We follow sound direction through the audio representations, their fusion, and the projector that passes them to the language model. A linear probe is a simple readout trained to recover direction from each representation.10 This lets us estimate how accessible that information is at each stage.

We play the same recordings from different directions around the listener, keeping distance and volume consistent. We then measure how accurately direction can be recovered at each stage of the model, using separate recordings for training and evaluation.11 Sound sources include knocks, human voice, footsteps, clapping, bells, and musical instruments.12

Where direction becomes harder to recover

Preparing figure…

The highlight follows measured direction error through the model. Hover over the plots, or drag “Capture point”, to inspect a layer; shading and whiskers show 95% confidence intervals.

Direction becomes easier to recover in the first spatial blocks, with average error falling to 10.7°. Error rises sharply at fusion and remains high through the projector and decoder, suggesting that precise direction becomes less accessible to a linear probe.13

The points below show where the probe predicts the sound is coming from for each of 25 recordings. The larger marker shows the actual source. As it moves around the listener, compare stages to see how the predictions spread after fusion.

Where the probe thinks the sound is

Preparing figure…

The source circles the listener; the smaller points show predictions for 25 recordings. Click a model stage to compare its predictions, or drag the rim to hold a direction; motion interpolates between measured 5° positions.

Can the model distinguish left from right?

We ask a simple question, “Is the sound coming from your left or your right?”, and compare the model’s answers with linear probes trained on its audio encoders, projector, and language-model layers, using the same recordings as before.10 We leave out sounds directly in front and behind, where left and right are ambiguous.

The probes learn from examples; the model answers directly. This lets us compare what information is available with how well the model uses it.14

Knowing left from right

Preparing figure…

The moving source illustrates the left/right task; the results beside it are measured accuracies. Click a readout to highlight its score; whiskers show 95% confidence intervals where reported.

Left and right, from every direction

Preparing figure…

The outline’s radius is accuracy at each measured direction; the outer ring is 100% and the dashed ring is chance. Click a readout to compare its outline, or drag the rim to inspect a direction.

Left/right information is accessible to a simple readout at the point where audio enters the language model. The lower accuracy of the model’s answers suggests a gap between having that information available and using it to answer the question.

Can the model distinguish four quadrants?

We extend the question to front-left, front-right, back-left, and back-right. This asks the language model to combine two spatial distinctions. We again compare what the probes recover at each stage with the model’s answers.10

We leave out directions that fall exactly between two quadrants.

Distinguishing front from back

Preparing figure…

The moving source illustrates the four-quadrant task; the results beside it are measured accuracies. Click a readout to highlight its score; whiskers show 95% confidence intervals where reported.

The blind spot behind the listener

Preparing figure…

The outline shows four-quadrant accuracy around the listener, with gaps at excluded boundaries. Click a readout to compare its outline, or drag the rim to inspect the differences behind the listener.

The audio-projector probe scores 18.7 percentage points higher than the model. The model often confuses back with front: it answers front-left for most sounds coming from back-left.

Distance

Can the model estimate distance?

We play sounds at distances from half a metre to 36 metres, from four directions. We examine how well distance can be recovered from the model’s layers.1015

We measure how far each estimate is from the true distance, whether estimates increase as the same sound moves farther away, and how much estimates vary between recordings at each distance.

Nearer and farther are not the same as metres

Preparing figure…

The moving source illustrates the distance experiment; the readouts show measured mean errors across four directions. Model error uses numeric answers only (82.5%); whiskers show 95% confidence intervals and the baseline is 9.0 m.

The probes can distinguish nearer from farther much more reliably than they can estimate the exact distance in metres. As with direction, distance estimates become less accurate at the projector.

We also ask the model to estimate distance in metres. Its numeric answers have average errors of 13–18 m across all four directions, far greater than the best probe could recover. Always predicting the middle distance gives errors of 9 m on those same examples. The model sometimes answers with descriptions such as “Far” instead of a numeric distance.

Motion

Which layers encode motion direction?

We next examine whether information about a changing sound position is accessible in the representations and used by the language model to describe movement.

We move each sound clockwise or counterclockwise around the listener along a quarter-circle. We vary where the motion begins so that every starting position occurs equally often in both directions. The ending positions are balanced in the same way. Identifying the direction therefore requires following how the position changes over time.16

At each layer, a probe reads the difference between the audio representations from the first and last thirds of the recording to predict motion direction.10 We compare it with the model answering “Viewed from above, does the sound move clockwise or counterclockwise around you?”

Following a sound as it moves

Preparing figure…

The source traces a quarter-circle, alternating clockwise and counterclockwise to illustrate the task. Click a readout to compare measured accuracies; whiskers show 95% confidence intervals where reported.

The best probe in the spatial encoder reaches 88.4% accuracy. This falls to 53.9% at the projector and 52.0% at the final decoder. The model’s answers score 49.7%, close to the 50.0% chance baseline.

Spatial reasoning

Which blocks predict the MMAU answer, and is the right answer predictable?

We follow how the language model forms an answer to the benchmark questions. Here, it needs to connect spatial information to the sounds and concepts the question refers to.

At each stage of the language model, we train one readout to predict the correct answer and another to predict the answer the model will give.1017

Both readouts are tested on the same questions, using recordings held out from their training.

The model’s choice becomes clearer than the right answer

Preparing figure…

The solid line predicts the model’s answer; the dashed line predicts the reference answer. Hover over the plot, or drag “Decoder block”, to compare their measured accuracy and 95% confidence intervals.

Near the end of the language model, its eventual answer can be predicted with 94.4% accuracy, while the best readout for the correct answer reaches 67.3%. The later representations make the model’s choice easier to recover, including when that choice is wrong.

Are directions easier to recover from SAE features?

Sparse autoencoders (SAEs) break the model’s representations into features. We test whether these features make direction easier to recover.10 At the projector, the original representations give higher accuracy on both tasks:

Readout Left/right Four quadrants
Best raw probe 100.0% 97.3%
Audio-projector probe · raw 93.3% 76.3%
Best SAE-feature probe 99.9% 97.8%
Audio-projector probe · SAE 80.0% 61.3%

The best probes are chosen using separate validation recordings.

SAE features generally make direction harder to recover in this comparison.

Can boosting SAE features improve answers?

We next identify SAE features that respond to left and right, then strengthen them to see whether the model’s answers improve.

We start with recordings placed fully left or fully right and look for features that respond consistently to each side. For other sound directions, we strengthen the feature for the side indicated by the model’s representation and let the model answer.18

Strengthening a spatial feature

Preparing figure…

The animation illustrates finding and amplifying a left/right feature by 10×. The measured answer accuracy rises from 83.3% to 85.3%; the animated feature heights are illustrative.

Boosting a left/right feature tenfold improves MiDashengLM-Spatial’s left/right accuracy from 83.3% to 85.3%, a gain of 2.0 percentage points. A tenfold front/back boost selected in our follow-up comparison leaves accuracy unchanged on the held-out questions.

Conclusion

We tested how well frontier models make sense of space from sound using the spatial understanding subset of MMAU-Pro. Even the best-performing model scored about 35 percentage points below the published human score. Our experiments on this model show that information about direction, motion, and distance can be recovered from its earlier audio representations using linear probes, but becomes harder to recover as those representations are passed to the language model. Our findings highlight the need for richer spatial understanding in audio models as well as the importance of how we assess that understanding.

  1. Spatial hearing: Sound localization and Auditory distance perception in humans. ↩
  2. Example question. This is an actual question from the MMAU-Pro benchmark. ↩
  3. Spatial-Omni. We use Spatial-Omni from version 2 of the paper (September 2026). We selected SO-AF3 MIX, the best-performing Spatial-Omni model on the MMAU-Pro spatial benchmark in the authors’ comparison (Table 17). ↩
  4. MMAU-Pro. Sonal Kumar et al. (2025). MMAU-Pro: A Challenging and Comprehensive Benchmark for Holistic Evaluation of Audio General Intelligence. ↩
  5. Evaluated questions. We score the same 321 questions with uniquely matchable reference answers across all four models. Four questions are excluded because their reference answers were not included in the option list. ↩
  6. Evaluation protocol. We shuffle the options, include the full audio in the audio conditions, and request an option letter. We also lowercase all answer choices. In the source annotations, 310/321 correct answers are placed first in the list of options, and in 210/321 questions only the correct answer begins with a capital letter. We keep the same shuffled option order across models and conditions. For this reason, our results may differ from other published evaluations, which may use shorter audio or different option ordering and capitalization. We score answers using deterministic matching. Lowercasing answer choices reduced stereo accuracy from 55.1% to 53.3% for MiDasheng (−1.9 percentage points), 49.8% to 48.0% for Qwen (−1.9 points), 44.5% to 42.4% for Spatial-Omni (−2.2 points), and 47.4% to 39.6% for Gemini (−7.8 points). Changes are calculated before rounding. “Original stereo” retains the original capitalization of the answer choices. Scoring accepts an option letter, case-insensitively (“B”, “b”, “(B)”, or “B.”), an optional “Answer:” prefix (“Answer: B”), or a leading letter followed by punctuation and text (“B. To the left” or “B: To the left”). It also accepts text that uniquely matches an answer choice after ignoring capitalization, punctuation, and extra spaces. Conflicting choices, out-of-range letters, empty answers, and unrecognized formats count as incorrect. We apply the same scorer to all models and conditions. ↩
  7. Published comparison. MiDashengLM-Spatial also leads the published MMAU-Pro spatial comparison at 55.4%, ahead of Gemini-3.6-Flash (48.3%), MiMo-V2.5 (43.1%), Qwen2.5-Omni-7B (41.2%), Audio-Flamingo-Next-Instruct (37.2%), Qwen3-Omni-30B-A3B-Instruct (36.6%), MOSS-Audio-8B-Instruct (26.8%), BAT (23.7%), and MiDashengLM-7B-1021 (18.2%). These published scores use the authors’ evaluation protocol; our comparison uses our evaluation described above. ↩
  8. Human score. The human score of 88.2% is reported on the full 325 questions in MMAU-Pro. ↩
  9. Benchmark conditions. Stereo sends both channels; dual-mono averages left and right and sends the same mix to both channels. Question-only omits audio. The random baseline averages uniform choice probabilities over the same questions. ↩
  10. Probe types. Direction: ridge regression predicting the sine and cosine of the angle. Left/right: L2-regularized binary logistic regression. Four quadrants: L2-regularized multinomial logistic regression. The raw-versus-SAE comparison uses these same classifiers. Motion: L2-regularized binary logistic regression on the change between late and early representations. Distance: ridge regression predicting metres. MMAU answers: separate L2-regularized multinomial logistic regressions for the reference answer and the model’s answer. ↩
  11. Recording setup. Direction probes use 100 training, 25 validation, and 25 evaluation recordings, each rendered at 72 positions in 5° steps, at 5 m with RMS-matched volume. All directions of a recording stay in one split. Left/right excludes 0° and 180° (1,750 evaluation examples); quadrants exclude all four cardinal boundaries (1,700 examples). The animation interpolates between measured directions. Shading and error bars show recording-level 95% confidence intervals. ↩
  12. KEMAR. We render each recording using measured directional filters from Bill Gardner and Keith Martin’s KEMAR measurements (MIT, 1994). The source recording stays the same, while front and back use different filters. Directly in front or behind, sound reaches both ears at roughly the same time and volume. Distinguishing these directions therefore depends on subtler frequency changes caused by the ears, head, and torso, captured in these measurements. ↩
  13. Could averaging hide spatial information? We repeated the comparison with earlier and later parts of each recording kept separate. The rise in direction error at fusion—from 13.5° to 48.8° with averaging—remained. Direction becomes harder to recover even when some timing information is preserved; the precise reason remains open. ↩
  14. Probe comparisons. Raw probes cover 81 capture points: both encoders, HSSC outputs, fusion, projection, and decoder blocks. Validation chooses regularization and the displayed best layer. Probes receive supervised task labels; the frozen model answers a fixed prompt. ↩
  15. Distance setup. Distances span 0.5–36 m in 0.5 m steps, at 45°, 135°, 225°, and 315°. The same 100/25/25 recording split is used. At 45°, validation-selected probe errors are 5.9 m; projector error is 13.0 m. The constant middle-distance baseline has 9.0 m error. Selected probes order distances correctly in 99.7% of comparisons. Model error is calculated only on numeric answers; ordering gives no credit to ties or pairs with missing numeric answers.
    DirectionNumeric answersMean errorCorrect distance ordering
    45°81.5%17.7 m11.1%
    135°87.6%13.6 m10.3%
    225°75.2%12.8 m5.4%
    315°85.8%14.1 m12.0%
    ↩
  16. Motion setup. Each full recording starts at 72 positions in 5° steps and traverses a 90° clockwise or counterclockwise arc, at 5 m with loudness matched. The same 100/25/25 recording split is used. Every start and end position is balanced across labels, giving an analytical 50% static-position baseline. Probes use the change in the model’s representation between the beginning and end of each recording to predict motion direction. ↩
  17. MMAU-answer probes. Separate multinomial logistic probes are fitted at all 28 decoder blocks for the reference answer and the model answer. They use the earlier original-choice MiDasheng run. Both targets share evaluation folds grouped by identical audio. ↩
  18. Feature amplification. Left/right features are identified from 90°/270° recordings; the side to amplify is inferred from unmodified activations. Strength is chosen on validation recordings. The ×10 intervention corrects 41 and regresses six answers among 1,750 examples; random-feature and opposite-side controls score 83.1% and 83.0%. ↩