Attractor Labs

Audio World Model

Sensing the World through Sound

Listen as you read. Each world fades in while it is on screen.

Our ears have evolved over millions of years to help us make sense of our surroundings. We can hear someone coming around a corner, sense the size of a room through its echoes, or follow a passing car with our eyes closed. Hearing lets us pick up cues from our surroundings that are hidden from our other senses, including vision. This makes it central to our ability to make sense of the world around us.

Sound sources on a rainy street

In this world

Click “Close your eyes” to listen as passing cars approach and recede.

Giving machines the ability to understand the world around them requires them to make sense of these cues too. At Attractor Labs, we build audio world models so that machines can learn about the physical world through sound.

What is an audio world model?

An audio world model learns a representation of the world and how that world evolves by listening. Modeling a world from audio involves both perception and generation: perception forms this representation, enabling understanding, prediction, and planning; generation brings that world to life.

Perception

Let’s consider a person walking through a room with a boiling kettle and a fireplace. A model needs to identify the footsteps, estimate where they come from, and distinguish them from the other sounds in the room. As the person moves, it needs to update their position while preserving their identity. If they stop walking or leave the room, the model should retain what it has learned and update its confidence as time passes.

The listener’s movement matters too. When the listener turns, a stationary sound source changes direction relative to them. A model following the world through sound needs to account for both the listener’s movement and the movement of surrounding objects.

Tracking sound sources

In this world

Drag or tap the rim to turn your head. Use arrow keys for fine adjustments and Home to face forward.Facing ahead
Floor
Drag the rim to turn your head, or click a floor material to change the footsteps. The shaded area illustrates uncertainty in the walker’s position.

This information needs to be encoded in the model’s representation. We can use it to estimate a sound source’s direction, distance, or speed. We can connect it to a language model to ask questions about what is happening, to a robot’s policy to help it choose how to act, or to a vision model to give it information beyond what it can see.

We can also use the representation to predict how the world might evolve. Given the current representation and an action, a model can predict the next representation: what the world might be like after that action. This lets us explore what could happen if we move or interact with our surroundings.

We are developing perception models to form this understanding of the world from sound and enable applications in robotics, wearables, and agentic AI.

Generation

Perception infers properties of a world from sound. Generation conditions sound on those properties: which sources are present, what they are doing, the surrounding geometry and materials, and the listener’s position and orientation. A world state does not specify every acoustic detail, so a generative model should represent a distribution of possible sounds consistent with that state.

The model needs to capture dependencies between sources. Ecological validity concerns which sounds plausibly coexist in a setting, their timing and relative levels, and the relationships between events. These dependencies should constrain both source selection and event timing. When using recordings, their background noise and reverberation also need to suit the intended environment. The constellation below illustrates source selection using different groups of recordings from a shared library.

Soundscape compositions

Out in the open wood, just after first light.

In this world

Click a soundscape to highlight its sources. Hover over a source to explore alternative recordings.

Generation also requires temporal state. A source should retain its identity and ongoing activity as its position and surroundings change. New events should depend on that history and on interactions in the scene: a foot contact initiates a step sound, while a continuously burning fire continues through a change in viewpoint. The model should allow variation in individual sounds while preserving these relationships over time.

The sound emitted by a source and the sound received by a listener impose different constraints. Contact with a surface affects the emitted footsteps; distance, geometry, and materials determine how that sound propagates. Sources in the same space should share a consistent acoustic environment, with reflections and occlusion determined by their positions. The model should update these propagation paths as the listener moves or a door closes, while maintaining the histories of the sources. The world below illustrates these changes at the listener.

Sound changes with the environment

In this world

Outside · birds, insects and a gravel path

Click the ground to move, or click “Inside the house” to enter. Click “Close the door” to hear outdoor sounds become muffled.

At Attractor Labs, we're beyond excited to be building machines that perceive the world like we do and enable creators to create worlds of their own. Reach out if you're building worlds, robots, wearables, or agents.