You, Me, and Accessibility in 3D Pt. 4: Designing for Auditory Engagement

Part 4: Current Page

Extended reality (XR) experiences captivate us by augmenting our own worlds or immersing us in new ones. XR headsets take over our senses with incredibly vivid imagery and visual interactions.

1_9rl2ccBnHwWpo7x1MVEmeg

By Caitlyn Haisel, Design Director at CARNEVALE

We are often so focused on the visual wonders 3D experiences can provide that we overlook a crucial aspect of digital immersion: audio.

Audio is a method of communication. Audio is an experience. It can provide important information to us like where we are, what’s happening, and what we’re doing. It helps us orient ourselves to our environment and make an experience more immersive. Audio is a tool that, as designers, we can use to craft experiences that are engaging, inclusive, and immersive to our users in XR.

Environment and Action Descriptions

Screen readers and features like Apple’s VoiceOver are pivotal tools to making 2D experiences accessible to users who are blind or have low vision, but how does that translate to 3D?

For anything communicated through visual mediums, there should be a method to receive the same information without using visuals at all. A key component of accessibility is providing alternate methods to use an experience.

Environmental audio descriptions could be offered to give users context to the space they are exploring. They could pan through the 3D space and hear descriptions of each element around them. Additionally, actions could be described so the user can understand the interactions available to them. Best practices from existing screen readers could be leveraged, such as giving the user the ability to skip through descriptions. This gives the user more independence to navigate and perceive 3D experiences, with or without being able to see it.

1_fAF-YBVeSD2TccHWKTop-w

Audible Action Feedback

Audio can be used to not only describe the composition of a 3D environment but also communicate important feedback when a user is interacting with something in that 3D space.

When a user successfully performs an action such as picking up an object, there are usually clear visual indicators. However, if someone is unable to see the visual indicators, the same confirmation should be provided via a different medium, like audio.

Audio feedback can span a wide range from explicit to intuitive. Highly descriptive audio can tell the user in clear detail when an action is performed, such as “object picked up.” Some audio feedback can become learned and eventually intuitive, like the Apple Pay tone that’s associated with a successful transaction. Some sounds can parallel real life, like what happens if a user bumps into an object or the sound a door makes when it opens and closes.

Different forms of feedback can and should be paired. Users should have no questions as to the status of an action they are performing in an XR experience. Providing information in multiple ways allows the user to receive it in the way they navigate the world best.

1_G88iEr-_lGjgwBD7xrq8kA

Spatial Audio

Spatial audio is sound design that mimics the way our ears naturally perceive sound in 3D. This allows the user to better orient themselves to their environment and can give cues about the location of elements or activities happening in their space. Audio cues can be paired with visual cues to draw the user’s attention to important actions or elements in 3D space.

For users who are blind or have low vision, spatial audio is particularly impactful, providing important information to navigate 3D spaces.

Spatial audio is an important communication tool to make an XR experience more accessible and truly immersive. With thoughtful execution and attention to how sound works in our physical world, it can make digital worlds feel real.

1_Rxsg-hhtFhEurbFbaQUgGw

Assistive Audio for UI

In addition to 3D space, many XR experiences also have overlaid 2D user interfaces that require interaction. Assistive audio should be offered to help users navigate through the UI with the same interactions as a typical screen reader offers.

The ability to turn on important accessibility tools like assistive audio should also be easy to access, navigate, and enable, not buried under layers hierarchy in a menu. How useful are accessible features if it’s not accessible to reach them?

1_UmSX3P8UKMhXzdRbCE2YuQ

Audio Adjustments

While audio can be an invaluable tool for creating immersive experiences, it can also become overwhelming. Users should have complete control over their audio experience and be able to fully to customize it to their needs.

Audio should be created in layers that can be turned up, down, or off completely. Music. Ambient background noise. Crowds. Footsteps. Dialogue. Actions. Descriptive audio.

For example, users who are blind or have low vision may turn on descriptive audio but want to minimize distracting background noise like music. They should also be able to decide the speed at which descriptive audio is spoken.

While virtually impossible in the physical world, users could have the opportunity to acutely tune the layers of their audio engagement in digital experiences.

1_0GIK4YKOXHgZQzCkKvyaCg

Voice Control

Creating engaging experiences with audio also extends to user input. Enabling users to perform actions, give commands, and navigate using voice control makes XR immeasurably more accessible by eliminating the need for the users to physically move, see a UI, or use a controller.

Voice control alone does not make an experience inclusive to everyone, but providing users a choice to engage with an experience verbally makes our experiences more accessible and adaptable.

Audio engagement should be designed with the same care and precision as the visual components of an XR experience. Without thoughtful solutions, we risk designing and building XR experiences that replicate the inaccessibility in our physical world.

As digital experiences become more immersive and blended with our lives, we are presented with the opportunity to include, or unintentionally exclude, more people. While the rules of accessible XR experiences are not defined yet, at CARNEVALE, we seek help to create a more inclusive future.