During February I participated in two accessibility focused game jams: Access Reset and Games from Blind Gamers jam 3. Games from Blind Gamers used audience rating, with Access Reset having judges, which gives me the chance to act on two different kinds of accessibility feedback: Expert and Player. 

Jam runner thanks:

Games for Blind Gamers jam 3:

@nightblade99

Access reset:

@cameron_keywood

@A11yGameJam

(keep an eye out for future jams!)

Access Reset Judges:

• SightlessKombat
• littlemotac_
• Arevya
• Tarja Porkka-Kontturi
• Radderss 
• The Wobbly Gamer

(Who you should be following as they’re all excellent content creators and disability advocates)

You can also checkout all the entries to both jams here:

• https://itch.io/jam/accessreset-jam/entries
• https://itch.io/jam/games-for-blind-gamers-3/entries

Prelude: Why do we need accessibility feedback?

With the growing set of resources for developers to use for designing accessible games, from the Game Accessibility Guidelines, to APX patterns and GA conference talks, why do we need to gather accessibility feedback from players? For me it comes down to two main reasons:

First the principle of “nothing about us without us”. This statement is an important part of modern disability advocacy, and part of its core is that accessible solutions cannot be built without lived experience of disabilities being included in the process.  

Second, every game's barriers and challenges are unique, because every game is unique. Just as usability heuristics haven’t removed the need for UX testing, verification that a design is working as intended is key (particularly as clearly describing to players with disabilities the accessibility features a game has before purchase is particularly important). 

Identifying issues

I’m going to divide issues identified in the feedback into two categories:

• Design issues: These occur when a design doesn’t fulfil its goals.

For example: Audio description of the level doesn’t allow the player to orient themselves and understand the position of the party.

• Implementation issues: These occur when a design works in general, but not in specific situations.

For example: Monsters of the Greenwood had a high contrast option, however this didn’t alter sprites to have sufficient contrast in all cases.

These categories aren't hard lines, but more of a spectrum which I find it's useful to sort issues into once they’ve been identified. (Also sometimes a system has so many implementation issues that it becomes a design issue.) Additionally it's best to present issues with associated quotes from players, however I’m not doing that here as I don’t have explicit permission from those who gave feedback to do so.

I’ll use the following format to present issues: 

Name	Effect	Severity (1-5)
Level description	Audio description of the level doesn’t allow the player to orient themselves and understand the position of the party.	4
High enough contrast	High contrast option doesn’t alter sprites to have sufficient contrast in all cases.	3

Player feedback

Starting with the player feedback for The Journey to Kepler-186f. This game is a light management game about a spaceship on a long journey, with its primary mechanic being assigning crew between different roles on a spaceship to repair the systems in response to events (asteroid impacts, solar flares etc). The game is controllable with mouse or keyboard and has text to speech (TTS) voicing of menus and events.

Player feedback, particularly in the “review comment” style format from the jam, can look quite messy as players' understanding of the game will be quite different from you and each other. I find it particularly useful for understanding the severity of issues, both in volume of feedback about a particular problem but also in the language used. If a barrier is present, players will tell you!

In this case the major issues reported were centred around the TTS system:

TTS on new selection	TTS doesn’t interrupt when player moves to a new UI item. This causes stacking desyncs between the game and player input.	5
TTS options	TTS options (speed, volume) don’t properly function with keyboard input	3

These two issues are particularly notable as they stem from my lack of lived experience with TTS systems. When testing internally I couldn’t parse the speech when it was interrupted by changes in what I had selected, so I’d assumed not interrupting was preferable, this isn’t the case. Similarly I couldn’t parse faster TTS speeds (unlike someone experienced with using it), so hadn’t deeply tested using those options.

Barriers were also found with menu design. This stems from a classic “you are not your player issue”, unlike players I already know where everything is in the menus and how it functions, because I put it there.

Cryo navigation

Cryo UI is confusing to navigate to and leave. It's also not clear to players under what condition the UI shuts itself.

2

The final key problem was a design problem in tutorialisation of systems, and is compounded by the issues we’ve already discussed with TTS rate and players having difficulty navigating between menus.

Unclear consequences

Players don’t understand the consequences of assigning crew and events which occur in game. They also don’t have sufficient time to explore consequences due to game speed.

4

Live playthrough feedback:

The streamers Pitermarch and TalonTheDragon played through all the jam entries and this gave a great opportunity for understanding barriers presented by the game. Watching players play is perhaps the oldest form of playtesting, and provides a high density of feedback and issue identification. It's particularly useful for specifying exactly how issues are occurring, and how severe the effects are.

In this case it helped narrow down two implementation issues and identify a design issue:

Tab voicing	TTS system doesn’t voice when the player moves from within a level to their currently selected Tab in the menu.	4
Pause tutorialisation	Pause option isn’t sufficiently tutorialised so players can’t use to take a break/take in information about the game.	2

The Tab voicing issue had been described previously as an issue with unresponsive UI, however being able to view it occurring clarified what exactly was going wrong.

Design issue:

Game speed

Progression of in game events outstrips the speed of information delivery via-TTS, even at higher TTS speeds.

5

Expert feedback

Access Reset was judged by a number of disability advocates. This feedback can be treated like consultant feedback, an expert in the field giving in depth analysis of issues. The expert perspective helps this feedback hit at a good level of generality, which can help in avoiding patchwork solutions to problems. Expert feedback also tends to point out positives more than player feedback, which is nice but also useful as it can highlight effective existing solutions.

Monsters of the Greenwood is an isometric turn based tactics game about fighting giant monsters, controlled with mouse, keyboard, or controller. TTS is used for menus and audio description for levels and their navigation.

Primary issues were in the audio description system and its interaction with different control sensitivities:

Audio description and speed	When moving around the map, the cursor move speed outstrips audio description speed and it additionally does not interrupt previous voicing on moving to a new tile.	5
Controller sensitivity	Controller input moves the cursor round the level too fast, an option and potentially some kind of acceleration might alleviate this	2

A good example of how experts spot issues beyond their specific context was in  an implementation issue with text legibility. This was an issue identified as a potential barrier even though it wasn’t affecting play for the expert in practice.

Text box backgrounds

Textbox backgrounds are slightly translucent, this makes text difficult to read under some circumstances.

2

Game startup issues were also identified, another good example of how experts consider multiple scenarios, such as repeated play or the need to adjust options.

Settings access	Settings menu cannot be accessed during the game.	2
Tutorialisation	Tutorial doesn’t make all system sufficiently clear (such as attacks and damage), players also don’t have a good sandbox to learn these things.	3

Learning for future games

Ideally we’d be acting on feedback as early as possible, as attempting to fix issues late in development (or even after release) often leads to expensive and difficult to implement fixes. I think an increasingly important aspect of designing accessible games going forward will be bringing knowledge gained from previous games forward to future games.

In my case I’m considering how to bring knowledge gained in these two projects to Cellular City, a puzzle game I’m working on. The three key things I’m taking forward are:

1. Accessibility information on the game page. This was a point of positive feedback for Monsters of the Greenwood, and is something Cellular City currently lacks.
2. TTS adjustment for menu navigation. The lack of TTS interruption on new selection (and bugs with keyboard adjustment of settings) are shared between Cellular City and The Journey to Kepler-186f.
3. Audio description adjustment. Cellular City shares an audio level description with Monsters of the Greenwood, so it directly shares issues of cursor speed and overlapping description of elements.