Clicking soldiers
Screenshot of a real time strategy interface based on mouse input.
Early mouse
Keyboards ruled the early days of computing. Most visually impaired still compute that way because of the difficulties the mouse brings them.
Fire-fighters
Fireman is a game that has many variations. It is usually played with the mouse.
Sensitivity
This stuff is easy and basic.
Wii gameplay
The Wii controller is essentially a mouse, with a lot of dimensions to keep track of... and more buttons.
Dungeon Keeper II
In games like Dungeon Keeper II and Black and White, the players hand is making roughly the same movements as the hand visible in the game.
Determinance
In the game Determinance the player moves his sword-arm directly by moving the mouse.
Gesture recognition
Black and White has a system that gives the player quick acces to his miracles by performing gestures. This system is also used to give orders to godly creatures that battle each other.
Wii target group
The target group of the Wii is not used to abstract connection between keyboard buttons and in game actions
A force feedback mouse
Force feedback physically shakes the mouse when the programmer specifics, for instance when the player is tossed around by a large monster.
by Brian Bors
The mouse is an accurate and versatile input device, it is used in games for the sighted as a pointing device to direct soldiers to specific spots, aim at enemies and turn the camera, things visually impaired gamers won't need to do often in games. Audio games usually don't use the mouse as an input device. In this article I will discuss why and examine why it could be handy to do use the mouse, and how to do it. I will back up some of my arguments with small downloadable examples.
Although recent games like LWorks' "Judgement day" and the soon to be released "Rail Racer" developed by Blind Adrenaline have clearly shown that the mouse can easily and successfully be used in audio games, it is rarely done. The first large commercial audio game that will probably not be able to be played without a mouse is Max Shrapnell, from Alchemy Game Studios, and that game is still in development. How come we can only name one of these games, while there are tons of game genres for the sighted that would be almost unthinkable without a mouse?
First off all it is important to understand that blind gamers do not use their mouse to control their computer. This is because using the cursor to click things instead of tabbing trough the different options to reach their goal, is nearly impossible for blind users, and an unnecessary challenge for the partly visually impaired. This has given the visually impaired community a distaste for the mouse, some even go so far that they are afraid they will click close buttons on programs - a very understandable fear when you are, let's say, surfing the web.
Audio games never have clickable icons, units, powers or menu options. They don't feature cursors, weapon aim points in the middle of the screen, or floating hands that pick up demons. Audio games don't have things sighted players use the mouse for, so the question the visually impaired community asks is not "Why don't we use the mouse?" but "Why would we want to use the mouse? Everything can be done with a keyboard." The community is simply not used to this type of input.
Finally, there are simply not that many audio games out there, so it could all simply bog down to the audio game designers saying. "We are planning this, but we simply haven't don it yet. We, as you, see no problem that can not be overcome in using the mouse in audio games. So shut up and stop telling us things we already know, Brian." That would make my day.
The problems mentioned by both gamers and game designers for the use of the mouse in audio games are, indeed, easily solved. Clicking things by accident is not possible when games are put into full screen mode. Sensitivity could be toned down (or scaled up) in the options menu of the game.
Clicking stuff by design is hard, but game designers that do incorporate mouse input in their audio games usually don't expect their players to click on certain thing. They might not even expect their players to click at all, they just want their players to move. The mouse buttons are no different than keyboard input, but it is the way the mouse can accurately input the hand movement into the game that makes the mouse so special. The mouse can generate more gameplay than programmers, of sighted and audio games alike, usually think about.
First of all, mouse movement has an analogue speed. A "left" key can only be pressed, or not be pressed, a mouse can move to the left with a high speed, or a low speed. This means that the mouse supports both high speed action movement (delivering punches to a certain direction with a certain speed) and low speed accurate precise movement (dismantling a highly sensitive explosive.) I have programmed an example that shows how this can be used in simple gameplay. In this downloadable example you need to catch jumping people in your net at the bottom of a burning building, there are two playing modes. One where you use the mouse and one where you use the keyboard. You only need the left and right button or need to move left and right with your mouse.
As you play the game, and get used to the controls, you will probably notice that using the mouse is not only more enjoyable, you tend to catch more people as well, because when they are close you can use your time to precisely line up with them, and when they are far away, you can get close really fast. You can download the first example game here.
Secondly. Engagement is enhanced when using the mouse. It is so much more immersive to swing your mouse to swing a sword, than to simply click a button at the right moment and hear a sword swing sound. Nintendo now uses that realisation to create immersive gameplay for the Wii, swinging a virtual bat/racket/bowling bal in real time by actually swinging a controller. But strategy god games like "Black & White" and "Dungeon Keeper" created by Peter Molyneux and "Determinance" created by Mode7 games have used the mouse as a tool to immerse the player for years. This form of interaction also increases adrenaline and will make the game more enjoyable because it give the player a feeling of "action" and "wild gameplay". Look at (or simply listen to) YouTube movies of people playing Wii boxing matches and you will know what I am talking about.
In this next example the player should listen to his breath, if he breaths in he is ready for another blow with his sword. But he could wait for the enemy to hit, if the enemy hits and the player hits right after that, he blocks the attack by hitting the enemy sword. The longer he survives, the faster the enemy will get. Again, there are two playing modes, in the first mode you use the spacebar to hit (doubles as a single button game), in the second mode, you have to move your mouse to hit. Try both modes and you will probably notice the mouse mode pulls you into the game more. The mouse mode is actually harder, but if the gameplay is fine-tuned towards mouse controlled, the difficulty could be toned down and certain gameplay elements (like, doing more damage when you swing your mouse harder, or swinging your sword in circles to disarm your opponent, see reason number four) will be possible to implant by the game designer. You can download the second example game here.
Thirdly. Sound panning, or 3D sound, can be analogously scaled when the player uses the mouse to move around or turn. This reduces the hacking sound jumps in volume or panning a player experiences when he approaches something in "steps" and uses his keyboard to move. "Super Liam" has a grid-based gameplay, sound can either hear 5 steps away, or 4 steps away. Of course this could be fixed by giving super Liam a fixed speed when the left or right mouse button is pressed, but this robs the player from the possibility to run by pressing the button in fast repetition.
Four (and counting). The mouse lets the player point in more than 8 directions. It lets the player make wave like movements, complex gestures to cast spells (again, an idea I stole from sighted games.) and if players are really adept at hearing, they can point to the origin of sounds more accurately. I have programmed a simple gesture recognition system, made easier by the use of sounds(with an increasing pitch) at every "corner" of the gesture. In the example the player is attacked by ogres and imps and must combat them with spells. These two spells are cast by performing various gestures, and variations of these gestures. You can download the last example game here. It still only uses 8 directions and could in theory be easily controlled by the num-pad, but when this system is programmed any further than its current state it would get increasingly difficult to be controlled by the keyboard. Once professional gesture recognition systems (that are far past the scope of examples games given away freely in articles) are mixed with an auditory interface, relaxed to ignore the boundaries of the screen, properly explained in a fun to play tutorial and finally used in audio games, they will give the player a simple, quick and immersive method of control that cannot easily be matched by keyboard input. And above all, it would be intuitive.
The fifth reason, it's intuitive. Most visually impaired gamers have had the ability of sight in some part of their lives, so they could be used to pushing buttons. But not all of them have had this experience, and than picking something up and throwing it away towards your enemy with the mouse, and therefore your hand, can be much more intuitive than pushing two buttons, because you don't throw rocks with your fingers, but with your hand. It's more "real" and therefore easier to grasp. This reduces the learning curve and even makes the game more accessible to the visually impaired counterpart of the target group Nintendo is hunting for with their intuitive Wii controllers, but on a less innovating scale.
The last and sixth reason I am going to point out is the possibility to use force feedback with a mouse. A force feedback mouse is a perfect tool to expand the way to output game information to the player without simply using more sounds, this reduces the number of sound heard simultaneously and will make the game more playable and accessible. It is perfect for separating the sounds of an enemy getting hit, and the player getting hit. When the player gets hit, his mouse will let him feel it. Try making a rocking keyboard...
Aside from all of this, a mouse could nudge game designers towards innovating gameplay concepts. Maybe clicking things can be seen as a challenge instead of something impossible, if audio output is given to the player about the cursors relative position to the button. Maybe interfaces could be build that resemble point and click adventure interfaces. Maybe game designers find new uses for the mouse that can be used in sighted games. Maybe we have overlooked uses for the scroll wheel. Maybe...
Concluding the article, I am not saying that every audio game should use the mouse. Depending on genre and gameplay mechanics a game could well be better controlled by the keyboard. I even hope that audio games will someday be played with a keyboard and a mouse simultaneously like in most mainstream PC games today. This article is simple a way to point out that the mouse is indeed an almost untapped input method for audio games. I also hope that I have taken away some of the players distrust for the mouse by writing this article.
Good luck, have fun, and lobotomize some trolls for me,
Brian Bors Contact@brianbors.nl
[Discuss this article in the AudioGames.net Forum]
[The links to the example games again, Fireman example 1, Sword swing example 2 and Gestures example 3]
(c) 2007 Brian Bors, Accessibility