Thinking Aloud method – a software prototype test with users

Description of the test setup:

The test was performed in the flat of a group member of our “Moove”- development team. Each user was asked to test the game for about 15 minutes. In advance, he received a short, written instruction, containing the following tasks:

The users had to choose a song without help or further instruction and were asked to play both available songs once in 2-player-mode.

We acquisited basic demographic data about the users who are 25 (male) and 23 (female) of age and 1,84m and 1,65m tall. Both users do not have any experiences with interactive displays so far. They are familiar with the game principle of guitar hero, but without having played it frequently. The man is a musician, the woman is not.

During the test, we journalized all important events to record the user’s comments and behaviour. After the testing period, in which the user only received questions that keep him verbalizing his thaughts or actions, we asked a set of questions to further examiine the user’s experience with the game. These are the mentioned questions:

  • How was the firs impression of the game?Has the choice of a song been easy and without difficulty?
  • Are all areas, containing the symbols which the player is expected to hit in time, good reachable? Is the challenge to hit these areas well balanced?
  • Is the game giving clear feedback about the benchmark of a movement, wether the symbol had been hit correctly or not?
  • Are you satisfied concerning the (visible) separation of player 1 and player 2?
  • Is it acoustically recognizable, if you repeatedly fail to hit the symbol correctly?
  • Do you think of the game being funny and entertaining or rather frustrating?

The major issues we discovered throughout this test and the severity rating we gave them are:

  • too many sheet music input -> too challenging
  • 2-player-mode is still confusing
  • sheet music, background music and “symbol-events” are not percetly synchronised
  • green areas of player 2 do not set apart from the green user -> confusing
  • the loose-score does not work properly so far -> challenge fluctuates between too easy and  too difficult

Possible solutions for these issues are:

  • reducing amount of sheet music coming in
  • changing color of the areas surrounding the player
  • adjusting the score -system ->in an earlier state adapted sheet music was necessary
  • arrangement of the symbol-areas in 2-player-mode

Heuristic Evaluation

To identify usability problems in the user interface design before completion of the application, we performed an heuristic evaluation. The following program flow chart shows the structure of “Moove” and formed the basis of our heuristic evaluation.

Overall game state

Performing the evaluation we examined the following heuristics:

  • user control and freedom
  • error prevention
  • flexibility and efficiency of use
  • aesthetic and minimalist design
  • help and documentation
  • missing elements
  • errors in the program sctructure

The last two heuristics had been phrased especially for our apllication, thus they are project specific heuristics. After having done the heuristic we evaluated and compared our results, rated the severity of each problem and discussed the most important issues that needed to be adressed:

1. User control and freedom

problem: The user can touch the particular area anytime without error report releasing an error report or using score points.
severity: major problem
relevance: principle of the game is missed
status: FIXED

problem: The buttons in the menu are too small, therefore its challenging to use them.
severity: cosmetic problem
relevance: only concerning comfort
status: FIXED

problem: control too imprecise and kind of bumpy
severity: minor problem
relevance: limiting the joy of use
status: not yet fixed

2. Error prevention:
problem: identification of multiple players in single-player
severity: minor problem
relevance: maybe confusing. If it is fixed, there probably will be no more possibility to change from single- to multi-player-mode during a session of the game
status: not yet fixed

3. Flexibility and efficiency of use
problem: sheet music appearing too fastly and in a too large amount
severity: usability catastrophe
relevance: game is just not playable
status: not yet fixed

problem: appearance of sheet music and song are not synchronized perfectly
severity: minor problem
relevance: nevertheless well playable and not too annoying
status: not yet fixed

problem: placing of the symbols around the player still needs adjustment for being more efficient but without underchallenging the player
severity: minor problem
status: not yet fixed

4. Aesthetic and minimalist design
problem: The look of “moove” is a little bit confusing. The background is supposed to be just black, therefore player can more easily focuse on the game
severity: minor problem
status: not yet fixed

problem: letters are too small and difficult to read
severity: cosmetic problem
status: FIXED

problem: sheet music could look more lovely
severity: cosmetic problem
status: not yet fixed

5. Help and documentation
problem: especially in the menu there is no guidance or instructions, telling users what to do. But we suppose the game to be usable intuitionally
severity:cosmetoc problem
status: status: not yet fixed

6. Missing elements
problem: 2-player-mode is missing
severity: major problem
relevance: 2-player-mode is supposed to be a feature of major importance, because this is how we promoted the idea for our application and moreover it just will be more fun in  multi-player-mode
status: not yet fixed

problem: there are only 2 song to choose from
severity: cosmetic problem
relevance: more songs will increase long-term-amusement, but implementing more songs later will be easy if main functions of the application are developped completely. So currently it is just not important
status: not yet fixed

problem: no highscore availablle
severity: minor problem
relevance: Without highscore, there is no possibility to determine the winner in 2-player mode. There are more important problems to solve in advance, but realisation is easy.
status: FIXED

7. Errors in the program structure
problem: there is no valid stop-signal indicating the end of a session. For this reason the game starts another session or continues incorrect.
severity: usability catastrophe
relevance: Game not playable for more than one session
status: not yet fixed

problem: sometimes the game starts without sound output, a reset is needed
severity: minor problem
status: not yet fixed

The following pictures show some of the imrovements we made to the software after having done this evaluation. Because of the short time provided for this evaluation and it’s implementation, we could not solve all problems so far.

Before evaluation:

Game

After evaluation:
Game_2

Before evaluation:

Menu

After evaluation:

Menu_2

First prototype of “Moove”

Our approach to realize „Moove“, the game we introduced to you in the previous posts, is based on two parts. On the one hand there is a Processing Sketch as the main application, containing the basic part of the game. On the other hand there is a MAX/MSP Patch, which provides the real-time-interaction between movements of the player and musical events. Therefore, MAX/MSP has to be linked to Processing, enabling an exchange of relevant data in both directions. More precisely, in this case Processing and MAX/MSP send and receive OSC-messages to interact with each other.

So far we realized a menu, in which you can choose the song you want to perform in the game. Recently, there are two songs to choose from. Each song hast to be on hand in four separate tracks, each including only the song’s drums, guitars, bass guitar or vocals. In addition, a midi-data is needed, which is used to trigger the sheet music to appear on the screen in the right time and in the right amount. If the player hits the right symbol in time, the song will go on correctly, if he misses to hit the symbol in time, there will be an unpleasant sound.

At this time, only single player mode is available and there are still some problems concerning the number and timing of sheet music that has to be hit.

Here you can see two pictures of the menu and game:

Menu
Menu

Game
Ingame

You can download the latest version of our prototype here:
http://dl.dropbox.com/u/29345526/moove_prototype.zip

Paper Prototype

Now we want to share our paper protoype with you, firstly the original one, secondly the version which considers user feedback.

The first picture shows our original paper-mock-up in 2-player-mode. A simplified picture of the user, based on the camera-signal, will be displayed. This will be something like a silhouette of his body in one distinct color. There will be a display for each player, showing the recent score. Left-sided at the top of the screen, there will be an exit-button. The colors used for this protoype are chosen arbitrarily and do not match the colours for the actual game.

The sheet music, flying into the screen and towards one of the symbols surrounding a player, indicates that the particular symbol has to be touched with one part of the body. Sheet music can appear either from the left side or from the right side and the player can use hands and feet:

01_erster_prototype

This paper protoype was introduced to 2 potential users, who were asked to share their opinion and give us some suggestions to improve the game.

After confronting some users with our paper protoype, we noticed that the screen and the displayed symbols and objects are somehow confusing to the user. This is because the symbols surrounding the players on the screen are hard to distinguish. The original choice of equal colours and the players being placed to closely seemed to be the reasons.

So we decided to add a division line (will maybe be solved differently later) and to change colours of sheet music and symbols individually for each player, as you can see in the following picture. We also increased distance between player 1 and player 2:

02_Verbesserung

The mentioned changes improved the clarity and visual appearance of the game, respectively the paper protoype. In picture 3 you see the players before sheet music starts to fly in the screen:

03_gif_01

Sheet music appears on the screen and the players prepare to start the game:

04_gif_02

Sheet music reached a symbol of each player and the players try to reach this symbol as fast as they can. Player 1 misses to reach the symbol in time and the sheet music continues to move over the screen. Player 2 hits the symbol in time, the symbol-colour changes and the sheet music dissapears. Player 2 receives one score-point:

05_gif_03

Next sheet music is appearing and player 1 hits the particular symbol in time. The symbol-color changes and the sheet music disappears:

06_gif_04

Storyboard

In the previous post on December 16th we shared the technical storyboard with you, which shows the basic idea of our gaming application. But it is also important to think about possible scenarios, in which the application could be provided and used. Therefore, we also want to share a typical storyboard with you to give you an idea of such a scenario.ImageImage