Get the latest Education e-news
  • Playtesting is Sovereign, Part 2

    - Lewis Pulsipher
  •  [GameCareerGuide here presents a continuation of Lewis Pulsipher's playtesting article, which continues the discussion by explaining what to do with playtesting feedback you get and how to evaluate it. You can Part 1 by clicking here.]

    What to Do With The Feedback

    "One of the early lessons to learn in writing is that feedback is good, but must be held at arm's length." - Brandon Sanderson (fantasy novelist who is completing Robert Jordan's "Wheel of Time" series)

    "Most of the letters we'd get were almost a standard form. They were like, 'Dear Sid. I liked your game Civilization. Here are the five things I would change to make it a much better game.'" Sid Meier about responses to the original Civilization video game.

    Game design, when taken to completion, is highly interactive. Playtesting sets good games apart from bad, and playtesting is (or should be) interactive.

    I include Brandon Sanderson's admonition here because it applies to some extent to game playtesting. Novel authorship is very personal, game design is much more a group effort, even if there is just one designer and many playtesters, certainly when there is a large team creating a video game. You must listen to playtesters, but you have to keep in mind that playtesters each have their own likes and dislikes, and the game designer is the person who must keep in mind the original "vision" of the game and follow it to a conclusion.

    On the other hand, game designers must practice not being ego-involved. Playtesting is an invitation to say what is wrong with a game as well as what is right. Comments are about the game, not about you. Playtesting is an invitation to say your game sucks, not that you suck.

    I am very low-key in beta playtesting, preferring to watch reactions of people rather than try to solicit opinions, in part because people often won't say negative things even when asked. I also try not to play with/against others, as 1) the designer playing in a game tends to skew results and 2) when I play, I do a worse job of playing, and a worse job of evaluating the playtesting, than if I did either alone. As I'm that strange sort of person who enjoys watching games as much as playing (a combination of people-watching and "what happens next?"), why play?

    Playtesters tend to be polite. It's hard to find out what they really think. I am skeptical that a feedback questionnaire will make a difference, though many designers use them. Rather, I sometimes try the "Six Hats" method (devised by Edward de Bono) when playtesting; specifically I'll ask players successively to put on their black hat (the judge), then the red hat (intuition and emotion) to see how they assess a game, and then the yellow hat (the positive side of assessing an idea) to see what they like about a game. With local playtesters I sometimes ask them to think of ways to make the game better (the green hat). Use Wikipedia or google "de Bono" or "Six Hats" for more information.

    Keep in mind also that people tend to like games they win. When the losers like the game you can be more certain of the value of the feedback!

    You don't need to "defend" your game to playtesters. It's your game, not theirs. They may not like what you want, and they can explain why, but in the end you have to decide what's best.

    When it comes down to it, should a designer in playtest stages do what he wants with the mechanics of play, or what the playtesters recommend?

    I believe I'm very receptive to what players suggest (or what I see that they would prefer, as they play). If people take the time to play my game, I ought to be receptive, else why bother? I think playtesters may be more likely to offer suggestions if they know the designer is receptive to them. They're certainly pleased when they play again and see that I've changed the game because of their suggestion.

    A playtester's comment may cause a change in a game, but not quite (or not nearly) the change they thought of. Playtesters point out potential problems, designers are responsible for solutions (though solutions, too, may originate with a playtester).

    This is where the "scientific" part of game design comes to the fore. The scientific method involves controlled experimentation toward answering a question, observation of results, new hypotheses, and further experimentation. Wikipedia's description of the scientific method (accessed 14 April 09) can be taken as a guide to what you're doing:

    To be termed scientific, a method of inquiry must be based on gathering observable, empirical and measurable evidence subject to specific principles of reasoning. A scientific method consists of the collection of data through observation and experimentation, and the formulation and testing of hypotheses.

    Your data is collected by observing playtesters, or by getting their accounts in writing or orally. Your hypotheses involve ways to make the game work better. You test your changes through more playtesting and see what happens.

    Yet scientific method will take you only so far. Experience helps, knowledge of games helps, background knowledge (e.g., of history and physics) helps, analysis and creativity help.

    What is obvious to experienced designers is not necessarily obvious to the inexperienced. Unfortunately, it is easy to find "wannabe" designers who have an idea and think they have a "golden egg" that will be a blockbuster game. One reason why this attitude exists is that they don't listen to playtesters, or that their only playtesters are family members who are too nice to say that the game needs changes.

    On the other hand, especially in the video game world, many players think they know "the secret," and are willing to expound at length (as shown by Sid Meier's quote above). There are all kinds of "fanboys" (and girls) who will accost the designer of a well-known, successful video game and tell the designer how it "ought" to have been designed. This is ridiculous, but doesn't stop it from happening. At some point, no matter how good your game is, you'll encounter such folks. If I had a dime for every person who said Britannia is "very unbalanced", even though playing data shows that it is very well balanced, I'd have a new tractor. The trick is to find out what the really good game players, and the ones who are willing to study and think about a game, have to say.

    There are elements in a game where there are two choices, and one will work as well as the other. Which is used is pretty much arbitrary, or so it seems to the designer. In those cases, it really is wise to choose as the playtesters choose, if only because they'll see, when they next play, that you're listening.


comments powered by Disqus