Game Career Guide is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Get the latest Education e-news
 
  • Playtesting is Sovereign, Part 1

    [08.10.10]
    - Lewis Pulsipher

  • "Blind"/External Testing

    The third stage is "blind" testing, where someone who is not involved in developing the game is given the game and must play it without any intervention from the designer. For video games, there may be people from, or hired by, the studio to observe this testing, but they should absolutely not try to influence the players in any way. For Civilization IV Firaxis recruited people from the Civilization fan community to test the game. In some cases it's better to have people who like the type of game you're making, but have no connection with the particular franchise or series. For example, if you're making "Halo IV," you want external testers who are Halo fans, but you also want people who like shooters but for some reason haven't played recent versions of Halo much.

    At least one company specializes in setting up external testing for studios that cannot or will not do it themselves. The slides for a presentation at the Triangle Game Conference about this kind of service can be seen at this link.

    For tabletop games this is a big test of the rules, somewhat akin to video game "bug testing". Are the rules clear enough that people can play the game from the rules? What questions do the blind testers come up with, and how can the rules be improved as a result? Unfortunately, nowadays people are often poor rules-readers, so I advocate use of video tutorials to help people learn how to play a game, yet those tutorials are rarely done until the game is "done".

    I know from experience with published games that there will ALWAYS be people who misread rules, sometimes willfully. 99 percent clarity of detail is about the best you can get using the English language. Similarly, most video games have many glitches until the first patch is issued several months after the game is released.

    Even when you don't intend to change the rules, rewriting them introduces unintended consequences . When you rewrite to change a rule, the repercussions are often larger. So a remarkable amount of testing is needed. This is something like the experience of programming, where you change the program to fix a "bug", but sometimes another bug results.

    In a sense, video games can jump to "blind" testing quickly, because by their very nature these games hide the rules from the players, enforcing them through the programming. This is a major advantage of video games over tabletop, that no one needs to read and understand a set of rules.

    The most interesting testing occurs when I have let a game "lie fallow" for a year or more. By that time I don't remember much of the rules, so reading them is almost like someone reading them for the first time. I confess I sometimes wonder what the heck I was thinking when I wrote this or that!

    In the video game world it is difficult to quickly and cheaply make big changes in a prototype. This is one of the problems that all makers of electronic games face, and a major reason why some video games are not very good. By the time the development studio has a prototype ready for testing outside of the company, it's too late in the schedule to make many of the changes that playtesting reveals are necessary or perhaps only "desirable".

    Rules testing and bug testing

    Software of any kind requires "bug testing", looking for failures in programming that cause the software to not work as desired. The equivalent for a tabletop game is rules testing. There are two parts. First, do the rules cover all possibilities? Second, is that coverage sufficiently specific and concise that almost everyone is going to understand what it means?

    Change, change, change-love it or fail

    "The most important thing to remember is this: To be ready at any moment to give up what you are for what you might become." W.E.B. Du Bois

    You must be willing to change your game again and again and again in a search for, not perfection, but something close to it.

    Sooner or later the Law of Diminishing Marginal Returns comes into play: the amount of time it will take to identify and successfully implement an improvement will not be worth the value of the improvement.

    Possibly the most important skillset of a game designer is the ability to analyze play of prototypes, identify problems, and provide practical solutions to anything that undermines the quality of the game.

    Analyze

    This involves self-criticism: you must learn to recognize faults even when you are the one responsible for them. The goal is not to "do nothing wrong", it's to "do a lot of things right". If you're only interested in doing nothing wrong, you'll probably end up with bland pablum, an unexciting and uninteresting game. You're better off working to make things go right, and if a few go wrong, that's worth all the right that occurs.

    Identify

    A game can appear to work pretty well, and still have flaws. The designer must identify those flaws even if no one else has recognized them! Because once the game gets out to the public, the collective minds of many players WILL identify those flaws. Again, "Nobody is as smart as everybody."

    I have had games that had not changed significantly in months of play, which I felt were "done", then I recognized I had to make a significant change.

    Solve problems

    Everyone can help here, and playtesters who can identify and solve problems are worth treasuring. In video game terms, increasingly everyone on the production team is expected to try to identify problems and propose solutions. This is yet another example of the maxim, "two heads are better than one". It's also a reflection of the "1-10-100 rule" fundamental to participative management schemes such as TQM (Total Quality Management). In any case, the game designer(s) must ultimately be responsible for deciding what solution to pursue. The game designer must be "in charge" yet still cooperate with the rest of the team and keep them feeling that they're contributing to the game.

Comments

comments powered by Disqus