Tips for running an effective board game play-test

The Invasion begins…As I find myself deeper and deeper in the design process for Valour, the more I’ve come to appreciate how tight so many commercially-available board games are. In that vein, I’ve been doubling down on the play test schedule, because (to my mind), there is no better way to smooth the experience than to have group after group of insanely smart people take on the design, find the weaknesses, exploit the loopholes, and identify the “fiddly” parts which should be unceremoniously ripped out.

After dozens of person-hours spent around the table beating up on my design, I’ve started trying to systematize the process so I can get the most out of every opportunity, and be as efficient as possible with my volunteer play-testers’ time.

Before the test

  1. Choose play-testers who are appropriate for the current stage of development that you’re in. You surely know a spectrum of gamers, possibly ranging from “Published Designers” all the way through enthusiastic novice: “Catan, that’s the one with the sheep, right?” All of these people are smart and have valuable insight and feedback for your design. I tend to like to match the development pipeline with that spectrum; i.e., when you’re trying to determine if a brand new design has any legs at all, play-testing time is not well spent explaining the concepts of victory points, area control, or resource management.
  2. Decide on which variant(s) of your rules you wish to test against. Whether this is your design’s first play-test or its hundredth, you have a set of rules. Make sure these are written down, so that testers can read them ahead of time, and refer to them without having to ask aloud in case that would broadcast their tactics. Most likely (especially in the early stages), testers will come up with variations on rules you’ve determined, or will find themselves in a completely intractable position you hadn’t accounted for. When entering the next play-test, make a cribsheet for yourself enumerating the specific mechanics you’re hoping to observe. Mine are named like “Changes from Revision 7”. It can also be valuable to test rules you discovered to be broken in previous testing sessions, to confirm whether the rule is actually broken, or if the circumstances were a fluke.
  3. Be prepared. Lots of things can happen during a test. Perhaps players are gung-ho about trying a variant they’ve come up with (that aligns with your vision), or they have feedback they want to jot down without interrupting gameplay. Make sure you have writing tools and scratch paper for each of your testers, as well as an assortment of spare bits.

During the test

  1. Remind your players that the game is unfinished. Especially as you move through the development process and begin introducing enthusiastic novices to your work. Players who are accustomed to playing finished works can occasionally become too attached to either their own victory, or a strategy you discover to be degenerate during the session which needs to be altered mid-game. A lot of frustration from testers can be avoided with a quick mention at the beginning that the victory condition for all is, in fact, the game coming out the other side stronger and more fun then it came in. Which segues nicely into the next point:
  2. Don’t be overly devoted to the rules as written. If you discover something is broken, or feel like you want to see a slightly different scenario tried halfway through a two- to three-hour play-test, you’ll be wasting an hour to ninety minutes of everyone’s time you could have been uncovering new insights. Talk it through with the testing crew, and make an executive decision about what change you’re going to make. Don’t be afraid of making material alterations to your prototype, either. I once heard at Protospiel: “If you’re not wiling to take a Sharpie to your prototype, it’s too pretty.” which I feel has (some) merit.
  3. TAKE NOTES. This ought to go without saying, but I’ve participated in play-tests where the designer had to go scrambling for a pen & paper as the first feedback started coming out. I’ll jot down notes using Evernote on an iPad if I’m on the road, or my laptop if I’m on my home turf. My notes tend to range from stuff that needs to be added to the rules: “Clarify on where army tribute has to come from (i.e.: board or supply)” to verbatim player reactions that need to be addressed: “without wine all you have to do is discard a role card? meh. *shrug*“.

After the test

  1. Follow up. Contact your testers the next day, if you can; oftentimes my play-testers have their most revealing insights after they’ve had a while to stew on things, or ‘sleep on it’. I tend to organize play-tests via email, so I’ll continue the scheduling thread, and give everyone the opportunity to chime in on everyone else’s feedback. I’ve had these threads last hours and days as testers square off about rule suggestions and how their strategies would have been impacted. This is also a great opportunity to thank them for spending time with your creation (and to determine how & if they want to be listed in the credits when you finally launch!)
  2. It’s all about the re-write. Qui scribit bis legit. (He who writes, reads twice) The notes you took during the test are just that — notes. But the plan is to synthesize them into a future version of the rules, right? I find it helpful to go over my own notes several times and process them into complete thoughts or ideas for modifications of the rules. I’ll do this with any notes written down by my testers as well, to make sure I’m understanding and absorbing everything. This way, it’s easy to connect the narrative across the entire arc of play when determining rule updates to make for my next revision.
  3. Take it all with a grain of salt. Everyone has a different preferred style of gameplay, and not every game is for every gamer. As you synthesize feedback into later revisions, try and understand where each tester is coming from. One of my most loyal testers strongly favors randomness as ways to increase re-playability, while another detests luck and wants to feel like he has complete agency over his strategy and victory. If I took each of their suggestions at face value, my design would oscillate wildly between the two ends of the luck spectrum and end up pleasing no one. At the end of they day, it’s your design, so you have to be happy with it, but don’t overdo it; if you ever want anyone to buy it, you’d better be designing with others in mind as well 🙂

Without a doubt, this only scratches the surface; play-testing and battle hardening a board game is definitely one of the trickiest endeavors I’ve ever undertaken. Was this post helpful? I’m curious what other strategies any designers out there have developed for running an effective board game play-test (or seen used while acting as a tester!) that you thought were effective. Post to comments!

  • Sounds like it could be good advice for user testing a new app or startup idea.

  • Mitch

    This is a great post – I would add one more thing for “Before Playtest”: make sure you have a serious-looking prototype.

    In fact, you mentioned that a fellow designer at Protospiel said that taking your prototype seriously with passable art, non-paper tokens and chits, and early attempts at game theme is a good habit to get into for designers.

    As a playtester, having a good board to play on and tokens to play with made playtesting that much more enjoyable as I was focused on the game mechanics and not how shoddy the game could have looked at that early stage in development.

    • woodardj

      Great point, Mitch! I’ve heard it described “If you don’t take your prototype seriously, how can I? [as a tester]”

      • Behrooz Shahriari

        I disagree. Maybe you’re testing out the mechanisms and haven’t decided upon the theme yet. In any case, art seems superfluous to me for the first playtest.

        I’ll make sure that things are functional. I’ll usually use paper in front of real cards in sleeves when hidden information is an issue, or if the prototype is a later one. But for a first prototype when hidden info isn’t an issue, I’ll just use paper instead of cards.

        And I tend to remake components and cards so often that spending time trying to make it look presentable might just motivate me to stick with something for longer than I should.

        Of course, this may be down to the type of game I make – my games have ranged from 2-50 minutes. If I were asking folk to devote 3 hours to something, I’d probably spend a bit more time making things look better.

        To me, the thing you should be spending time on isn’t art or iconography. That time should be spent thinking about the ramifications of the prototype, maybe ripping up some ‘cards’ or brainstorming different ideas.

        • woodardj

          There is a thin line to be walked, for sure. The opposite corollary to “If you don’t take your prototype seriously how can I” is: “If you won’t take a Sharpie to it mid-playtest, it’s too pretty”.

          I’m definitely in favor of really rough prototypes when you’re fleshing out mechanics; that said, even at the stage Valour is, I’m finding that if I use words and text notation on player action cards, the cognitive effort for players to read it (instead of recognizing icons) is detracting from their ability to choose quickly.

          It’s definitely an issue with a lot of schools of thought. Behrooz, do you tend to test with the same group of players, or do you recruit different people every revision? I’d be curious to see how that correlates.

          • Behrooz Shahriari

            I don’t tend to recruit players. I do most of my testing at a dedicated meetup group for playtesting.

            If icons are required for faster recognition, then yes – that can become a functional issue. I’d think of an icon if it was required to facilitate play (probably within the first 5 tests, once I’ve worked out what the game needs).

            Once the game is good enough, I’d start demoing it at an Expo or taking it to ‘real’ boardgame clubs. At that point, it should have some sort of placeholder art (even if it’s just stick figures).