Tips for organizing playtesting beyond first impressions?

[runevision]

We probably all know the value of playtesting. I've read lots about it, and have had plenty of success testing "initial impressions".

However, I don't have a well working approach for having playtesters go back for additional sessions. How do you all approach this?

What I mean is, not all games are short enough to be played to completion in a single session. (My game is not particularly long but is a physically demanding VR game, so session durations are often shorter than what's common for non-physical games.)

Dumping a new player into a late part of the game doesn't work, as there is a learning element and skill progression that means the earlier areas are a prerequisite for playing the later areas.

And while it's not too hard to find volunteers to test the game, it's harder to find volunteers who are prepared to try it once, and later come back and try again, in order to cover the whole game. For some reason I haven't been able to find any articles about best practices and advice for this specific aspect of playtesting. I'd be very happy to hear about tips and experiences from people who have organized things like this in the past.

Lots of games are long like this. What's the normal way to handle this?

For anyone curious, this is about playtesting for my game Eye of the Temple.
http://eyeofthetemple.com/

[JesperSB]

Look like a nice game... Indiana Jones...

Normally I use Reddit/Gamdev... they have a Feedback friday.
https://www.reddit.com/r/gamedev/

They test your game, and gives feedback, and given that most of them are GameDev them-self... it is good feedback.

But your game is a VR.?!?.
And mine are small games, like 1 hour or less.

Invite 10 people, and let them test it, then get their Email/contact, and contact them later.
Or make a non VR version, and have a Web version. Then people can test it from home.

I make Stand-alone PC games for Steam, but I have a simple WebGL version on my own page, that people can alpha/beta test.

[Mechanical Fruits]

Playtesting is why cheat codes where invented

You just need some shortcuts to be able to jump in the timeline and to give players whatever tool/weapon/food etc.. they need to complete that part of the game.

This way you can jump around as you like and it is really essential for testing to avoid spending/wasting hours getting where you need to be in the game.

[JesperSB]

When I get to the beta testing I add a Button called "GOD Mode", where you can enter all levels, and test all things.

And a possibility to start the game at a higher level, so Beta testers don't have to do the Beginner guide again.

[jonaz.dk]

Rune, what kind of testing are we talking here? Local testing with you present or off-site play at home beta sorts?
I will assume local testing until I hear otherwise. With you present and observing to get the best possible data from the test session.

So the basic problem in trouble in repeated testing of the game of the later stages. From people who have prior experience with the game mechanics.

Is monetary or otherwise compensation for peoples time off the table for a second playtest?
I know you are a one man army and that imposes certain limits.

Have you considered have an eg. ITU intern organizing test and QA for you? That could free some time up on your hands from bugging people around. (Don't know how that would fit in your setup.. was it half-time at Unity?)

Or just brute force marathon through the whole game in a single session when you have everything you need is in there.
Possibly with you standing there with a bullwhip to encourage a proper testing session.

[runevision]

Rune, what kind of testing are we talking here? Local testing with you present or off-site play at home beta sorts?
I will assume local testing until I hear otherwise. With you present and observing to get the best possible data from the test session.

Yeah, both are on the table, and I have been doing both. My experience is that local testing works better too. Off-site can work well if people record videos of their screens and share with me, ideally thinking out loud as well, but that's hard to convince people to do consistently. The problem with getting people back for additional sessions exist in both cases.

So the basic problem in trouble in repeated testing of the game of the later stages. From people who have prior experience with the game mechanics.

Is monetary or otherwise compensation for peoples time off the table for a second playtest?
I know you are a one man army and that imposes certain limits.

That's true, it's a shoestring budget kind of situation. Small compensation is not completely off the table but would be a last desperate effort. Do you know if it's a common approach for indies to compensate game playtesting? I'm very curious how others have handled things.

Have you considered have an eg. ITU intern organizing test and QA for you? That could free some time up on your hands from bugging people around. (Don't know how that would fit in your setup.. was it half-time at Unity?)

I have Mondays off at Unity. So far time spent on organization is not the bottleneck, but if it does becomes that in the future, I could look into it.

Or just brute force marathon through the whole game in a single session when you have everything you need is in there.
Possibly with you standing there with a bullwhip to encourage a proper testing session.

That's an option too, though I think the amount of people dropping off will be high. It's an intense game to play, much more so than when sitting down with a gamepad.

[JesperSB]

Pizza is always a good motivator for students.

[runevision]

Playtesting is why cheat codes where invented

You just need some shortcuts to be able to jump in the timeline and to give players whatever tool/weapon/food etc.. they need to complete that part of the game.

Right, I do have such cheats to make life easier for testers, but it doesn't help with giving them the skills and knowledge needed, so they still need to have played from the beginning regardless.

[vonknut]

Try "Ungespiluviklere.dk", they might be so interested in the business, but not yet having "work for money", that they maybe can be lured to test for longer sessions.

Another way is to use the same testers over time. So they test the game as its developed.
But during several sessions over time.

[runevision]

Another way is to use the same testers over time. So they test the game as its developed.
But during several sessions over time.

Oh yeah, using the same testers over time is what I meant! With this thread I wanted to hear if anyone have any concrete experience with that - with getting the same people to come back for a second and third time etc. How to motivate people to do that and any tips for making it go smoothly.

[buchardt]

Hi Rune,

At Playdead we started out with a super simple setup, where we gave people a very simple questionnaire before and after a session.
Among other things we asked them for their favorite and least favorite. And if they could add one thing to the game what would it be!?
When the first guy answered 'a shotgun' we learned the hard way that gamers do not know what they want. They think they do.

Any way because of Limbo's puzzle content we couldn't use testers twice for the same content, and we also felt that a straight up 4-5 hour session was too much(even though we did conduct them)
But we ended up accepting that someone could play through first 40%, then come back and play the last 40% another day, and yes that might mean that they missed 10% in the middle.
But since we'd didn't have tests every day, the game would usually develop quit a lot in between session(which all in all was the goal =)

We kept testing very very lightweight at Playdead. No more than 4 testers - so one observer could keep track on everyone.
Try to make that any feature owner was present to evaluate their own work through the lenses of the test. Most dev are easier to convince to change things if they've sat through a session with a struggling gamer =)

So my advice I guess is to see if you can find some natural place to end a session, and then pick it up roughly at the same place for a future test session.
You do have to keep track of each testers progression, and when you evaluate second session, you need to keep in mind how and what they've played before.

/ Peter

[runevision]

Thanks Peter, that's great to hear about how you did it at Playdead!

Where did you "recruit" volunteers? Were multiple sessions of the same person usually days, weeks, or months apart?

[jonaz.dk]

Oh by the way. One of our Swedish interns had played your game at Nordic Game Conference. He was really impressed and liked it a lot.
And he would like play it again he said when this topic came up.
I can hook you guys up.

[runevision]

Oh by the way. One of our Swedish interns had played your game at Nordic Game Conference. He was really impressed and liked it a lot.
And he would like play it again he said when this topic came up.
I can hook you guys up.

Sounds great! Thanks! If he'd like to playtest the game and will be able to travel to Valby to do it, he's very welcome to contact me at rune [at] runevision.com and let me know! That goes for anyone else too for that matter.