Good experiences with outsourced QA

User avatar
EKnudsen
Level 7 - Spellcaster
Posts: 77
Joined: 04 Nov 2008, 19:29
Location: Copenhagen

Good experiences with outsourced QA

Unread post by EKnudsen » 26 Jun 2015, 10:33

Hi spiludviklers (figured that would be the correct term).

Does any of you have good experiences with outsourcing QA to professional QA companies? And if yes could I get you to share those companies, I guess it could be beneficial not just for me (us) but for other people considering outsourcing that part of their work.

Cheers,
/E

BoWunder
Level 0 - Null
Posts: 4
Joined: 29 Jun 2015, 12:06

Re: Good experiences with outsourced QA

Unread post by BoWunder » 30 Jun 2015, 00:26

As a QA/UX Manager myself, I can only offer an opinion from the other side of the fence. What I can say is thate outsourcing QA to another company is a very viable solution, as it adds objectivity and a new look on things (which is always a good thing when it comes to QAing parts that are meant to be secure, I.E. account/login information etc.).

That being said, the main factor is objectivity - you could hire a professional QA firm to do the work for you, or you could hire yuor own QA person who can do it for you, as long as they're objective (Tests the game and reports their findings, and otherwise aren't a direct part of the development of the game), both can be used. Just know that a QA company will most likely have many clients, whereas a QA Manager working for you directly only have one - you. In short, the latter can contribute 100% of his time on your game (and doing related research as well as finding tester), whereas a QA company will have multiple clients to juggle with, though usually they also have the manpower (and resources) to do so without suffering much in the quality of their work.

That said, I cannot really povide any good answers until I know more specificly what you mean by "QA" (The term have been used wildly and so, I always makes it a point to know what someone specificly means when saying "QA"). If it includes UX (User Experience), then both are good solutions, though having your own QA means he/she may better know what your goal with the game is, and thus more efficiently design tests. The negative point of having your own QA is that he may get "used" to testing the same game over and over, and thus suffer a drop in objectivity - which is why having a QA who does UX is essential (in my book atleast), as he continually works with the objective findings brought through external-user/target user tests.

User avatar
buchardt
Level 17 - Jade Raymond
Posts: 174
Joined: 09 Apr 2010, 11:25

Re: Good experiences with outsourced QA

Unread post by buchardt » 30 Jun 2015, 10:46

A few years back I started Playdeads QA efforts. That has resulted in a full time QA guy and a part time guy who helps out.
Back when I started I would have said that that was enough. But then I went through my first certification process! That was X360, which was actually peace of cake compared to the later PS3 certification(and PS Vita which I did almost alone)...

We found at Playdead, that if we concentrate on making sure the game runs well and bug free, then we use 3rd party QA to verify all the stuff we don't like to spend resioucres getting good at. Being "certification requirements". So what we do is that we keep the game rather bug free all the way until we're close to a serious release candidate and than we do a first pass with external QA. Who'll return to us with a lot of platform specific bugs and of course some more "grind" based bugs from the game.
This means that we now have a solid list of eveything that the game needs to "be able" to do to pass certification. At this without spending so much time reading up on it.
Then we do 2-4 extra passes with external QA.

So yeah we've had good experience with using external QA(https://www.vmc.com/games/), but be ware that they really wanna sell you a huge package. So a good idea here is to be super critical on what actually is needed. When we first approached VMC they wanted to throw 5000 test cases at LIMBO. And I was like WHAT!!?
Then you find out that 2000 test cases if for online multiplayer, and 1000 is for dance-mat support(joking)... And then you end up with a nice set of tests that does the thing.

I can't see how any "decent" size developers can ship a game without help from external QA. But then again there's a lot of buggy game out there, and they have to come from somewhere...

- Peter

User avatar
EKnudsen
Level 7 - Spellcaster
Posts: 77
Joined: 04 Nov 2008, 19:29
Location: Copenhagen

Re: Good experiences with outsourced QA

Unread post by EKnudsen » 30 Jun 2015, 12:13

Thank you for the input.

@Bo: I am mostly looking towards technical QA, traditional bug finding and testing on different platforms, configurations etc. I intend to keep user experience testing in house for now.

@Buchardt: Thanks for the link to vmc

Keep more links coming if you have good experiences (or some experiences) to share.

JeppeB
Level 9 - Conjurer
Posts: 98
Joined: 04 Feb 2009, 18:16

Re: Good experiences with outsourced QA

Unread post by JeppeB » 01 Jul 2015, 12:58

EKnudsen wrote:Thank you for the input.

@Bo: I am mostly looking towards technical QA, traditional bug finding and testing on different platforms, configurations etc. I intend to keep user experience testing in house for now.

@Buchardt: Thanks for the link to vmc

Keep more links coming if you have good experiences (or some experiences) to share.
Can definitely recommend Testology in UK, which have helped us keeping our games "almost" bug free for quite some years now. Good guys.
Jeppe Bisbjerg
Director of Publishing

User avatar
EKnudsen
Level 7 - Spellcaster
Posts: 77
Joined: 04 Nov 2008, 19:29
Location: Copenhagen

Re: Good experiences with outsourced QA

Unread post by EKnudsen » 06 Jul 2015, 14:13

Awesome, thanks

thowans76
Level 0 - Null
Posts: 2
Joined: 13 Jun 2016, 12:25

Re: Good experiences with outsourced QA

Unread post by thowans76 » 13 Jun 2016, 12:29

Yes, I have some experience with such company as NixSolutions. You can easily get more imformation about their QA service here http://www.nixsolutions.com/services/qu ... -services/. They are professionals and I can recommend them.

thowans76
Level 0 - Null
Posts: 2
Joined: 13 Jun 2016, 12:25

Re: Good experiences with outsourced QA

Unread post by thowans76 » 26 Jan 2017, 14:34

Outsourcing it is very interesting. Many big companies use outsourcing in their business. But there is also such thing as outstaffing https://qubit-labs.com/outsourcing-vs-outstaffing/ It is not the same, but it is also a good solution.

User avatar
BubbleWrapBoy
Level 8 - Summoner
Posts: 81
Joined: 26 Nov 2010, 12:44

Re: Good experiences with outsourced QA

Unread post by BubbleWrapBoy » 28 Jan 2017, 03:39

thowans76 wrote:Outsourcing it is very interesting. Many big companies use outsourcing in their business. But there is also such thing as outstaffing https://qubit-labs.com/outsourcing-vs-outstaffing/ It is not the same, but it is also a good solution.
Man... This, right here, is the reason people like Trump gets elected and Britain leaving EU. On their web it even says 'outstaffing provides the ability to hire great professionals without having to pay high price and taxes;'. Disgusting.

AlexPradt
Level 0 - Null
Posts: 4
Joined: 01 Apr 2021, 20:02

Re: Good experiences with outsourced QA

Unread post by AlexPradt » 02 Apr 2021, 12:18

BubbleWrapBoy wrote: 28 Jan 2017, 03:39
thowans76 wrote:Outsourcing it is very interesting. Many big companies use outsourcing in their business. But there is also such thing as outstaffing https://qubit-labs.com/outsourcing-vs-outstaffing/ It is not the same, but it is also a good solution.
Man... This, right here, is the reason people like Trump gets elected and Britain leaving EU. On their web it even says 'outstaffing provides the ability to hire great professionals without having to pay high price and taxes;'. Disgusting.
The best solution now is to create an offshore development center in Eastern Europe. It has helped my company a lot to cut costs.

Post Reply