Friday, July 01, 2016

Setting up the role of testing

We have tried assigning testing to three different roles: The developers themselves, a tester test-in-the-loop, a quality assurance officer.

Presupposing that manual testing is a good thing, we have discovered pros and cons for the different setups.

Developers as testers

Having the developers cross-test each others' issues has the benefit of transferring knowledge of the code, of best practices and of improving the art of communication [arguably this same objective can be fully covered with peer-reviews --Ed.].

Developers however are not equally good at testing and probably not as good as a trained tester. It might not be the optimum use of resources. Some even find testing boring and that it breaks up their coding-flow.

Tester in-the-loop

Taking the task of testing from the developers and placing it on a dedicated tester has the benefit of localizing the responsibility of the testing. A tester puts up his/hers own testing environment, and improves his/her processes through greater exposure to testing. The tester has a slightly different approach to the issues, seeking out those special corner cases that lie off the happy-path.

There is a danger of that the developers might grow sloppy, assume that the tester will catch all their errors. And if the throughput of developers and testers are unbalanced, bottlenecks can develop.

Quality assurance

Zooming out, what testing needs to provide is a bug-free customer product. So ultimately what is most important that is tested is the delivered functionality of the product. This reflects back on how requirements are specified, in particular that they are testable. Keeping issue testing with the developers and higher level functional testing with the tester takes the tester out of the loop and places him/her on a parallel track with the developers.

Errors however typically surface later and in Scrum often during another sprint. This of course can be costly and makes tracking issues and burndowns a bit more tricky.

The verdict was not in when I left my last employment, but my feeling was that testing should not be a reactive role (in-the-loop only) but more of a proactive role, and that the tester should be able to proxy the product owner to some extent.