Stop 1: This stop is when all cases are executed within the set time and the system has met the acceptance criteria. Most of the times this stop will occur after several releases.
Stop 2: This is the most common one since most of the time Testing is done, but the system is not OK (acceptance criteria have not been met). The decision now has to be made by the client if he still want to go to the next stage, or that he will give the testing team a new assignment (time, money etc) for retest.
Stop 3: This is the most uncommon stop since testing is not yet completed (not all cases are executed for example), but the acceptance criteria already have been met.
The completion criteria are for the test process itself and the acceptance criteria for the quality of the test object. IEEE829 speaks of the completion criteria as "suspension and resumption criteria". But almost always, we don't use them in our planning and reporting of test activities. This might be because the client only wants to know: can I go into production (Go/No Go)? This is a question we should not be answering for him as testers. We only should provide him with the information on which he can base his own conclusion. So if he asked for 5 pounds of quality and we found that the object only was about 4,5 pounds, it could be that he is willing to take the 0,5 pound of risk and go into production anyway. But it should be his choice and not (even as an advice) the testers'.
Coming back to the post of Michael; Nice and funny post, but if he would have set up his completion and acceptance criteria correctly, would this approach still stand? I agree that most of the times there is no strategy or we haven't thought the criteria trough properly. So to conclude the quote from Michael can be said for having these criteria as well:
If we don't care now, why were we testing in the first place? Have we lost track of our priorities? If someone has checked out, why? Sometimes businesses get less heat for not knowing about a problem than they do for knowing about a problem and not fixing it—might that be in play here?
Posts
I'd hold that "acceptance and completion criteria" are covered as part of the Mission Accomplished heuristic.
ReplyDelete---Michael B.
If the acceptance, completion or exit criteria are set correct and management supports the criteria. We always know when testing should stop or resume!
ReplyDeleteProblem with these criteria is that most of the time new criteria's are added.
Ewald Roodenrijs (www.testingthefuture.net)
Part of the reason for testing is that we don't know if the acceptance, completion, or exit criteria are set correctly. At the outset, we might think we have the right criteria, but at any time, testing may reveal information that challenges our concept of correctness.
ReplyDelete---Michael B.
@Michael -- With testing you mean the complete process I presume? So the review of requirements, functional design etc as well as the criteria set by the management? Then yes; that is the reason for testing!
ReplyDelete@Ewald -- If it's not on paper it's not a criteria :) And if it is, the assignment must be revised. Somewhere along the way a baseline must be made where no alterations are to be made (in my opinion)
ReplyDeleteReinder,
ReplyDeleteyou are correct. I mean when it's on paper.
-Ewald
Somewhere along the way a baseline must be made where no alterations are to be made (in my opinion)
ReplyDeleteI encourage you to think this through: no alterations are to be made? To use the kind of example that Jerry Weinberg likes to use: what if someone dies? What if there's a fire in the server room? If you think those are ridiculous, try this one: what if a competitor comes out with a product that renders our current product so lame that it would be better not to release? What if we discover that a competitor is coming out with a product in three months, such that if we release in one month, we could get a big head start on dominating the market?
Or, to put it another way, what do you see or hear that helps you to believe that a tester's opinion is more important than the business' opinion? Why should the business be asked to lock itself down for the convenience of the tester?
---Michael B.