Are you positive about being negative?
Are you positive about being negative?
I don’t believe there is a distinction between Positive testing and Negative testing. I don’t believe that there is such a test as a “Positive” or “Negative” test.
There, I have said it and the heavens haven’t opened up and I haven’t been hit by a thunderbolt.
You often hear the question “What is negative testing?” and you get various answers.
Some definitions I have read/heard:
- Those test that are designed to return an error, using invalid input
- Tests that are designed to cause the system to crash.
- Tests that are designed to make the system do something “off spec"
With reference to definition 1, so if I have an integer field and I put in an alpha character I expect to get an error, yes? So what is negative about that test? I have made a positive input and expect a positive response, i.e. an error message.
With reference to definition 2, so, if I set up corrupt data I expect the system to crash, yes? Again what is negative about that? I have made a positive decision to try to force the system into an error condition to see how it handles it. Where is the negativity?
With reference to definition 3, so, I am doing something to the system that might make it do something unusual. Is this negative? I have designed tests that try to force the system to do something it shouldn’t. This is the closest that I can come to saying it is a “negative” test, but even here I am making a positive decision.
If one of the primary reasons we test is to find bugs, then we should be designing tests that are targeted at finding bugs. Any means at the tester’s disposal should be used to find those bugs, use whatever techniques you have in your toolbox, whatever tools you have and you will execute “just tests”, not positive, not negative.
What does it matter? My concern with classifying tests as positive or negative is that it is an artificial classification that clouds tester thinking. There are no positive or negative tests, only tests that exercise the system looking for bugs. I don’t understand the reason why testers want to distinguish between positive and negative, what benefit does it give?
I have heard people say “we are running out of time, only run your positive tests”. But which are those? I have a set of tests, I can’t differentiate them according to positive or negative. What people usually mean by that statement is “Only run those tests that exercise the system the way the user will normally use it”. OK, I can identify those tests, they are sometimes called “Happy path” tests, and we always create those, but they are a small percentage of the tests we prepare, I estimate less than 10% of tests fall into that category. Are they positive tests? By some definitions, yes, but what if one (or more) fails? Does that make it a negative test? Again, does it matter? A failure is a failure whatever classification is used for the test.
I don’t want a tester to go off and run a set of “positive” tests, I want him/her to execute tests that exercise the system in such a way that have a reasonable degree of probability of finding bugs. That should be by exercising the system in the way the user will normally do it and doing other things as well.
I guess that I am not going to suddenly stop all the testers in the world from using “Positive” and “Negative” tags for tests, all I can do is try to stop my own testers!