Pairwise Software Testing

Pairwise software testing (also known as "all pairs") is a methodology which enables testers to large number of scenarios in as few test cases as possible. By definition the coverage is not as good as exhaustive software testing, but we believe that you can catch over 90% of defects this way, in a fraction of the time required to perform exhaustive testing. With pairwise software testing you make the assumption that your defects will be driven out by testing each unique pair of values. This pair can be tested with any other pair of parameters. So, you can mix and match pairs of tests, so long as each unique pair of parameter values appears once in the test set.

How effective is it?

Let's say you have a website where you offer life assurance to individuals, and the user can enter the following personal information, all of which affect processing:

Sex (male/female)
Age (18-25, 26-35, 36-45, 46-55, 56+)
Smoker (no, under 10 a day, 10-20 a day, 20+)
Employed (yes/no)
Ever had an operation (yes/no)

If you were to exhasutively test just this page, you would need 2x5x4x2x2= 160 different scenarios. Whereas this doesn't look too bad, that's 160 tests for only 5 different parameters. Think about how many different places in your systems you have a variable that affects the code path, and you can begin to see that exhaustive testing is simply impractical. If you had ten varaibles each with ten different possible values, then exhaustive testing would mean you had to perform 10 billion tests. Just how much time do you have ?

With pairwise testing, the ten billion can be whittled down to as few as 210 separate tests.

So how does pairwise software testing work?

The methodology is based on the assumption that most defects are created as a result of no more than 2 test parameters (test values) being in a certain combination. Let's explain this by example.

Consider a group of three parameters (ie three different variables), each able to take 3 values. We'll call them T1 to T3. Valid values for each is "Read", "Blue" or "Green". Now, under exhaustive testing this would require 3X3X3 = 27 tests .

Using pairwise, we must follow the rule that each pair of parameters must be tested all ways - the pairs being

T1+T2, T1+T3, T2+T3

For each test (eg T1+T2), you must test T1 for each of the three values as well as T2 for each of the three values.

Q) So, how many tests would pairwise require to test all these combinations ?

A) Nine.

Don't believe me ? Have a look at this.

T1 T2 T3

Red Red Green

Red Green Red

Red Blue Blue

Blue Red Blue

Blue Green Green

Blue Blue Red

Green Red Red

Green Green Blue

Green Blue Green

So how did you get down to that few tests ?

For a full and scary explanation of the pairwise methodology in software testing, including examples, please have a look at http://www.pairwise.org/