Basic Logic - The Null Hypothesis

Since the Research Hypothesis cannot be proven, the next logical step is to see if we can fail to disprove it. The Research Hypothesis (H1) is IF A THEN B (i.e. there is a relationship between A and B). The opposite, or Null Hypothesis (H0), is that there is no such relationship. This is the logical NOT of A -> B.

Expanding the truth table for the implies operator to include H0:

For NOT A -> B there is only ONE condition that does not produce a contradiction: the condition where A is NOT related to B (shaded row). The Null Hypothesis is therefore a statement that there is NO relationship between A and B.

This is the goal of the negative proof – to show that the opposite of the research hypothesis produces a contradiction and is therefore an invalid argument. So, if B = 1, then the Null Hypothesis cannot be Valid (H0 is rejected), therefore the research hypothesis cannot be Invalid!

This is also called modus tollens, negative proof, or indirect proof.

Note, however, that the Null Hypothesis cannot be proven True – for the same reasons that H1 cannot be proven True - because the validity of the argument still says nothing about the truth of the premises!

So when testing, if we get the expected result (B = 1), we reject the Null Hypothesis, because in every case where B=1, the value of NOT A -> B is False (0). This still does not prove the Research Hypothesis, since the fact that B = 1 does not prove that A = 1; A could still be 0. However, rejection of the Null Hypothesis makes it possible for the Research Hypothesis to be True.

If, however, we do not get the expected result (B = 0), we fail to reject the Null Hypothesis. This still does not reject the Research Hypothesis - it is possible that either A = 0 (we didn't actually create the test condition we intended), or A = 1 and the relationship A -> B truly does not exist (an actual software defect has been found!). Remember, the existence of the relationship A -> B was our original premise, and it could be False!

So, if we get the expected result, either the software works, or something is broken and is giving us a false result. And, if do not get the expected result, then something is broken, we just don't know what. The generic term for this is an Anomaly. This is why it takes many tests to confirm the Research Hypothesis - we need to build up a large enough body of data to believe that all those expected results are caused by working software.

So, one test failure is enough to show that something is broken, but it takes many test passes to convince ourselves that the software is actually working correctly.