«

Jan 08 2018

A Simple Function to Return the Result of a “Negative” Checkpoint

Negative tests check that the application or system under test (AUT/SUT) can handle gracefully invalid inputs, such as an invalid username at login, or an invalid ID number when creating a new customer. There is some controversy around the term “negative” for such tests, as pointed by Alon LInetzki here. To me, the term “Negative Testing” never appealed or seemed right. After all one is seeking to VALIDATE the SUT’S response to an input that is defined by the SPEC as invalid (most times perhaps merely by exclusion). However, whether it appeals to you or not, negative tests must be part of the repertoire of all testing professionals. This includes also test automation practitioners, who are required to implement in their code such tests, usually as checkpoints. For example, an invalid username or password at login would be validated by an appropriate message displayed by the application. In such a case, the login process would fail, but the (negative) test should pass.

This feat of negative tests sometimes lead to a quite cumbersome coding style in which for each checkpoint a complex If…Then…Else construct is written to handle the case. In the event of a negative test, it is not enough to check for the result of the process; it is also necessary to check it in the context of the input type, i.e., whether it is a valid (positive) input which should lead to the main (or vanilla) business flow, or to a different flow in which the application reacts (hopefully) appropriately to the invalid input.

For example, the code might look like this:

If(DoLogin(strUsername, GetPassword(strUsername))=micPass)Then
    If(boolNegTest)Then
        intStatus = 1 'micFail
    Else
        intStatus = 0 'micPass
    End If
Else
    If(boolNegTest)Then
        intStatus = 0 'micPass
    Else
        intStatus = 1 'micFail
    End If
End If

It is however possible to use a simple and concise formula to achieve the same, as shown in the following function:

Function StatusByTestType(ByVal intStatus, ByVal boolIsNegativeTest)
    StatusByTestType = CInt(intStatus = Abs(CInt(boolIsNegativeTest)))+1	
End Function

This formula exploits the fact that in VBScript the Boolean value of False equals 0 (zero) and True equals -1 (minus one), so that the following status and checkpoint (test) type pairs result in:

 

intStatus boolIsNegativeTest Result
0 -1 1
0 0 0
1 -1 0
1 0 1

Making it possible to reduce the above code to:

'For the sake of the example let's assume that DoLogin returns 0 on success and 1 on failure
intStatus = StatusByTestType(DoLogin(strUsername, GetEncryptedPassword(strUsername)), boolIsNegativeTest)
Reporter.ReportEvent intStatus, "Login Test", "Finished with " & GetStatusAsText(intStatus)

With the utility function GetStatusAsText helping to fill-in the appropriate message:

Function GetStatusAsText(ByVal intStatus)
    Select Case intStatus
        Case 0
            GetStatusAsText = "success"
        Case 1
            GetStatusAsText = "failure"
        Case 3  
            GetStatusAsText = "warning"                                    
    End Select
End Function

By using this scheme it is possible to reduce the amount of code per checkpoint, hence making the code more generic, efficient and readable.