Background and motivation

A common frustrating experience is having your script pop-up an error dialog in some long-forgotten function, without one having the slightest clue in its regard. What usually follows is a series of trial-and-error breakpoint runs to determine which action the script was actually trying to perform when executing the function call that triggered the error.

All programming languages suffer from such problems, but in late bound languages like VB Script this problem is really challenging. This is because in VB Script compilation is done on the basis of a line-after line interpretation during run-time, in contrast to languages such as C/C++, in which compilation at least guarantees that the function calls are syntactically correct.

For example, a faulty call to a function (wrong function name or number of parameters) would pass unnoticed in VB Script when part of a function, whereas in C++ it would trigger a compilation error.

The only way to get the error message in such cases is by actually performing the code, that is, calling the wrapping function, or by testing the inner function call on a standalone basis. In any language, it is not long until the calls to functions and object methods become too deeply nested for us humans to grasp.

Having said that, programmers do debug their code, even when bugs and error occur within extremely nested function calls. They manage to do so via a dedicated mechanism: the call-chain (sometimes referred to as the call-stack).

The call-chain shows us exactly which function called the currently executing function, which function called that function, and so on up to the root of the call hierarchy. This allows us to analyze the error’s path, and see where things started to go wrong (usually 2-3 function calls before the function we are currently in).

An example of a Visual Studio call-stack for an error can be viewed in Figure 1 below. Please notice that it shows both the functions and code lines that precipitated the error.

Figure 1: A Sample Visual Studio Error Call-Stack

Just imagine how many debug runs you could have saved just by knowing the exact order of calls which lead you to the current function. Or, how valuable would it been to know which parameters were sent into those function calls.

Possible Implementations

Now, it is possible to implement a call-chain in QTP with intensive reporting – just by sending a line to the report from any function as you enter and exit it. However, this will render your report cluttered and hence difficult to grasp. Moreover, QTP reports are inaccessible during the run-session, and that is exactly why the call-chain data is crucial.

Is the inevitable conclusion that there is no way to implement a call-chain in QTP effectively?

Well, the answer is no. There is a way, indeed, though it’s not trivial. A call-chain mechanism is just one of the rich features provided by SOLMAR’s flagship product: My System^TM. Developed by our people at SOLMAR’s R&D Division, My System^TM brings the latest advances to boost ROI on testing automation (We shall have much more to say on that in future posts; stay tuned!). Meanwhile, I thought that I could share some of my techniques and thoughts on the subject.

Though the examples given here are not meant to be used or to be an out-of-the-box solution, I do believe they can point you in the right direction for developing robust solutions of your own.

The Concept

Developing an effective call-chain requires four basic components:

1. A place to store the call-chain data.

2. A mechanism for updating the call-chain when you enter a context (by context I mean a function, sub or action).

3. A mechanism for updating the call-chain when you exit a context.

4. A mechanism for viewing the stored data mid-run (we won’t get into this one – it is basically just a simple window to peek at the stored data)

If you have a robust mechanism for updating the call-chain when you are exiting and entering contexts, and an easy to use storage for the call chain, then implementing the call-chain will be quite straightforward.

Storing the Call-Chain Data

An excellent mechanism for storing call-chain data is a stack. A stack can store data (i.e., the names of the contexts you happen to enter during the flow) in a FILO manner – First-In Last-Out. This will make sure your call chain contains all the relevant data needed to understand what got you until this point, and nothing else. Here is an example of how the stack mechanism would work:

Let’s say our code went though these functions:

Function Func1

    Call Func2

End Function

Function Func2

    Call Func3

End Function

Function Func3

    Dim x

    Call Func 4

    x = 1/0

End Function

Function Func4

    'Ended

End Function

If we call Func1, it will call Func2, which would call Func3, and it would call Func4. At this point, our call-chain stack would look like this: (“Func1”, “Func2”, “Func3”, “Func4”). Now Func4 will end, returning the control to Func3. Since our call chain is a Stack, once the code will exist Func4, it will remove the last item from our stack, make it: (“Func1”, “Func2”, “Func3”).

Line 14 will result is an error, where we could debug, and take a look at our stack, which would hold just enough data to understand how we got into Func3, and no irrelevant data (for example, the fact the we have entered and exited Func4).

If you’d like to know exactly how to implement a stack, you can take a look at Dani Vainstein’s excellent article on the matter – here.

A Naive Approach to Updating the Call-Chain Data on the Run

Well, after we got our own storing mechanism for the call-chain data, we better start recording it. The naive approach would be to simply perform an add command once we enter a function, and a remove command every time we exit it. So it should look something like this (assuming our data is stored in an oChainData stack storage, with add and remove commands):

Function Func1

    'Add to call chain

    oChainData.Add "Func1"

    '[…

    ' Function body

    ' …]

    'Remove from stack

    oChainData.Remove

End Function

Notice that we don’t have to specify the function name for our Remove command, as the stack mechanism always know to remove the top level item.

Problems with the Naive Approach

That is all fine and well, but usually our code is more complex than this – usually our functions have multiple exit gates, as shown in the following example:

Function Func1

    'Add to call chain

    oChainData.Add "Func1"

    'Code…

    'If structures etc…

    oChainData.Remove

    Exit Function

    'More Code…

    'If structures etc…

    oChainData.Remove

    Exit Function

    'More Code…

    'If structures etc…

    oChainData.Remove

    Exit Function

    'Now the regular exit gate:

    oChainData.Remove

End Function

When our function has multiple exit gates, it is just a matter of time before we forget to update the call-chain, and that would just make our whole stack misleading and useless.

And even if our code is perfect (and it never is), we still run the risk of a mid-function error that will throw us out, before letting us update the call-chain data. But such errors are exactly our motivation to be willing to put an effort in maintaining a call-chain mechanism in the first place!

A More Robust Solution

We have seen that our naive solution fails at some basic everyday cases, but perhaps we can overcome this with more complex scripting.

What these examples show is that we need an automatic component that will “listen” to our function, and tell us (or rather the call-chain) when we enter and exit it. Implementing the whole package (monitor both entering and exiting a function) might be hard, but we can do the exit part quite easily.

In order to achieve that, we could use the termination event of an object. When we write a class, anything we write within the Class_Terminate sub of the class will be executed when the class object is destroyed. So, for example:

Class KillSwitch

    Sub Class_Terminate

        Msgbox "Object terminated"

    End Sub

End Class

Set oKill = New KillSwitch

Set oKill = Nothing

'Now the message box will appear

As you can see, the object “knows” when it is destroyed. We can use this to our advantage – instead of popping up a message box, let us make our object update the call-chain. Since the object will be destroyed whenever we exit the function (either purposefully, or due to an error), we don’t have to worry about “forgetting” to update the call-chain. It could look something like this:

'The blueprint for the updater

Class ChainUpdater

    Sub Class_Terminate

        oChainData.Remove

    End Sub

End Class

Function Func1

    Dim oUpdater 'This will hold the exit kill-switch

    oChainData.Add "Func1" 'Update the chain-data

    Set oUpdater = New ChainUpdater 'Create an instance of the kill switch

    'Now, no matter when we exit the function, the kill switch will be activated

    'And the chain-data will be updated

End Function

Summary

This article introduced the concept of a call-chain and provided substantial reasons for its centrality in the automation development effort. Although the article did not provide a detailed example of a call-chain, I hope that it has successfully outlined how to build the main components that are required in order to implement such a mechanism.

The main concept explained how we can keep track of the chain-data. We have seen that the naive approach (manually update on entering and exiting a context) is impractical. We examined an alternative approach by which call-chain update is achieved automatically through an object’s kill-switch mechanism.

As aforementioned, SOLMAR Knowledge Networks has developed an automation framework with a built-in call-chain mechanism. While it is much more detailed and effective, it builds upon the same basic principles demonstrated in this article.

As a closing word, let me encourage you to share your thoughts and comments about the call-chain concept, about this article, and if you happen to dare and design, to share with the community your own call-chain mechanism!