Loading ...
December 9th, 2008 by 
Meir Bar-Tal
This QTip is based on an answer I have given recently in the AdvancedQTP Forums.
It is not rare to find QTP users and developers feeling helpless while contemplating a test run session that goes on and on, seemingly till eternity, without any sign of progress. Such a frustrating experience would lead in most cases to killing the QTPro.exe process from the Task Manager (if possible), or in extreme cases (which are all too frequent) even to restarting the machine. Of course, the report data gathered so far would end-up corrupted, and precious time thus would be lost. Another aspect of this problem is that there is no guarantee that taking such drastic measures and re-running the test would yield different, more profitable, results.
It is for these reasons that QTP developers should carry in their toolbox a device or method to avoid such an awkward situation. At first this seems to be a quite complex challenge to conceive such a method, and as will be shown below it indeed requires combining deep QTP-specific knowledge with a blend of general architectural design considerations.
The problem stems from the fact that VB Scripts are processed by WSH (Windows Script Host) line-by-line. However, in order to break the operation of a QTP action arbitrarily (that is, without an error being detected, for which a recovery scenario could be used), you’d need to monitor the time taken by the action and compare it, let’s say, at each step with the timeout you define.
For example, let us suppose that an Environment variable MAX_ACTION_TIMEOUT is defined. Now, at each and every step you would like to see if the time has already reached the timeout, but this is absurd, since your scripts would be filled with tons of if statements that have nothing to do with your automated test proper.
Hence that we need to find a way to signal QTP to ExitAction, without having to recur to the aforementioned absurd (and tedious) possibility. Having said all that, I will show in what follows how is it possible to implement such a method quite (surprisingly) easily.
The method is the following:
-
Set the environment variable MAX_ACTION_TIMEOUT with the value of X milliseconds (so if you have 40 minutes then it would be equal to 40*60*1000=2400000 msecs).
- In a vbs file define a global variable (another possibility is to have another environment variable) to store the actual elapsed time, e.g.:
- In the same vbs as afore, write a simple function that just checks if the actual elapsed time did not exceed the timeout, as follows:
- At the beginning of each action, put the following statement:
- Now comes the point of the whole idea. In order to avoid writing If…Then statements within your actions, you need to override the basic QTP methods you use in your project and register these new versions using RegisterUserFunc (see in the QTP reference). All your new versions of the functions would look pretty much the same. At the top of each overridden function the following statement will appear:
Dim INT_ACTION_STARTTIME_MSEC
1: Public Function CheckIfActionReachedTimeout()
2: Dim intElapsedTimeMsec
3:
4: intElapsedTimeMsec = Timer-INT_ACTION_STARTTIME_MSEC
5:
6: If intElapsedTimeMsec > Environment("MAX_ACTION_TIMEOUT") Then
7: Reporter.ReportEvent, micFail, Environment("ActionName"), _
8: "Error: Timeout exceeded. Action aborted because it had executed for " & _
9: intElapsedTimeMsec & " msec (timeout: " & Environment("MAX_ACTION_TIMEOUT") & "."
10: CheckIfActionReachedTimeout = True
11: ExitAction
12: Else
13: CheckIfActionReachedTimeout = False
14: End If
15: End Function
INT_ACTION_STARTTIME_MSEC = Timer
If CheckIfActionReachedTimeout() Then Exit Function
And hence if the answer is positive, both the action and the function will be exited. As promised, the method is cost effective, requiring a minimal effort to implement.
Posted in Error Handling, Meir Bar-Tal's Blog
December 11th, 2008 at 3:43 pm
Nice article Meir! Your content is absolutely helpful to us QTP users.
Font size Suggestion:
I kindly request you/webmaster to increase the font size in this page( may be the entire site) a little bigger for easy reading :)
December 11th, 2008 at 4:15 pm
Thank you, Ramesh.
As to the font size, it is not really a problem, since you can increase it by zooming in your browser! :)
We’ll take your suggestion into consideration, though.
December 23rd, 2008 at 12:46 pm
There exists an alternative way: if one is using Your call-chain implementation: to check the variable within ‘Class_Terminate’ handler of a chain call object :). It seem to me easier to implement (when the it is already decided to use Call chain implementation anyway).
Also, using both approaches (call stack or registerUserFunc) there exist a way to stop test from outside without waiting timeout to be reached: check (time to time, not every time check is called) that some ‘flag’-file exists in, say Test’s directory. To stop test, just copy flag file to the test’s folder.
December 23rd, 2008 at 2:46 pm
Thank you for your comment.
The article only shows the basic idea, which was based on a reply I’ve given in the General Q&A forum. I agree that there are alternative ways of implementing it. However, the RegisterUserFunc would be required anyway to implement the call-chain mechanism effectively.
The method you’ve suggested with a flag file is, in my view, awkward. First because, if I understood correctly, you are talking about manually copying the file (which might be impossible under certain circumstances, such as when the script blocks keyboard input). Second, A flag in memory is much faster to check, and this way you avoid problems that may arise with a completely unrelated component - the File System. Third, the method was not intended to be about how to check some flag, but explicitly to check the time elapsed within an action to reduce the problems involved with QTP getting stuck.
December 24th, 2008 at 12:14 am
Thanks for an answer. Actually, I did’nt have in mind to contradict the proposed solution. I just wished to share some other base ideas ;-) on the same subject.
Please, elaborate, why you feel that we need to use RegisterUserFunc to implement call-chain mechanism properly?
As for me, explicit call chain implementation is required when we can’t get information on all calls in chain (when we need to) in any other way. QTP Test Object methods themselves don’t need a call chain. Function in GUI Layer, calling QTP TO methods can properly handle errors in test object methods by using exception handling mechanism (on error resume next/on error goto 0). From the other side, call chain implementation drastically affect performance (especially when I call some function in loop, say, 100000 times - normal situation when processing, say, huge tables). So, IMHO, call chain should be used where it’s use is more that overheads (such as performance defradation) only.
Now on flags…
Actually, my idea with flag does not meant to contradict to an article. It’s just its further development (imho). I agree, that checkin’ EVERY time the flag in file system would drastically affect performance. However, with proper implementation (e.g. using timers of any kind), it’s not a real problem. The idea was to let user (or batch run tool) STOP test if he/it wish to (and can’t do it in normal way), not just waiting until it will be finished because of timeout. For night run, timeout is a great stuff! For controlled run, ability to stop test event if it can’t be done in normal way is important as well. Frankly, the idea emerged in the times of QuickTest Pro 6.5 - 8.2, when there were severe problems with stopping test in normal way (’cause we could’nt debug in libraries that time). Now situation get much better, and, agree, there is no serious need to do so now. However, if it’s so easy now (using a call chain), why not ;)?
December 24th, 2008 at 8:16 am
1) Call-Chain: Though not the subject of this article, I will try my best to give a proper reply to your arguments. Regardless of the call-chain, in my practice I override ALL QTP methods, to prevent QTP from trying by brute force to continue the run (see also my article on Implementing a GUI Layer with Classes and other articles in this blog). Because On Error Resume Next .. On Error Goto 0 would not catch a situation in which the tool gets into a deadlock while trying to perform an operation on a GUI object (which is also a “normal” QTP behavior), you need to take preventive measures to avoid such a situation (such as checking if the objects are loaded - i.e., exist - beforehand). This enabled me to develop an Exception Handling mechanism that completely replaces the need for Recovery Scenarios - the impact of which is even more felt with regard to performance and are not completely reliable).
As to your claim about performance degradation, I have 2 things to say: First, if you got a specific function like GetCellData, which is called iteratively, then I agree it is logical to check for the Table existence only once before the loop begins and then let free (and do not add such a function to the call chain). Second, regardless automation - just from a testing methodological point of view - I think that checking every single value in huge tables is simply over-testing. Taking random select samples of data that represent the test cases would give you almost 100% confidence with reduced cost viz-a-viz data preparation and execution time, as well as a reduced risk of loosing your results due to corruption in case of crash (which is not too rare).
2) “Controlled Run” is in my view a polite expression for semi-automation. Such a mode should be used in development and maintenance of automated tests, but not in production. To control the run, it’s better to implement a controller object, as we have done in My System. To stop a run at will, well, you probably know that QTP sometimes (and it’s quite often!) simply insists in getting on and on even if you press stop (it didn’t improve that much since version 8.2)… So maybe an external signal is in place, indeed. However, I would use a script executed remotely (from a remote launching tool with a GUI) and change the value of an OS environment variable on the remote machine, instead of copying a file locally.
Cheers,
Meir
December 24th, 2008 at 10:36 am
Another good way to exit current action. On the other side, you should know exactly how much the action need to run in order to define the timeout parameter.
December 24th, 2008 at 10:45 am
That’s true, but there’s no bread without sweat. It is obvious that a timeout can be set only based upon expectations based on experience, or a good risk management scheme that takes into account how fast the test results must be delivered.
December 24th, 2008 at 3:19 pm
Thanks for a detailed reply, Meir!
As for call chain and excessive re-implementation of QTP built-in methods - I think that it’s very interesting discussion, and if you kindly agree to discuss it in Q&A forumns, it would be great :). I have my arguments, but, agree - this is not a right place.
2) 100% Agree. Setting environment variable (System Env. variable, not QTP one) would be more elegant solution than file-flag - thanks for a bright idea. As well, of course, I agree your ‘controlled run’ arguments - that’s exactly what was meant: stopping tests from outside (if required) when debugging/maintaining test.
3) On performance and looping. Disagree. There exist situations when we may need to check all the data on the screen. If you wish, I can provide examples from our practice - in Q&A forums (here: http://www.advancedqtp.com/forums/index.php/topic,2624.0.html)
February 26th, 2009 at 10:02 am
Nice. Why not using Recovery Senario and call this function every step?
Eyal
February 26th, 2009 at 10:24 am
Hi Eyal,
A recovery scenario is used for exception handling like when an unexpected pop-up dialog opens or a VBScript runtime error occurs. It also carries its cost in terms of performance and many times I observed it failing.
The proposed mechanism’s effect on performance is negligible and better meets the requirement of monitoring the time taken by an action to perform in a more reliable fashion.
Regards,
Meir
July 3rd, 2009 at 2:59 pm
How do you subtract milliseconds from seconds (Timer)?
intElapsedTimeMsec = Timer-INT_ACTION_STARTTIME_MSEC
July 15th, 2009 at 12:29 pm
Hi, this is good, but why dont we use it as the recovery scenario? Did you find any difference in it?
July 15th, 2009 at 12:50 pm
See my reply to Eyal above to the same question.
July 15th, 2009 at 1:46 pm
Hi Mikhail,
Quote:
“How do you subtract milliseconds from seconds (Timer)?
intElapsedTimeMsec = Timer-INT_ACTION_STARTTIME_MSEC”
As you recall I just assigned Timer to INT_ACTION_STARTTIME_MSEC, and I now deduct the previous from the current Timer to get the elapsed time. You’re right; actually the name of the variable is misleading. In practice I multiply it by 1000 - I omitted this detail from my example. I will amend it ASAP.
July 21st, 2009 at 12:44 pm
Thanks Meir.
Being a nasty validation engineer from hell ;-), I’d like to note also yet another gotcha:
Timer “returns the number of seconds that have elapsed since 12:00 AM (midnight)”
So if we start the test right before the midnight, at 12 AM the clock bells and our Cinderella finds herself shockingly looking at reset timer…
Rather small probability, but it’s still there…
But overall, the technique is really good, thanks again.
July 21st, 2009 at 2:22 pm
Thank you Mikhail, for your comment.
Being a system architect from hell ;-), I tend to miss some trivial issues related to code implementation.
I take note and the defect will be handled.
July 21st, 2009 at 2:42 pm
Hi Mikhail, thank for your comment.
Being a nasty system architect from hell ;-), I tend to miss some trivial issues related to code implementation.
I take note and the defect will be handled ASAP.