Possible ByRef catastrophes

The example from part 1 (regarding passing objects as ByVal parameters) might lead us to believe that when passing an object to a function, we shouldn’t care less if it’s passed ByVal or ByRef, as we’re passing the object’s reference in any case. Any change to the object within the function’s scope will inevitably effect the outside world, so there doesn’t seem to be any difference between the two methods. However, there is one special case that makes all the change in the world.

Let’s imagine two new functions:

Sub DoIt(ByVal oDictionary)

    oDictionary.Add "Did It", True

    Set oDictionary = Nothing

End Sub

Sub DoIt2(ByRef oDictionary)

    oDictionary.Add "Did It 2″, True

    Set oDictionary = Nothing

End Sub

Set oDic = CreateObject("Scripting.Dictionary")

Call DoIt (oDic)

MsgBox oDic.Count 'What will this print?

Call DoIt2 (oDic)

MsgBox oDic.Count 'What will this print?

Let’s see what happens. In the first function, oDictionary will be a copy of the oDic variable, but as we’ve seen, they both point to the same object in memory. The value will be added to the actual dictionary object, and the change will be reflected in oDic as well.

What about Set oDictionary = Nothing? We’ve severed the link between oDictionary and the actual object in memory. However, oDic still points to that object, as oDictionary is a completely new pointer, which just happens to point to the same object as oDic. Set oDictionary = Nothing effects oDictionary as a variable – not as a conduit, and therefore it has NO effect on the actual dictionary object or oDic.

So when the garbage collector scans for objects to destroy, it will skip the dictionary object (as it has a variable pointing at it) and move on. And finally, MsgBox oDic.Count will print 1.

In the second function, oDictionary points to the original oDic variable, so there is of course the new value will be reflected in oDic. However, what happens here when Set oDictionary = Nothing is executed? oDictionary isn’t a copy of oDic, it IS oDic. So Set oDictionary = Nothing is actually like saying Set oDic = Nothing. And now were in a pickle.

Since we’ve severed the link between our only pointer to the dictionary and the actual dictionary object, we now got a lost object consuming memory and resources with nothing we can do about it. If we’re REALLY lucky, the dictionary held no pointers to other objects. This way after Set oDictionary = Nothing, the garbage collector will scan the memory, find our dictionary object, but won’t find any variable pointing at it or any pointer from it – and it will be destroyed. The data will still be lost forever but at least the system resources will be freed.

Do not underestimate the importance of freeing system resources. Just imagine a Scripting.FileSystemObject keeping a pointer to a file (and thus locking it), or a DB object holding an open connection to the database, and you’ll realize that there are far worse things then mere data loss.

Just for a nice closure on the whole catastrophe title, the last MsgBox oDic.Count command will throw an error, as oDic no longer points to any object which can perform the .Count action.

So be careful not to get sucked into the object reference ByRef catastrophe.

Procedure calling and ByVal / ByRef

This might seem to be the most nitpicking and technical section of the whole article, but it’s imperative to know the issue. VBScript will seem to behave so inconsistently, that it might drive you mad, were you ever to encounter such situation.

Well, what is procedure calling? It’s the different ways we can call functions and procedures and send parameters to them. This is the point where you ask yourself "is there more than one way?" and "why does it matter?". Consider our sub from a previous example:

Sub DoIt2(ByRef oDictionary)

    Set oDictionary = Nothing

End Sub

We’ve seen that when we call the procedure, it will nullify any pointer sent to it as a parameter. Or will it? It seems that the result depends on the exact way the procedure is called. Copy the two ways to QTP and try it for yourself:

Set oDic = CreateObject("Scripting.Dictoinary")

DoIt2(oDic)

Msgbox oDic.Count 'Surprisingly, this will output 0

Call DoIt2(oDic)

Msgbox oDic.Count 'Only now will oDic be nullified, and we’ll get an error.

What has happened? It seems that even though we specifically set the procedure to receive the parameter as ByRef, oDic was passed as ByVal in the first call. That makes absolutely no sense – if the call we made is somehow illegal, it should just fail. Instead it produces inconsistent, very confusing results.

After some major internet digging, I’ve found the answer in Fabulous Adventures In Coding - Eric Lippert’s Blog.

While DoIt2 oDic And Call DoIt2(oDic) Are the same, DoIt2(oDic) is NOT. The latter sends oDic as ByVal, no matter what’s specified in the actual function / procedure. In short, the design principle that governs the behavior (as Eric explains it) is as follows:

Arguments are passed ByRef when possible. If there are “extra” parentheses around a variable then the variable is passed ByVal, not ByRef.

So, why are the parentheses in DoIt2(oDic) considered "extra"? It seems that VBScript regards DoIt2(oDic) as a call to the DoIt2 procedure without parentheses, but with extra parentheses around the first parameter that’s sent (=oDic). I can’t say I fully understand WHY this is the way the command is interpreted (though Eric kinda explains it), but at least now you know about it, you can avoid it.

Here are some quick examples for the problematic and unproblematic procedure calls (adapted from Eric’s blog). Assume DoIt receives one or two parameters according to the relevant case:

Unproblematic (Parameters are passed as defined in the target procedure):

Doit x

Call Doit (x)

Call Doit (x, y)

DoIt x, y

Var = DoIt(x)

Var = DoIt(x, y)

Problematic (y will be passed according to the target procedure; x will be passed ByVal regardless of the target procedure):

DoIt(x)

Call DoIt ((x))

z = DoIt((x))

DoIt (x), y

DoIt ((x)), y

Call DoIt ((x), y)

z = DoIt ((x), y)

Summery

In both parts of the article we’ve seen how seemingly insignificant details can have a dramatic effect on our script flow and execution. Moreover, we’ve seen how the script behavior can be easily seen as inconsistent, and you can imagine how confusing debug could become.

I hope that now that you’ve read about the underlying VBScript logic in these situations, you’ll be able to address them with relative ease.