testjutsu

One of the things I really like about the Logical Functional Model is the concept of removing verification code from the execution code. Another is updating verification data on the fly to reduce the likelihood of false positives.

These concepts are especially appealing since QTP’s in-built verification method is not worth using. Verification points are script-specific. You can’t update and re-use them multiple times within a script or share them across multiple scripts. Maintenance nightmare.

Instead, what I have done is to write a custom verification layer. Here’s how it works. At the start of my control script, I instantiate the VerificationManager, which is a class I have written to take care of verification code.

If a function that I am executing needs to do any verification, I pass the class to it by reference UPDATE: I’ve switched how I call this now. I’ve turned function libraries into classes. Each class instantiates the VerificationManager as part of its own setup and the functions can then call the VerificationManager without having to handle it as a parameter. There are several reasons for this – I’ll go into these in an upcoming post.

Within the function (now class method), there are 2 calls to the VerificationManager, one to set up the expected data and another to retrieve the actual data.

Where you call these depends on what setup you need to do beforehand. If you’re getting expected values from the application under test, you’ll want to do so before you execute the code you want to verify. If you’re grabbing your expected data from the data table, then you can do it just before you call the code to retrieve the actual data.

That probably looks like word salad on first read through. If you haven’t already, I strongly suggest you take a look at the verification chapter in Michael Hunter’s description of the LFM. It took me a couple of read-throughs to really get it, but once I did, I was impressed.

How does the VerificationManager work?
It’s as generic as I can make it. It has 4 methods.

• CalculateExpectedState(someAction)
• Verify(someAction)
• Diff
• UpdateMasterData

CalculateExpectedData and Verify both make calls to files called StateGenerators. It is the role of the StateGenerators to hold the logic and fetch the data required to do the verification. In simple terms, they are function libraries that are loaded at runtime by the VerificationManager.

You’ll notice that the parameter for each of these methods is the same. They call the same StateGenerator and this parameter tells them what name to look for. The VerificationManager loads the StateGenerator specified and executes one of two functions – either a ‘setExpectedState’ or ‘getActualState’ function.

Each call to the verification manager will use a different name. There will be one StateGenerator for each call. The StateGenerators all have 2 functions, all of them identically named. What they contain differs based on what they need to verify. There might be a series of calls to the global data table to grab data that needs to be checked, or there could be logic to calculate an expected value from a set of other values.

The setExpectedState function creates a new data sheet to hold expected data. If any of this data comes from the GlobalDataTable, then the new data sheet has an identically named column. Likewise if there are any new columns that need to be added to the Global data table for future reference, then this happens here too.

Once the code to be verified has executed, the call to Verify happens. Verify grabs expected values from the application under test and adds them to the second line of the expected data column. It then calls Diff, which (as the name suggests), compares each value of row 1 with the values in row 2.

The diff method logs both successful comparisons and failures by making a call to the standard QTP reporter. Right now, I’m going for simplicity; there’s a lot more you could do with this if you wanted. You could add weights to each verification item and log a severity message based on the weight of the failed item. You could call a custom reporter (which is a good idea, given that any time you tell QTP of a failure, it treats the entire script as failed). You get the idea.

Lastly, the Diff method calls UpdateMasterData. This routine simply loops over any difference and checks the global data table for columns of the same name. If they exist, the global data table is updated with the new value, so that if any comparison is made against it later in the script, it will be made against what we see in the system during this run.

I’ve asked my current employer for permission to post the code, as it’s completely abstract and contains no proprietary data. No word on whether they’ll go for that yet, but if so, I’ll update that here. Hopefully there’s enough here already to be useful.