testjutsu

I like to engage in good healthy discussion. I don’t mind taking a contrary position or having an argument. I want my friends and colleagues to challenge me. It helps me learn and think in different ways. I’m not a zealot about it. I’m okay with people being wrong on the internet, but every now and again there is the invitation for dialogue that I find difficult to pass up. It can be tough though when the person who has requested said dialogue is reluctant to engage.

I had just such an experience recently over on UTest’s blog where Rex Black of ISTQB fame was interviewed for ‘Testing the Limits’. Mr Black was hopeful of conversation between himself and readers of the UTest blog. Being a reader and having a number of questions I was happy to oblige.

Unfortunately the dialogue has, at least thus far, not been what I was hoping for. Now I don’t want to be unfair. Mr Black is a busy man. He did go into quite some detail on a number of the questions I asked. He indicated his lack of time a number of times during his replies. I had a number of other questions and made several statements though that I was hoping would prompt further dialogue. I’d like to think he will at some point he will take a well earned breather and spare a few minutes to come back and continue.

He was kind enough to point me in the direction of their psychographic analysts. I did put in an a request to them for more information, but have thus far received no response. Should that change, I will be sure to amend this paragraph. (Mr. Black, if you are reading this, perhaps you would be so kind as to have a word on my behalf and have them forward the relevant materials)

In the event that the conversation moves over here, let me list the areas where I would like to continue discussion:

What do you think the value is (to the certificate holder) of a certificate that pretty much anyone can achieve after a multiple choice exam that involves no practical examination of testing?

Would the testing industry not be better served by an examination process that rigorously tests the candidate’s ability to apply in pratice the theory and techniques they have learned, even if such a process was time consuming, difficult and did not scale well?

Wouldn’t the ISTQB serve the industry better by helping busineses realise that excellent testing is difficult to do well and that not everyone is cut out to be a professional tester?

There were a couple of other questions that I was waiting to ask. I wanted to get the current set squared away before I continued, but since I am not sure when this discussion will continue, let me state them here.

You say that ISTQB’s competitors have obvious commercial interests. Is the ISTQB a not-for-profit organisation? If that is the case, what happens with the $100 or so that examinees pay for their certification. 160,000 x $100 is not a small sum of money, even over 10 years.

I should say that these last couple of questions are not only for Mr. Black, but are open to anyone from the ISTQB who can authoratively answer them.

Once again, I’d like to invite you to read through this PDF, at least slides 8 to 14, but the rest is very worthwhile also. There are questions and assertions there that I think you really need to answer, for the sake of your credibility if nothing else.

The floor is yours.

The new job (well, not so new now, but in any case my latest job) uses quite a bit of tech I’d not dug into previously. As far as source control goes, I’m used to centralised SC like SVN (or even CVS in a pinch), so getting my head around a distributed model (Git) took some doing (more work required there, but that’s fodder for a different post). Working with a new language and framework as well – I hadn’t touched Ruby on rails up to now, which was stupid of me – I’m finding it pretty freaking awesome for the most part.

It’s taken me longer than I thought to come up to speed. I’ve been scraping the rust off my unix-fu and with unfettered access to the codebase and the command line, I feel like a long-lost missing limb has been reattached. Between that and the whole ‘learning new stuff’ thing, I’ve not had much time for anything else. There seems to be light at the end of the tunnel though. I’m grokking more of the code base and starting to make some useful code contributions.

One of the things that I’m trying to do at the moment is put a little bit of structure in the way that the business group and the development group communicate. Right now there is lots and lots of communication, which is great. The confusion comes in where the lines blur between the business telling dev what they want and going to the extent of speccing out everything in such minute detail that there’s not much design work left to be done by the dev group. That might not really sound like a problem to some, but in practice I’ve found that it can make the dev guys feel micromanaged or like their opinion is not required/valued – especially when design-level decisions are architecturally difficult to implement. If non-tech design reaches too far and gets it wrong, it can mean wasted time and effort either for them when the developer rejects it, or for both if the developer codes it in order to prove the point. The thing I love about my current place is that there are no passengers, and this stuff tends to get worked out, but I think I can do more.

This is where I am hoping that Cucumber will help out. If I can put together a series of tests that step out the current behaviour with a DSL that both the business side and tech side can build their communication around, then I suspect we will find it a little easier to step through the desired behaviour in enough detail to be clear about functionality while leaving the dev guys to work out the implementation. It’s not the only reason I want to use Cucumber, but I do have high hopes in this area. It probably won’t hurt that people will be able to see this stuff turn into checks that can be executed automatically. The key is still in the communication. I’m looking to make the clarity of that communication easier.

Cucumber is new tech to me as well. Conceptually I get it, but it’s different when you start to get your hands dirty. After a lot of reading, I decided to go with Cucumber, Capybara and at this stage, I think a combination of Selenium and TestUnit (Ruby’s in-built unit testing framework). I’m a Capybara noob as well, so I’m up to my elbows in new stuff and loving it. Mostly. Sorta. Apart from the frustrating bits.

Almost the entirety of the literature I’ve found on this setup is for BDD – which is to be expected, given that’s what its for. All of the setup and how-to stuff is for building stuff that doesn’t exist yet. Great if you’re building from scratch, but not super helpful if you have an existing code base you want to fit tests to.

I had an utter bastard of a time getting Cucumber and Capybara to work at all. Granted that it’s probably because I’m working with code that does a few unusual things, but I didn’t expect quite as much screwing around as I had to do – so, I’m going to detail it here in case there are any other poor bastards out there who are trying something similar. Hopefully this will save you some frustration.

My setup is (as of writing) Ubuntu 10.4, using rails 2. I already have selenium installed, and the installation of that is not arduous, so I’m going to omit that part.

I installed the following gems:

libxml2 libxml2-dev libxslt1-dev
gherkin cucumber cucumber-rails nokogiri capybara database-cleaner

I’ve listed some that should be pulled in automatically by other gems’ dependencies. For instance, gherkin should be installed when installing cucumber, but for some reason I had to do it manually. YMMV.

After installing the requisite gems, it should be a simple matter of generating all of the cucumber goodness from the root of your rails project

(ruby) script/generate cucumber –capybara
(you can add other options if you want to have them built in)

If all your bits and pieces are installed correctly, you should see a bunch of files and directories get created.

Here are some things that tripped me up:
I thought I might do some work with rspec as well, so I pulled those down. If you’re thinking along the same lines, then Cucumber seems to barf if you install anything beyond rspec 1.3.3
I also had to mess around with the dependencies for the json gem in my project as cucumber seems quite particular about wanting 1.4.6

By default, cucumber will clean the database after each run (as is generally considered good automation practice). In my case however, I have enough stuff in the db that it doesn’t make sense right now to load from scratch or mock the bits I need. Maybe down the track but for the moment I’ve just switched this off in the environment file (more on this shortly).

Anyway, moving on. From root, there should now be a features directory. edit features/support/env.rb
You can either make your settings changes in here, or in another file and require it from here. This file is automatically generated, so if you plan on making a lot of changes here or you think you’ll have cause to regenerate this file, the latter might be a better option.

Here are the changes I put in

Capybara.default_selector = :css
Capybara.run_server = false
Capybara.app_host = <url of my testing area>
Capybara.javascript_driver = :selenium

One of the things that really screwed me around was not realising that Cucumber starts its own webserver and runs on that. I tried pointing it at the server instance that Rails kicks off, only to find that things were falling over and there was nothing in my logs to tell me why. I felt like a complete moron once I did find out, but then again, the docco in this area appears to be awfully light so I don’t feel too bad. Anyway – be warned. If you don’t want Cucumber to use its own webserver, you need to tell it so explicitly.

As I said, there’s bugger all out there right now on retrofitting scripts for an existing code base. It sounds like it’d be easy, right? The code and functionality is already there, so skip the red/green/refactor bit and go straight to green, right? Not really.

Rather than conceptually pre-fabricating your functionality, you get to look through your existing functionality, decide what is most important to test, work out where you need to retroactively install test hooks and build use cases that serve a business purpose, while not getting lost down a rabbit hole around functionality that currently exists, but perhaps doesn’t work quite how it should. Yay! Given that there is the potential for test to ‘just light up green’ when you write them, I’ve also found it a good idea to write checks that fail so you know they’re actually doing something, then correcting them – it sort of keeps the spirit of BDD/TDD even if you’re not driving any development just at this point.

Retrofitting Cucumber tests to your site is not as simple as it might sound. I am hopeful that it will be rewarding. It’s a little too early to tell just yet, but the signs are positive. If there’s any interest, I may follow up with a post about some more of the specifics of the actual hands-on retrofitting bit. If you have questions, fire away.

For those of you who weren’t at CAST2010 and maybe aren’t so acquainted with some of the things the Association for Software Testers is up to, you might be interested to learn that Cem Kaner is heading up a special interest group to create a dictionary of terms relevant to software testing.

For those people who are passionate about their testing terminology, why not give them a hand?

I would love to see this proliferated far and wide. I’d much prefer a wealth of definitions to choose from and a place to point people than a cult of money grubbers peddling their one true way.

I haven’t seen the dictionary get much airplay yet and granted it’s in its early stages, but my google-fu failed me the other day when I did an idle search and then I stumbled across it here. I figured I’d do my bit and get the word out. If you don’t have time to volunteer, pass the link on to someone that might.

I’ve been busy lately. In a good way, at long last.
I recently started at a new company and have been flat out getting acquainted with how the new place is put together. I have to say – I am loving the new role. I know when I turn up that before the day is done I will have learned something new or I will have made a positive difference somewhere – usually both.

Which brings me to the topic of this post. I’m not a big fan of bitching about ex-employers (and doing so is not the point of this post), but I feel compelled to point out some of the things that have struck me as I shift into this new role.

I did not enjoy working at my previous place of engagement. It’s a shame to say, because some of the people I worked with were very smart andvery talented. Unfortunately the company seemed designed to make any change a herculean effort. I’d be hard pressed to purposely design a more inefficient way of working. I’m not going to give you a laundry list of specifics (though I could). Suffice it to say that there were uphill battles to be fought on pretty much any front you turned to.

I should have seen the writing on the wall. I remember thinking two weeks in ‘Can I really bring myself to care about this company enough to stay and do what needs to be done?’ Still, I was optimistic. I had come from a job where I’d been more successful than I could have hoped and like Alexander before me, I was going to cut through bureaucratic Giordian knots and lead the way to a shining future of efficient development, clean code, strong inter-team relationships excellent configuration and change management, not to mention a kick-arse testing group.

Sadly I acheived none of that. I decided I’d stick it out for 2 years and see how much change I could make. Maybe it was like turning an ocean liner around. Once you got past the momentum built in one direction, surely change would come more quickly. Nope. Didn’t happen. What’s worse is I found myself conforming. Keeping my head down. Not saying anything when I should have made a stand.

As the two-year mark rapidly approached, I had what some substance abusers call a moment of clarity. I was ashamed at my behaviour. If my peers saw me behaving like this, they would not be proud. Far from me making that place better, I had allowed myself to become worse. I decided it was time to leave.

No sooner had I made the decision, an email came out of the blue. A guy emailed me, said he’d read my blog and wanted to have a chat about testing over lunch. He worked a few train stations away and hey, I’ll talk about testing all day if you buy me lunch so I was all for it.  I happened to mention during the meal that my current contract was almost up. One thing led to another and I found myself with an offer from a company that looked to me very much like it was doing things that I wanted to be a part of.

Long story short, I’m now working at said company with a bunch of super smart people doing work that I think makes a positive difference in the world. I like the work that I do and perhaps the most importantly I am myself again. I work and think and act from a place of integrity and I can see that the people I work with do the same.

I guess what I’m trying to get at is life is just too bloody short to work in a job that makes you miserable. Not only is it time you will regret never being able to get back, but you may not like the person you become. No amount of money is worth that.

If you find yourself looking at the clock wishing it would tick faster, if you wonder why you’re doing what you do when nothing seems to change, if late on Sunday afternoon when the prospect of work tomorrow makes you die a little bit inside – your place is no longer there (if it ever was).

I’m not saying ‘quit your job tomorrow’, but if you’re stuck in a work situation like this, do everything in your power to find something that is more you. There is something better out there. Go find it. Please.

Adam Goucher’s post on New things to steal from the Japanese is the impetus for this one. At CAST I gave a lightning talk on borrowing Japanese language as a teaching aid.

Using foreign loanwords is probably harmless enough if you’re using them to jazz up a story, but if you’re going to use them as teaching concepts especially in the sense of you being the teacher and having enough understanding to relay these concepts to others then more care is warranted.

There were a couple of points that I wanted to make in my talk (though I’m not sure how well I did in the five minutes that I had). Firstly, there seems to be some sort of mystic profundity attached to the Japanese language. Pick a common concept and use the Japanese word for it and all of a sudden it’s full of ancient wisdom.

When I first started learning Japanese, everything seemed terribly profound to me too – probably a combination of kanji seeming unintelligible and me watching far too many samurai movies. Being a little older and a little bit more experienced, I can safely say that, like western teenagers, Japanese teenagers can talk for a half hour solid and say ABSOLUTELY NOTHING.

Japanese language is not necessarily any more profound than your native language. English has some wonderful, seldom-used words that you could choose to bring back into common parlance.

The other thing is that Japanese language is not necessarily as straight forward as it seems. There are many words that do not have a one-to-one translation and many words that have more than one meaning based on the context you use them in. That cool word you borrowed to spice up your new teaching material may have other meanings, than the one you intended, or deeper meanings that you don’t quite understand.

Shu-Ha-Ri is a particular favourite of mine when it comes to borrowed terminology. On the surface, it describes the progression of learning from the novice stage (shu), where you follow an instructor’s directions over and over again without necessarily understanding why (wax on, wax off, Daniel-san); then an intermediate stage (ha) where you begin to explore what your new-found skills mean and how to apply them in practical situations; to an advanced stage (ri) where your learning becomes self-directed and you no longer require a teacher in order to evolve your skills.

Sounds simple enough in concept, right?

Okay, so let’s take a closer look at the three kanji used.

守 – Shu – means to obey, but also to protect or defend.

破 – Ha – means to rip, tear, break or destroy

離 – Ri – means to detach, separate or release

Armed with this knowledge it becomes a little tougher to reconcile each of them into the neat little definition above. To add to this, allow me to make a gross generalisation about Japanese culture compared with Western culture.

I have observed that westerners tend to have a mindset of ‘make me understand, then I will do’
In Japan there is frequently a pattern of ‘do and eventually you will understand’

The ‘do’ part may take several years before understanding arrives. Case in point, I have done kendo for around 18 years. I felt that I had a solid understanding of the basics maybe 6 years ago. Some of my peers would probably tell you I’m still not there yet.

I’m not saying that one way of thinking is right and one is wrong, or that one is better than the other, but that you should understand that these difference is there. They are part of the culture and it has a direct bearing on language and the understanding of concepts such as Shu Ha Ri

Ri may not come to you without many years of effort. It is unlikely that a several week or even several month course will take you from Shu, through Ha to Ri. Not in the most simplistic sense of the term. Got someone telling you that you can be a black-belt from scratch in six weeks? Run far, far away from that person.

As far as teachers go, I would suggest that if you are tempted to borrow from the Japanese, or another language that you don’t speak for that matter, that you spend some time studying the culture and the people and at the very least be aware that your understanding of the term may well be incomplete before you have your students start calling you sensei.

I started this post a few times before I realised I was actually dealing with two subjects rather than one. At CAST2010 I took Fiona Charles‘ workshop on Speaking Truth to Power. It turned out that this was one of the subjects that I was writing about and hadn’t managed to articulate at all (Thanks again, Fiona).

The other subject is the situation where you know that a particular subject, product, action etc is not a good one (at least in your context), but don’t necessarily have the familiarity with it to explain why. In order to do so, you’re going to have to learn at least enough that you can speak from your own experience why such a thing is not good.
I call this the ‘know thy enemy paradox‘ – In order to explain why you shouldn’t need to know/use/do something, you need to learn enough about it so that you can coherently explain why.

It would be nice if we could simply take the advice of our learned peers or mentors. No one I respect in the testing industry is going to pan someone or something without a valid reason. If my interest does not lie in that direction, I am happy to take them at their word. As far as directing our own learning goes, this is entirely possible. It becomes another matter however when your line manager proposes that you use one of these things (say, an expensive testing tool or certification or even an inappropriate testing methodology), especially if its use is to be tied to some KPI. This is where speaking truth to power comes into play.

Let us take a hypothetical situation. Your manager tells you that the company will pay for you to go and get certified as it is something that will help you in your career (now say thank you and get out of my office). You haven’t been testing long, but you’re a keen reader of testing blogs and you’ve read on some of them that some testing certifications may not be all they’re cracked up to be. Of course, you don’t have any direct experience (positive or negative) with the particular certification proposed.

You can find online a lot of information spruiking this certification method. There are also vocal groups saying ‘no, don’t do it!’.

If you need to argue with your manager about the pros and cons of certification, then it is not going to be enough to point at online info and say ‘Read this’. It will come down to ‘pointy haired manager’s glossy whitepaper from salesguy trumps your bit of paper from the internets’. As far as your manager is concerned, it’s just stuff that some dude said. It’s not your own experience (and by extension, not your own argument).

If you find yourself in this situation there are a few things you can do. If you are being pressed for an answer, an effective response is ‘Let me get back to you on that’. This will give you time to put together the information you need to argue your case.

Take the time you need. Study the material and the alternatives (I’ll come back to this). Put it into language they understand (probably money, but this is not always the case). Have whatever facts and figures you need prepared before you present your case.

When you present, use newspaper style. Your first sentence is your conclusion. Your first paragraph is a summary of your story and anything that comes after is as much detail as is required. If you see a room of nodding dogs (I mean agreement, not falling asleep), stop selling.

There’s a danger in the ‘know thy enemy paradox’ for the more experienced tester also, as there seems to be no shortage of stupid trying to pass itself off as smart and taking the time to find out enough to refute the never-ending streams of stupid can leave you less time than you want for learning to be a better tester.

There are a few things that I have found that help. If I’m not gearing up to take them down, I only need to know enough that I can say ‘I disagree, and here’s why’. That’s probably not going to take you long (I’m talking minutes, not hours or longer).If you find yourself spending longer, ask yourself ‘Do I need to be reading this, or am I just getting off on outrage porn?’

If I need to convince someone else, I’ll look for ways to construct a cost/benefit analysis. If you can show you’d get more benefit from a preferred alternative and put that into dollar figures along with a compelling story, then your going to stand a much stronger chance of your voice being heard. If you can, speak to someone with direct experience with the subject in question. If your manager has mistaken beliefs, you may need to spend a bit of time refuting them. Maybe have those in your back pocket in case you need to pull them out.

If you are gearing up for a longer conflict well, that’s probably a topic big enough for a separate post (probably by someone with more battle scars than me).

I spent the last week in Grand Rapids, Michigan, attending CAST2010 (Conference of the Association for Software Testing). It is difficult to describe the value of a conference like this. It is a collection of some of the most respected minds in software testing, of my peers and of people full of passion for the craft.

I attended an awesome workshop by Fiona Charles on ‘Speaking Truth to Power’. There were some fantastic presentations by testers from all over the globe. There were a series of lightning talks (I made one on the use of Japanese language as a learning mechanism). If that wasn’t enough, there were some excellent hallway conversations and exchange of ideas and the opportunity to meet some very talented minds. I met old friends, and many people with whom I had exchanged emails, but never met. It was all over far too soon and I am very much looking forward to the next one.

If you only attend one testing conference in 2011, I urge you in the strongest possible terms to make it CAST2011.

Here is what some other attendees had to say about CAST2010

Michael Hunter
Selena Delesie
Mark Vasco
Carsten Feilberg

Edit:
For those that want a small taste of what CAST is like, the keynote speeches are available online

Tim Lister
Cem Kaner

In the spirit of recent puzzles/challenges, I thought I might throw in one of my own. In high school, my class was given an exercise to come up with a coded message that the rest of the class would have to work out the cipher to.

I didn’t take it that seriously at the time and put in some sort of lame effort at home that afforded me the most time in front of the Commodore-64 instead. Most other kids did the same. Clearly the teacher had been hoping that someone in the class was a secret genius and would put together something brilliant. His disappointment when this did not occur was palpable.

I got thinking afterward that it was a wasted opportunity. I quite like puzzles, I like being creative, but I’d never put the two together. I spent a little bit of time coming up with something that might have been worthy of the assignment. I never did anything with it, or showed it to anyone, but I was reasonably sure that no one in that class at least would be able to figure it out. I should have put my money where my mouth was then. I might have been able to turn a profit betting people they couldn’t solve it in a given time limit. Ah well, hindsight is 20/20 as they say.

Instead, I present my code and the message it contains and invite you to work out the key. If you do figure it out, please don’t post the solution, but  I’d be really interested to hear about how you went about attacking the problem, so by all means leave a comment.

The message: the quick brown fox jumps over the lazy dog

IMPORTANT EDIT: In a moment of paranoia I double-checked my key and noted that there is a small change to the decoded message. In the coded text, ‘jumps’ is actually ‘jumped’. To those of you who have started, I apologise profusely. This means that there is an extra letter to the message than you thought you were dealing with, and rather than an ‘s’, you have a ‘d’ and an ‘e’.

The revised decoded message is:

The message: the quick brown fox jumped over the lazy dog

The coded message:
n g o c w u o y j   r
  f l l l q m v   e k
  o i y k b v   r x a
  k s e a t   d i b o
  v h a v   e u i p l

edit: James Bach has rightly pointed out that I could have worded my challenge a little more clearly.

There are an infinite number of keys that can associate your output with your input. For instance, the key might be simply to look at the first letter in the top left, and “n” means “quick brown fox…”

My intention is that people would attack this problem as though they did not know what the message was in an attempt to resolve the letters presented into the answer provided. I had a particular key in mind when I put it together, so the letters do resolve into the message given.

Enjoy.

Update
Okay, so a few people have asked for hints.
Here’s the deal – I’ll post a few hints, but I’m curious to know what you’ve tried and what you thought. I’ve posted one hint below. I will post more, but I’d like to know – How did you approach the problem? What strategies did you employ? What effect did the hint(s) have?

Hint 1

Hint 2

Hint 3

Congratulations to Rushmila Islam, who has at least worked out the message component, if not the complete cipher (which is trivial once the message is in-hand). Here’s another pangram (one with all the letters this time).

The coded message:
k e i f v a l p q u w
  b m a g n z i e c e

  m g d r w o o t a h
  y j a w u x a g e s

I kinda wish I’d put this one up first now. It looks trickier
Rushmila (or anyone else) – care to post the decoded message?

To borrow from Groucho Marx – QTP, I’ve had a wonderful time, but  this wasn’t it.

So thankfully I’ve been able to step away from QTP for the moment. Given that QTP doesn’t recognise Firefox so well after v3.6, and since we use a firefox plugin for most of our mobile testing (FireMobileSimulator), yours truly gets to switch to Selenium instead. I’m thankful for having had the opportunity to work on QTP, mostly so I have a better understanding of its limitations and shortcomings. I can argue more coherently against what seems to be in most cases, a ridiculous waste of money.

I view QTP a bit like one of those unfortunate bears in a Chinese bile farm. You can see how at birth it had the potential to be something majestic and powerful, but instead it languishes in a cage, irreparably twisted and deformed by years of abuse by the ignorant.

In comparison, Selenium while not without challenges of its own, has been by and large a real joy to use. For starters, having any number of fully fledged languages to work in is almost unbelievable after having toiled in VBScript land for far too long. In fact, the first problem I faced was which language do I go with?

I ended up choosing Java and JUnit, mostly because it’s what the current dev team codes with, and I simply cannot be arsed copping the flak I would get from management for introducing another language into the picture (as wonderful as jruby and jython are, I’m sure).

I didn’t realise quite how much thinking in VBS had stunted my thinking in other languages. I do a bit of coding in my spare time, but as far as automation coding goes, it took me a good few days to get comfortable again remembering the power that a real language gives you. I was all set to start importing test data from excel when a timely tweet (and subsequent blog post) from Adam Goucher reminded me that Java has these nifty things called Properties that you can import. (This is why tools like twitter should be allowed in the workplace btw – okay my RSS reader would have picked it up, but not before I’d wasted a lot of time).

It’s been a bit over 3 years since I last looked at Selenium. It doesn’t ‘feel’ all that different, but it does seem a lot easier to work with than I remember it being. Within a couple of days I’d whipped together a script to cut down some checks that took 4 hours manually to a little under 2 minutes. W00t.

I added loggingSelenium to my test setup and now have some pretty colours that light up for people who are impressed by that sort of thing. I’m now in the process of putting together a few more tests and a framework that will support them and their inevitable expansion.

There are a bunch of setup how-to’s out there, so I won’t be doing that, but I will drag together some of the more useful links that I’ve found. Stay tuned

On the suspicion that my last post amounted to a bunch of word salad for a few, let me see if I can add some clarity by describing the basic hierarchy I’m using for my framework.

At the top level, I have a suite control script that looks like this:

doSetup   ‘Calls to set up data and environmental variables and such like

Call Script1
Call Script2
…
Call Script n

Each script that is called is some sort of major transaction that an end-user would recognise.

If you missed the update I made to my last post, I’ve moved a number of function libraries to classes to lessen the amount of coupling required between the ValidationManager and the scripts. There is a small overhead involved in that the relevant class has to be instantiated at the start of the script. For debugging purposes, I have also added a commented-out debug section that calls the setup function that the suite control script does. Each script looks a bit like this:

‘#begin debug
‘doSetup    ‘uncomment this if debugging this script
‘#end debug

Transaction = loadTransaction    ‘This is the instantiation of the class that holds the Transaction’s functions

Transaction.function1
Transaction.function2
SharedFunction
Transaction.function3
…
Transaction.function n

For functions that are shared across multiple transactions, I have left them as a standard function library. On the plus side, the Transaction namespace aids clarity and lets you know where your functions live. On the downside, you can no longer access them by using the right-click->go to function definition option. No biggie imho.

The transaction classes aren’t that much different from your standard function library other than the fact that they’re wrapped in a class definition and there’s a bit of setup stuff at the start.

Class Transaction

Dim VerificationManager

Private Sub Class_Initialize()
VerificationManager = loadVerificationManager   ‘Here’s where the verification manager is instantiated
EndSub

Public Sub Function1
do.something
VerificationManager.CalculateExpectedState(“someAction”)
do.someAction
VerificationManager.Verify(“someAction”)
do.somethingElse

End Sub

…

End Class

I’ve still not gotten the all-clear to post the verification manager code, but as mentioned previously, it is a class also. There are 4 methods; 1 for setup and 3 relating to verification.

CalculateExpectedState calls a file called “someAction”. someAction calls a setup routine that creates a new dataSheet, also called someAction and populates it with expected data.
Back in the class method, the action takes place, at which point we return to the verification manager when Verify is called.

Verify calls the someAction file which calls a second routine that gathers data from the app and does any calculations required and stores the results in the someAction dataSheet. Verify then calls a diff function that compares the expected and actual results and reports. Finally, verify calls UpdateMasterData, which compares the actual data with the master data sheet and replaces anything that was changed.

At this point, there is a new data sheet created for each different call to the verification manager. This has the potential to get out of hand as the number of verification points increases. I’m thinking of using a single sheet and dumping the results to a file at the end of each verification. I’ll see what else occurs to me as I go.

I promised to talk a bit about what I’m doing with data. I still need to cover that. I’ve been having fun trying to define heuristics to govern where the data should drive the scripts and where I should handle variance programatically. I think I have something that works. We shall see.