Exploratory testing versus Test cases – Why versus?

Posted on Posted in Everything, Software Testing

Jonathan Kohl beat me to the punch on this blog entry by a good margin, but since I was already working on this thing, I’d like to go ahead and basically say exactly what he said in probably a far less cogent fashion.  A lot has been written about the good and bad things about Exploratory testing and the writing and (mis)usage of test cases.

I don’t see a lot written about harmonious use of both. I’m sure people do it. I know I do. Is it like airing dirty laundry? Do context driven testers not want to admit to using test cases in case James Bach reads their blog and decides to tear them a new one? Do factory testers (sorry – what do you BDUF testers call yourselves anyway? Factory tester brings to mind a grey-clad downtrodden peon, which I suppose is appropriate, but not very flattering) – anyway, do you lot not want to admit to doing exploratory testing in case your overseers beat your for not following the script? Am I missing something? Is this just not a big deal?

Johnathan advocates using a blend of both and uses the terms prescriptive and descriptive testing, which quite appeals to me. Michael Bolton’s recent highlighting of the difference between testing and checking as something we should be more conscious of helped me to frame this better in my mind.

There are things that you can plan ahead for. There are risks you can identify and tests that you can design to mitigate against them. I see no reason why you shouldn’t use test cases for these if that is what you are comfortable with.

One of the issues I have with test cases is that some people see them as a magic artefact that covers all possible problems in a given area, because one or more tests have been written that mention it.  They fail to see that many test cases are highly focused on verifying a very specific item somewhere, and just because we go through a bunch of steps to get there does not mean that those steps have also been verified.

This leads me to wonder if I have been thinking about scripts all wrong.

I recently read Michael Hunter’s work on setting up test frameworks. One of the things that was a lightbulb for me was separating the verification from the action. Very very obvious in hindsight, but if you think about it – this is basically a use case and a checklist.

We have a series of steps from point A to point B

We have a list of things that we want to make sure are as we expect them.  Why do we need to mash them together? Why do we need to repeat the list of steps over and over and over so that we can list a different verification point next to it?

An argument I hear for test cases is that if you write them well enough, then anyone can execute them. I really hate the way that thinking goes, mostly because I read it as ‘anyone can do testing, you just follow the scripts’. That being said, a clear set of instructions and a checklist could mean that you could get a non-tester to do checking, even of mission-critical information. That, I might hold with.

Checklists and Check scripts? I think this is something worth exploring further.

8 thoughts on “Exploratory testing versus Test cases – Why versus?

  1. Ben,

    Great post. I completely agree! I wrote a blog post on a related topic yesterday. See:

    http://hexawise.wordpress.com/2009/11/04/checklists-good-test-scripts-bad

    (My post is more nuanced than the url might suggest). I am personally very interested in reading more about this exact topic. I’m not aware of much out there, but I see a need for it. Specifically, how are practitioners working towards a blended approach? What lessons learned can people share? etc.

    – Justin

  2. Hi Ben,

    I posted a similar query on the Linkedin Exploratory Testing Group, haven’t posted the URL as it won’t come up if you’re not a member (and signed in) but there’s a few interesting comments.

    Cheers

    Tony.

  3. Hi Justin,
    I think one of the things that is often misunderstood is that exploratory testing is an approach to testing, rather than a technique. The resistance I have met to ET has largely been around this (even after I’ve explained the difference – perhaps my explanation is lacking).

    I think for a test manager or test lead, understanding what your team prefers to work with is important. Foisting ET on them and expecting them to run with it when they’ve previously only been exposed to scripted testing can be counterproductive if you approach it the wrong way.

    Would love to throw this subject around some more with like minded people. I think there’s a rich vein of material to mine.

  4. Hi, im a software tester (manual testing). Could you please give me a sample test plan template with repsect to Exploratory testing and not the conventional test plan? The template should descrbe functional test plan for exploratory testing based on one or more Tours like landmark, rained out etc.

    Thanks,
    Raghunandan.

  5. Raghunandan, what have you done to seek out this stuff already? Your desire to learn is admirable. It would be more so if coupled with proactivity. A quick online search would lead you straight to James or Jon Bach’s extensive writings on the subject.

    I strongly suggest you get in contact with Pradeep Soundararajan and speak with him about Weekend Testers.

Leave a Reply

Your email address will not be published. Required fields are marked *