Framing your tests, Framing your audience

Test Framing was a subject that was big at CAST2011 and indeed on a number of blogs that I frequent in the lead up and aftermath. It is a term that James Bach coined and Michael Bolton has run with. They describe it thus:

To test is to tell two parallel stories: a story of the product, and the story of our testing. Test framing is a key skill that helps us to compose, edit, narrate, and justify the story of our testing in a logical, coherent, and rapid way.  The goal of test framing is to link each testing activity with the testing mission.

This to me is a worthy skill to have. Ask a tester why they execute a certain test at a certain time and often they will struggle to tell you. It doesn’t mean they are testing randomly or that there is no strategy to their approach, but that they are not yet skilled enough to articulate their reasoning from conception to execution; or if you like, they are not yet able to frame the reasoning for their actions in a way that is compelling.

Here’s where my brain nags at me. I like this concept of test framing and yet it seems to be missing something. Something fundamental; something without which the act of framing loses much of its meaning.

I said ‘not yet able to frame the reasoning for their actions in a way that is compelling’, which begs the question:
Compelling to whom?

I think Test framing as James and Michael describe it is part of a larger set of framing skills that are fundamental to being a skillful software tester. From the perspective of training software testers and answering to anyone who asks about the specifics of the testing you are doing, Test framing is a key skill. There is much more to framing than this however.

Much of what I’ve seen written about test framing to date implies an audience, but does not say a great deal about the audience explicitly. Simon Morley has written an excellent blog post on Decision and Analysis Frames in Testing. If you haven’t read it yet, please do. Simon’s writing hit upon the importance of understanding the filters through which your audience views the world (the frame through which they are looking) and that this can be different depending on the role your audience plays. He writes:

…the way we present information can affect its interpretation – depending upon the frame that a stakeholder is adopting.

Think of a frame as a filter through which someone looks at a problem – they’re taking in lots of data but only the data that gets through the filter gets attention (the rest may end up in the subconscious for later or isn’t absorbed), “I have my show-stopper filter on today so I don’t notice the progress the team has made…”

So, being aware of the potential different types of frames that each project member might have as well as some traps associated with frame formulation is important.

He goes on to write about the different kind of frames the audience and the tester might employ (it is worth noting the differences between them). He also writes about problems with framing, such as Functional Blindness (I see parallels here to the Dunning-Kruger effect), the Sunk Cost fallacy, overconfidence and distortion through measurement and numbers.

Understanding the audience’s part in framing information is a key part in effective communication. My presentation at CAST2011 was on telling a compelling testing story, so framing, especially in terms of the audience is a subject that is very interesting to me. Framing of your testing (and the information it reveals) is inseparable from your audience. Even if you are only framing your information for you – you are still your own audience.

Each of us has different filters through which we view the world. Even within a single role (tester, programmer, project manager etc) individuals can have vastly different outlooks on the same subject. Largely this has to do with the way we model the world. We create a mental representation for everything we interact with. Every object, every person – we have a mental representation about what each of them means to us based on our previous experiences and our previous models. Our experiences shape our models and our models become the filters through which we experience the world.

Necessarily then, every individual has different models (and these differences may be subtle or profound), and so we all have different frames of reference through which we interpret the world around us.

To understand frames, you need to understand models. Not just the model that we have of our system under test, but the models we have of the world around us, the model we have of our audience especially. The map is not the territory. How we view ourselves is not how others view us. The reality (who we actually are) is different again. Same goes for our audience and indeed any other object or concept you care to name.

When we present information about something to someone, we are in fact presenting our interpretation of that information through the filter of our models, through the filter of our audience’s models to them.

Understanding the model(s) your audience is using can help you build information that they need to know about. Understanding your audience’s frame of reference when receiving information can help you frame that information in a way that they are receptive to. Moreover, you can decide if your audience needs information that will augment the models they have and whether or not you need to reframe them in order to present it.

You may present information to your audience that you thought was vitally important only to have them react as though it were inconsequential. Why is that? There are lots of possible explanations, but it’s likely that either they are working from a model which renders your information inconsequential, or it could be that you have a richer model than they do and they do not understand the importance of your information because the limited nature of their model does not allow them to.

Here is where reframing can be helpful. If your audience’s current model creates a filter whereby your information is not useful or valid it does not necessarily follow that your information is invalid per se. Were your audience to look at your information from a different perspective it may be that they would find if valid and important after all.

I feel like there is a lot more here to be pondered and discussed, but for the sake of brevity (hah) I will leave off here for now.

Leave a Reply

Your email address will not be published. Required fields are marked *