Software Testing

An open letter to the blog publisher of

To the publisher(s) of the blog post entitled ‘Book burners threaten (old) new testing standard’ on on August 20, 2014:

(I have attached an image of said blog’s text in case it should change or be removed in future)

At CAST2014, a number of like-minded professional testers got together after a very insightful presentation by James Christie on the subject of the proposed ISO 29119 standard. Out of this meeting of minds, two things emerged. One was a manifesto drafted by Karen N. Johnson about our beliefs as professional testers ( The other was a petition initiated by the International Society for Software Testing (ISST) to demonstrate a lack of consensus by professional software testers to the proposed standard ISO 29119 (

The petition exists to show that there are a significant number of software testing professionals who have significant, reasoned and substantial objections to the publication and subsequent adoption of the ISO29119 standard and therefore there is no consensus in the software testing industry that this standard is valid.

ISO’s own guidelines define consensus as :

ISO/IEC Guide 2:2004, definition 1.7

“General agreement characterised by the absence of sustained opposition to substantial issues by any important part of the concerned interests and by a process that involves seeking to take into account the views of all parties concerned and to reconcile any conflicting arguments.

NOTE Consensus need not imply unanimity.”


Over the past week, signatories of this petition and other concerned parties have been circulating the petition and encouraging others to sign it. As of writing, it has upward of 250 signatories.

Your blog seems to be a fairly low-brow effort to understand and respond to the concerns raised by the petition. I see few redeeming qualities upon closer reading. It denounces this petition in what I can only describe as insultingly inflammatory fashion. You appear to be using a number of fallacies to support your attempt at an argument. Let’s go through them.

You begin with a fairly vague appeal “Testers have been waiting many years for ISO29119”. I wonder, which testers specifically are you referring to? Aside from consultants waiting to sell services based on ISO certification and anyone related to the drafting of these documents, who exactly is clamouring for the publication of these documents?

Next up – guilt by association.

You are calling the ISST and the signatories of this petition ‘book burners’. There have been a number of groups known throughout history for burning books and one would be hard pressed not to think first of the Nazis. To the best of my knowledge, neither the ISST, nor any signatories have actually burned any books (Actually, members of ISST read a lot of books and quite a few also write them). If your intention was indeed to draw parallels between the two groups, then I find this repugnant and highly unprofessional. If your intent was otherwise, then by all means, please leave a comment here (my blog unlike yours is open for discussion) and enlighten me.

Since you bring up the subject of books, let’s take a quick look shall we? The published volumes of the ISO29119 standard have bibliographies that refer predominantly to other ISO/IEEE publications. As far as I can see there are three publications referred to that are external sources and one of those is to a publication of ISTQB.

Here’s a small fraction of a list that I think could have been referred to or at least recommended as further reading:

  • Perfect software and other illusions about other illusions about testing – Gerald M Weinberg

  • Adrenaline junkies and template zombies – DeMarco, Hruschka, Lister et al

  • Mistakes were made (but not by me): Why we justify foolish beliefs, bad decisions and hurtful acts – Carol Tavris

  • Introducing ethics – Dave Robinson

  • You are not so smart – David McRaney

  • Why software gets in trouble – Gerald M Weinberg

  • Antifragile: Things that gain from disorder – Nassim Nicholas Taleb

  • Lessons learned in software testing – Bach, Kaner, Pettichord

  • Bad software: What to do when software fails – Cem Kaner

  • Seeing like a state: How certain schemes to improve the human condition have failed – James C Scott

  • Tacit and explicit knowledge – Harry Collins

  • Leprechauns of software engineering – Laurent Bossavit

  • The structure of magic Volume 1 & 2 – Bandler, Grinder

  • Lateral thinking: Creative thinking step by step – Edward De Bono

  • Secrets of consulting – Gerald M Weinberg

  • An introduction to general systems thinking – Gerald M Weinberg

  • Becoming a technical leader – Gerald M Weinberg

  • The psychology of computer programming – Gerald M Weinberg

  • Kuhn vs. Popper: The struggle for the soul of science

  • Please understand me (2) – David Keirsey

  • Frogs into princes – Bandler, Grinder

  • Sherlock Holmes – the complete novels and stories – Sir Arthur Conan Doyle

You get the idea. There is a good deal more out there that software testers should familiarise themselves with. I’ve left out tomes that refer to specific technologies. They are easily found and I leave them as an exercise for the reader.

For more, see


Returning to your blog post – you falsely assert that our issue with the standard is that

not everyone will agree with what the standard says.

This is at best a gross oversimplification. The text of the petition does not explicitly state what specific disagreements and opposition the signatories have, it simply states that such opposition exists and must be considered. The specifics are not difficult to find. There are a number of other professional testers who have written wellreasoned arguments about their opposition to software testing standards and that number is growing.

You go on to build the following strawman argument

…they don’t want there to be any standards at all. Effective, generic, documented systematic testing processes and methods impact their ability to depict testing as a mystic art and themselves as its gurus

Let’s look at the word ‘effective’ – Effective for what? One might assume for the orderly execution of software testing, but I would hate to put words in your mouth, so please, once again enlighten me as to what specifically you mean by effective and do please back this up with proof that this standard actually achieves this.

As for the rest of the sentence, what has the ISST or any other signatory of the petition said or done that leads you to believe that they gain from depicting testing as ‘a mystic art and themselves as its gurus’. I challenge you to prove this statement or withdraw it and make an apology.

Furthermore, I challenge you to publish your real name next to your blog post and stand behind it and defend it as best you are able – or, retract it and post an apology with your real name attached.



Ben Kelly

Professional Software Tester

Founding member of the International Society for Software Testing

3 thoughts on “An open letter to the blog publisher of

Leave a Reply

Your email address will not be published. Required fields are marked *