Software testing? Still not going away.

Tuesday, November 8, 2011

So at the risk of flogging a dead horse, I have some more to say on James Whittaker’s STARwest keynote. My previous post was written based on notes of a presentation that I (at the time) had not seen. Rosie Sherry posted the link to the recording of the presentation (thanks Rosie!) so it was with some interest that I fired up the browser and tuned in.

The intended audience seemed to be testers who are enslaved by process and useless dogma, though he pitched it as being for testers who are lacking respect. I suppose the latter comes from being the former. I expect that many of those people will be feeling some trepidation having heard this presentation. That’s probably a good thing, but the doom was layered on pretty thick. I thought it was overkill personally.

In my previous post, I wasn’t shy about criticizing the content as I saw it. I’m not backing away from those criticisms either. They stand. That said, there were some cool and useful things that Mr. Whittaker did talk about. It’s just a shame that the vast majority of the points he made were very blinkered and didn’t really extend beyond web tech (especially web tech for large companies). I spent enough time going over that last time. There’s more nonsense I could have a go at, but it wouldn’t serve any real purpose. If you’re curious you can watch it yourself.

I think it’s a little irresponsible to say ‘you’re users are going to suffer anyway, so just get it out there’. That works in some situations – mostly when users are expecting it. When you have customers paying money for a product that they expect to work well, that strategy isn’t so great. The users (rightly) get upset when the software doesn’t do the job. He did justify this point later with ‘use dogfooders and releases to trusted users’, but that’s not going to work across the board.

At Google, the majority of the tools they produce are free for the end user. Their model is based on building things that people will use (for free) and building advertising into it. If it doesn’t work perfectly, oh well. It’s not like you’re paying money for it.

Try that on with a product where people are forking over money. It doesn’t have to be a lot of money. Even $10 per month is enough for someone to become irate when what they expect to work does not. As a company releasing software, you have a duty of care to make sure that software does the job.

Yes of course there will be bugs. You can’t make it ‘perfect’. That’s not the point. The point is that you have taken reasonable steps to make sure that it does the job intended and doesn’t do what it wasn’t. Would you release CAD sofware for engineers with cursory testing because you can easily patch it later? What about drivers for machines used in radiation therapy? Pacemakers? Traffic control? Problems with these don’t just mean users get pissed of and bitch about it on twitter. It means people die or are maimed horrifically.

Yes some of the stuff that Mr. Whittaker talks about is cool and is made by developers and will serve to bring testers closer to both the end user and to developers. That’s awesome and I’m all for that. That doesn’t mean that testing as a sapient vocation is under threat. Fake testers would appear to have a use by date and I’m all for that too. That day can’t come soon enough for me.

Computers are getting better at assisting us to test and yes of course we have the developers who wrote that code to thank for it. For example, the tools Mr. Whittaker spoke about to help end-users report bugs are very cool indeed. I’d love to put something like that in place for the end users of my company’s products. I also agree with him that we should be embracing our users in order to assist us with testing. Select groups of early adopters, dogfooders, people who understand that what they are using is not perfect yet and that we appreciate their feedback. Why wouldn’t you want to tap into that?

There were other worthwhile truisms that Mr. Whittaker brought up in amongst the hand wringing and gnashing of teeth.

Developers can test.
This is true. Testing is part of a developers job. Being able to look at requirements or a story and think about not only how the code should work, but how it should handle failure is something developers should be doing. They should also be harnessing frameworks out there to build checks that save them time down the track. Agreed. That’s part of their job and if they’re doing it, that’s full of win for both them and the testers.

Stop doing stuff you can build a robot for.
I’m on board with this too. Anything that doesn’t require a human to evaluate and that can reasonably be automated, should be.

Testing is not the sole domain of testers. It doesn’t matter who does it, only that it gets done.
I agree with this one too, by and large. You can get into semantics about what testing is and what skill sets are particular to dedicated testers, but yes as long as testing is done to the extent it needs to be in a given context, then fine.

The thing that irked me the most about this presentation is the fallacious premise that you have to have produced something corporeal and tangible in order to be of any worth. This is simply not true. Testing is a cognitive process throughout which you produce information that (among other things) helps bring a piece of software from conception to fruition.

There are places for testers who code and there are places for testers that don’t. The advances that Mr. Whittaker is talking about will make life difficult for (so called) testers who don’t think; for the those who blindly follow a busted process because someone called it ‘best practice’ and they decided to stop learning when they graduated from whatever school they last graduated from. On the other hand, for testers who love to learn, who enjoy new challenges and are not afraid to get their hands dirty there is a bright future. It may be that their job title might move from ‘tester’ to something else. Who cares? They’ll be bringing the same incisive critical thinking and analysis to whatever you decide to call them and they’ll still be testing. Bring it on.

 

 

10 Comments

  1. Scott Barber says:

    Finally, I’ve posted my blog reaction to the reactions to the keynote (on a bump on a log in a hole in the bottom of the sea) .

    On the Alleged Death of Testing

    – Scott

  2. Good post -

    >>> Anything that doesn’t require a human to evaluate and that can reasonably be automated, should be.
    Tough thing here is decide “what” and “how” – that requires humans. Mr Whittaker would probably give that to end user to decide or give it crowdsourcing.

    This whole drama about “developers can test” or “developers are getting better at test” – is what irritates me. How on the earth that will reduce the need for any non-developer testing?

    Mr Whittaker feels that – “we know where the bugs are ” (this – he talked about in eurostar 2011) and “fixing the bugs is more important than finding them. Hence testing is dead. He famously announced that Google has eliminated “title” “tester” from their software roles.

    Pretty frustrating… while agreeing that some non-sense going in the name of testing has been dead long ago.

    Shrini Kulkarni

    1. Ben Kelly says:

      Hi Shrini,
      I can’t speak to what Mr. Whittaker would or wouldn’t do, but yeah doing test automation well can be tough. It requires skill and understanding about what testing is and not only how to make automated checks useful, but also how to make that information visible and digestible. It’s not necessarily just about fixing code bugs that crop up.

      Developers’ increasing testing ability being touted as making testers obsolete is a pretty blinkered viewpoint. It may work for Google and maybe some other companies, but that’s not the majority of software development. Not by a long way.

      Alberto Savoia was onto something when he talked about building the right it over building it right. That makes a lot of sense to me. To me, that speaks much more to the death of fake testing than testing as those of the context-driven school of thought would describe it.

  3. Sunil says:

    Finding worthy blogs is not easy now and it’s a pleasure to run across your site.

    1. Ben Kelly says:

      Thanks Sunil,
      There are plenty of good software testing blogs out there. Unfortunately there are a far greater number of bad ones. Probably one of the reasons why so many new testers find themselves on the path to zombie testing.
      Ultimately I think that those who are dissatisfied with bad testing will ultimately be dissatisfied with bad testing blogs. I can only hope that this is motivation enough to persevere in the search for the worthy testing blogs out there.

Leave a Reply

Pingbacks & Trackbacks

  1. Testing [is] Dead is it? Do I now cease to exist? | Duncan Nisbet - Pingback on 2011/11/18
  2. Попинайте труп. Он оживает « QA – грамотно - Pingback on 2011/12/01
  3. What is Trustiness — iamdez - Pingback on 2011/12/21
  4. The Context Problem Part I talking about testing | Testers Tower - Pingback on 2012/01/10
  5. The Social Tester » Blog Archive » Testing is dead, that’s what they said in the news - Pingback on 2013/03/05