I’ve been ranting a bit in various fora lately. I very much want to see fake testing die. What do I mean by fake testing? You might describe it as test theatre, in the same way that the TSA performs security theatre. It’s a well-known pantomime that we’re all forced to go through allegedly to keep us safe. We all know it’s bullshit, but it seems no one really has the power to change it.
Fake testing is the blind worship of process and procedure (over the sapient and conscientious application of skill). There’s been some talk of its demise recently and while I want to believe we’re heading in the right direction, I also want to keep in mind that this is not an easy problem to solve and if we’re not careful we won’t solve the problem, we’ll just shift it elsewhere.
Let me give you an example.
At GTAC, there were some very interesting tools demo’ed. I looked at BITE (Browser Integrated Testing Environment) and thought – yeah, there’s another nail in the fake testing coffin. I particularly like the concept of making it easy for the end-user to report bugs that contain information that programmers can use to solve issues quickly.
As I continued to watch, something nagged at me. A sense that something was not quite right. Not quite deja vu, but certainly the sense that if it were this easy, we’d have done it before.
Shortly thereafter, I was chatting with good friend and colleague Jared Quinert. I had forgotten how good he is at taking a subject, stripping back all the bullshit and laying its foundations bare. If you ever get the chance to work with, or hire him. Do it. Seriously. As usual, Jared locked onto what I’d been feeling uneasy about without me even having to explain myself. Having shared the link to the talk I’ve linked to at the start of this post, our conversation went something like this:
Jared: Record playback software is apparently central to the life of a manual tester…
Ben: Yeah I chuckled at that too.
Jared: Recording scripts, converting to English doesn’t solve the abstraction problem. Waiting to hear them elaborate on re-recording being cheaper than maintaining…and they’re still just talking about QTP features, but nobody has piped up with this QTP can re-learn missing controls…
Jared: Tool running in the browser probably creates other issues too I would think.
Ben: Likely yes. That and when you have several thousand tests and you need to re-record a couple hundred, the appeal wears thin.
Jared: The abstraction is the problem, not the approach and as always, I’m skeptical that the test can simply be replayed without any issues when there is dynamic content.
Jared: I think I get annoyed by the ignorance they have of other tools, which is why programmers seem to keep reinventing the automation wheel.
Nerds have far too much faith in machines and solving problems through programming. Some good things come out, and some massively misguided efforts come out as well. But ultimately, these guys will decide our future, because they will dangle an appealing, human-free testing future, which will be doggedly pursued by management. If there are two choices for how work is designed, and one gives management more control, they’ll choose the latter.
Effectiveness is secondary. They’re not solving my problems. They’re solving developer problems. And management problems that arise due to poor work design. But they’ll probably win. Those of us who are skilled enough and market well will retain decent jobs, but there may not be many of those roles. Mostly people will forget how to do the skilled work. When they say ‘I wanted to help the manual testers’, I always hear ‘I wanted to help the shit testers (Even though they didn’t ask for any help)’.
The fact that Google has released the client but not the server will be a deterrent for some, but it won’t take long before someone puts together a workable open source server and the crap automation tool + low paid maintenance worker farce will have a new player. It might mean that that you won’t need quite as many fake test drones in your organization, but in order to make a dent in the plague that is fake testing, we need something a little better than crap + 1.