EverythingSoftware Testing

That’s not a bug, it’s like that by design.

notabug.jpgWhen I hear this phrase it is occasionally followed by a sort of forlorn sense of resignation (and the urge to say ‘well, your design is shit’), but more often by hope that a healthy, spirited discussion is about to take place – especially when that phrase is subsequently followed by ‘No user would ever do that’.

I’m not saying I’m the gods’ gift to usability design or anything like that, but I do think that the phrase ‘It’s supposed to be like that’ is often code for ‘You might have a valid concern, but it doesn’t neatly fit into the model that works for me, so it’s your problem, thankyounext’. To me, it’s an argument on an intellectual par with ‘eh eh eeeeeeh!‘.

Rather than being helpful, it sets up a contrary position that doesn’t invite further (positive) dialogue. Of course, you might use an answer like this as an educational tool or test to see how well people cope with it, but as a valid answer to a concern that a bug exists, it has limited value that I can see.

I am also not saying that the person who invokes the ‘not a bug’ incantation is necessarily wrong. They may have a perfectly valid point as to why something is not a bug, and that indeed the design does make sense. However, to arbitrarily shut down someone’s question closes them to the possibility that the design is flawed, or could be improved (which is also a possibility, no matter how much they may wish otherwise).

Take for example the ‘maximise vs. snap-to-fit agument‘ – That the Windows OS ‘maximises’ windows, and the Mac OS snaps to fit the content. I don’t necessarily want to get into which is ‘right’, but more to have a look at the presupposition in the argument around it. Specifically, the comments of Alex Chamberlain that Jeff Atwood quotes:

This is a textbook example of how Microsoft’s programmers got the original Mac GUI wrong when they copied it for Win 3.1, and never bothered to fix it: there’s no zoom button on Mac OS windows because it’s unnecessary. What you’re mistaking for a “maximize” button is actually a “snap window to size of contents” button. Far more useful and elegant. Once again, Microsoft has no taste and no clue when it comes to the GUI. All that money and Gates has never been able to hire a decent human factors person.

If this is indeed the entirety of the quote, then it is perhaps one of the most unhelpful stances you could take on the matter. Windows is wrong. Mac is right, and more useful and more elegant, M$ is teh suxxorz. Here’s the dogma I’ve been indoctrinated with, I expect you to swallow it without question, end of story.

The presupposition that MS got it wrong leaves no room to explore whether the difference is an improvement. Different isn’t necessarily wrong. Different is different. Sometimes, different is better whether it’s intended or not.

Why is snap to fit far more useful and elegant? What is it about the windows interface specifically that is clueless and tasteless? There are things that I personally am not a fan of in both environments. Why is maximise unnecessary? What specifically needs fixing?

I can see the relevance of having the window maximised. If like me you’re particularly uncoordinated, and you want to use the scrollbar, you might overshoot the thing two or three times before catching it, but with a maximised window you have unlimited screen real estate in at least one direction.

I don’t immediately see why snapping to fit the content is either more elegant or more useful – I’m about to go and have a look, but the difference is, I’m not immediately dismissing the possibility that it could be.

As a software tester, you’re going to eventually come up against people who are threatened or otherwise upset when you challenge their ideas and opinions. Don’t take it personally. They’re just trying to do their job as much as you are yours. Don’t be intimidated either. If you don’t have many years of experience, it doesn’t necessarily follow that you don’t have anything useful to offer. If you have questions, ask.

If you can show the person you’re asking that you’re on the same side, and trying to provide them with a service, then your audience is likely to be more receptive and arguments like ‘follow the gourd!’ can be avoided.

2 thoughts on “That’s not a bug, it’s like that by design.

  1. Ok, first let me fess up and say I’m more developer than tester, so let me take the devils advocate on this one. The principal issue that I see with the “It’s not a bug…” conversation occurring between tester and programmer is that it typically suggests that the tester is trying to influence the design at a stage in the development process where design changes are expensive. It highlights reasons for including testers at the design stage, and throughout the development process, rather than late in the implemenation. If I have someone testing a piece of code that is due for delivery, I want it tested as it has been written (for good or bad), I’m not looking to enter into dialog as to how the tester would have written it. If testers want to become involved in design, let them do it at design stage.

    I take your point about developers feeling threatened and upset; there is a popular stereotype of the devloper as a pizza fed, coffee slurping nerd with a fragile ego. This is probably true in many cases, but IMHO also applies equally well to QA people, who often feel undervalued and hence easily threatened. As a tester, how would you feel if a developer questioned part of your test strategy as testing was nearing completion? No one likes to repeat work, even if it would improve the final outcome, this is true of developers, testers, managers and even the tea lady.

    If its a bug, pragmatically you have to balance the cost of repair against the cost to the end user. The ‘is it a bug or a feature’ discussion occurring on a regular basis suggest scope for improvement of the development process.

  2. Hi Shane,
    You bring up some interesting points. Let me see if I can address some of them. I should say that I am doing my best to remain ‘SDLC agnostic’ fwiw.
    Also, excuse the extreme length of this comment, it’s somewhat more sizeable than I originally intended.

    “If I have someone testing a piece of code that is due for delivery, I want it tested as it has been written (for good or bad), I’m not looking to enter into dialog as to how the tester would have written it.”

    I am not suggesting that a tester finding such a bug should be looking to tell the developer how they should be coding. I do think that a tester raising such a point is providing an alternate point of view that deserves consideration rather than flat dismissal.

    Yes you have time and budget constraints, and some issues are more critical to resolve than others, but if a tester finds a potential issue or improvement in design or implementation at *any* stage of a project, whether or not you have scope to handle it then and there – wouldn’t you want to at least consider it?

    Is the cost of repeating work outweighed by the potential cost of a user interface with extremely poor affordance? (or goodwill engendered by an excellent interface?) Do you as a developer have the domain knowledge to make that sort of decision? These are a few of any number of questions that you might want to think about.

    You can put such an issue on the backburner, list it as a non-critical fix if it’s not worthy of action, but at least examine it from more angles than ‘bugger, I have to cut more code’.

    If you tell testers to ignore certain kinds of bugs, that’s a recipe for trouble.

    For example, you run the risk of a malignant bug that looks benign on the surface going not undetected, but unreported because you didn’t want to know. If the tester has an alternative to what they suspect is a design flaw, shouldn’t you consider it?

    “If testers want to become involved in design, let them do it at design stage.”

    Saying ‘let a tester be involved at the design stage’ I have several issues with. Firstly, in my experience testers often do not have a choice as to when they are involved. Secondly, design describes a broad range of activities some or all of which may or may not happen (or be appropriate).

    So if you’re saying that a tester should find flaws in design at the design stage or forever hold his piece, then that is something I cannot agree with.

    In terms of stereotypes, I can’t speak for anyone other than myself, but I take people as I find them. If someone gets defensive, I try to manage it as best I can. Some testers certainly do feel undervalued and overworked. I don’t know that it necessarily follows that it makes them easily threatened, but I take your point. People are people, be they testers, developers or other.

    In response to your question, how would I feel if a developer questioned my test strategy as it was nearing completion? I’d like to think I would welcome it. If I’m totally honest about it, I’d be embarrassed if they came up with something I think I should have covered and didn’t. More so if it meant I had more work to do.

    That said, I absolutely would not tell the developer that it’s too late in the day to worry about it because we’re well into the test cycle and repeating work now would be too much time and effort.

    I absolutely would determine how much of a risk the new information posed, the cost of mitigating the risk and if necessary the time available to do so.

    I don’t look at my workload in terms of having to repeat it or not. The nature of testing is that it is often repetitive anyway. I look at it as the identification and mitigation of identifiable risk.

    “No one likes to repeat work, even if it would improve the final outcome, this is true of developers, testers, managers and even the tea lady”

    No it isn’t.
    I might agree with the statement ‘no one likes to repeat work *unnecessarily*’.

    If someone comes up with a better way and it’s going to be an absolute bitch in terms of time and money to implement at this point – sure bench the idea.

    If someone comes up with a usability issue that looked okay on paper but turns out to be a possible showstopper – even if that is also a bitch to fix you can’t dismiss it because it’s late in the day and you have to repeat some work.

    If it means as a tester I need to do a bunch more testing because you as a developer need to cut more code, so be it. Bring it on. If my role as a tester means providing a reasonable assurance of value to someone, you can bet the house I’m going to take every opportunity to do it.

    If the powers that be don’t want to take the opportunity, then fine, but to expect a tester to sit idly by when they see potential problems (or improvement) (or problem) because someone in another department might have to cut more code? That doesn’t make much sense to me.

    Can you explain to me what you mean by ‘cost of repair vs. cost to the end user?’ Are you speaking in terms of cost to fix impacting on the sale price of the product to the end user? I think that if the ‘is this a feature or a bug’ conversation happens repeatedly, then it becomes a matter of managing expectation as far as you are able.

    How is the user expecting to use the product? Is that different to how the developer has coded it? Is it different to how the tester is testing it? If you can determine a baseline to work from, then you can check in to make sure you’re on the same page. If a tester finds a problem, you can examine it in relation to your initial understanding, supplemented by whatever new information the tester has uncovered and decide on the most appropriate course of action given the other parameters of the project (time, money, etc).

Leave a Reply

Your email address will not be published. Required fields are marked *