testjutsu

I like to engage in good healthy discussion. I don’t mind taking a contrary position or having an argument. I want my friends and colleagues to challenge me. It helps me learn and think in different ways. I’m not a zealot about it. I’m okay with people being wrong on the internet, but every now and again there is the invitation for dialogue that I find difficult to pass up. It can be tough though when the person who has requested said dialogue is reluctant to engage.

I had just such an experience recently over on UTest’s blog where Rex Black of ISTQB fame was interviewed for ‘Testing the Limits’. Mr Black was hopeful of conversation between himself and readers of the UTest blog. Being a reader and having a number of questions I was happy to oblige.

Unfortunately the dialogue has, at least thus far, not been what I was hoping for. Now I don’t want to be unfair. Mr Black is a busy man. He did go into quite some detail on a number of the questions I asked. He indicated his lack of time a number of times during his replies. I had a number of other questions and made several statements though that I was hoping would prompt further dialogue. I’d like to think he will at some point he will take a well earned breather and spare a few minutes to come back and continue.

He was kind enough to point me in the direction of their psychographic analysts. I did put in an a request to them for more information, but have thus far received no response. Should that change, I will be sure to amend this paragraph. (Mr. Black, if you are reading this, perhaps you would be so kind as to have a word on my behalf and have them forward the relevant materials)

In the event that the conversation moves over here, let me list the areas where I would like to continue discussion:

What do you think the value is (to the certificate holder) of a certificate that pretty much anyone can achieve after a multiple choice exam that involves no practical examination of testing?

Would the testing industry not be better served by an examination process that rigorously tests the candidate’s ability to apply in pratice the theory and techniques they have learned, even if such a process was time consuming, difficult and did not scale well?

Wouldn’t the ISTQB serve the industry better by helping busineses realise that excellent testing is difficult to do well and that not everyone is cut out to be a professional tester?

There were a couple of other questions that I was waiting to ask. I wanted to get the current set squared away before I continued, but since I am not sure when this discussion will continue, let me state them here.

You say that ISTQB’s competitors have obvious commercial interests. Is the ISTQB a not-for-profit organisation? If that is the case, what happens with the $100 or so that examinees pay for their certification. 160,000 x $100 is not a small sum of money, even over 10 years.

I should say that these last couple of questions are not only for Mr. Black, but are open to anyone from the ISTQB who can authoratively answer them.

Once again, I’d like to invite you to read through this PDF, at least slides 8 to 14, but the rest is very worthwhile also. There are questions and assertions there that I think you really need to answer, for the sake of your credibility if nothing else.

The floor is yours.

The new job (well, not so new now, but in any case my latest job) uses quite a bit of tech I’d not dug into previously. As far as source control goes, I’m used to centralised SC like SVN (or even CVS in a pinch), so getting my head around a distributed model (Git) took some doing (more work required there, but that’s fodder for a different post). Working with a new language and framework as well – I hadn’t touched Ruby on rails up to now, which was stupid of me – I’m finding it pretty freaking awesome for the most part.

It’s taken me longer than I thought to come up to speed. I’ve been scraping the rust off my unix-fu and with unfettered access to the codebase and the command line, I feel like a long-lost missing limb has been reattached. Between that and the whole ‘learning new stuff’ thing, I’ve not had much time for anything else. There seems to be light at the end of the tunnel though. I’m grokking more of the code base and starting to make some useful code contributions.

One of the things that I’m trying to do at the moment is put a little bit of structure in the way that the business group and the development group communicate. Right now there is lots and lots of communication, which is great. The confusion comes in where the lines blur between the business telling dev what they want and going to the extent of speccing out everything in such minute detail that there’s not much design work left to be done by the dev group. That might not really sound like a problem to some, but in practice I’ve found that it can make the dev guys feel micromanaged or like their opinion is not required/valued – especially when design-level decisions are architecturally difficult to implement. If non-tech design reaches too far and gets it wrong, it can mean wasted time and effort either for them when the developer rejects it, or for both if the developer codes it in order to prove the point. The thing I love about my current place is that there are no passengers, and this stuff tends to get worked out, but I think I can do more.

This is where I am hoping that Cucumber will help out. If I can put together a series of tests that step out the current behaviour with a DSL that both the business side and tech side can build their communication around, then I suspect we will find it a little easier to step through the desired behaviour in enough detail to be clear about functionality while leaving the dev guys to work out the implementation. It’s not the only reason I want to use Cucumber, but I do have high hopes in this area. It probably won’t hurt that people will be able to see this stuff turn into checks that can be executed automatically. The key is still in the communication. I’m looking to make the clarity of that communication easier.

Cucumber is new tech to me as well. Conceptually I get it, but it’s different when you start to get your hands dirty. After a lot of reading, I decided to go with Cucumber, Capybara and at this stage, I think a combination of Selenium and TestUnit (Ruby’s in-built unit testing framework). I’m a Capybara noob as well, so I’m up to my elbows in new stuff and loving it. Mostly. Sorta. Apart from the frustrating bits.

Almost the entirety of the literature I’ve found on this setup is for BDD – which is to be expected, given that’s what its for. All of the setup and how-to stuff is for building stuff that doesn’t exist yet. Great if you’re building from scratch, but not super helpful if you have an existing code base you want to fit tests to.

I had an utter bastard of a time getting Cucumber and Capybara to work at all. Granted that it’s probably because I’m working with code that does a few unusual things, but I didn’t expect quite as much screwing around as I had to do – so, I’m going to detail it here in case there are any other poor bastards out there who are trying something similar. Hopefully this will save you some frustration.

My setup is (as of writing) Ubuntu 10.4, using rails 2. I already have selenium installed, and the installation of that is not arduous, so I’m going to omit that part.

I installed the following gems:

libxml2 libxml2-dev libxslt1-dev
gherkin cucumber cucumber-rails nokogiri capybara database-cleaner

I’ve listed some that should be pulled in automatically by other gems’ dependencies. For instance, gherkin should be installed when installing cucumber, but for some reason I had to do it manually. YMMV.

After installing the requisite gems, it should be a simple matter of generating all of the cucumber goodness from the root of your rails project

(ruby) script/generate cucumber –capybara
(you can add other options if you want to have them built in)

If all your bits and pieces are installed correctly, you should see a bunch of files and directories get created.

Here are some things that tripped me up:
I thought I might do some work with rspec as well, so I pulled those down. If you’re thinking along the same lines, then Cucumber seems to barf if you install anything beyond rspec 1.3.3
I also had to mess around with the dependencies for the json gem in my project as cucumber seems quite particular about wanting 1.4.6

By default, cucumber will clean the database after each run (as is generally considered good automation practice). In my case however, I have enough stuff in the db that it doesn’t make sense right now to load from scratch or mock the bits I need. Maybe down the track but for the moment I’ve just switched this off in the environment file (more on this shortly).

Anyway, moving on. From root, there should now be a features directory. edit features/support/env.rb
You can either make your settings changes in here, or in another file and require it from here. This file is automatically generated, so if you plan on making a lot of changes here or you think you’ll have cause to regenerate this file, the latter might be a better option.

Here are the changes I put in

Capybara.default_selector = :css
Capybara.run_server = false
Capybara.app_host = <url of my testing area>
Capybara.javascript_driver = :selenium

One of the things that really screwed me around was not realising that Cucumber starts its own webserver and runs on that. I tried pointing it at the server instance that Rails kicks off, only to find that things were falling over and there was nothing in my logs to tell me why. I felt like a complete moron once I did find out, but then again, the docco in this area appears to be awfully light so I don’t feel too bad. Anyway – be warned. If you don’t want Cucumber to use its own webserver, you need to tell it so explicitly.

As I said, there’s bugger all out there right now on retrofitting scripts for an existing code base. It sounds like it’d be easy, right? The code and functionality is already there, so skip the red/green/refactor bit and go straight to green, right? Not really.

Rather than conceptually pre-fabricating your functionality, you get to look through your existing functionality, decide what is most important to test, work out where you need to retroactively install test hooks and build use cases that serve a business purpose, while not getting lost down a rabbit hole around functionality that currently exists, but perhaps doesn’t work quite how it should. Yay! Given that there is the potential for test to ‘just light up green’ when you write them, I’ve also found it a good idea to write checks that fail so you know they’re actually doing something, then correcting them – it sort of keeps the spirit of BDD/TDD even if you’re not driving any development just at this point.

Retrofitting Cucumber tests to your site is not as simple as it might sound. I am hopeful that it will be rewarding. It’s a little too early to tell just yet, but the signs are positive. If there’s any interest, I may follow up with a post about some more of the specifics of the actual hands-on retrofitting bit. If you have questions, fire away.

For those of you who weren’t at CAST2010 and maybe aren’t so acquainted with some of the things the Association for Software Testers is up to, you might be interested to learn that Cem Kaner is heading up a special interest group to create a dictionary of terms relevant to software testing.

For those people who are passionate about their testing terminology, why not give them a hand?

I would love to see this proliferated far and wide. I’d much prefer a wealth of definitions to choose from and a place to point people than a cult of money grubbers peddling their one true way.

I haven’t seen the dictionary get much airplay yet and granted it’s in its early stages, but my google-fu failed me the other day when I did an idle search and then I stumbled across it here. I figured I’d do my bit and get the word out. If you don’t have time to volunteer, pass the link on to someone that might.

I’ve been busy lately. In a good way, at long last.
I recently started at a new company and have been flat out getting acquainted with how the new place is put together. I have to say – I am loving the new role. I know when I turn up that before the day is done I will have learned something new or I will have made a positive difference somewhere – usually both.

Which brings me to the topic of this post. I’m not a big fan of bitching about ex-employers (and doing so is not the point of this post), but I feel compelled to point out some of the things that have struck me as I shift into this new role.

I did not enjoy working at my previous place of engagement. It’s a shame to say, because some of the people I worked with were very smart andvery talented. Unfortunately the company seemed designed to make any change a herculean effort. I’d be hard pressed to purposely design a more inefficient way of working. I’m not going to give you a laundry list of specifics (though I could). Suffice it to say that there were uphill battles to be fought on pretty much any front you turned to.

I should have seen the writing on the wall. I remember thinking two weeks in ‘Can I really bring myself to care about this company enough to stay and do what needs to be done?’ Still, I was optimistic. I had come from a job where I’d been more successful than I could have hoped and like Alexander before me, I was going to cut through bureaucratic Giordian knots and lead the way to a shining future of efficient development, clean code, strong inter-team relationships excellent configuration and change management, not to mention a kick-arse testing group.

Sadly I acheived none of that. I decided I’d stick it out for 2 years and see how much change I could make. Maybe it was like turning an ocean liner around. Once you got past the momentum built in one direction, surely change would come more quickly. Nope. Didn’t happen. What’s worse is I found myself conforming. Keeping my head down. Not saying anything when I should have made a stand.

As the two-year mark rapidly approached, I had what some substance abusers call a moment of clarity. I was ashamed at my behaviour. If my peers saw me behaving like this, they would not be proud. Far from me making that place better, I had allowed myself to become worse. I decided it was time to leave.

No sooner had I made the decision, an email came out of the blue. A guy emailed me, said he’d read my blog and wanted to have a chat about testing over lunch. He worked a few train stations away and hey, I’ll talk about testing all day if you buy me lunch so I was all for it.  I happened to mention during the meal that my current contract was almost up. One thing led to another and I found myself with an offer from a company that looked to me very much like it was doing things that I wanted to be a part of.

Long story short, I’m now working at said company with a bunch of super smart people doing work that I think makes a positive difference in the world. I like the work that I do and perhaps the most importantly I am myself again. I work and think and act from a place of integrity and I can see that the people I work with do the same.

I guess what I’m trying to get at is life is just too bloody short to work in a job that makes you miserable. Not only is it time you will regret never being able to get back, but you may not like the person you become. No amount of money is worth that.

If you find yourself looking at the clock wishing it would tick faster, if you wonder why you’re doing what you do when nothing seems to change, if late on Sunday afternoon when the prospect of work tomorrow makes you die a little bit inside – your place is no longer there (if it ever was).

I’m not saying ‘quit your job tomorrow’, but if you’re stuck in a work situation like this, do everything in your power to find something that is more you. There is something better out there. Go find it. Please.

Adam Goucher’s post on New things to steal from the Japanese is the impetus for this one. At CAST I gave a lightning talk on borrowing Japanese language as a teaching aid.

Using foreign loanwords is probably harmless enough if you’re using them to jazz up a story, but if you’re going to use them as teaching concepts especially in the sense of you being the teacher and having enough understanding to relay these concepts to others then more care is warranted.

There were a couple of points that I wanted to make in my talk (though I’m not sure how well I did in the five minutes that I had). Firstly, there seems to be some sort of mystic profundity attached to the Japanese language. Pick a common concept and use the Japanese word for it and all of a sudden it’s full of ancient wisdom.

When I first started learning Japanese, everything seemed terribly profound to me too – probably a combination of kanji seeming unintelligible and me watching far too many samurai movies. Being a little older and a little bit more experienced, I can safely say that, like western teenagers, Japanese teenagers can talk for a half hour solid and say ABSOLUTELY NOTHING.

Japanese language is not necessarily any more profound than your native language. English has some wonderful, seldom-used words that you could choose to bring back into common parlance.

The other thing is that Japanese language is not necessarily as straight forward as it seems. There are many words that do not have a one-to-one translation and many words that have more than one meaning based on the context you use them in. That cool word you borrowed to spice up your new teaching material may have other meanings, than the one you intended, or deeper meanings that you don’t quite understand.

Shu-Ha-Ri is a particular favourite of mine when it comes to borrowed terminology. On the surface, it describes the progression of learning from the novice stage (shu), where you follow an instructor’s directions over and over again without necessarily understanding why (wax on, wax off, Daniel-san); then an intermediate stage (ha) where you begin to explore what your new-found skills mean and how to apply them in practical situations; to an advanced stage (ri) where your learning becomes self-directed and you no longer require a teacher in order to evolve your skills.

Sounds simple enough in concept, right?

Okay, so let’s take a closer look at the three kanji used.

守 – Shu – means to obey, but also to protect or defend.

破 – Ha – means to rip, tear, break or destroy

離 – Ri – means to detach, separate or release

Armed with this knowledge it becomes a little tougher to reconcile each of them into the neat little definition above. To add to this, allow me to make a gross generalisation about Japanese culture compared with Western culture.

I have observed that westerners tend to have a mindset of ‘make me understand, then I will do’
In Japan there is frequently a pattern of ‘do and eventually you will understand’

The ‘do’ part may take several years before understanding arrives. Case in point, I have done kendo for around 18 years. I felt that I had a solid understanding of the basics maybe 6 years ago. Some of my peers would probably tell you I’m still not there yet.

I’m not saying that one way of thinking is right and one is wrong, or that one is better than the other, but that you should understand that these difference is there. They are part of the culture and it has a direct bearing on language and the understanding of concepts such as Shu Ha Ri

Ri may not come to you without many years of effort. It is unlikely that a several week or even several month course will take you from Shu, through Ha to Ri. Not in the most simplistic sense of the term. Got someone telling you that you can be a black-belt from scratch in six weeks? Run far, far away from that person.

As far as teachers go, I would suggest that if you are tempted to borrow from the Japanese, or another language that you don’t speak for that matter, that you spend some time studying the culture and the people and at the very least be aware that your understanding of the term may well be incomplete before you have your students start calling you sensei.

I’d like to say thank you to everyone who participated in this little puzzle exercise and most especially to everyone who provided feedback. I hope you had fun with it. Well done again to Rushmila Islam who was the first to work out the puzzle.

I have posted .the solution at the bottom of this post. YOU HAVE BEEN WARNED.

I learned a lot more than I thought I was going to. Firstly and probably the most obviously, I learned a lot about cryptography that I didn’t previously know. Thanks to James Bach for the collection of links.
Braingle was probably the best of the bunch. Thanks also for the link to Sebi’s puzzle.

I found Jürgen Plasser‘s answer fascinating. I hadn’t encountered measuring entropy of text as a way to inform possible encryption techniques. It’s something I’d like to look into further.

It was interesting to see how close people got with their responses. Many people had several correct pieces of the puzzle, but not quite enough to solve it completely.

I think most people gave me way too much credit for the amount of crypto knowledge I’d have had as a high-school student. I didn’t have access to teh intarwebs back then, and time that I could have spent in the library was more likely to be spent playing role-playing games or painting miniatures or similar valuable life skills.

One of the things that this puzzle made abundantly clear to me was the value of information before and after it becomes known. Without the key information, it was a real puzzle. Far more so than I had originally anticipated. Having all the information at hand, the puzzle seems obvious and simple in hindsight.

It drove home for me just how difficult it is to convey the value of a tester to those who don’t understand what good testers do. Even the distinction that testers are a source of information (so is a book. so what?) doesn’t necessarily convey the effort that is involved at producing this information.

Some people went to a great deal of effort; tried some very cool things. That was the most gratifying thing for me. The answer was ultimately irrelevant. I got to see how you guys think. As testers, we are often showing others the fruits of our effort, but not the beauty of the effort itself. Having been able to do that here, at least in part, I think is pretty cool. Thank you again to everyone who shared.

SPOILER ALERT – Explanation of the cipher follows.

The message was encoded with direct character substitution with a few added extras. The message reads top-down, right to left. The first two characters provide the character offset – character 1 – character 2 = offset.

The spaces themselves mean nothing, but exist primarily for obfuscation though I did put them in diagonally in the first puzzle to give a clue to the direction of the word flow. Spaces in the actual message are represented by vowels, so in this sense, vowels do double duty in that they can represent their offset character or a space.

I played around a bit with iteration using JUnit today. I have some generic tests that behave differently depending on the values fed to them. I don’t want to have iteration code living alongside each test (maintenance nightmare), so I wanted to use JUnit’s @Parameters tag to pull in my test data via a Preferences file and do the iteration for me.

It took me longer than I expected to get right, mostly because static methods annoy me and trip me up a bit. (Your parameters method must be static in order for it to work).

However, get it working I did. On the off chance that it saves someone else some time, here’s how it works:

import java.io.InputStream;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Enumeration;
import java.util.Properties;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;

@RunWith(Parameterized.class)
public class paramTest
{
	final static String dataFile = "/dataDriver.txt";
	private String words;

	public paramTest(String words)
	{
		this.words = words;
	}

	@Test
	public void verifyThing() throws Exception
	{
		System.out.println("key: " + words);
	}

	@Parameters
	public static Collection data()
	{
		Collection returnList = new ArrayList