Reducing the Cost of Testing - Interview with Markus Gärtner

Next up in my series of interviews with authors of Reducing the Cost of Software Testing is Markus Gärtner


It should be a short book - do less of it and/or hire cheaper testers. Why is there a whole book about it ?

Software Testing is a broad field. A very broad one. It is florished from diverse fields as software testing, programming, software design, psychology, philosophy, epistemology, ..., and management. So, when we ask how to reduce the cost of software testing, one possible answer is to do less testing. But considering the broad field of testing, that answer doesn't help you at all, because that answer does not tell you how to do that, what the liabilities of your approach might be, and what other things you may want to try out.

In the book, Matt Heusser and Govind Kulkarni got together many of the thoughtleaders in the field of software testing to provide a broad collection of answers. In fact, you will find 20 different chapters providing different answers on what software testing actually costs us, what we should do in our projects, and how we should do it.

I'm a tester, I find bugs - why do I need to worry about reducing the cost of testing ?

Testing is our profession. That means that we as software testers are asked about our professional opinions and inputs to the whole software development cycle. Managers make decisions based on our opinions and input to them. That is why we should provide the information of the actual cost of our actions. As professional testers we have the responsibility to inform managers and stakeholders about what we are doing, why we are doing it, and what it will actually cost them, so that they can make the right decisions. If you don't care about the cost of your actions, maybe you are wasting your company's time and money. I call that unprofessional, if you're unaware about it.

Your chapter was about Exploiting the Testing Bottleneck - why did you decide to write about that?

Initially I was considering to use some of the insights I got from the Quality Software Management series by Jerry Weinberg, and write a chapter on how to manage software development projects so that we can actually reduce the cost of testing.

When I started to write my chapter, I found myself on a different track though. Inspired by Elisabeth Hendrickson's talk on Agile Testing practices and principles, I started to explain that benefits you get from working on an Agile team that applies good Agile Development practices like Continuous Integration, TDD, and Exploratory Testing. As I digged deeper, I found more and more pieces, I wanted to incorporate. In the end, I had the difficulty to find a proper name for what I had written down.

As testing is mostly perceived to be a bottleneck on the software projects I had been part of until that time, I found that name quite easily. In Goldratt's The Goal he explains that you should identify bottlenecks in your process, and exploit them, in order to improve your work. The chapter name "Exploiting the Testing Bottleneck" is a deliberate pick on that, because I truly believe that we would test better on any projects, if we seeked to exploit the (perceived) bottleneck in first place, rather than blaming the messengers.

How long did it take you to write ?

Roughly 32 years, but I think that I need to explain that answer. I consider the question "How long did it take you to write your chapter?" similar to the questions "why does testing take us so long?", "how long does it take you to test that?", or "when will be done eating?" I incorporated some portion of my past life into the chapter I wrote. There are nuggets like the insights I got from "The Goal", Lean thinking, systems thinking, and Agile software development in there.

If you consider learning about the stuff I wrote about to be part of the "writing process" of my chapter, I would say roughly 32 years. If you think of the time of writing my chapter as the amount of time I actually worked on the text of my chapter, I would say, roughly one week of work, starting in early May 2010 during my parental leave from work, polishing it up over the summer holidays in August, and finishing "my" first final draft in early October 2010.


Are you happy when you re-read it or would you like to change it ?

For most of my writings I apply a fire-and-forget principle. I write it down, I hit the publish button on my blog, and then I start to forget about it. By forgetting I mean not to revisit it later. This makes a review process very hurtful. So, while I re-read it, I incorporated some of the insights I got from review comments, already. So far, I haven't received a physical copy of the book, so I haven't had the opportunity to re-read it there, but I didn't dare to read the electronical copies we got before the book went to printing. I am probably too scared about that, and I know that it will hunt me down in a few months, maybe.

This does not hold for all of my writing. Sometimes I re-read some of my stuff, and find quite a few shortcomings. I know that I will be hurt to find so many tiny glitches and problems I left in there. So, I prefer to keep the illusion of a good enough chapter for the moment, and deliberately taking the time to re-read in a few weeks or so. Whenever I feel like hurting myself with it. :)

Now that you are a published author, do you have any plans for future books ?

Oh, I was a published author before the book. Few of readers might know that, but I published my diploma thesis back in 2008. It's in German, it's about hand gesture recognition in a human-machine interaction, and I explained the approach to my mother - who is a non-technician - at the time I worked on it. I think, I sold 5 copies of it so far, but publishing the book didn't change anything for me, I think.

That said, I work on a book on ATDD right now. I submitted a first draft to the publisher. I am also in contact with a German publisher through whom I might publish a book on Agile Testing in German in the future, and I also got lots of ideas for maybe a second book in English. Personally I consider my chapter in "How to reduce the cost of testing?" as one milestone on a longer publishing road.

Will your copy of the book be put in a special place ?

I will put it on my book shelf right beneath Beautiful Testing. Since we just moved into a house, I don't have that book shelf filled, yet. So, the special place might actually end up as the first book on my shelf - depends on what reaches me first: my copy of the book, or my energy to unpack my library. :)

Putting you on the spot - what is your favourite chapter in the book ( and why ) ?

Oh, I can't take a single pick there. So, let me give a bunch. I loved reviewing Michael Kelly's "Session-Based Test Management" as well as "Opportunity Cost of Testing" from Catherine Powell. Curtis Stuehrenberg hit a nail for me with "Clean Test: Suggestions for Reducing Costs by Increasing Test Craftsmanship" , but it might be just the mention of craftsmanship in the title.

Michael Kelly explained thoroughly what Session-based test management is about. From his chapter I got the insight that lead me to the idea of inspectional testing - a way to explore the functions of the product in order to inform good test planning. Every tester that claims to do Exploratory Testing has to read this.

Catherine Powell explained the concept of opportunity costs in a way that I could understand, and follow-up on it. It was a real eye-opener.

For Curtis' chapter, I loved the mention of craftsmanship. Having been involved since December 2008 with the Software Craftsmanship community, I actually envied him to use craftsmanship in his title. I was considering to rename my chapter when I saw that, but it would have been obvious who would have spoiled whom about it.

But, I have to admit that I haven't read all of the chapters. I missed to read the one from Michael Bolton and the one from Jon Bach so far, but I am quite sure that I will love these chapters as well, and will incorporate them in my list of favourite chapters.

Until this book came out, what was your favourite testing book ?

This is a hard question. I have read some of the most tedious software testing books, I think. Among the testing books that I loved are Lee Copeland's "A practitioner's approach to software testing design". I also loved reviewing Lisa Crispin's and Janet Gregory's "Agile Testing".
But my favoourite testing book before "How to reduce the cost of testing" is "Lessons Learned in Software Testing" by Cem Kaner, James Bach, and Bret Pettichord, as well as "Perfect Software... and other illusions about testing" from Jerry Weinberg.

( Me - what a great choice, Lessons Learned was one of my very first books and set me off on my testing career )