Reducing the Cost of Testing - Interview with Matthew Heusser

Next up in my series of interviews with authors of Reducing the Cost of Software Testing is Matthew Heusser


From a Linked In discussion to a book - quite a jump. What triggered the idea that the discussion could be made into a book ?

I have no idea. It was all Govind.

That’s a sort of secret in our industry: You can get shockingly far not inventing stuff, but by listening, paying attention, connecting the dots, and taking action early. Govind came to me with the book idea; all I did was take him seriously and then help him execute.

The book is about Reducing the Cost of Testing - this implies there is a a lot of waste that could be cut ? If so, why is this, why is there so much waste ?

There is waste all over the place in the corporate world. It is ridiculous.

Consider, for example, the telephone game we all played in grammar school. First one person says a sentence to the person next to him, and around the circle we go. By the end, the message is nothing like what it started out as. In many companies, that is the software development process - with a translation process from a human to a spec, and another from spec to design, design to code, code to test, and so on. I just don’t have a better term than ridiculous.

Now take a walk around a modern corporation with a clip-board. For everyone you see doing something that really looks like work, make a check-mark. For the ones that don’t, make a check-mark in a different column. Do it every fifteen minutes for a few hours, and you’ll find more waste than you might expect. Now am I saying that people don’t need breaks now and again? Certainly not. But a lot of people are just plain not working very hard. Often, no one is working very hard -- and that is due to systematic issues that management can do something about.

Chapters from different testers in different parts of the world - how was organising this ? was it like herding cats ?

Oh it was tough. You have to realize that most of the contributors are not professional writers, and that it is incredibly easy to create a fifty-word outline, but, for a non-writer, creating five thousand solid words is, well, hard. When we had to track down the contributors and ask them “when are you going to be done?” or, worse, tell a few we had quality concerns ... wow. That was no fun.

There’s an old saw that, as a publisher, you can have friends, they just won’t be anyone you work with professionally. It’s a joke, but I certainly understand where it came from.

Do you have a favourite chapter ?

Being both immersed in the problem and arguably doing the most research, and more than a little bit human and selfish, I’m a little partial to my own chapter. Michael Larsen’s , Michael Kelly’s and Scott Barber's chapters, along with Catherine Powell’s appendix, also provide some of the most direct/concrete advice in the book.

What was the hardest part of putting the book together ?

Oh wow. Everything. Offhand: Dealing with authors who didn’t deliver, or delivered substandard work. Dealing with the authors who stepped in at the last minute, who wrote a chapter in a weekend -- you have to admire that, but gosh did we have some polishing to do.

Probably the most painful for me personally was the communications problems that came from people genuinely mis-understanding, often because I had failed to make things clear in the first place. Knowing that more than a bit of it was my fault makes it hurt more.

You also did the foreword for "Clean Coder" by Robert "Uncle Bob" Martin and you've had a chapter in "Beautiful Testing" Are books and writing important to you ?

For me, writing is a creative outlet that is just enough like testing to sharpen my mind, and just different enough to be a distraction. That I can do it anywhere on my own schedule, without travel, certainly helps.

When someone tells me that they took one of my ideas and applied them to success, that makes my day. When they are more critical, it makes me think and improve. Either way, I win.

You did this book, had a day job, family, blog and got other articles published - do you only sleep 2 hours a night ?

The rumor seems to be that I have super-powers, but the truth is simply that I have made a conscious choice to make software testing an important part of my life. I also have a wonderful life-partner in my wife, who does many things that allow me to sneak in a few more hours for testing here or there.

It’s not two hours, but yes, you’re right, I do not sleep enough, and that’s a problem.

What's next in the pipeline for you - another book ?

How did you guess? (Laughs) I am currently gathering resources for a book on classic essays in software testing. I am open to recommendations ...




Thanks for the replies Matt - and to show how busy he is, take a look at his profile

Joining the club

When new members request membership of the Software Testing Club, one of the questions is:

What would you like to get out of the Software Testing Club ?

Pretty straightforward question but I have been amused at some of the answers.

I would not like to get out of the software club but in case you want me to leave this club then simply a mail will do.
- seems The Terminators reputation has spread.

I LIKE SOFTWARE TOO MUCH
- too much to do what ? test it ? and please dont shout

My dream software..its a secret, will let you know later
- I'm still waiting

Its too early...
you only test at night ?

dsdsdd
asdf asdf 123

Nothing
so you wont mind if I dont approve your membership then ?

Reducing the Cost of Testing - Interview with Markus Gärtner

Next up in my series of interviews with authors of Reducing the Cost of Software Testing is Markus Gärtner


It should be a short book - do less of it and/or hire cheaper testers. Why is there a whole book about it ?

Software Testing is a broad field. A very broad one. It is florished from diverse fields as software testing, programming, software design, psychology, philosophy, epistemology, ..., and management. So, when we ask how to reduce the cost of software testing, one possible answer is to do less testing. But considering the broad field of testing, that answer doesn't help you at all, because that answer does not tell you how to do that, what the liabilities of your approach might be, and what other things you may want to try out.

In the book, Matt Heusser and Govind Kulkarni got together many of the thoughtleaders in the field of software testing to provide a broad collection of answers. In fact, you will find 20 different chapters providing different answers on what software testing actually costs us, what we should do in our projects, and how we should do it.

I'm a tester, I find bugs - why do I need to worry about reducing the cost of testing ?

Testing is our profession. That means that we as software testers are asked about our professional opinions and inputs to the whole software development cycle. Managers make decisions based on our opinions and input to them. That is why we should provide the information of the actual cost of our actions. As professional testers we have the responsibility to inform managers and stakeholders about what we are doing, why we are doing it, and what it will actually cost them, so that they can make the right decisions. If you don't care about the cost of your actions, maybe you are wasting your company's time and money. I call that unprofessional, if you're unaware about it.

Your chapter was about Exploiting the Testing Bottleneck - why did you decide to write about that?

Initially I was considering to use some of the insights I got from the Quality Software Management series by Jerry Weinberg, and write a chapter on how to manage software development projects so that we can actually reduce the cost of testing.

When I started to write my chapter, I found myself on a different track though. Inspired by Elisabeth Hendrickson's talk on Agile Testing practices and principles, I started to explain that benefits you get from working on an Agile team that applies good Agile Development practices like Continuous Integration, TDD, and Exploratory Testing. As I digged deeper, I found more and more pieces, I wanted to incorporate. In the end, I had the difficulty to find a proper name for what I had written down.

As testing is mostly perceived to be a bottleneck on the software projects I had been part of until that time, I found that name quite easily. In Goldratt's The Goal he explains that you should identify bottlenecks in your process, and exploit them, in order to improve your work. The chapter name "Exploiting the Testing Bottleneck" is a deliberate pick on that, because I truly believe that we would test better on any projects, if we seeked to exploit the (perceived) bottleneck in first place, rather than blaming the messengers.

How long did it take you to write ?

Roughly 32 years, but I think that I need to explain that answer. I consider the question "How long did it take you to write your chapter?" similar to the questions "why does testing take us so long?", "how long does it take you to test that?", or "when will be done eating?" I incorporated some portion of my past life into the chapter I wrote. There are nuggets like the insights I got from "The Goal", Lean thinking, systems thinking, and Agile software development in there.

If you consider learning about the stuff I wrote about to be part of the "writing process" of my chapter, I would say roughly 32 years. If you think of the time of writing my chapter as the amount of time I actually worked on the text of my chapter, I would say, roughly one week of work, starting in early May 2010 during my parental leave from work, polishing it up over the summer holidays in August, and finishing "my" first final draft in early October 2010.


Are you happy when you re-read it or would you like to change it ?

For most of my writings I apply a fire-and-forget principle. I write it down, I hit the publish button on my blog, and then I start to forget about it. By forgetting I mean not to revisit it later. This makes a review process very hurtful. So, while I re-read it, I incorporated some of the insights I got from review comments, already. So far, I haven't received a physical copy of the book, so I haven't had the opportunity to re-read it there, but I didn't dare to read the electronical copies we got before the book went to printing. I am probably too scared about that, and I know that it will hunt me down in a few months, maybe.

This does not hold for all of my writing. Sometimes I re-read some of my stuff, and find quite a few shortcomings. I know that I will be hurt to find so many tiny glitches and problems I left in there. So, I prefer to keep the illusion of a good enough chapter for the moment, and deliberately taking the time to re-read in a few weeks or so. Whenever I feel like hurting myself with it. :)

Now that you are a published author, do you have any plans for future books ?

Oh, I was a published author before the book. Few of readers might know that, but I published my diploma thesis back in 2008. It's in German, it's about hand gesture recognition in a human-machine interaction, and I explained the approach to my mother - who is a non-technician - at the time I worked on it. I think, I sold 5 copies of it so far, but publishing the book didn't change anything for me, I think.

That said, I work on a book on ATDD right now. I submitted a first draft to the publisher. I am also in contact with a German publisher through whom I might publish a book on Agile Testing in German in the future, and I also got lots of ideas for maybe a second book in English. Personally I consider my chapter in "How to reduce the cost of testing?" as one milestone on a longer publishing road.

Will your copy of the book be put in a special place ?

I will put it on my book shelf right beneath Beautiful Testing. Since we just moved into a house, I don't have that book shelf filled, yet. So, the special place might actually end up as the first book on my shelf - depends on what reaches me first: my copy of the book, or my energy to unpack my library. :)

Putting you on the spot - what is your favourite chapter in the book ( and why ) ?

Oh, I can't take a single pick there. So, let me give a bunch. I loved reviewing Michael Kelly's "Session-Based Test Management" as well as "Opportunity Cost of Testing" from Catherine Powell. Curtis Stuehrenberg hit a nail for me with "Clean Test: Suggestions for Reducing Costs by Increasing Test Craftsmanship" , but it might be just the mention of craftsmanship in the title.

Michael Kelly explained thoroughly what Session-based test management is about. From his chapter I got the insight that lead me to the idea of inspectional testing - a way to explore the functions of the product in order to inform good test planning. Every tester that claims to do Exploratory Testing has to read this.

Catherine Powell explained the concept of opportunity costs in a way that I could understand, and follow-up on it. It was a real eye-opener.

For Curtis' chapter, I loved the mention of craftsmanship. Having been involved since December 2008 with the Software Craftsmanship community, I actually envied him to use craftsmanship in his title. I was considering to rename my chapter when I saw that, but it would have been obvious who would have spoiled whom about it.

But, I have to admit that I haven't read all of the chapters. I missed to read the one from Michael Bolton and the one from Jon Bach so far, but I am quite sure that I will love these chapters as well, and will incorporate them in my list of favourite chapters.

Until this book came out, what was your favourite testing book ?

This is a hard question. I have read some of the most tedious software testing books, I think. Among the testing books that I loved are Lee Copeland's "A practitioner's approach to software testing design". I also loved reviewing Lisa Crispin's and Janet Gregory's "Agile Testing".
But my favoourite testing book before "How to reduce the cost of testing" is "Lessons Learned in Software Testing" by Cem Kaner, James Bach, and Bret Pettichord, as well as "Perfect Software... and other illusions about testing" from Jerry Weinberg.

( Me - what a great choice, Lessons Learned was one of my very first books and set me off on my testing career )

From acorns to skyscrapers

Just over 4 years ago I was member number 8 of the Software Testing Club - membership is now over 9500 and rapidly closing in on 10,000

I've posted 506 times to discussions, declined numerous request for memberships from lovely fun girls looking for love, been threatened with having a DOS attack on the site and been threatened with the involvement of lawyers after an argument with a member who thought the biggest thing in testing was going to be copyright laws.

Thanks to Rosie and Rob Lambert and Stephen Hill the site has gone from being just an online site to a site that has offered eBooks, posters and the fabulous newspaper, The Testing Planet.

It has also gone offline as well with STC meetups at London, Birmingham, Oxford, Bristol, Liverpool - and Winchester later this month.

and now it's going trans-Atlantic with the announcement that on November 10 there is a Chicago STC Meetup

From a twinkling in Rosie Sherry's eye to this - WOW.
I'm glad to have been part of it - and who knows what's next....

( well, Rosie and Rob and Stephen probably have lots of ideas for what's next !! )

Reducing the Cost of Testing - Interview with Catherine Powell

The book How to Reduce the Cost of Software Testing is now out - a great effort by a bunch of testers to put a book together and get it published.

Wanting to know more about how and why they did it, I interviewed some of the authors.
First to respond was Catherine Powell

It should be a short book - do less of it and/or hire cheaper testers. Why is there a whole book about it ?

You're right, it is easy. Except for all the ways in which it's really hard. Our original title for the book was "How to Reduce the Cost of Software Testing Without Mindlessly Shooting Yourself In the Foot". (It was too long.)

The point is that there are consequences of our decisions, and the book is about making informed choices that reduce the cost of testing without unacceptable side effects.

To take one of your simple examples: we could reduce the cost of testing by laying off all the testers. This would absolutely drive the costs of testing down. It would also result in buggy, unpredictable software that your customers don't like. Your development costs go up - fewer features per developer hour - because they're working on bugs. Your revenue goes down because people don't really want to buy buggy or hard to use software. Your cost of hosting goes up, because you don't know how the system scales, and you have to just increase everything. In the end, you're worse off than if you'd kept your testers around.

The book is about placing software test - and the costs of testing software - in context of the entire software development lifecycle. It's about taking a facetious answer ("just do less!") and providing manager, executives, and testers with the tools to make those decisions intelligently.

I'm a tester, I find bugs - why do I need to worry about reducing the cost of testing ?

You're a tester. You do a whole lot more than find bugs. You gather information, you understand systems, you figure out how things work and how they don't. And, if you're like most testers, you worry about your job. Heck, most professionals do. This book is about the value that test brings to an organization, and about realizing that value in a cost-effective way. Those are exactly the skills that you as a tester will need to thrive in the future of software development.

Your chapter was about Opportunity Costs - why did you decide to write about that?

So much of what we do in testing is about making tradeoffs. It's about recognizing that we could test forever but that we don't actually have forever; instead there's a release coming up fast. I wanted to write about that decision making process using a tool that is recognizable to the business: opportunity cost. Opportunity cost is what we give up by not doing something. Because we went to the store, we didn't have time to go to the gym; the opportunity cost of going to the store is the benefit we would have gotten from going to the gym (weight loss! better cardio!). Opportunity cost provides us with a way to expose the risks and costs of not doing something, and not doing something is a choice we frequently face in software testing.

How long did it take you to write ?

That's a tough question. The chapter on opportunity costs took a few days. The first draft came together in about a 6 hour period, but it was (like many first drafts) terrible! Fixing it took rather longer.

The appendix on 25 Tips to Reduce Testing Today was faster. Matt Heusser and I worked on that and Immediate Strategies to Reduce Test Cost together - you'll notice the strategies and the tips align - and it was really about the quick hits and small changes that together make a big difference. I don't know if it actually took less time, but it was certainly fun, and a lot of fun to think about what the most immediate thing could possibly be.

Are you happy when you re-read it or would you like to change it ?

I'm still happy with it. There are some tweaks I would make, but I'm still very excited about bringing together business concepts with testing needs and helping those two worlds work better together.

Now that you are a published author, do you have any plans for future books ?

I haven't really decided! At the moment I'm mostly writing for my blog and shorter articles, but there may be a book waiting to emerge.

Will your copy of the book be put in a special place ?

Does the bookshelf count as a special place?

Putting you on the spot - what is your favourite chapter in the book ( and why ) ?

That's a tough one. The chapters are all helpful in very different ways. This is a book to me that you don't read all the way through and put it away. You read different chapters as you go through various situations, whether it's doing budgeting (and reading Michael Larsen's Trading Money for Time), or building up a test team (and reading Anne-Marie Charrett's Cost of Starting Up a Test Team), or trying to move faster (and reading Petteri Lyttinen's You Can't Waste Money on a Defect That Isn't There).

Until this book came out, what was your favourite testing book ?

I'm into the classics, so my favorite is Hetzel's Complete Guide to Software Testing. I pick it up when I need to be reminded of the fundamentals. Parts of it are a bit dated, but most of it is still solid guidance.