A Factory of Bugs

The latest GR Testers meeting was at The Factory and the topic was 'defect types' - or to give it a grander title, Bug Taxonomies.One of our newer members had asked about different defect types at the previous meeting so we used this as a topic for this meeting

Introductions over, the topic for the evening was announced and I kicked things off by asking those present who knew what a Bug taxonomy was and if they ever made us of it.
No raised hands and a few blank faces

I used my copy of A Practitioner's Guide to Software Test Design to show some examples and to explain about the different taxonomies that have already been documented such as:
- the 500 listed in Testing Computer Software
- 60 high-level ones from Vijayaraghavan that are specific to a shopping cart
- ones specific to OO by Robert Binder
and I referenced Binder and his two approaches to testing

1) Use reqts and spec to guide creation of test cases “non-specific fault model”

2) “specific fault model” - taxonomy of defects guides the creation of test cases

We then discussed how to use bug taxonomies - can they help answer the question:

"if there is a bug in this part of the system how would you find it ?"

- if you are aware of the potential defects then does this knowledge help you find them more efficiently ? Would you miss them if you did not know about them ? If you're unaware of memory leaks then you're not going to write a test to see if your app has them.

Conversation took one of the many detours that make the chats at these events so interesting as usability was discussed and how usability bugs could get short shrift ( because the product has shipped and the PM got a tick for doing so ) or how usability can be great but the back-end is rubbish as it's built on a 30 year old system or how usability problems get ignored because "the customers are using the program wrong"
Seems there is enough material here for a topic on it's own.

Next detour was to discuss what makes a defect a defect - a trap cunningly laid by Pete Walen to see if people were aware or not about safety language and the possible legal ramifications of actually using the word 'defect'
It was interesting to me to hear different people using different words for bugs and some being quite adamant ( sometime because it is official company policy ) that 'bug' and 'defect' were different things, as was 'issue'.

There was a bit of confusion about using bug taxonomies as couldn't a bug be in several ? If there was a deadlock bug then wouldn't this also count as a usability issue if the customer couldn't update their records ? And weren't all bugs in effect usability issues ?

This was resolved by explaining that by using taxonomies you would run tests to see if that form of defect existed in the system. To test for a deadlock issue you wouldn't be running a usability test, there would be tests specifically for a deadlock. Then, maybe if deadlocks could happen and could not be fixed there would be a usability test to see if the error for a deadlock was handled correctly.

The evening was wrapped up by asking around the table to see if anyone had encountered 3 types of bug not usually found in a standard bug taxonomy - Heisenbugs, Mandlebugs and Schrōdingbugs

Next meeting is on automated vs manual

Meeting notes can be found here - thanks to Pete Walen