Found a showstopper bug fairly late in the process which has got me wondering about how it could have been caught earlier
The defect description :
The app has an address book - one of the fields in the address book is for a mobile (cell for any US readers) phone number. Put a number in there with spaces in - some people like their numbers formatted that way.
From the order screen of the app, find this address and select it for use in the order. The order can have extras where you can include the mobile phone number so the customer is notified about the stages of their order. However, do not select this extra option - so the phone number is not needed and does not appear on the screen. Only the mailing address is used.
Process the order - the screen hangs.
Go to the address book and remove the space from the phone number in the address book, save the address, find the address from the order screen - order processes.
Go to the address book again, put the space in, press the 'use this address' button that is on the address book screen, process the order, all works fine.
From the order screen search for the address, find it, process. Hangs
So what testing technique ( apart from persistence and some luck ) would have caught this ?
Not having access to the code I don't know if a code review would have - the code seems to behave differently when called from 2 places and somehow the fact that there is a space causes the hang ( maybe there is some exception being thrown that is displayed to the user if the phone number is actually being used )
Would a detailed script have caught this ? It would have to be a detailed scenario - I can imagine scripts which check saving values to an address book and scripts that check that an entry can be used to populate an order - but a script that will check an order that uses an entry in the address book with spaces in it that is not actually used in the order ??
So it will go into my memory banks and be one of those tales told when testers gather to tell tales about bugs found. I'll pass the defect details onto the other testers on the team to see how they might have caught it. But in the end it will be another bit of local tester folklore - how can this be passed onto newbie testers ?
Posts
1 comment:
Phil,
Sounds like two bugs to me. Or at least, the bug manifesting in two different areas of the system.
If I understand you correctly this is basically a bug due to bad data. The lesson for the newbie tester would therefore be that just because data is OK in one(or the first) part of the system doesn’t mean that if it’s subsequently used in other areas that it doesn’t cause problems.
I think a data / structure review approach would’ve found that this kind of bug. What I mean by that is take some good/bad data and then follow it through the system. What stages does it go through the system, changed or unchanged. What modules within the application use the data?
We found a similar kind of bug through exploratory testing. This was in a mapping application where you digitized a line which was self-intersecting (crossing itself) – warning message displayed, no problem. Created a non-self-intersecting line, no problem. Edited a non-self-intersecting line and intersected it – system hangs. It didn’t help that this was only for a certain type of line that you had to choose before editing...
What we learned from that is that we mix the order in which to use functionality. Just because one approach works doesn’t mean that using functions in a different order works.
What I would say is that you DID find the bug. If you’re working in an environment that encourages people to follow their hunches and give them the freedom to explore rather than counting bug cases against requirements you have done something to find these kind of bugs. The atmosphere in which you test has a lot to do with what you get out of your test effort.
Post a Comment