Unfortunately Anne couldn’t attend so Rob did the keynote on bug triage by himself. It was a slight deviation from the planned keynote, but Rob chose to talk about his Elevator Parable instead of testing lessons from the delivery room labour triage.
So Rob began by telling us his “story of the elevator parable” (courtesy of Rob Sabourin), the crux of which is he got into an packed elevator in his place of work, turned around to face the door and on the way up he heard 2 people talking about his project (Rob tells a great story, i’m not even going to try to recreate it here!)
He didn’t recognise the voices & shiver went down his spine & caused him to have a cold sweat when they started talking about a new (big) feature request & a Blue Screen Of Death.
Who were they & why were they talking about his project? He decided to treat it like a bug & triage it accordingly.
This is where Rob got the audience participation going – we had to use the colour cards we’d use in open season for asking questions to prioritise the bug with the information we had available (the context). 2 strangers talking about his product and crash related to a new feature.
We overwhelmingly prioritised it as a P1 (84% P1 approx).
Rob started playing “messages” which “had been left on his answerphone” – each one providing more information, giving better context. We would reprioritise after each message. All the messages in the keynote can be read in the full Elevator Parable story
Message 1: from boss – 3 months to deliver new feature instead of 1 week. More time to identify bug & get it fixed. Priority dropped to P2 (71% P2 approx).
Message 2: from finance VP – new feature is worth a load of cash. Priority back up to P1 (61% P1 approx)
And so on for another 2 or 3 messages, each one giving increased context, causing us to change our minds on the priority.
We also assessed the severity on the same scale after each message.
What the exercise demonstrated is that context impacts our decisions. We started with a P1 when we had little context & finished on a P2 after we had heard all the messages.
Having context enables us to make informed decisions about which bugs to push to get fixed backed up with reasoned arguments & justification – I.e. Bug advocacy.
Rob used the opening credits to M*A*S*H as a metaphor for bug triage – after the chopper has landed & the casualties have been offloaded on to the jeep (Jeep?), the doctors are triaging them on the jeep as they are being driven to the hospital.
They are assessing in patient in turn, assigning a priority & severity, so that when the casualty reaches the hospital it’s clearer how they need to be treated. Compare this to bug triage – as the bugs are submitted, they are assigned a priority & severity to ascertain how they should be treated – do they need to be fixed now or later?
Rob also won over some of the audience with a clip from Star Wars where Han & C-3PO advise R2-D2 to let Chewbacca win the game of chess & what the consequences of him losing might be (loss of life or limb)
This was a metaphor for decision making in triaging. Certain stakeholders use their position in an org – those folks who have a certain style which results in them getting overly angry. Sometimes, you just have to let the Wookie win. You need these stakeholders buy-in to the decision making process to try & prevent these stakeholders “escalating” problems around you.
Also, be certain about your decision making process, be efficient in your decision making & follow through on your decisions.
Rob has created bug triage quadrants to help prioritise bugs:
Some key takeaways from Robs keynote include:
- Context is alive! Deal with it accordingly
- Re-evaluate your bugs as different information comes to light
- Be slick with your decision making process when triaging bugs
- Bugs are not good or bad – they are important, some have high priority, some have high severity
Applying what Rob was talking about in terms of bug triage to my context – I’m currently working in an agile environment & we don’t have a bug tracking tool as such.
We aim to find business logic related bugs on the Programmers machine before they commit the code. If we believe there is a problem, we can ask the BA or even better, the Business stakeholders for their opinion if needs be. I can use the M*A*S*H metaphor – the chopper coming into land is the bug being identified. The chatting to stakeholders is the triaging on the jeep. Fixing the bug after the chat is the stitching up of the patient in the tent.
This is only suitable for bugs found locally on the Programmers machine – 1 casuality on 1 jeep – whats the priority there!?
Other bugs we experience include those bugs found once the code of the feature is in our Integration environment – we might have several of those. Which do we fix first – multiple casualities on 1 jeep.
There’s also some bugs found in Production environment (not great!) – these bugs could be from any features we have released into the wild – multiple casualities on multiple jeeps. These bugs require more triage & prioritisation – not all of them are service impacting (P1) & as such don’t need fixing immediately.
Do we have a Wookie in our organisation? I’m not sure if we do. And if there was, who would they escalate to? We all work on the same floor!
Overall, the keynote was rich with visuals & audio, informative & included great audience participation. It certainly has made me more interested in Robs Just In Time Testing course (link to JITT articles in zip file) as well as a wealth of other information available on his site amibugshare
Video of Rob Sabourin Lets Test keynote is available on Youtube.
PS – if you’re interested, here’s how the Lest’s Test audience prioritised the bugs after each piece of information:
Round 1 – straight out of the lift
p1 = 84
p2 = 14
p3 = 3
Round 2 – After message from boss – more time – 3 months instead of 1 week. Reprioitise bug
P1 = 28
p2 = 71
P3 = 5
Round 3 – After call from vp finance – feature is worth how much!?
P1 = 61
p2 = 28
P3 = 6