Lets Test Session: Scott Barber – Testing Missions in Context: From Checking To Assessment

Scott’s keynote was about understanding what is asked of us (Testers) & how those requests fit into the bigger picture of what the business / organisation actually wants to achieve.

Scott Barber Lets Lest Keynote

Picture courtesy of Huib Schoots

Scott had so much energy in his keynote that the people in the room had to sit up & pay attention!

During an introduction to “Testing Missions in Context: From Checking To Assessment” to Scott & what the keynote was going to be about, he was rudely interrupted by an incoming message a la Mission Impossible.

The message included an unfeasible request, with all blame being laid on Scott if any bugs were found in production. This prompted the question “sound familiar?” from Scott which resulted in a mexican wave of nods & smiles. Scott broke compared the message to the requests we (Testers) we recieve & highlighted that this was a task, not a mission.

Scott used an example from his time in the army where he was given a ‘mission’ which he thought he had nailed, but unfortunately he had failed – he had not been aware of the “Commanders Intent”. This intent was the bigger picture. Yes, Scott had successfully completed his ‘mission’ but really it was only a smaller task in a higher mission he had no idea about.

This example was then compared in the SW development context, such as not knowing about /  making assumptions about the higher mission. This was supported by some definitions & tips on how to avoid falling into the trap of thinking you’ve successfully completed your ‘mission’:

slide from Scott Barbers Lets Test keynote

I like these definitions. Thinking about testing tasks & assignments makes you realise what you are doing is not the be all & end all of the testing phase of the project. There is always a higher reason you have been asked to do something – try & get to the bottom of why you have been asked to do something!

The keynote went onto to focus on the individual sections highlighted in the above slide.

For Purpose, Scott talked about how businesses don’t want to have to pay for testing, how it is a necessary evil & how our testing needs to provide more value than if the business had not paid for testing at all.

This was a great point to make – we need to ensure we are always delivering value & looking for ways to increase our value. We don’t want to be just another checkbox exercise: “Test team recruited? check!” This undermines all that we have to offer.

For Roles, Scott provided a tool/model to help us think about the higher missions which includes different example questions we can ask of the business to obtain a greater understanding of why we’re doing something, rather than just diving headlong into our tasks. I like this tool/model for demonstrating product states & ideas through a products lifestyle – helps to visualise the bigger picture. Fortunately I’m on a team where the stakeholders & business sit next to the development team & are intrinsic in our day-to-day development. This really helps to get an understanding of where the product is currently at, where its going, how we get there & why.

Scott then went on to talk about Assignment as it perceived in the testing domain – they are often (incorrectly) seen as a mission themselves & that assignments tend to have a person identified who is responsible for delivering the assignment. Assignments are (generally) for individuals whereas missions are (generally) for groups. Nice way of differentiating between the missions & assignments I thought.

It was also whilst talking about assignments that Scott brought up the great quote “No plan survives the first bug” (taken from Moltke’s theory of war – No plan survives the first contact with the enemy).  What this essentially means is all the planning you’ve done up fornt becomes irrelevant when a showstopper bug is found, or a 3rd party (internal or external) changes their mind about a key piece of design or functionality.

Having assignments provides the individual with flexibility / freedom should the context change.

For Tasks, Scott had created a heuristic model for test task categorization:

scott barbers heuristic model for test task categorisation

To give this slide some context, Scott thinks of testing tasks as fallible.

Check = Pass/Fail, needs to be done.

Test = Search for info, experiment. What happens if? May have a desired outcome, not necessarily an expected outcome (that results in checking)

Assess = What teachers do with students – wouldn’t just give P or F on end of year grading without any feedback. Instead, we make qualitative & quantitative measurements & assign ranges.

Demonstrate = show the positive

Field Study = fancy term for UAT

Scott tied all the elements of a testing mission together to demonstrate of how to build a testing mission. This included the question of is there intrinsic value of having a title of “Tester” or being a member of a “Test Group”.

I can answer this question: For me, I’m moving away from the title of Tester. Yes I test, but I do lots of other stuff that isn’t currently classed as testing. Perhaps if the “standard” definition of testing was to change then I might think differently. At the moment, I am a person on a development team therefore I am a Developer! Seriously, I have highlighted some of the stuff I get up to as a Tester previously, but as well that, I now coach other Testers (as well as other roles) & I’m still learning to code (Java)

So the title of Tester does provide some intrinsic motivation for me, but its not the title I strive for – its the role & its responsibilities I look for. My title hasn’t really changed over the past 12 years of my testing career, but my actions, tasks, responsibility & accountability have definitely changed.

Scott makes 3 great points about the use of titles (I agree with them!):

Titles detract from collaborative work
Titles detract from focus on business value
Titles (currently) enhance executive misunderstanding & mirco-management

Scott wrapped up the keynote with his new ideas:

Software System Readiness Assessment: Model

Software System Readiness Assessment Types (download the xls as well!)

Which lead to his…

Software System Assessment Report Card:

Scott Barbers Software System Assessment Report Card


Unfortunately I can’t tell you much more about the models, not because its a secret, more because I can’t remember & Scott didn’t go into much detail.

This is very much Work In Progress & I for one will be following Scott to see what he has to say about it! I’ll be happy to sit through more on what he has to say on this model.


Overall, another fantastic keynote from Lets Test.

Key takeaways for me include:

  • Stuff we do as Testers is not a mission, its a task or an assignment of a higher mission
  • It pays dividends to get some context of that higher mission to help you be more effective
  • Titles do not necessarily provide intrinsic value


UPDATE – Pete Walen has written a series of posts related to the value of testing which supports Scotts keynote