Posted By Duncs ~ 1st November 2015
This is the 2nd post in my series on Truth. It builds on the first post by hopefully demonstrating how my opinions on truth & “Living Documentation” have changed after some research into what is truth.
This research was done in preparation for the fabulous MEWT conference. Here is a link for the slides of my “Single Source Of Truth Is A Lie” talk – this post aims to follow the structure of that talk.
There can be no such thing as a “single source of truth” as truth comes in many forms. At one abstraction, there can be considered to be 3 truths; Physical (the product), Psychological (our beliefs) & Social (our groups).
As I see it, the aim of collaborative requirements gathering techniques (such as the 3 Amigo’s) is to discover what people believe as true themselves (psychological) & within their groups (social) in order to create a set of requirements or needs (physical) in whatever form that artefact takes.
Even with these collaborative techniques, there is very little chance that we could elicit all the “truths” from the stakeholders in that meeting, largely because there will be truths that they are not even aware of yet.
Therefore, the closest we will get to a (physical) single source of truth will be a very shallow representation or model of the actual product we are trying to develop.
In the talk, I set up the idea that we had an idea of the product should look like (yeah, yeah, I leaped to the solution space):
Through collaborative requirements gathering (e.g. 3 Amigo’s) we hope to create a representative model of the desired product to serve as a “single source of truth”:
I then set about tearing down that misconception through taking a dive into the definition of “truth” & what it is comprised of.
The majority of definitions of truth are pretty weak, so I avoided most of them until I came across one I could actually work with:
“conformity with fact or reality; verity” (dictionary.com)
(& a definition of “verity” )
So if you break this definition out, it could read:
Truth is compliance with a thing that is known
Yeah, I thought that as well.
Instead of trying to work out a definition of truth I thought I would dig deeper into what truths are.
It was here that I mapped James Bach’s Heuristic Test Strategy Model to the 3 types of truth: Physical (Product), Psychological (Quality Criteria) & Social (Project):
The Explicit & Tacit elements were called to me by the mighty Vernon Richards (thanks Vern!). I’ve not pulled on that thread much more yet. (EDIT I have since had a fascinating chat with Thomas Ponnet regarding my categorisation of quality criteria & project environment being solely in tacit knowledge – more on that in a later post)
I’ve bundled the Social & Psychological truths into a human bucket to help my argument later…
Then I outlined the tests of truth. Go check them out. I’ll wait…
To me, these tests of truth are heuristic, in that they are fallible methods for solving a problem and interestingly each of these tests could be considered an oracle.
This didn’t occur to me until open season of the talk, when one of the MEWT attendees kindly pointed this out. The idea was also abstracted to
“as Testers, we shouldn’t be reliant on a single oracle”
When the idea that oracles as (fallible) sources of truth was played back to me – a fact I was fully aware of – it really helped break the twisted model I had of a “single source of truth” which was just confusing me.
After the tests of truth, I put forward my argument that in the 3 Amigos session we are aiming to create an explicit physical truth from tacit human truths. This process is flawed from the outset because we as humans are flawed & we cannot (or do not) make our tacit truths explicit enough that we all share the same truths.
The result is that we end up with a model that on the surface resembles the truth, but in no way resembles the actual product that was desired.
The automation framework around the product serving as the “living documentation” or “executable specifications” can only be the shallowest representations of the product itself and as such cannot be considered to be a source of truth to be reliant upon.
Yes it’s red, has 4 wheels. 2 doors, a spoiler & a Ferrari badge – but can I get in it & drive it? Possibly not…