Metrics. How numbers facilitate communication.
Everyone wants to be happy. But can you measure happiness? Is is possible to assert how much happier you are on a particular day? Have you heard anyone say “today I’m 10% happier than a week ago”? I doubt that. But even if it was the case, would it make any sense? After all, no one expects a mathematical approach to feelings. This is different, however, with software.
Expert intuition
Success can be measured with numbers. And all those who think otherwise are just afraid of the truth. ~ Jo Nesbo
This is where we also encounter an approach based on gut feeling instead of numbers. Still, it would seem that referring to numbers would be the natural way to communicate in the world of technology. My trainings on system architecture often begin with the question: what do you mean by “good architecture”? What makes it good? Usually, the answer depends on the profile of a particular company. Some features are enumerated more frequently than others, so we may assume that an architecture should be:
- maintainable,
- flexible,
- tailored,
- testable,
- secure.
Now that we compiled the list, I’m asking to take a piece of paper and write down the definition of one of the above, e.g. testability. Then, confusion sets in as each person reads out their definition:
“Testability means the possibility to verify each component of the system”
“Testability is achieved when the duration of tests is short”
“Testability is understood as a high percentage of automated test coverage”
What happened? A second ago everything was agreed and now some discrepancies arise. We’re lucky that it happened during training in controlled conditions and not when everyone got back to work. If this was the case, one team would add tests to increase the coverage and the other would remove them so that compilation would take as little time as before. This is a recipe for conflict. The simple solution is to ensure equal levels of awareness.
The same levels of awareness
We can use numbers to make sure expectations are uniform. Agreeing on the scale applied in the case of a particular feature promotes a common understanding of such feature. Let’s analyse the following examples:
“Time in minutes necessary to perform automated tests in continuous integration environment”
“Percentage of critical paths with end2end test coverage”
“Number of functional blocking errors discovered in production environment”
“Percentage of business processes which can undergo acceptance testing without the use of GUI”
Needless to say, these scales significantly differ from one another. It is difficult to come up with a single ultimate definition of a particular feature. It always depends on the technology (it will be different for web applications and embedded systems), the stage of the project (it is just starting off or it has been developed for 12 years) and the client’s expectations (safe to fail or fail-safe). It is important, though, that the understanding of a particular idea is uniform within a group.
Metrics
Now that we established the scale, we can move forward and define metrics. We usually define three values, i.e. current, goal and wish. Time metrics for performing tests may be, for instance, the following:
- current value = 8 minutes
- goal = 6 minutes
- wish = 4 minutes
Measuring the changes in metrics, we can see where our project is headed and whether we managed to achieve our goals. This means we reached the point at which we can easily say that, according to our definition, “following recent changes, the application is 21% more testable”. It’s not “gut feeling” but numbers that are indicative of correct or incorrect completion of tasks. This translates into an increased transparency for the business (that is our clients). Instead of saying “you’re always tinkering and refactoring”, they say “your work now yields quantifiable results”.