Like many people I am a sucker for good books. The crisp feeling of a new book with its unexplored chapters is irresistible, enhanced by that “new book” and rich coffee smell you can only get in the best bookshops.
So it was that I revisited my old University bookshop in Cape Town and for nostalgia’s sake, purchased a brand new copy of “Software Engineering, a Practitioner’s Approach” by Roger S Pressman. This was prescribed for one of the courses in Computer Science. The book helped me through several years of software development in my own company; until it found its place on a shelf in my study, untouched for months, until today.
In the world of modern software engineering over the past few years we have seen the rise in popularity of the Agile manifesto as a response to the problems associated with cost, productivity, functional fit and quality of modern software. Several Agile practitioners now also advocate Kanban as an approach to transforming development team projects by smoothing the flow of work.
Out of curiosity I checked the index of my 2001 copy of Software Engineering, a Practitioner’s Approach” for the word “Agile”.
“Agile” wasn’t in the index! Nor was “Kanban”!
That surprised me, surely before 2001 people successfully wrote good software and apparently before Agile was part of mainstream software development process? Or perhaps the specific academic text was flawed in that it did not consider Agile worthy of any real attention. (In the 2001 edition at least).
What is Kanban?
As a process engineer I have always been interested in simple approaches to streamlining the flow of materials in a manufacturing process. One such philosophy is Kanban, a method utilised in the motor vehicle assembly lines in Japan (Toyota).
A Kanban system is characterised by the highly visible boards showing work in progress (WIP) in a production line. The visible WIP boards are intended to ensure that inventory is moved only when needed. It is intended to reduce inefficiencies in changeover at the bottlenecks in the process by smoothing flow and minimising work in progress.
Kanban in software development
Kanban is applicable to software development teams too, albeit that it requires some mental abstraction to see the similarities with manufacturing. Yet, you guessed it, my copy of “Software Engineering, a Practitioner’s Approach” also had no mention of Kanban in the index at all.
I could not believe that in 2001 in the middle of the technology boom, neither Agile nor Kanban were mainstream.
Yet, good software was being engineered in 2001. A great deal of good software in fact. And despite the minor technical challenges at Y2K; most enterprise software actually worked well.
Of course we must question whether or not software development has advanced or actually gone backwards in the last decade? And why?
Are the emergence of approaches such as Agile and Kanban in software projects a response to a lack of contemporary software engineering skills? Or more complex development environments? SAAS, cloud, mobile? Or higher expectations / requirements by users, poorly defined processes or a volatile workforce who keep moving between projects leaving chaos behind them?
The answer can be found I believe in understanding the essence of the approaches themselves. By looking at what root problems Kanban seeks to resolve we can get clues as to what is wrong with the modern software industry.
I will work off the principles as explained by David Anderson in his excellent article: The Principles of the Kanban Method.
Principles of Kanban
David describes three foundational principles and then 5 core principles of Kanban in a software environment.
The three foundational principles are:
- Start with what you do now
- Agree to pursue incremental, evolutionary change
- Respect the current process, roles, responsibilities and titles.
Now why would these be important? Is it because our software development environments in practice are subject to radical and sudden changes, that disrupt work processes and existing roles and responsibilities to the point where you break things badly?
I think disruptive change is sadly a reality in many contemporary situations.
I know some bad managers who believe in intentional disruption. Their philosophy is that the “comfort zone” needs to be constantly challenged and broken down in order for people to be most effective.
Yet this philosophy is totally opposite to Kanban, which advocates small incremental improvements; allowing software professionals time to adjust and adapt to their new environments while remaining productive.
David continues with each of the five core principles as follows:
Visualise the workflow
Why is this necessary?
Is it because the measures of what constitutes value creation through the process of writing software by a software developer are poorly aligned to actual value?
For example the number of bugs you fix per day is not adding much value if the bugs are trivial, have no impact and are in infrequently accessed parts of the software. A developer who spends 2 weeks on solving a really difficult bug in a core system function arguably adds as much value as a developer who solves 20 minor bugs in the same two weeks. Measuring bug fixes per day is clearly not a good metric when it comes to understanding real value add.
But remember what you measure you improve. Many companies record their bug lists as some form of “backlog” and developers are rewarded for reducing the size of the list regardless of the magnitude of each bug.
The way we measure developer productivity and value creation in the workflow will drive behaviours.
David argues that we need to map and understand where the real value is created in the development process. Only by getting this right will we be able to identify the true work in progress and see the true bottlenecks in a software development team in order to apply Kanban optimisation techniques.
This is best described by an example.
If the software is full of bugs then the outstanding “backlog” of “urgent” bug fixes will swamp the developers, preventing productive work on new features. I have seen this happen often in practice.
Kanban advocates holding back the work until there is capacity to handle it in development. This “triaging” of bugs is the role of the product specialist / manager and the support consultants. If they are absent the developer is directly faced with the raw and irate customer.
Releasing all bugs in an unprocessed, unfiltered mode into the development team hinders productivity, resulting in rushed processes and more bugs in the final product. In other words, an ever increasing cycle of fixing bugs that never seems to be resolved.
In a physical production scenario, Kanban seeks to achieve a smooth measured flow through the process.
Eli Goldratt in his “Theory of constraints” suggested we set up a drum, rope and buffer to achieve this smooth flow. He was concerned mainly with inefficiencies in discrete manufacturing.
In the software development environment the principle is slightly more abstract but fundamentally the same.
But measuring flow is not straightforward unless we understand the parameters of quality and risk in the finished product.
Agile practices such as daily scrum and weekly reviews can create the equivalent of the “drum beat” and thereby act as a timer for the weekly calendar in a software team.
Not all real world software development situations are managed against a scheduled timetable – both the short term (weekly iterations) and the long term (formal operational releases of software versions) are sometimes left to circumstance. There can be no smooth flow without planning and routine.
Using the “rope” and the “buffer” to regulate flow through a software development process are also possible provided the project co-ordinator fully understands the principles involved.
When our development manager implemented a weekly “Scrum” calendar in our company, productivity in the software team went up significantly. By simply creating predictability in the weekly cycle the “flow” through the development process was being better managed.
Make Processes Policies Explicit
What is measured can be improved.
David recommends that the development processes be formally documented with clear interim deliverables explicitly defined, in order that you can measure interim status (deliverables) in the software development lifecycle.
By accurately measuring these interim deliverables, you can manage them. The absence of a clear policy (documented process) is a sad reality in most small software teams today. This seems to be in contrast to the software engineering approaches before 2001, when extensive process documentation was the order of the day.
A shared understanding of the policies, value, prescribed processes, measures and task status is the basis for collaboration in software teams.
The reality is that in many development teams there is actually no consistency in the understanding of what these parameters are. Therefore team collaboration has no chance of arriving at an optimum outcome.
When people speak different “languages” in terms of the process, instead of aligning to a common objective each developer starts pursuing their own separate agenda.
Without a common language describing the interim deliverables and work in progress a smooth flow cannot be achieved.
So, can Kanban help software development teams become more productive?
In summary, the principles of Kanban, while they were developed for discrete manufacturing processes can definitely be applied in software development teams to improve quality and throughput.
It might be worthwhile asking how Kanban would help in your situation by honestly assessing whether your status quo is aligned or not with the principles above.
Most software teams are under such pressure that they are in a state of delusional harmony; without a clear understanding of workflow, work in progress, process policies and without effective collaboration. Somehow they just manage to deliver code and get by, but are patently unable to increase throughput no matter how much pizza and coffee is provided.
The problem comes when you try to ramp up capacity (throughput), even by a small amount. Then the team harmony dissolves and the team productivity goes way down.
And this is probably what is happening in our software industry today. The reality is that the customers will continue to raise their expectations and demand more and more from fewer resources.
If your development team is showing signs that it is at maximum capacity and unable to scale up to an increased demand then it will be a good idea to spend time applying Kanban principles in the context of your own team. These after all are the same techniques that made the Japanese motor manufacturers world leaders.