Sunday, June 19, 2016

Lean manufacturing vs Scrum - motion waste (part 4)

This is a continuation of the previous post in this series.

In short, the number one goal in lean manufacturing is eliminating waste. In this series I'm looking at the seven categories of waste identified in lean manufacturing, how these translate to software development, and how scrum can help eliminate that waste. This is the fourth post in the series.

In this post I want to focus on motion. This is one of the seven categories of waste targeted by lean manufacturing.

In manufacturing motion refers to motion of people or machines, but not work product (work product that moves around is transportation). For example, if a person on an assembly line needs to get up and walk 10 steps to get a specific tool every now and then this is motion waste. Motion waste isn't so much about the wasted effort of moving, but more about the wasted time. Because while you're moving to get that tool, you're not adding value to the product. Motion also provides opportunity for work disruption that wouldn't exist if you avoided the motion in the first place.

Similarly, in software development, motion waste is when people are moving around between places where they might add value rather than actually adding value. This could be travelling between your office and a client site, walking from a meeting room to your desk, walking to a colleague's office to ask a question, etc. I also believe virtual motion is potentially a big waste that is often neglected. If you are busy working on an IDE on your computer and you need to remote desktop to a separate server from time to time, this can also be considered motion as the action of switching between environments doesn't add value, it wastes time, and it provides an opportunity for disruption.

How does this tie back to scrum?

Scrum isn't really very prescriptive about motion, but it does put a lot of emphasis on team work and collaboration, a lack of which is often a leading cause of motion waste. It is advisable as far as possible to put a team in the same room to remove physical barriers to teamwork and collaboration and to also put the team's scrum board in the same room.

Scrum also provides a fixed framework of events (scrum planning, scrum review, scrum retrospective, and daily scrum) that helps to eliminate excessive meetings, another major cause of motion. The scrum master's role also becomes very important in eliminating waste. The scrum master is responsible for a lot of things that can directly affect, and therefore minimize, motion waste. The scrum master ensures that scrum events take place, the location and timing of which can have a great impact on motion waste. The scrum master also has responsibility for removing impediments and physical or virtual impediments in the tools and workspace can often necessitate motion.

In summary, scrum can help eliminate wastes that can be classified as motion because:
  1. Scrum events provide a fixed structure that helps eliminate unnecessary meetings that often cause motion.
  2. Teamwork emphasis, which should in practice be extended to putting team members, as far as possible, in the same physical space.
  3. Emphasis on identifying and removing impediments that often cause motion waste.

Thursday, June 9, 2016

Lean manufacturing vs Scrum - inventory waste (part 3)

This is a continuation of the previous post in this series.

In short, the number one goal in lean manufacturing is eliminating waste. In this series I'm looking at the seven categories of waste identified in lean manufacturing, how these translate to software development, and how scrum can help eliminate that waste. This is the third post in the series.

In this post I want to focus on inventory. This is one of the seven categories of waste targeted by lean manufacturing.

It is somewhat counterintuitive, even in manufacturing and especially for those with financial backgrounds, to see inventory as anything but an asset. Yet, inventory can be your greatest enemy. I'm not going to go into the manufacturing theory in detail here, but in summary inventory is a major waste. It basically means that you've tied up resources (including cash), into something that is not being used. Aside from tying up resources, there are also additional costs in handling of inventory from its storage and transportation to simple administrative costs. But perhaps the greatest problem with inventory is that it tends to hide other forms of waste. Inventory creates an illusion that things are well when they're not.

In software development any partially completed work is inventory and should be minimized. Partially completed tasks are mostly problematic because they hide the truth. There are usually complex integration or quality issues hidden in those partially completed tasks that you will never know about until you finally get around to trying to complete them.

To explain this I'll revert back to a manufacturing analogy. Imagine you produce custom furniture and have just won an order for a large batch of chairs. To minimize cutover times on machine setups you produce a large batch of legs, then a large batch of backrests, then a large batch of seats, then you assemble. Come assembly time, you realize that many of the legs are faulty and cannot be fitted onto the seats without some rework. Now you have more machine setup time, more work on the legs, and finally you can start assembly again. Now look at all the waste. Aside from the cost incurred because of effort, storage space, etc. and the waste associated with the rework, the inventory was hiding a quality issue. If you had made four legs, one backrest, one seat, assembled and repeated, you would have discovered the quality issue the moment it started happening, but instead you continued to build up inventory that you thought would be an asset, but quickly turned into a liability.

This is a very simplistic example of something that is usually much more nuanced and complex in manufacturing and even more so in software development, but it highlights how inventory can hide other problems.

Back to the world of software. To make matters worse software projects have traditionally made extensive use of progress reports. Progress reports are great perpetuators of the lies of inventory. People want to look good in progress reports so they overestimate progress. They also tend to vastly underestimate the amount of work remaining on any given task. And progress reports tend to accumulate a false narrative up to a climax where the whole project crashes down. At the start of a project there will be zero tasks with any progress, when you get to the 80% completed stage, this tends to be represented not by 80% of tasks completed, but by a small number of tasks completed and a large number of tasks almost completed (typically with progress percentages in the 75%-95% range). If this sounds familiar, you're not alone.

Consider the problem with the chairs again. If you were asked to estimate progress after the complete batch of legs had been done, you might have broken the work up into 4 tasks (legs, seats, backrests, assembly) and you probably would have said that manufacturing of the components is more effort so will carry more weight. Thus, by the time the legs were done the overall completion estimate might well have been about 30% (100% legs, 0% seats, 0% backrests, 0% assembly). At the start of assembly you would be forgiven for thinking that you were 90% done. But because of the quality issue hiding in inventory, you would have been completely wrong and 50% would probably have been a much more accurate number.

Unfortunately, these kinds of errors only cut one way. There are always hidden quality issues or complexities that you didn't realize would exist, but there are never golden eggs that make your life magically easier. Legs that work so much better than expected that you only need to fit two per chair or legs that assemble themselves simply don't exist. Legs that break or don't fit do exist.

How does this tie back to scrum?

Scrum puts a lot of emphasis on task completion. You must finish a task before starting the next one. A feature must be completed before moving on to the next one and the goal for a sprint is to complete features, not to have them at some point of perceived progress. This is important because it eliminates all the risk associated with work in progress.

In scrum there are no progress estimates and the only measurement of progress is when a feature is completed. Look back to the chair example: If progress was measured by completed number of fully functional chairs (i.e. the only thing of real value to an end user), you would have been sitting at 0% progress when you first started assembly. This sounds harsh, but it would have been a good warning sign of the trouble that lay ahead. If the batch was for a 100 chairs, you would have been at 1% after completing the first fully functional chair and by the time you got to 80%, the remaining 20% would most likely have been a walk in the park.

The definition of done becomes very important here (and there are lots of online resources that delve much deeper into this topic) as it directly affects how we perceive progress. A software feature isn't done when it just needs to be tested, or when it just needs to be integrated with something else, or when it just needs to be deployed. It's done only when there are no justs left. Exactly what the team's exact definition of done is depends on the context of their work, but it is important the definition of done is agreed upfront and is transparent. This definition of done is very central to scrum and an important part of managing software development without getting nasty surprises.

Scrum also puts a lot of focus on delivering shippable product at the end of each sprint. Artefacts like requirements specifications or design documents have no real value to the customer, so delivery is focused on working software. Before you have working software delivered and being used by real-world end users, it's not delivering real value.

To summarize, I think scrum is great at eliminating inventory waste and does so in many ways. Particularly:
  1. Emphasis on task completion and how a task needs to get done before starting the next.
  2. There is no reward or recognition for partially completed tasks.
  3. There are no progress estimates that give a false sense of security.
  4. Delivery of shippable product in each sprint is emphasized.
  5. Working software is emphasized over work artefacts that don't have inherent value.
Scrum isn't prescriptive about everything that could impact inventory waste though, and it is important that the team also keeps an eye on this. I think what is especially important here is the definition of done. Scrum puts a lot of emphasis on this, but ultimately the team has to decide what their definition of done is and if this definition allows any scope for partial work to be considered done, the end result is inventory waste.

Friday, June 3, 2016

Lean manufacturing vs Scrum - transport waste (part 2)

This is a continuation of the previous post in this series.

In short, the number one goal in lean manufacturing is eliminating waste. In this series I'm looking at the seven categories of waste identified in lean manufacturing, how these translate to software development, and how scrum can help eliminate that waste, or not. This is the second post in the series.

In this post I want to focus on transport waste, one of the seven categories of waste targeted by lean manufacturing.

Transport doesn't sound like something that's really relevant in software development, but there are some typical wastes that happen in software development projects that are analogous for various reasons.

Transport waste in software development doesn't happen when physical material is transported between locations, but rather when focus is shifted and tasks are transported. I like to split this into two categories:
  • Task switching
  • Task reassignment
Task switching is similar to transport waste in manufacturing because you're really moving your thought process around when you switch tasks. This doesn't require physical movement, but the mere act of moving between different streams of thought wastes a lot of time.

Software development can be very complex and, especially when done well, requires deep levels of concentration. To exit that state of focused concentration and switch to something else, which in turn most likely requires a similar level of deep concentration, is not trivial. If you don't take the time to focus properly you're likely to make lots of errors and what you produce is likely to cause more problems than it fixes, in other words it will create more waste.

Modern culture has created the false impression that humans can "multi-task", but this is a myth. There are lots of studies in psychology on this topic and it falls somewhat beyond the scope of this post, but the findings show that people do not multi-task, they just task-switch. I'm not going to go into more detail here, but this is nicely summarized in this blog post by Aza Raskin.

The bottom line is, switching between tasks costs time. The time spent switching between tasks will make everything go slower and is waste.

Another form of waste in software development is task reassignment. This is similar to transport waste in that you're essentially transporting work from one person to another, either internally in the team or to new team members that are joining the team. It's almost intuitive (especially for most project managers) when a project is in trouble to throw extra resources at it. But this is fundamentally flawed. Software development complexities mean that ramp-up time for new people joining a project will be big and often prevent the existing team members from effectively continuing with their work. Adding and removing people from a project, or moving tasks between people on the team, creates a special type of waste because of handovers. The goal with these changes is usually to make things go faster, but the result is usually that things actually go slower, at least in the short term.

How does scrum address this?

Let's start with task switching. There is a lot of emphasis in scrum on getting things done and especially on not starting anything if you still have partially completed tasks. Implemented properly by a team, this eliminates task switching waste very effectively. It's therefore important to make sure team members are doing one thing at a time. Daily scrums are especially useful for this purpose to keep this focus.

Ideally, the team should also focus on completing one thing at a time, i.e. nobody should start working on feature two if there are tasks waiting on feature one. Two partially completed features at the end of a sprint will do no good. This may seem counterintuitive, but when a team gets it right it does wonders to bring focus and increase flow.

On to task reassignment. Teams change over time, especially if you're busy with long term projects, and in most cases it would be naive to assume that a team can remain completely static for the duration of a project. Also, if something isn't working on a team you have to try and fix it. In extreme cases that may involve changes to the team structure. But this should be a last resort and should come from within. Scrum puts a lot of emphasis on team autonomy and generally a change shouldn't be made to a team unless the team demands it, however scrum doesn't really dictate when our how this should happen, so the onus is on the team to handle this responsibly.

The iterative nature of scrum does provide an advantage though, in that when task reassignments happen the impact is smaller. Since the project team will always be working on a relatively small subset of overall requirements, the potential damage in task reassignment is somewhat limited. To be clear, it still causes a lot of waste, but the potential size of that waste is much bigger if you're using a waterfall-type approach.

In summary, scrum directly addresses transport waste by encouraging task completion before moving on.

More indirectly and incidental scrum also reduces the potential risk in task reassignment due to its iterative nature.
Where scrum doesn't have a direct impact and the team needs to make sure they apply good common sense and engineering practices is in watching for individual task switching at a micro level (e.g. individual team members should apply discipline and not be distracted too often by things like e-mail) and by making sure task reassignments are only done when absolutely necessary.

Friday, May 20, 2016

Lean manufacturing vs Scrum - intro (part 1)

This is the first post in a blog series in which I intend to explore the relationship between lean manufacturing and software development, and how scrum can, or can not, help eliminate the seven wastes of lean.

We use scrum for software development at Xpedia. If you don't know what that is, check our methodology write-up on the subject or if your attention span is shorter try our slide deck.

Before starting, it is important to note that scrum has a strong aspect of continuous improvement, especially through retrospectives at the end of each sprint. While the retrospectives are an integral part of scrum, the improvement could be totally unrelated to scrum and more often have to do with the team's environment and specific challenges. These continuous improvements should therefore help eliminate waste of all flavours, so in an indirect way one could argue that scrum addresses all categories of waste. Similarly the important role of a scrum master can be argued to have a role to play in eliminating any sort of waste, but again this is rather indirect and probably depends heavily on how good the scrum master is.

For purposes of this series though, I'm going to ignore indirect benefit though these mechanisms and rather focus on the direct waste-elimination benefits of core scrum components as I see it.

You may also be familiar with the happily acronymed lean software development (LSD). There is a relation with scrum, but it's not quite the same thing. The terminology for LSD (yes, I have to call it that now), originated (according to Wikipedia at least) with a 2003 book, Lean Software Development: An Agile Toolkit, by Mary and Tom Poppendieck, who specifically took the principles of lean manufacturing and applied these to software development to come up with LSD. However, scrum predates this by several years and by 2003 was already fairly well established. Of course, the Poppendiecks knew scrum well when they wrote their book and it contains lots of nods to scrum, but it seems their background in various manufacturing environments led them to more directly apply the principles of lean manufacturing to software development in general and not just specifically through scrum. I'm going to probably lean rather heavily on the Poppendiecks' work throughout this series and will definitely also make use of various other sources, but will rather list these in the final post to make sure I include them all. I do recommend this book though, definitely a worthwhile investment of your time if this stuff interests you.

Before LSD, the overlap between scrum and lean manufacturing principles seems to be a bit more coincidental. Various authors did pick up on this, but to my knowledge the Poppendieck's seem to have been the first ones to actually take lean manufacturing principles and apply them directly to software development.

Let's get to the seven wastes now.

Lean manufacturing focuses primarily on eliminating waste and specifically identifies the following 7 types of waste:
  • Transport
  • Inventory
  • Motion
  • Waiting
  • Overproduction
  • Over processing
  • Defects
Others have attempted to add to this list over time, but mostly lean manufacturing focuses on these seven. Proponents of scrum may think this was stolen from scrum, but I assure you lean manufacturing is somewhat older than that! 

In the following posts I'm going to explore the extent to which there is an analogous waste in software development  and how this is minimised by following a good scrum process and also in what areas you'll need to think for yourself and cannot rely on scrum alone.


Wednesday, May 18, 2016

I'm back!

I've been lazy with this blog for too long. Going to start posting like crazy soon. Watch this space!