Scrum is a great framework for building systems, it is simple, elegant and effective. However, it has one limitation that most teams quickly run into: small team size. Invariably, most major initiatives and programs require more than 7, plus or minus 2 people, in order to complete the work in a reasonable amount of time. Or, some organizations face the flip side of this where they run so many projects that the idea of dedicating half a dozen people full time to one initiative is overwhelming. In either case, these organizations are facing the challenge of managing a product pipeline. Let’s take a look at some of the techniques available to manage this within an Agile program. Before going further, I should offer my gratitude to Mike Dwyer and Bob Sarni for sharing their thoughts on this with me. They both brought a lot of expertise that helped me further my own thinking on this topic and this blog is a continuation of that thought process.
Old Models Don’t Hold Up
When looking at organizations trying to manage a portfolio of work with Agile teams, we frequently find one of two common patterns that no longer supports the organization
- Each item run as a project: I’ve written before about the pull some organizations feel to solve all their projects with individual projects. Indeed the very success of prior projects can become their undoing as everything becomes a project, and stakeholders then expect to see all projects start so they can see progress towards their goals. This ends up leading to a mental model where executives may love Agile teams, but they feel that they couldn’t run their entire portfolio using it, because they don’t have 20-30 teams to manage the 20-30 projects they are running at any given time
- Large Project Broken into Silos: some organization undertake the exact opposite approach. Rather than having numerous tiny projects, they run large, monolithic ones that encompass more work than a small group of individuals could ever achieve. In these cases, we frequently see smaller build teams within the overall program, and they are frequently crudely cleaved around a functionality, application, or technology stack. This can introduce many problems, as teams may not be able to individually deliver much work, small sizes of each can introduce lots of variability and the cleaving of work across teams may also mean that the most valuable stuff is not confronted first (sure, we don’t need to build the more for the XYZ component right now, but that’s what the XYZ team is tasked with doing!)
Building Blocks within a Pipeline
Thus the challenge becomes managing a pipeline whereby work can be fed to teams in digestible sizes that are properly prioritized and elaborated. There are a number of different approaches and workflows that follow an idea from conception to prioritization and delivery by a team. Looking at these different flows it occurred to me that this is really just workflow encompassing some combination of a series of transformation steps. At the risk of being accused by my coworkers of “going meta” too much, I figured it would be worth looking each of these steps individually.
A good program manager can use these steps like building blocks to assemble a pipeline process to offer the necessary transformations to keep their project teams happily working on the most important work at a sustainable pace. Indeed my experience has shown that many of the lean concepts like managing work in process, cycle time and value stream apply quite nicely.
Purpose: The prioritized list ranks work according to some measure such as value, time needed, or even a weighted importance across several dimensions.
Where is it used: within Agile teams, prioritized lists are probably one of the most commonly used portfolio management tools. Teams have a backlog of prioritized work to be taken on over time. When looking at a large portfolio, the same concept applies, by prioritizing work, we can figure out what should be sent through the pipe first. This tool should be at the very beginning of a product pipeline.
Common Applications: business value or expected order of delivery are probably the two most common lenses used for prioritizing work, but there are a number of other techniques ranging from the wisdom to of a product owner to dynamic group decision models such as multi-voting or even Innovation Games® such as Buy a Feature.
Purpose: Removes items from the backlog that do not meet a certain criteria (strategic alignment, value / cost, etc.)
Where is it used: frequently used at the beginning of a pipeline to prevent clearly extraneous things from entering. Also used after work is further elaborated or decomposed to remove things that, upon further elaboration, don’t meet a certain criteria or to eliminate parts of a larger item that was just decomposed
Common Applications: The most common applications of a filter would be some sort of ROI or strategic alignment tool where projects or their components must be justified by in some way aligning with goals or meeting an expected ROI threshold. As such, we frequently would see a filter after some sort of elaboration step.
Purpose: Breaks apart large deliverables into smaller ones that are more likely to fit within a team’s backlog or iteration
Where is it used: decompositions are generally used once a piece of work has made it through some level of due diligence and prioritization. The point is that this work is most likely to be undertaken, at least in part. While breaking deliverables into smaller pieces requires analysis work and carrying costs, it helps make estimating more accurate, provides smaller pieces of work that can be fed into teams and allows a better prioritization of features so that some sub-components can be delivered faster and wasteful gold plating can be removed.
Common Applications: Story slicing or the functional decomposition of large epics into user stories are common approaches for breaking apart stories. Story mapping is another incredibly powerful technique here for breaking apart what, at first blush, appeared to be an atomic component.
Elaboration & Sizing
Purpose: I am including these two steps together because my experience has been that they are invariably tied together. Elaborating deliverables is required to estimate how big they are, and the act of estimating invariably ferrets out numerous details of what would need to be delivered.
Where is it used: this is a fundamental step of any pipeline taking advantage of progressive elaboration, where we add more detail as time goes on. Indeed, this concept is even used in most traditional projects where we move from high level business requirements, to functional requirements, to system requirements and finally a design. Likewise, elaboration and sizing may happen several times within a portfolio process where we add “just enough” detail to determine a high level size in order to know if a feature could fit in a 3 month release or not. As it gets closer to delivery within a release, it may need to be further elaborated so that a team can take it or some piece of it into a specific sprint.
Common Applications: There are numerous forms of parametric sizing ranging from planning poker to more abstract ones like t-shirt sizes or, one of my personal favorites, dog sizes – really, you haven’t led a planning session until you hear people argue about whether or not a piece of work is a “standard poodle” or a “great dane”. Regardless, most techniques revolve around a similar concept, discuss the details of a piece of work enough compare it to other similarly sized pieces of work.
Purpose: Many pipelines need a level of evenness of flow. The idea of a grid is to allocate a certain percentage of the pipe to different types of work in order to ensure that one things doesn’t overwhelm the others. This could be allocations of the pipe to different initiatives based on funding, thereby ensuring that each funding sponsor gets “their share” of the development output. It could also be used to implement the lean concept of leveling the work in order to ensure that work flows consistently. Lastly, it may be used to ensure we don’t accumulate technical debt or neglect important parts of our portfolio such as maintaining current systems or technical infrastructure.
Where is it used: Grids are frequently used once work has been broken into smaller pieces and has been prioritized.
Common Applications: It effectively can split one backlog into multiple. For example, if a program allocates 20% of its through put to maintenance work, then the top maintenance items will be pulled from the backlog to fill the 20% allocation, even if they are not top on the list. It can also be used to level work in the case where a team or teams must manage multiple types of work such as stories that involve only UI or stories entirely focused on business logic. Teams may even have specific allocations for technical improvements or refactoring in order to ensure they have some protected allocation of time to eliminate technical debt and run at a sustainable pace.
Putting it Together
I’m finding that most portfolio management processes can be found using combinations of these different building blocks, with each step a transformation along a value stream. Just like when using a task board, I would expect to see the precise flow over time as people mature, but let’s take a look at a couple example to demonstrate the point.
Let’s start with our challenge of too many requests. Traditionally, each request may very well become it’s own project requiring overhead and all the accompanying costs. This model below demonstrates how we would put request through a methodical process to elaborate, prioritize and limit the work that gets to the team. At the end of the process, we have a single consolidated backlog from which an individual team could pull work. The net effect is that the team would have the highest importance work across the entire portfolio, allowing the organization to reconcile the work of a program with a single team.
So here we see a development pipe walking through the following steps
- Requests come in through multiple sources
- Immediately apply a strategic filter and try to remove any requests that are not important
- For those requests with merit, break them down into smaller stories
- Prioritize all the decomposed stories, so that you have an individualized list of priority stories / features across all requests
- Filter out those low value features or stories
- Add detail and estimate the size of the higher priority decomposed stories
- Prioritize those stories based on their net value (benefit – cost)
- Allocate 25% of the pipe’s through put to each of the four initiatives
The key step I’m missing here is feedback, and as my coworker Mike would remind me, we should have feedback along every single step of the way to ensure that what we’re doing is indeed valuable and moving us towards our strategic goals. I’m not sure I’ve figure out how to fit that into a diagram like this, but I’m still working on it. At this point, I’m just trying to articulate a common set of components I may use when helping a client define and manage their program process for managing multiple requests. Like a Kanban system, right now I’m just trying to offer a tool for visualizing the flow, if we can’t see it, we can’t do anything.
I would still consider this a work in process, but I would welcome feedback from others. Are there other critical transformation steps you see missing? Does a diagram like the one I laid out at clarity or make such a process more confusing? I suspect I’ll be writing a bit more about this soon, but I hope you enjoy the idea as I mature it further and thanks again to my colleagues for helping me to develop some of these ideas.