User stories is one of the most searched for terms on our website, so we decided to ask our 27+ year veteran developer, manager, and consultant David Wylie what makes a good user story, who writes them, and when. We also talk about acceptance criteria, story splitting and what a user story is not. Our interview with David also became one of our most downloaded podcasts; below is the full transcript.
Howard: Welcome to another in depth edition of Agile Amped. I’m your host, Howard Sublett. Today, my guest is David Wiley. David oversees the delivery efforts of SolutionsIQ’s Agile software development teams. Using Agile management, he collaborates with customers on prioritization, successful delivery, and ongoing enhancements to software and applications systems. He has worked as a software developer, a tester, analyst, project tester, and consultant during his 27+ year career. In this episode, David and I discuss user stories, what they are, why they exist, who writes them and when. He makes it clear that user stories are simple yet hard, and are placeholders for conversations. He gives great advice to people that may be overthinking this sometimes misunderstood concept. Again, thank you for joining us. Now, on to the conversation. David, thank you so much for taking time with me to record today for this session. I sure appreciate you, buddy.
David: Happy to do it, Howard.
Howard: I asked you to join and talk about user stories mainly because it’s written in the top three or four things that people search our glossary on at SolutionsIQ, and they’re interested in this. I thought it might be a little bit interesting to spend a little bit more time and deep dive on the topic of user stories. Can you kick us off in what, basically, what is a user story?
David: Sure. It’s a placeholder for a conversation. It’s a small statement of value or a feature that people want. The earliest uses of this came out of the extreme programming community where they had these literally index cards where they would jot down a few words. That would remind them of a conversation they had had with their onsite business user. As it’s grown and morphed over time, it’s really a very popular component of all the Agile practices now because they’re a very efficient way of communicating what needs to be done by the system.
Howard: I believe you. Why do you think that that is as opposed to the alternative way that we used to do this?
David: I used to write those large requirements documents in spec. We would spend months or years writing those documents. What we found out was that no matter how well we interviewed users, by the time we implemented it, they either had changed their minds about what they wanted or had forgot to tell us about stuff that they wanted. The chaos study shows us that systems over produce features like crazy. That’s partially because of the way we’ve documented requirements. Then we also started to do these horrible things like trying to stop people from changing. We had these change review boards that had to sign off on any change once a requirement was baselined. It was just not fitting in the world, even 20 years ago. It certainly doesn’t fit in our world today.
Howard: These are not e-specs. They’re not giant documents. When we talk about a user story, they’re really simple. They fit on an index card usually, don’t they?
David: Right. That’s where they came from, index cards. People literally had the requirements for a project in a shoebox in the team area. Currently, everybody uses PostIt notes. We love the concept of having the small form factor because it enforces brevity. You have to be very economical with how you state who you want to provide the benefit to, what the feature is, and how they’re going to get benefit from using it. This small form factor’s really good. It supports hosting on walls and windows. By making it visible to people, it really supports great conversations by everybody that’s involved. Every time you write a document, it tends to force people to look at a screen and having one typer. Also, you just tend not to get involved. People just don’t read the big documents, much to the frustration of the great business analysts in the world.
Howard: These start out with the word user.
David: Sure, and we, in systems, sometimes forget that the work we are doing is to benefit some user, user of the system. When we start to think about that, we do a better job of showing empathy for the system users about who’s going to use the system, what role they’re in, and therefore what kind of features we want to create for them. Users are the first ones. There’s a lot of energy around who are those users. Are there personas and persona mapping? Are there just generic users that we can understand? It’s really a way of conveying who’s going to use the features that we’re developing. Therefore, it allows the development team to be human and show their empathy for the user, and therefore create the appropriate solution.
Howard: Right, so there are several templates that I’ve seen. People have different varying degrees on it. I’m curious your take over the templates that you’ve seen and what you like, how the original one was, “As a blank, I want to whatever goal desired, so that …” Then I think Cohn came in and said that the “so that” was an optional statement within that. Then I’ve seen a lot of, “In order to receive certain benefits as a user, I want a certain goal desired,” that doesn’t put the user first. There’s several different myriad ways of writing user stories with the same desired outcome. Do you have a particular format or template that you begin with when you’re working with teams?
David: Yeah, the basic concept that even predated Mike, and by the way, Mike Cohn, for anybody that’s listening, is a seminal writer in this field and has great content and highly, highly recommend him. The basic format of “As a role, I want a feature so that I can get a benefit,” is very powerful and is great for people that are new to this. It gives them a structure that they can support and understand and know how to fill out. It’s a great starting point. In the world, it’s definitely a Shu practice. It helps us understand what the role is and what the feature is that they’re asking for. The “so that”, I know that Mike says that the “so that” is an optional thing. We, on our teams, find the “so that” very powerful because, especially if you’re working on an enhancement to a current system or you’re not doing groundbreaking stuff, the “so that” can sometimes be met by the current system by just configurations of a system or other features that may not be known to the person making the request. The power of that is that we can meet the benefit without writing any new software. Of course, no code means no bugs. We love to hear that benefit. The other thing that’s impactful for that benefit statement, it helps align the developers to why they’re doing the work. When they understand the benefit of what they’re doing, they’re much more empowered and interested in doing the delivery. I’ve had multiple teams that say, “Will you tell me? I don’t know why we’re doing this. This is stupid. Why are we doing it?’ When they understand the benefit, then they can get behind it. It makes the people that are capturing those feature requests work a little harder. It’s a basic gray template. That said, I’ve seen some people go overboard with it. They really start to stand on their head to make this template work. Then they start getting into odd behaviors that really don’t make sense, which I can talk about a little bit. I’m also reminded by some of my XP friends, my extreme programming friends, that two or three words on a card is often enough, too. If you have a very close relationship with your product owner or your business representative, the two words may be enough for us. I know what to do. We’re going to add date to report. I know which report it is. I know what date you want because we talked about it and I have context. I know the system. I know your world. Therefore, it’s enough for me to go and do the work. You really need to find what works for you in your context.
Howard: It’s interesting. The way you said something a little bit ago hit me and it resonated, I think, in a way it never had. If you take the “as a user”, that part of it, and skip the “I want X” and keep the “so that”, there’s the two bookends of that are the things that actually give meaning and purpose to the work being done. The ask is the thing in the middle, which actually is the thing we tend to focus on a lot when we’re talking about a new feature, but for the people actually doing the work, who is this going to impact and what’s the benefit that they’re going to get from this is the most important thing in the entire user story.
David: I couldn’t agree more. It’s very powerful. In fact, we often get frustrated with our business people. They know they want a benefit. We get frustrated that they can’t tell us exactly what feature they want. When we get the full understanding of what the context of the user, the role is, and what the benefits are, we can have an adult conversation about how we can deliver that benefit.
Howard: How do you answer the question to the person that’s writing their very first user story about I’ve got this monster idea, say and epic idea. How do I go about writing this first one, and how do I break it? How do I split it? How do you begin?
David: Great question, and the trite answer is: Begin. Write it down. When you have the idea, whether you’re interviewing one of your stakeholders or you’re driving in the car or you’re walking the dog and you have this idea, write it down. Again, it’s going to bring us back to what do we want to do? What are the ideas? One of the basic things in scrum is that anybody can add items to the product backlog. Anybody can write stories. Whether you’re a developer or tester or a stakeholder or the product owner, anybody can write these things down. It’s important to write them. The first recommendation I’d have to people is don’t worry about if they’re too big or too broad or not well understood. We can get to that. One of the basic pieces of advice is don’t try and split it too early. If it’s big, let’s talk about it. Let’s see what it is because the next phase of, once we gather these requirements, is we’re going to prioritize them. If a bit requirement is out there, but it’s lower priority, let’s not invest time in fully understanding it yet. Let’s deal with the higher priority stuff and break those things down before we get to these lower priority. A big, low priority item should stay big. That’s a great thing because we’ve minimized our investment. If we decide not to do it or we decide to delay it, the impact on the investment is trivial. If they’re big, that’s fine. Don’t worry about them. There’s some words that we’ve started to use. They’re not uniformly used about what is bigger than a story. We’ve come across the concept of epics or themes. Again, I’m going to rely on what Mike Cohn has said about epics is that epics are a big request that are probably on what a team can deliver in a time box, in the iteration. There’s a scale definition of what an epic is. It could be that an epic actually is composed of other epics, and so you need to break these things down. When you start to break them down into smaller chunks, into these user stories that a team can start to consume, you may want to talk about a collection of user stories. That’s where we come up with a theme concept. We can say, “The delivery of these five stories are part of the theme of doing checkout from your shopping cart, and that we know by delivery of these things what we’re going to start deliver theirs.” Most users, especially at the higher levels of executives, tend to think and communicate in terms of epics. They have very broad brush wishes of what they want to get done. Often projects are funded related to, “Here’s three epics I want you guys to deliver.” They don’t talk about them in terms of epics, but that’s what they are for the teams.
Howard: It’s weird how you brought up the term “epic” because I’ve worked with teams that they end up in thrash and almost in-fight over the terminology used on what you call the really big, gnarly story. That’s so not important …
Howard: … in the grand scheme of things. It’s just a story on a larger scale and the smaller ones almost because little fractals of it or little divisions of it, in a way, but they’re all stories. They may not be consumable within a . Sometimes it’s helpful for people to call them different things. It seems like every client right now wants to call them something different just so they can identify big themes, big seasons of their product cycle, whatever they want to call those.
David: Yeah, and do whatever works so that you can then go through and prioritize. Then, because at its base, a user story is a means of communication and we’ve got to remember, and we often forgot, I often forgot, as when I wrote requirements documents, the requirements document has no value in and of itself. All it is, is a means of communicating. If, when we need to have these stories at the right level, that we’re communicating from some person who’s sponsoring us, the business person or a senior architect, or someone who wants some value, we need to communicate that to the people that will be doing the work. As long as that communication occurs, then we’re at the right level.
Howard: Right. You mentioned a little earlier about who writes a user story. It’s not just the product owner. Anybody can write them. In your experience, do they ever sit down with developers? Do developers, testers, do POs, do maybe stakeholders, do people write them collaboratively? When within a product cycle is the time to write a story? I see times where they go, “This is the session where we write user stories,” and others are more fluid and they write them just in time.
David: That’s a great question, Howard, because there is actually multiple times when you write a story. When you’re first getting started, you might want to populate a user backlog by going through something that we call a User Story Workshop. That’s where we get a lot of stakeholders together and some representatives, if not all the technical team as well. We talk about what is required from the system. What are the big epics that we’re trying to deal with? What is the timing? What are the priorities? We can populate the backlog using that way. We can write a bunch of stories. We can write the epics. We can break the high level ones down so that the teams can get started. Then within, especially within Scrum, there’s a couple of options, opportunities, within our Scrum events to write more stories. The classic one that people sometimes forget is during the Sprint Review when we’ve shown the product owners and other stakeholders what we’ve gotten done, that often drives conversation that says, “Now that I see what you guys have done, I want something else. I want something new that we need to deal with,” or the team will get feedback that’s like, “Well, that was great you guys did this, but you forgot to do this piece of it.” The team may originate a story during that time. That’s a second, so a User Story Workshop then, during Sprint Review, where you’re starting to look at what backlog’s coming next, part of Sprint Planning, but that Sprint Review really causes people’s juices to get flowing about what else they may need. Then the third one that comes up is what we call Backlog Refinement. We used to call it Backlog Grooming. That’s an event that happens during the iteration, during the Sprint, where the product owner usually runs that meeting but they say, “Hey, here’s what’s coming up in the backlog. Let’s talk about it. Do you guys understand it enough? Do we have enough detail? Do we have enough acceptance criteria? Can you guys estimate it?” During that session, the team may come up and say, “Hey, this is too big. Let’s split the story. Let’s create two new stories. There’s a new story created or, “Now that you’re saying this, it sounds like we may need something else because we …” One of the challenges or one of the criticisms of this approach is that we miss the bigger narrative. We start to deliver incrementally, so we’re delivering all these little pieces and we’re missing the forest. We’re delivering the trees, but we’re missing the forest, right?
David: We want to be able to have a flow and an understanding of these. That’s another. We need to understand and fill in those blanks. There’s a time, once the stories are generated initially, there’s a couple techniques that people use. One is user story maps or user journeys. This is where you can start to put how does a user access this system from beginning to end and what is the flow and what are the alternate flows and what are the level of detail that we might need to deliver on these. That’ll also help us understand what stories are missing on what we have next. It’s also an amazing and excellent time, using this technique, to look at what’s our first deliverable. Can we get the MVP, the minimum viable product? What of these stories could we use? Having these stories in small chunks on sticky notes allows us to have this conversation, a very fluid conversation, early in the process. It’s looking for opportunities to ship early so we can deliver value and, even more important for the team, is opportunities to get the feedback quickly. The user story map is another way to understand what’s there and what might be missing.
Howard: Right. You mentioned something about splitting in there. I know that that’s a really struggle for most teams and most Product Owners trying to figure out how do I divide this story. How do I get it to the right size? What is the right size? How do I know when it’s small enough, granular enough? Do I go smaller? There’s several techniques that I know you’ve worked with that help Product Owners and teams know how to split different types of stories.
David: Yeah, absolutely. It’s interesting. It is a normal challenge for people that are new to the process. We need to understand what those, what to do with those. There’s no way that we can get work done in two weeks. Our work is too hard and too big. There’s some techniques around how we can break it down because what we’re looking for, ideally, is to deliver value, but also to look for opportunities to give feedback. Are we on the right track? Again, I’ll got back to Mike Cohn. He’s got a little acronym called SPIDR, S-P-I-D-R, of ways to split. Spikes, the S is for spikes, being able to learn and understand more about this so you can maybe make it smaller. P is for paths, paths of the user through the system. I like to look at the happy path. If we forget all the edge cases, could I just deliver a happy path quicker in something less than the full story is currently understood. There’s a way to look at interfaces. For example, is this a system interface? Is it a mobile? Does it have a mobile front end? Does it have a web front end? Could we deliver one of those instead of all of them? Another one is data. There are subsets of data. Could we assume that some data exists? Could we mock out some data and allow us to deliver something that we can at least get feedback on? The last is rules. He says if we can relax some of the rules, again, this is like going through the happy path, could we deliver without all the rules that somebody has envisioned and still get some feedback? SPIDR is a way to do that. Again, the ones that are hard to split, some of them, some stories, are complex and some stories are compound. The compound stories are relatively easy. When you start to see it, and for example you’ll see I need this and this and this and this or this comma, this comma, this comma. Look for ands and commas in your request.
Howard: Great tip.
David: That’ll give you an idea of where to split. Then I almost feel bad about saying this, but some stories are just not splittable. Please, as Mike Cohn says, feel guilty if you get to that point. Don’t stand on your head to split. If you need a story that takes two sprints to deliver, just go do it. Just go do it, but feel bad about it because you tried to do it.
Howard: There must be a Scrum confessional booth that you go to and you say your Hail Marys and …
David: That’s right. Split them if you can. There’s lots of ways. Talk to more experienced people. Look for simple paths through the system. It’s one of those personal challenges. I’ve never met a story I couldn’t split. If you have a story you can’t split, give me a call. I would love to help you out. I’d love to help you try and split that story.
Howard: Wow, throw down the gauntlet there.
David: That’s right.
Howard: You’ve got a really great technical background. What do you do about technical stories, because I know that they’re … I’ve seen that before, when people think that there is no user. There’s no person in this user story, so it’s, “As a mainframe, I …” How do you advise people on what they call technical stories?
David: Well, in the one that, frankly, I see some people in our Agile community try and wave their hand at it and says, “Well, that’s just because you don’t understand user.” I’ve been on multiple projects that are funded and driven by an architect or a dev manager. They’re trying to move from old hardware to new hardware or they’re trying to go from a complex database to a simple database. They’re trying to move from database X to database Y. Those are very definitely technical stories. The benefit is we keep current functionality available. Hopefully we’re going to make it there’s technical reasons to do things, security, performance, architectural flexibility, that you need to do things. It depends on how you’re breaking up your teams, also. If you have a team that just does services and you have a team that just does front end, they’re going to have dependent stories on each other.
David: You’re going to say, “As a front-end user, I need a service that tells me the location of the customer so that I can provide adequate deals for his location.” They’re going to say that’s a request. Think hard about who the user might be. That helps, but sometimes you’re just doing something for some other team. That’s just part of the deal. Remember, we’re trying to convey requirement of change to the people that are making the change. I’m okay with that. I don’t want to do it a lot, but sometimes it’s project dependent. Don’t get lazy and say, “Well, as a developer, I need.” There’s a rare case where you as a developer can say, “I need the system to do something for me.” Even that has exceptions. You could say as the developer, “I need log files so I can debug problems when they occur.” Those kinds of things do happen as well. Most of the time when we’re getting our funding for changes from somebody and those people are the people we’re trying to provide benefit to.
Howard: See if I can broach a bigger topic there. There’s usually a backside of those cards. Some teams, they use a backside to write things like acceptance criteria. Can you give us a 1,000 foot overview of what’s an acceptance criteria and how does it relate to a user story?
David: Sure. We go back to Ron Jeffries who came up with the great mnemonic, the three Cs. It’s a card, it’s a placeholder for a conversation, and then there’s confirmation. How do I know when I’m done? So card, conversation, confirmation. The confirmation criteria is sometimes known as acceptance criteria. In the XP days, we would just flip the card over and write three or four or five things down. By the way, there’s a quick relationship back to splitting, that when you have more than five or six acceptance criteria, you might need to split that story. That’s another. They fit there too. Acceptance criteria, some people say, one of our trainers says, “That’s where requirements went to live in Agile.” It’s the details. I need this to be blue. I need a dropdown list. I want you to pull it from this table and edit for these criteria. They’re the details that says, “If you will do these things, then we’ll call this done.” Again, we call them ACs, acceptance criteria. Mike Cohn’s saying he likes conditions of satisfaction, essentially saying, “If you’ll do these things, then I’m going to accept. If you don’t meet these conditions, I’m going to reject the story. It’s a way for the details to come out. Again, you want them fairly straightforward. You want them understandable. If you’ve got 17 acceptance criteria, it’s probably too big of a story.
Howard: Absolutely. I know that for you, the cards are simple. The user story is something simple. It’s not a 100 page e-specs. It’s not all of the details. You really can’t rely 100% on that particular document in order to know what to build. It’s a placeholder. Any advice for people out there that may be moving from one position of writing those 20 to 30 to 40 page e-specs that are writing their very first ones and how to navigate that?
David: Great question. The first is get started. As with any kinds of communication, remember your audience. If you’re helping, if you’re a proxy Product Owner, a BA, that’s talking to people and you’re helping a product owner define what needs to be done, start small. Put some words down and then talk to the people who are going to consume it, the developer, the testers, and operations people that are going to consume this thing and say, “Is this enough information? What other details do you need?” Sometimes we would think that we knew better, that they’re going to need this information whether they know it or not. That’s just not the case. In the old days where we would do detailed design specs, where we would not trust our developers and testers to do anything, and then we’d have to specify to the Nth degree what to be done and we really disempowered our developers and testers and operations people from doing, being smart people. I like to say, “You’re hiring smart people. Let them be smart people. If they’re not smart people, fire them.” Get smart people, let them do their work, and give them that ability to do so. Get started, remember your audience, keep it small. Now, there is time, Howard, where some people in some environments say, “Well, our requirements document is required. We have to do this given our current business, our current regulatory environment. We have to create a requirements document.” That’s fine. Then one of our deliverables as a team should be the requirements document. There’s some great advice that says keep them still fairly straightforward, even those documents. Start with the basic themes, the high level epics, if you will. Think about how a user role is going to use those themes, and then break down the stories that are going to be done to implement the themes for that role. They’re very transitional between a document and stories in that way. The advice that we have is try and avoid use of actual acceptance criteria, the 188.8.131.52. don’t go down too low on those because, honestly, the consumers of those documents when you meet a requirement, an auditor, somebody that’s not on the development team, they don’t need the acceptance criteria. They just need the bigger picture. What are you building? Who’s going to use it? Try and avoid the detail on those things. Again, if that’s a requirement and it delivers value for you, then the team should help you create it, also.
Howard: Yeah, I’ve seen that as part of a definition of done on a team where they had a heavy regulatory compliance that that’s just part of the code check in that they have to update certain documents for that.
David: Absolutely. There’s another one is how low a level do we need to go. I had a customer that had a subject matter expert on their team. She was very experienced. Her role was in QA. She was used to seeing this level of detail, that Nth level of detail. For the first six months that they did Scrum, they would do story grooming for three or four hours every day because they had to get this level of detail. Otherwise she was not satisfied bringing those stories into the Sprint. It took a while to build up trust with her that she was working with smart people and they were going to deliver everything that was required to be functional and be verifiable. It took them six months to get her to the point where we don’t need all that detail on every story. The other trap that I see people falling into is that they go down, that they confuse a story and a task. A great rule of thumb I’ve seen is that a story is something that’s big enough that needs to be done by more than one person. It’s got value and when I deliver it, it delivers the thing but it’s more than one person doing something for a couple of days, a couple of hours. A task is one person or one pair working on something. It’s a small chunk of work. It’s build a code, change the database, write the test, go to a meeting. They’re tasks related to that delivery of stories. Don’t confuse they’re story and a task would be the other advice.
Howard: Absolutely, and keep it about a conversation. It’s a placeholder for a conversation. Don’t rely solely on that card to transfer all of the knowledge that’s in your head and what you think you want about the product to the people that are going to be working on it.
Howard: It’s got to be a placeholder for that conversation. David, thank you for taking some time with me today. This was a lot of fun. I actually, I learned some things about user stories. I didn’t think I needed to learn something, but I did.
David: Howard, the more I learn the less I know. You have to be humble in this profession because there’s so much that you can learn and be better at. It’s crazy how much I’ve learned in the last five years with working with amazing people.
David: My last two bits on user stories, remember that nobody has perfect knowledge. When we write these things, the beauty of writing them on sticky notes is you can wad them up and throw them away. When we change our mind or we change the priority, it’s cheap to change. That’s really what Agile’s about is being responsive to change. User stories are just a great way to get there.
Howard: Absolutely. All right, buddy. Thanks again.
David: Thanks Howard.
Howard: Thanks again for listening to this in-depth edition of Agile Amped. If you enjoyed this episode with David and myself, we’d appreciate you taking just a few minutes to add a rating and a review. Also, don’t forget to subscribe at iTunes or SoundCloud because you don’t know what might be next.