The product owner and the product-shaped hole

What the product owner needs to worry about isn’t in the product backlog

The XP customer is always right

I’ve been involved with agile development since my first XP project in 2000 where a dream team supported by Kent Beck helped get XP going in a San Francisco startup. I started at the company as a member of the XP customer team. I struggled with that role – felt lost. I had deep subject matter expertise, so I guess I belonged there, but the lack of clear and compelling vision for the product drove me crazy. I left work every day ready to scream.

In my past jobs I had been a product manager – been able to declare what the product should be. I was a prima donna, and the company I worked for supported me well since my products were successful. In this XP project I was part of a collaborative team of prima donnas. If that sounds like an oxymoron, it’s because it is. The process for customer seemed to be: Prima donnas fight, alpha prima donna wins, others shut up, then customer(s), speaking with one voice, tell the developers what to build.

My feelings were mixed. I loved XP – loved the productivity and energy I saw in this approach, but felt the customer side of things was a mess. The team of customers, although they had deep subject matter expertise and experience, had no clear vision or discipline in their approach to figuring out what to build. In truth, as a customer on that team, neither did I. I completely sucked. I saw that the alpha prima donna process approach didn’t work. There had to be something better.

In my past life I was also an engineer and UI designer. (note to readers, engineer with graphic arts background = UI designer.) I hated that XP didn’t let me be everything – subject matter expert, product manager, UI designer, and engineer. Happily for me, just before leaving frustrated, Rob – the XP coach, and one of the smartest people I’ve ever met, let me change my role to developer. I loved being a developer on the team. I loved the new skills I was learning, the increased discipline, and rigor I’d not known to apply to engineering before. I’d been using the loudest prima donna approach there as well – mostly.

I happily served my time on this XP project as a developer. Although I felt the product concept and release strategy was flawed, it wasn’t my problem, the hole was in their side of the boat.

That boat eventually sunk – or more accurately took on lots of water and limped to shore in Greenland with most employees thrown overboard, and the last remaining employees stock options in ruin.

I learned some fabulous lessons, and was left with a few problems to solve.

1. Keep doing agile. XP and Agile methods brought a level of rigor, expertise, and predictability to software development that I’d never seen before – that along with a more satisfying collaborative environment for the engineers at least. Embedded in XP-style engineering was a culture of craft, of continuously striving to increase skill.
2. The XP customer role needed help. I now understand that role to be the most critical – like the captain of a ship. That captain can drive it into an iceberg if he wants – happily smiling as it takes on water. The team does what he asks, sometimes oblivious to the fact that they’re in the boat too. After all, the customer is always right.
3. Find effective agile customer practice. I must figure out something to put the same level of rigor into that customer practice as I saw in XP engineering practice – at least if I was going to stay in an agile world.

Glad I got all that off my chest. Thanks for listening.

It’s the product, stupid

For those who didn’t know, there’s been a slow moving and quiet agile process brand war. Scrum has overtaken XP as the dominant approach. And much like Christianity absorbed Pagan holidays to help it spread through Europe, Scrum has relaxed its one calendar month sprint rule I’d originally learned for shorter XP-style 2-week sprints, and adopted XP’s user stories to fill its backlog. Scrum trainers have begun to gather and teach some of the best practices originally found in many other agile approaches at the beginning of this decade. Luminaries from other approaches (like Alistair Cockburn and Ron Jeffries) are joining everyone under the ever widening Scrumbrella. Me too. I want to be like Mike.

One of the concepts I particularly like in Scrum is the product owner role.

The role name “Customer” has always troubled me. “Customer” is the relationship between people or groups. To the engineers, people asking for software to be built are “customers.” To a company who builds a product, the people who buy the software are their customers. And for many products, the people who buy the product aren’t actually the people who use it.

In software development there are people who make decisions about what the product should do while taking into account what people who make buying decisions want, what people who use the product what, what stakeholders who pay for the software to be built want, and what others who maintain and support the product want. Sound tough? It is.

But it’s all about the product, and the very tough decisions someone needs to make – decisions rife with uncomfortable tradeoffs, risk, and uncertainty. By using the term product owner Scrum acknowledges that there is a product here, a piece of software, and someone with the tough job of “owning” decisions about that product. It’s a more accurate than customer.

Product owners learn lots of agile process but scant product design and product management practice

I often quote the simple distillation my friend Alistair made when talking about requirements.

Product owners have a lot of design decisions to make. They’re responsible for the success of the product. It seems appropriate that we leverage valuable design practice that’s already out there.

It’s unfortunate that much of the training and support available for those in the product ownership role doesn’t teach much in the way of design practice. It’s getting a bit better, but it has a ways to go. Until then, what passes as product owner indoctrination is lots of instruction on working inside a Scrum and agile process. That’s important – but it leaves product owners with the same problem I first encountered in 2000 with XP.

We need a hero?

I find lots of product owners leading with pure intuition. Not that intuition doesn’t work – it’s just unpredictable. It works fabulously well for a select few with strong experience in and understanding of their domain, their market, and a sixth sense about what that market needs.

The ideal Scrum product owner or agile customer is often described as super-human. They have a strong understanding of their customer, their users, deep domain expertise, and the time and willingness to work directly with the development team and write user stories that describe the vision of the product they hold in their mind. They can motivate others to see that vision, and to work hard to help achieve it.

I don’t know about you, but I’m imagining someone wearing spandex and a cape.

Those in the engineering and testing team aren’t expected to have such super powers. Instead they rely on learning lots of new skills. Armed with those skills, they build ever more capable tools that help them write better code, and validate it quickly and repeatably. They respond quickly, roll with change, and produce high quality software. When you see a good agile team at work, they actually look superhuman – but they’re not. They’re just good.

Batman is armed with brains, skills, and a really cool utility belt. Superman just came from another planet where compared to earthlings, he has superpowers.

So for those in the product ownership role, and for companies wishing to use agile development, you have a choice. You need a hero. You can search for that Superman product owner, or you can take the smart people you have, give them the skills, and help them identify the tools that make them look and act superhuman.

That’s my point.

There’s a frame of mind I bring to agile and Scrum product ownership -’ a mental model that I borrow from the design community that gives me the thinking structure to learn new skills and build new tools that allow me to look superhuman at times. This is a toolset built over the last eight years working in agile development and collecting techniques and experience from lots of other smart folks.

So here’s the mental model.

You’ll need more than a backlog

As a product owner, you’re often pushed into battle armed with only a backlog full of stuff to build. While that’s a necessary tool needed to communicate with the team, you’ll need more.

Building the backlog from XP style user stories became a Scrum practice early in this decade. Mike Cohn wrote the user story bible, and it’s a good one. In it he described the now popular user story format:

Writing stories in that format gives us some clues as to what’s important to know: our users, what they want to do, and what goals they have or benefits they home to receive.

A nasty thing happens when you write a bunch of these stories. You end up repeating the user part over and over – many people get tired of thinking about it and just start writing “as a user” – which is as good as writing nothing at all. Many folks have a hard time with the “so that” part – in that at times it seems obvious or unimportant – especially for very small stories about pushing buttons, or ensuring a phone number contains digits and not letters. Really – do we need to justify why phone numbers should be numbers and not letters?

The database guy in me gets twitchy writing these sorts of backlogs. They’re denormalized – which is a geeky way of saying we end up repeating the same stuff over and over and run the risk of being inconsistent as we do.

And, while each individual story makes some sense, when confronted with a backlog full of stories written this way I can’t make sense of what the software is supposed to about. It feels a bit like the story of the blind men and the elephant – this user story is about a rope, and this one is about a tree trunk, this one’s about a snake, and that one about a wall. How does the reader get an elephant out of that?

So let’s start from the beginning.

Identify your users.

No really. Name the kinds of people who use your system. And don’t stop there – gather details about those users relevant to your product design, details like how much subject matter expertise they have, how much they know about computers, what they’re doing when they’re using the system – if they’re likely to be interrupted or distracted, and most importantly why they use the system. What benefit do they gain for using your software? How do they define success? There’s lots more valuable information – stuff relevant to how you build your system.

And, if you don’t know the answers to some of these question, go out and meet your users, spend time with them, and make good notes about what you hear and observe. Take that information back and use it to fatten up your user information. Distill all that into something like a persona to keep it straight in your head, and help explain it to others.
It’s important to remember that you’re not doing this for fun. You should be able to easily identify characteristics of your users, their goals, and use that directly affect the decisions made in your product.

But, don’t stop there.

For each type of user, describe their use

You’ll need to understand your user’s likely “use” of the tool you’re designing. Start with the way they do things currently today using some other product, your old product, or some combination of other tools – some electronic and some manual. You’ll use this to understand how to create a product that meets their goals, works within their knowledge and skill constraints, and lets them accomplish what they need to.

Identify your customers

Users are great… but let’s face it, it’s about the money. If this is a commercial product, someone will likely buy it. These are your customers. Don’t mix them up with your users – even if they’re the same people. The thought process we go through to make a buying or product adoption decision is a different thing than our day-to-day use. I ranted about this in my old entry designing software for two headed people.

You’ll need to distill the most relevant bits about your customers into profiles or personas. Understanding your different types of customers and what motivates them to buy is often called market segmentation. If you’re building a commercial product, and don’t segment your market, you run the risk of satisfying no one. Alan Cooper describes the approach of honoring and acting on all customers input equally as “customer driven death spiral.”

Identify your organization’s goals and your resulting product goals

At the end of the day, if you’re a product owner, you work for a company that’s paying your salary, and for the construction of the software. Why are they doing this? What do they hope to gain? What organizational goals does building this software help realize?

The trick when setting goals for your product is to understand the difference between output, and outcome. Good product goals point to the intended outcome.

• Output is the software built, the stuff you ship.
• Outcome is what you get after the software ships, sometimes quite a while after. It’s the financial benefit, the increase in customer satisfaction, the decrease in customer service calls, or the improvement in efficiency.

No one really wants a few hundred more megabytes of code on a hard drive some place. What we really want is the benefit. It’s the product owner’s job to know what that benefit is, make sure everyone else does, and make sure all the software we build somehow increases the odds of getting it.

Pause and answer this question: For the product you’re working on right now, how will your company benefit after the product has shipped? If you understand how, do others?

Prioritize before you prioritize

There’s an important dependency chain here that product owners need to understand.

This is the sing-a-long part of the essay. Sing the following to the melody of “There was an old lady who swallowed a fly”:

• We wrote the story to build the software,
• We built the software to support the use,
• We supported the use to help the user,
• We helped the user to earn benefit for the customer,
• We benefited the customer to get the sale,
• We wanted the sale to support the business strategy,
• But we don’t know why we adopted the strategy, perhaps we should.

OK, it’s a bit of a leap, but I want you to see the dependency here. The minute you lose sight of it, you’ve lost the context you need to make good decisions as a product owner.

Given a business strategy, it helps us choose the most important customer segments to focus on. That’s not trivial – and takes understanding of who possible customer segments are, what they need, what our organization capabilities are, and what we as an organization want to be. I smell a hedgehog here someplace.

Given most important customer segments, we can look at their possible users and uses – and select the most critical users to support to help us earn and keep customers.

For those important users, we can look at their goals and possible uses and prioritize high the uses that help them meet their goals.

In an ideal world the user stories we write are about users and their use.

As a product owner, you need to understand all these things, before you can effectively build and prioritize a user story backlog. And that’s not all. Depending on your industry there may be regulatory concerns that come into play, SOx requirements, architectural concerns, or the CEOs latest passion – all of these factors may affect priority.

I find this isn’t commonly understood by product owners, or those who teach them. I find that even when understood, product owners lack the skills to capture, distill, leverage, and communicate this information.

The product-shaped hole

The interesting thing about business strategy, customers, users, use, and all those other possible constraints are that they don’t actually say what software to build – not directly anyway.

When I think of all this contextual information, in my head I see pieces coming together and filling in – building sort of a fabric… and in that fabric is a hole – a missing piece shaped exactly like the product I need to build – a product that fits perfectly into that context.

For me, my job as a product owner is to find that product-shaped hole.

Finding the product-shaped hole is: understanding all of the relevant context that helps me shape the product to build. The trick becomes not trying to understand too much – just the stuff that’s most relevant to the decisions we need to make so we can begin making them confidently. (It’s easy to get lost in irrelevant context. Try not to do that.)

Some get frustrated talking about users, and customers. For them it’s more interesting to leap straight to talking about the software we’d like to build. I don’t blame them. Talking about software is fun. And for me at least, building it is even more fun. But, when we don’t know what features to include, because we’re not sure who our users are, how they intend to use the product, or if they’re part of our target market or not, the easiest thing to do is put everything into the backlog that anyone asks for. Then we’re left with the daunting job of prioritizing this mess.

Sketching the solution using release planning and envisioning

For me the problem shaped hole starts by understanding business strategy as some concisely distilled product goals. Good goals describe what the desired benefit is after the product ships. Even better goals are supported by metrics that help to measure that benefit.

Simple personas are a good way to distill users, details about them that affect your product design, and the goals that motivate them to use your software and consider that use a success. You can do the same thing for you customers as well – those who make the buying decisions.

Finally, I model use a bit abstractly, thinking both about what users are trying to accomplish, and some details about how they might do it. This work is done using user stories shaped into a story map that helps me, as a product owner, understand the products use – the full business process – and easily explain it to others to boot.

Now I can begin to fill the hole by planning incremental releases of a product that fits tightly within that context, no more, no less. I can slice a story map with confidence that I’m considering all relevant users, the whole business process, and the entire value stream.

To be more confident my thin slice is a good one, I write user scenarios or “blockbuster stories” – longer narratives that cut through the entire release and demonstrate what a day in the life of my users and business will look like when the software is released. It’s a good way to be sure I’m building a release that really delivers value.

And, since a picture is worth a thousand words, now’s a good time to sketch the user experience as a simple sketched story board that describes the users’ journeys through the system from beginning to end, showing a bit of what the UI could look like along the way. Users, business people, testers, and engineers, all begin to “see” what the software could look like. All begin to develop a common shared vision of what the product could be.

Each planned incremental release is a prospective solution. Releasing it, getting to users is a chance to really be sure it’s a genuine solution, as well as to begin the flow of value received from using the product.

But, if you’ve worked in agile development before, you know the tough work has just begun. You’ve got a lot of detailed decisions to make to fill in your vision. There is a lot to learn along the way. It’ll take skill and bag of good techniques and tricks to help you sprint to delivery.

Sprint to release to fill the hole

The Scrum model starts the sprint with a sprint planning meeting. But, for a product owner, your sprint planning started days before – maybe longer.

As a product owner you’ve been working with your backlog identifying the most important stories to work on first. For those most important stories, you’ve split them down into the smallest valuable pieces. You’ve discussed them with users, UI designers, subject matter experts and others that can help you fill in the details – supply you with the context you need to make detailed decisions about the software.

If you’re an interaction designer you’ve leveraged your story map to sketch high level interaction design and navigation, and you’ll leverage that high-level design thinking to do more detailed UI design that supports the stories in the next sprint. Ideally you’ll find time to prototype and test the UI. (See the salesforce.com example for black belt design and testing.)

When the sprint planning meeting does come you’ll need to be ready to discuss the story, acceptance criteria and support the team as the plan the details of how to build software during the next sprint.

You’ll need to be available to answer questions and help keep development and testing moving smoothly. You’re doing this while planning and doing more detailed research to support upcoming sprints.

After a few sprints go by, you’ll have a the beginning of a product that if you can’t ship it, you’ll ideally be taking out to customers to test it – to validate it with them, to help gain confidence that product you’re building will get used.

Sprinting towards release isn’t simple. There’s a bit of strategy to determining what stories help you see the whole shape of the software as soon as possible – the functional walking skeleton. This first earliest version helps you confirm early that you’ve found all the necessary functionality for the software – early enough to course-correct if you haven’t.

As your grow the software iteratively and incrementally, you’ll see it begin to tighten up – move closer to releasable. While it’s true that the Scrum calls for “potentially shippable software” at the end of each release, it’ll be your job to assess releasability each sprint. You’ll need to communicate to the team where the product is, how close to releasable it is, and if it isn’t what areas of the software need the most attention.

I, and others in the scrum community, have begun to use a chess metaphor to describe an opening, midgame, and endgame when sprinting towards release. Different strategies are applicable at different times. Start by laying functional foundation; continue filling in and building to full functionality while vetting with users; finish by refining the software while validating tougher stuff like performance and scalability. While each sprint is important, as a product owner, you’ll need to keep your eye on the product release.

Land and expand

Once the product has released, that existing product becomes part of your context. You’ll need to begin paying attention to what your customers and users do with the product. Do they behave as you expected? Are you getting that outcome, that business value you hoped for?

You’ll need to begin balancing inevitable requests from users for changes to the product with your future plans for releases that further your company’s strategy or help you pursue new target markets. And, I’m sure it won’t happen to you, but sometimes products have bugs that need to be fixed with smaller short releases that interrupt longer running releases that add more functionality.

If it seemed challenging getting to that first release, you’re just getting started.

Telescope and Microscope – strategic and tactical product owners

For larger products and in larger companies I often see product owners that play a more strategic role. At a workshop for product managers, I heard Luke Hohman say:

“the only true thing we can say about strategic and tactical in a business context is that one refers to the long term, and one to the near term.”

Strategic product owners tend to be looking forward into the future, forward towards new markets, next problems, and next releases. Strategic product owners tend to focus outwards in the company towards the executive team.

Tactical product owners tend to focus on the current release in progress, working hard with the team to make sure the product goes out on time, is as valuable as it can be, and is aligned with the business outcomes targeted for the release – or better outcomes if they’ve been identified.

Guy Kawasaki has written and spoken about using microscope thinking to focus on important details today, and telescope thinking to look into the future – to bring the future close in. Some product owners move nimbly between telescope and microscope. In some organizations the responsibility is split among multiple people.

Rethinking product ownership as product design

Effective product ownership requires understanding and synthesizing lots of context. A good product owner is a business advocate, a customer advocate, a subject matter expert, a user experience expert, an engineer, a top notch communicator, and a visionary leader. That sounds like a bit of super-hero to me. In the absence of a super hero, you’ll need to build skills, to build a team with skills.

Product owners are decision makers. Design is a discipline focused on making decisions based relevant context, iteratively refining decisions, and validating product decisions with those that will actually use the product.

This is my wrap-up, and it’s sounding a bit like a call to arms. And, possibly it is. I started this story with concerns about agile customering or product ownership – concerns that common agile practice didn’t offer much for the unlucky person saddled with the role. But, the truth is it’s getting better. As agile development and Scrum grow in adoption, more practitioners, smart practitioners, are learning more, are evolving useful practice for product owners.

With this essay, I hoped to point out the problem, and lay down a bit of structure for the solution. Through the year this year, I’ll be working to co-teach Certified Scrum Product Ownership courses with friends who currently teach certified Scrum courses. I’ll also be teaching dedicated Agile Product Design courses for those who already feel they’ve got the basics of agile development under their belt, and wish to add some skills to those in product ownership, business analysis, and user experience design. These courses will help those in agile coaching roles to add more tactics to their portfolios to better help their customers. I’d love to see you at a class.

If you’re currently a product owner, and you’ve developed practice that’s working effectively for you, I’d love to hear from you. My personal goal is to help improve software development generally. I’m actively seeking practices that I, and others, could use to improve life for those in the product ownership role.

Quoting Fred Brooks: “The hardest single part of building a software system is deciding precisely what to build.” In agile development, much of that deciding responsibility falls to product owners. It’s the essential difficulty that Fred describes in his No Silver Bullet essay. This is an area ripe for innovation and practice improvement.

Agile Product Development

Agile development is more culture than process

Why thinking of agile as culture and not just process explains resistance and difficulty in teaching and learning the approach

Culture, not process

I’d been working with Josh and lots of others at his company for many months aiding with their agile adoption. Finally, things were beginning to go smoothly. The team was productive and happy – and we were making good progress on the product we were building. One day when passing in the hall, Josh said to me:

Josh is responsible for user experience at company I work with regularly. Like a lot of UX people, he was concerned about adopting an agile process since it didn’t seem to value the attention and time spent understanding users, reflecting on their needs, and iteratively working through possible product solutions. Agile approaches seemed to want to dive in, leap quickly to a possible solution, and begin its construction.

Josh is an interaction designer, one of the best. He places great value in understanding his users. You could say it’s part of his design culture. For agile to succeed in his organization, especially in Josh’s eyes, we couldn’t abandon that value. The agile process we’re using there today retains that value, and a number of other values held by design culture. The team has also adopted new values that came in with agile culture – notably valuing visible working software. The result is a process that accommodates all these values – or at least works very hard to try to.

Culture is process

Sitting with my friend Jonathan at lunch last week, we were talking about the process changes he felt like he was being cornered into making. He was adding more teams, and teams were growing. Things needed to change. Jonathan was justifiably concerned that all the new process being added would erode at the fluid communication and teamwork he’d worked so hard to promote. “How do you keep those things in your process?” He wondered. After talking a bit we decided that those things weren’t really process points – rather they were part of his company’s culture. These were the things he and others in his company valued.

Once in a while I feel particularly wise. It’s fleeting, and sometimes I get a “false positive” – stupid ideas that sound wise. But I said this, and it sounded wise enough to Jonathan.

Now, that statement “culture is process” may not make sense to you – so I’ll tell you why it makes sense to me.

As a member of a culture you value particular things, communicate in particular ways, and I believe participating in a culture often implies a particular process. Here’s what I mean. To participate in American culture, we’re all aware of this ideal process flow to achieve success:

1. Go to school and work hard
2. Go to a good college
3. Get a good job
4. Get married
5. Buy a house
6. Have 2.5 children
7. Save for retirement
8. Climb the career ladder
9. Retire to a warmer climate with golf course
10. Spoil grandchildren while reminding your children of all their bad behavior when they were this age

Now that’s the ideal process – which of course no one follows exactly, and some folks dislike. I suspect many of you recognized the process as you read it, although I’m not sure of any place it’s written down. We just know it. I’m not a fan of this particular process. I often work hard to deviate from the process – but often find myself comparing my current performance with the ideal American life process performance.

Processes vary by area of the country, social status, and likely lots of other factors that sociologists are aware of. Moving to other countries will cause the process to vary. For instance, folks I know from India might replace the “retire to a warmer climate” step with “Move in with eldest son’s family.” I’m sure there’s lots of other variations in the Ideal Indian Life process. But, I believe that behind that "process" step lies a cultural value enforced by norms or expectations of others in that culture.

My big point here is that while culture doesn’t dictate a precise process, cultural values supported by teaching structures embedded in cultures and enforcement structures such as norms and mores all lead to a commonly understood process.

"I’m not buying it."

At the coffee shop discussing this with my friend Alistair, he flatly disagreed on the point that culture is process.

Now Dr. Cockburn does hold a doctorate in software methodology, so you should believe him. But, I’ve chosen to suspend my judgment on the issue and pretend for the next couple of months that agile development isn’t a process, but a culture that generates sort of a "process prototype." For a long time I’ve had a trouble believing that agile development is a process anyway.

Assertion: business and software development cultural values motivate common software process

Let’s bring this back to software development and its smaller, more local culture.

I’ll assert that in traditional software development individual skill and competency is highly valued. It’s important to be demonstrably good at your job. I’ll assert the best way to demonstrate competence is to understand what’s required of you in your organization, and to perform it flawlessly. An ideal process for that might assign individuals specific work-products to create, give them time to create the work products, then judge individual’s success on the quality of that work product.

Since it takes lots of people and lots of work products to build software, we might create a “project plan” work product that strings the creation of all these other work products over time into a dependent sequence. This may be starting to sound a bit familiar.

Since individual competence is valued, each person strives to create their work product on time with high quality. Happily if we’re unable to do that, it’s easy to blame others upstream for deficiencies in the work products they constructed and you leveraged. And, for the person who created the “project plan” work product, to prove that was high quality, they’ll need your work to go exactly as they predicted. In fact, their value in the traditional software development culture is based on it. You not finishing your task on time affects their perceived competence.

Lots of dysfunctional process emerges from us valuing personal competence at the expense of overall group success. I suspect we carried that concern for personal competence with us from our primary and secondary education where no one’s report card reflected the grade of the entire class. At least I don’t remember getting a poor grade in class when the rest of the class did poorly. On the contrary, when they did poorly, it just meant that I was likely to be comparatively better. Life is graded on a curve.

There are a number of other things valued in traditional business and software development culture that are likely at odds with best process outcomes. It’s worth pondering a bit.

We can’t escape our values

Winston Royce first drew what we now call the waterfall model in 1970. He didn’t draw it as an example of how we should develop software – but as an example of how not to do software development. That common waterfall model is one of the first figures in a long paper followed by many other figures. Adjacent to that figure is this quote from Winston:

I honestly don’t think anyone really believes a waterfall-style process works. The guy who’s credited with first drawing it in 1970 didn’t. So why then does it keep emerging as the “right” process in so many organizations for so many years?

I’ve irreverently referred to waterfall development as a “CYA Process.” The coolest thing about a waterfall process is that it allows me personally to succeed, to demonstrate skill and competence, while the end result of the process is a dismal failure. Cleverly built into a waterfall process are a variety of scapegoating mechanisms that allow us to blame other people, or outside influences for failure. Given that important value of demonstrating personal competence, and keeping me personally safe, a waterfall process seems like an ideal process choice.

I suspect there are other values that motivate poor process. I’d love to hear from you. What seems to be valuable in your organization that seems to result in detrimental process or behavior?

Teach culture first, then process and techniques

After working through the cultural discussion with other agile colleagues, we were left with the realization that we spend a fair bit of time teaching process and techniques, when it’s the culture that matters most.

Given that we understand that agile development is a culture, if we were teaching culture, how would we do that?

This line of though lead me to a little digging around into the aspects of culture.

Language

Every culture has its own specific language – and agile language is pretty rich.

There’s on old joke told about England and the United States – they’re often described as two countries separated by a common language."

Agile language sounds quite a bit like traditional software development language. So much so that those new to agile may hear the words and think they understand.

There’s a number of agile homonyms words that look and sound the same as traditional software development terms – but have differences in meaning in an agile context. For example:

Now, in a culture of individual responsibility and blame, software estimation is a pretty tense hand-wringing affair. In an agile context, estimation - or what I feel better calling "sizing" - is actually a pretty fun and useful activity.
There’s a long list of other words used in agile that don’t mean what you might thing if you don’t come from agile culture including word like: requirement, test, customer, working software, deliverable, team, meeting, and lots of others.

And if agile homonyms weren’t problem enough, there’s tons of new vocabulary that cause those new to agile to be as lost as anyone might feel in a foreign country. And, may cause reasonable people to ask "we’re doing software development here – why do we need to redefine all the terminology we already had?"

• Backlog
• Product Owner
• Sprint
• Scrum – the process and the daily…
• Velocity
• Story Point
• Burn-down chart
• Product Backlog
• Planning Game
• Yesterday’s Weather

Stories, heros, myths, legends, and jokes

Stop a person you know who’s been involved with agile development for a while. The old-timers can tell you where the term Scrum came from, what the C3 project was, and about the meeting of 17 at a ski resort in Utah. These are stories – many of them evolving to the status of myth due to frequent retelling.

I’ve often thought of agile as a cult of personality – except without all the bad political implications. There’s certainly a long list of colorful personalities – all of them intelligent, charismatic, and compelling. Some of agile’s heroes include: Kent Beck, Ward Cunningham, Martin Fowler, Alistair Cockburn, Ken Schwaber, Jeff Sutherland] not to be confused with Big Dave Thomas, Uncle Bob Martin, Mike Cohn, Mary and Tom Poppendiek, and I could carry this list out for a long time. Each of these folks have made valuable contributions to the big agile story. We often explain agile by describing these people, their ideas, and their approaches to doing things. Speaking for myself, I often tell second hand stories I’ve heard from others - sometimes wondering how accurate they are… that’s the myth part.

And, there’s even a joke or two embedded in common agile culture. As well as the commonly observed trend for agile teams to build up their own supply of stupid jokes.

Values, norms, folkways, laws, taboos

We can’t talk about agile without talking about values. The agile manifesto itself is a statement about what that group of creators valued – not about process. The supporting 12 principles are littered with phrases for things that agile culture values – such as:

• welcome changing requirements,
• satisfy the customer through early and continuous delivery,
• business people and developers must work together daily,
• build projects around motivated individuals,

and I could go on and on about those too.

All of these value statements describe what’s important to agile culture, and we can see how they likely have process implications… but we can’t see exactly what the process is.

XP’s foundation is a set of 4 values, Scrum, the now dominant agile process, describes 5 values – which look sorta like values to me.

Recently Brian Marick, in a keynote talk at the Better Software Agile Practices conference, discussed what the Agile Manifesto left out. He focused on new emergent values - things agile people care about that didn’t get mentioned. Brian noted: skill, discipline, ease, and joy (who can get enough joy?).

I was lucky enough to join him on the stage where I added "purpose" and "angst" to the set of values he suggested. Purpose, because it’s important to me that I’m helping to build software I know will ultimately help people, and "angst" because I keep my edge by always being a bit dissatisfied with the status quo. Anxiousness keeps me aware and keeps me looking for opportunities to make things better - which I value in others as well.

Agile projects enforce their values with established norms. Agile folks have even come up with some slang used to dig at people who don’t look like they’re respecting agile values. For instance:

• Agile values doing only what’s necessary. If an agile developer catches another developer over-complicating design, especially to include functionality not yet needed he might say "YAGNI" - short for "You aint gunna need it." Kind of an update of the popular KISS - keep it simple stupid.
• That simple design ideal is enforced with the mantra "Do the simplest thing that could possibly work."
• Agile values not trying to be too predictive - to minimize up front design in favor of short planning and execution cycles. If your up front planning or analysis starts to look to heavy, you might be accused of "BDUF" - or "big design up front" - which is bad.

The agile community as a whole will frown on teams that don’t work together, and don’t deliver working software, Frowning and discouragement is as rigorous as agile culture gets with enforcement - although an agilista’s Zen slap can sting your ego a bit.

Beliefs, attitudes, assumptions, mental models, symbols

Agile practitioners have a large body of beliefs bottled up in their story and terminology. Agile folks don’t believe they can effectively predict the future, or estimate development time. Many agile folks believe in emergent architecture, and in growing software incrementally.

Mental models like the simplified scrum framework can be sketched quickly by lots of agile folks. The two loops often called the scrum snowman model have been turned into a simple symbol that’s the logo for the ScrumAlliance.

Rituals, rites, ceremonies, and celebrations

Daily standups or daily scrums, planning meetings, pair programming, and product demonstrations are all commonly taught "ceremonies." Healthy agile teams treat them as celebrations, and invent lots more celebrations. I think it’s an excuse to overeat.

Certified Scrum Master has become a bit of a rite of passage that the ScrumAlliance has tried to add a bit more ceremony to by adding additional testing and subsequent rites of passage like Certified Scrum Practitioner, and Certified Scrum Coach. So much for the secret handshake. I remember back in 2001 when it was a bit of an Extreme Programming right of passage to have attended an XP Immersion. At one of the first XP conferences people proudly wore their Ward number on their shirt to indicate how close they’d been to pair programming with Ward Cunningham. Sorta like the 6 degrees of separation with Kevin Bacon, a Ward number of "1" meant you’d paired with Ward, a "2" meant you’d paired with someone who’s paired with Ward. I have a "2." That’s a story about an agile hero and a bit of a right of passage.

Technology, artifacts

And finally, cultures develop their own technology and artifacts. Agile teams use burn down charts, task boards, and story cards. And a large number of tool vendors have created software products to support agile development.

So What?

The long pondering on culture, and the closer look at aspects of culture have caused me to look closer at the way I teach agile development – and anything to do with software development.

1. Underscore agile values that motivate practice

I find it more necessary to highlight and underscore the values that motivate the process or technique I’m speaking about.

2. Identify organization values that compete with agile values

When I sense concern from someone about a particular agile process point or practice, I work to identify the agile value the process supports and to identify what’s valuable to the person with the concern. I’m looking for a conflict of values.

3. Be sensitive to culture shock

I’m more sensitive to culture shock. The terminology, stories, and ritual that agile is chocked full of are bound to make anyone feel a bit uncomfortable. In particular, people who’ve spend a great number of years working in software development may be feeling something they haven’t felt for many years – uncomfortable doing their own jobs. They may feel like someone has drugged them, put them on an airplane, and dropped them off on a foreign country without their consent. They may have incorrectly assumed they were just learning a new process.

Agile Product Development

The new user story backlog is a map

Why the flat user story backlog doesn’t work, and how to build a better backlog that will help you more effectively explain your system, prioritize, and plan your releases.

This is Gary.

Gary and I worked together for a day to build a user story map - a better version of a product backlog. Building a user story map helps us focus on the big picture - the product as a whole instead of getting myopically focused on an individual story.

When it comes time to prioritize, Gary did so with the entire context of the system in view. In Gary’s case he originally set out to build Mimi - short for Music Industry Marketing Interface. However, when it came time to prioritize we focused on the band member promoting a gig in a club. It was too big for a single story - too big for an "epic" really… it took most of the map to think about all the details of designing a promotional mailer, building up an audience list, then sending out the promotion, and tracking responses. By the end of the mapping exercise, all the other music related things Mimi could have done seemed like a tall order. What’s more, building a piece of commercial software that made the job of sending out mailers easily and quickly seemed like a good idea. And it turns out it was.

Mimi shipped after a lot of sweat and effort from Gary, Dave Hoover, and the fine folks at Obtiva. As of this writing, Mimi currently sends close to four million messages per month. The user experience is a work of art. It’s a sexy piece of software. And, the user story backlog we built that first day of discussion still describes the same high level view of Mimi we saw then - minus the stories we moved out of the release of course.

Flat backlogs don’t work for me

One of the more troubling things, for me, about common Agile development practice is the idea of the flat user story backlog. It just seems like a dumb idea to me. Well not completely dumb, in that if I’m going to build software incrementally in small pieces, I’ll need to figure out what piece to start with. The backlog puts those pieces in build order - from highest value to lowest value - which is great if you’re a project manager - which I’m not.

I’m writing this from the plane as I fly back from a client site. We had what I feel like was a pretty successful couple days of story writing and release planning. I had fun - and I was pleased to see the group really get into it and feel good about what they came up with. I’ve been using a practice I [now] call story mapping. I first wrote about this approach in the article "How You Slice It." The article was published in January 2005 - but by the time it was written, I’d been using the practice for a couple years.

Years later, I still love the approach… so much so that I’m in danger of seeing it as a "silver bullet" - which is an early warning sign that I’m becoming a dogmatic fool. However, I’m seeing more and more people arrive at similar approaches - so at least I know I’m not too crazy.

Let me describe what’s wrong with story writing, then generally what a story map is - and why it solves problems for me.

The flat backlog is poor explanation of what a system does

Arranging user stories in the order you’ll build them doesn’t help me explain to others what the system does. Try handing your user story backlog to your stakeholders or users when they ask you the question "what does the system you’re building do?"

For my money, trying to understand the system - the whole system - is the difficult part of software development. One of the most common complaints I hear from Agile teams is that they lose the big picture - that is if they ever had it in the first place.

I tried to explain to a co-worker once what I thought was wrong with a story backlog. The story goes something like this:

"We spend lots of time working with our customers. We work hard to understand their goals, their users, and the major parts of the system we could build. Then we finally get down to the details - the pieces of functionality we’d like to build. In my head a see a tree where the trunk is built from the goals or desired benefits that drive the system; big branches are users; the small branches and twigs are the capabilities they need; then finally the leaves are the user stories small enough to place into development iterations."

"After all that work, after establishing all that shared understanding I feel like we pull all the leaves off the tree and load them into a leaf bag - then cut down the tree."

"That’s what a flat backlog is to me. A bag of context-free mulch."

I need that context in order for me to really tell a story about the system.

For a new system, the flat backlog is dismal at helping me determine if I’ve identified all the stories

Given a stack of index card, or a spreadsheet full of user stories, I can stare at them for hours, discuss them for hours… but I always walk away with the nagging feeling that there’s something I’m missing. Of course I know I can’t think of everything - but I just wish I felt a little more comfortable.

How often are you blindsided by overlooking important pieces of functionality when scoping a new piece of software?

Release planning with a flat backlog is difficult

For me, for the typical project I get involved with, the number of stories in a backlog is dozens at a minimum - often over a hundred. I work hard, especially when we’re just starting out, to keep the stories high level. Even then it’s common to end up with a 120 or so stories in a first cut of a backlog. Stepping through each one of these and making an in-out decision is tedious. Some of the most miserable hours of my life have been spent sitting in meetings with groups prioritizing their backlog. Uuugh.

Build a story map

Arranging user stories into a helpful shape - a map has worked well for me.

It’s a simple idea really. A small story map might look something like this:

At the top of the map are "big stories." I call them user activities (borrowing the term from UX people like Larry Constantine and Don Norman). An activity is sort of a big thing that people do – something that has lots of steps, and doesn’t always have a precise workflow. If I was building an email system (which I’d never be foolish enough to do) I might have an activity called: "managing email", and "configuring email servers", and "setting up out of office responses."

A story for an "activity" might read: As a consultant I want to manage my email so I can keep up with clients, colleagues, and friends.

But that’s way too big of a story to put into an iteration or sprint.

That story breaks down into other stories like "send message," "read message," "delete message," "mark message as spam" - stuff like that. I call these user tasks. (Again a word used by UX people.) For lots of Agile people "tasks" refer to the things that developers do to finish user stories. Really a task is something that someone does to reach a goal. A user task is what users do to reach their goals, developer tasks are what developers do to create stories, Ant tasks are what ant does to… well… do whatever you’re doing with Ant.

I simply arrange the small things under the big things in a bit of a grid form.

I’m always imagining time moving left to right (because it does in my western world. My counterparts from top down or right to left worlds can translate as needed). For instance when arranging stories in the map, if a person using the system typically does one thing after another, then I’ll put the early thing on the left, and the later thing on the right. I do this with the big things and the little things - the activities and the tasks.

When teaching this, people often tell me "the users can perform these in any order. What order should I put them in?" I’ll ask them to "explain to me what the system does at a high level - just tell me the activities." They then recite them to me. "That’s the order" I say. In fact, the order you’d explain the behavior of the system in is the correct order. We’re building a map that lets us tell a really big story about the system. Build the map in a way that helps you tell the story.

Keep your epics - but stop calling them that because it bothers me

The really big stories can be considered "epics" as Mike Cohn describes them. They’re stories - just really big ones - too big to estimate and build. When an epic gets in your backlog, and it’s time to discuss it in more detail, I often see people remove the epic from the backlog, decompose it, and replace the pieces they’ve identified. This is where I cringe. Recall my cutting down the tree and keeping the leaves in a mulch bag story above.

That big story was context. It was my simple way of thinking about the whole activity that people were doing. It’s my quick way of explaining to others what the system is about.

And, I hate that word "epic." I haven’t written beowulf here. There’s no hero with a magic weapon slaying a monster. It’s just my user "managing email" - a relatively basic thing from my user’s perspective. At least the terms "activity" and "user task" give me some idea of what kinds of stories they are. That said, I’m not fond of the term "user story" either, but I’ve come to accept it. It beats the crap out of requirement. I still have trouble looking a C-level exec in the eye and explaining to him that "we’ve broken your system down into a series of epics and stories." Upon saying something like that I imagine he’s hastily rethinking my daily consulting rate. All the sticky notes and index cards aren’t helping either.

Walk your map to test it - to get the big picture

Given a fully assembled map, I have something I can hang on a wall, or lay out on a tabletop and actually have a discussion with. I can walk the map from beginning to end with a user, stakeholder, or developer and tell a story about the users of the system and what they’re doing. I can skim along the top of the map, and just touch on the high points. I can dig down into the map to discuss the details.

Talking through the map with users and others helps me find things I’ve missed. I often hear "you’ve missed a couple steps here" from users when doing this.

I can annotate the map with pain points or opportunities. As I talk through the map with a user it’s common to hear him say, "this here is really a problem with the system today."

When working with users they often correct me. Yesterday I was working with a potential user of a new system - she was a compliance officer for a major company. I write down a process step as she said it and placed it on the table. I’d been using green cards for big things, yellow for smaller things. She quickly corrected me - "no, that should be a yellow card." In less than an hour’s discussion she had caught onto the modeling approach and could, in very few words, supply me with a lot of information about the thing she was doing with the system. She knew yellow was smaller than green, and could tell me that the thing she was doing in the system wasn’t as big as I thought it was.

Building and walking story maps leaves me more comfortable than ever before that I "get it" - that I haven’t missed something big.

Your software has a backbone and a skeleton - and your map shows it

I find that the big things on the top of the story map look a little like vertebrae. And the cards hanging down look a little like ribs. Those big things on the top are often the essential capabilities the system needs to have. I refer to them as the "backbone" of the software. I stole this term from Dr. Dan Rawsthorne who might use the term slightly differently than I do.

When it comes time to prioritize stories, I don’t prioritize the backbone. It just "is." I do prioritize the ribs - the stories hanging down from the backbone. Place them high to indicate they’re absolutely necessary, lower to indicate they’re less necessary. When you do this, you’ll find that all the stories placed high on the story map describe the smallest possible system you could build that would give you end to end functionality. This is what Alistair Cockburn refers to as the "walking skeleton". I always try to build this first.

Plan using your backbone

When it’s time to plan releases, it’s usually not important to prioritize backbone items against each other. For instance if I were to build a high level backlog for a car it might look something like this:

• engine
• transmission
• brakes
• suspension
• …

It would be stupid to ask stakeholders to prioritize that: "what’s more important, the engine or the transmission?" – or "we don’t have enough time in this release, could we release without brakes and add them later?" These items are essential - and we’ll need all of them to deliver a minimum viable product - and MVP.

Where the prioritization comes in is below this level: 4-cylinder engine or 6-cylinder engine? brakes with anti-locking or brakes without? sport suspension or not? It’s how we build up those backbone items - prioritize their characteristics that matters.

When you prioritize a story map, you’ll move cards or stickies up and down to indicate high or low. Lately I take a long strip of masking tape and line off the story map - creating horizontal swim lanes for each release. Then I move the stories up and down into each lane, and even vary their height in the lane.

Keep your map displayed to communicate the big picture

Building a story map helps you initially understand the functionality. Ask yourself when in your project do you not need to understand your functionality any longer? I know it’s not a fair question, but I do find some folks that spend a great deal of time building this sort of thing to understand the problem, then tossing it all out in favor of putting stories into a flat-backlog. Cutting down the tree and loading the leaves into a mulch bag.

I find a story map hung as an information radiator becomes a constant point of discussion about the product we’re building. When the project is running, it becomes our sprint or iteration planning board. We identify or mark off stories to build in the next iteration directly on the map. During the iteration we’ll place just the stories we’re working on into a task wall to managing their development - but the story map lives on the planning wall reminding us what the big picture is, and how far we’ve come.

When we’re building software incrementally, story by story, we’ll choose them from the story map left to right, and top to bottom. We’ll slowly move across the backbone, and down through the priorities of each rib. We’re slowly building up the system not a feature at a time, but rather by building up all major features a little at a time. That way we never release a car without brakes.

A different backbone may be in order for adding features to an existing product

When adding features to an existing product, it already has a backbone - and sufficient functionality to have released. I find it useful to stilly identify the backbone to help me get context - to help me see where new features are being placed.

On some projects adding just a few features to a large existing product, it’s been difficult to talk people into building up a story map for just a few features. In these cases, I’ll simply prioritize the new features, then for each feature build a little story map to prioritize the user stories that make up that features. Each little story map may have 10 or so cards - but they’re still arranged left to right, and top to bottom so we can focus on building a little-tiny-walking-skeleton of the features as early as possible.

It’s a pattern - not an innovation

Although I tried not to, I became indignant when I heard someone describe this concept back to me because a major consultancy was now teaching this as best practice for managing backlogs. "They stole my idea!" I thought. But then I had to remind myself that I’ve seen lots of others doing very similar things. I know when I started at ThoughtWorks many years ago, I hadn’t been working there long before I ran into Luke Barrett who’d been building almost exactly the same model collaboratively with users the same way I had.

I always remind myself of the litmus test for a pattern. If you explain someone a concept and they say "what a cool idea!" it’s not a pattern. But if they say "we’re doing something like that too!" it’s a pattern. I’ve seen this often enough now that I believe it’s a pattern.

When teaching I like showing Indi Young’s Mental Model that arranges user research in a similar left to right flow, with feature ideas placed adjacent to the user behavior they’re relevant to. I like Todd Warfel’s Task Analysis Grid. It’s like one vertebrae of a story map with lots of cool context, and all the user stories neatly prioritized below. Todd uses color to indicate priority, not height.

Learning more about story maps

If you’d like to learn more about this, consider showing up at one of the conferences where I’ll be teaching the practice. This year I’ll be at Spool’s UI13 in October, SD Best Practices 2008 in November, and Better Software’s Agile Development Practices 2008 in Orlando. A half day class should give you all you need to know to get started.

If you’re impatient, you can try to figure it out from my presentation slides. And, go back to my original article on building story maps.

Agile Product Development

Twelve emerging best practices for adding UX work to Agile development

How experienced UX practitioners have adapted to work happily in Agile environments

Agile development originated from a place where user experience practice was weak

In my last blog essay I explain how I see that the type of product we build has big effect on the process we follow to build it. In the lower left of my simple 4 quadrant model I place software built internally for use by an organization where usage is mandatory for its target users. I explained that this is where Agile processes originated from. I also explained that strong user experience practice rarely exists here. If your company is an exception, I’d love to hear from you. But, given this starting context where user experience practice isn’t well understood, no one should be surprised that user experience practice isn’t well understood in Agile development - at least not out of the box.

I’m often asked "does user experience practice work with Agile development?" The short answer is: "of course it does!"

After first addressing that question, I need to place the person asking somewhere on my 4-quadrant software context model to do a quick assessment of what life is probably like for them in their company. If they’re in anyplace other than the upper right quadrant, I suspect they’ve worked hard to build up a successful UX practice in their company that works inside their current process - whatever that is.

There’s an important point to understand:

Accept and adapt

In a presentation at a CHI panel discussion from Craig Villamor of salesforce.com, he explained the five phases of agile adoption for user experience people:

1. denial
2. anger
3. bargaining
4. depression
5. acceptance

You may recognize these five phases from someplace else. But, there’s an important point here: if you’re a user experience professional, you will need to adapt your current practice to work within an Agile value system and lifecycle. The sooner you get through the first four phases and begin that adaptation, the better.

My early Agile experience came from the lower right of my diagram: commercial compelled use products. I had real customers that paid my company for our software. That customer base was diverse. We had competitors that we needed to outperform. It was that context and the lack of rigor around user experience that lead me to adopt and adapt user experience practice from Constantine & Lockwood, Cooper, and Holtzblatt & Beyer, and others.

Before Agile my process was a combination of graphic arts skill, hangin’ out with users, and command and control. It worked pretty well. But, a faster paced collaborative Agile world put me in a position where I needed to be more transparent with what I was doing. I needed to collaborate effectively with others. I needed to rally a team to work together to move faster. Agile development isn’t about personal efficiency - it’s about throughput of software and value. My individual ad hoc approaches didn’t scale well. My personal throughput had it’s limits.

In my first Agile environment, I went through Craig’s five phases of adoption. I sometimes forget how miserable my first year in Agile development was. For any co-workers of mine from 200-2001 who might be reading this - I’m sorry for being such a crazy back then.

Eventually I adapted, and I’ve never been more content.

I really loved the control over the development process that Agile brought. Not unilateral command and control, but control that comes from a group of people working productively together showing consistent visible progress, and all enthusiastically working to meet the goals of our customers. Agile development has made me more confident than ever that I’ll get a good product to my users as efficiently as possible.

Best Agile user experience practices begin to emerge

There’s a description of a "pattern" that comes from the design patterns community:

I’ve got lots if ideas on what I think best user experience practice should be like in agile development, and I’m not too shy about sharing them. (Just attend one my tutorials at a conference and you’ll see.) But, I have to admit that they’re my adapted approaches - approaches I adapted for use in my Agile context. They’re not necessarily patterns. Over time I’ve seen a number of common patterns emerge from various successful user experience people across a number of companies. These are UX people that have gone through Craig’s five stages of UX practitioner Agile adoption, and have come out the other end with successful practice, and a real appreciation for Agile development.

Recently I attended a CHI workshop on Agile development and User Centered Design. While listening to this seasoned group of Agile UX people I begin to see patterns of practice both in the room, and that I’d heard before from talking and working with other UX people.

These are the common practices that I observed. I’ll list them here to spare you the suspense then give lots more detail on what I mean below. I’m confident there are lots more - these are just the ones I noticed. Here they are:

I think you’ll see that many of these are good practice, Agile or not.

Also take a look at Austin Govella’s Agile Principles for UX. You’ll see a couple points of overlap, and some different and important points Austin makes.

1. Drive: UX practitioners are part of the customer or product owner team

This means they have an active hand in deciding what is built, the overall business strategy the drives what’s built, and the tactical prioritization of work done first. In some successful Agile organizations the UX team is the Agile customer team or product owner team.

For UX practitioners not familiar with the terms customer or product owner in the Agile context, those terms refer to a process role and not necessarily the actual customers who purchase a commercial product. However there is a bias in Agile environments to get end-users more frequently and directly involved. Do this also. More on that in #6.

2. Research, model, and design up front - but only just enough

In spite of what a dogmatic Agile trainer may tell you, companies that are successful at incorporating UX work and Agile do some up front user research and modeling that results in things like personas, workflow models, task analysis grid, or even something like Indi’s Mental Models.

However, they have learned how to compress the timeframe for this work by:

• aggressively prioritizing the types of users researched to the those that are the highest priority, and shortening the amount of time spent with users of lower priority
• modeling quickly, in a lightweight and collaborative way often involving other team members to help analyze data and construct time consuming models like affinity diagrams
• defer some less urgent research to be completed while the software is being constructed

Since the UX team is ideally part of the customer/product owner team, they have a hand in determining incremental release strategy - that is determining what coherent collection of features release first. Leveraging this understanding, high level scenarios and/or wireframes are usually built to communicate screen flow, general look and feel, and general user interaction patterns.

All this research, modeling, and high level design often occurs in weeks, not months. Organizations such as Yahoo have been effective at packaging and leveraging user research across projects to shorten the time between initial concept and beginning construction. UX practitioners should leverage the time while the organization is finding production staff, such as developers and testers, to begin research and modeling.

Some Agile teams have used the term iteration 0 or sprint 0 – as a first development timebox set aside for this initial research along with getting the development environment setup and ready to go, and doing some initial architectural prototyping or "spiking" - the rough equivalent of high level design from an architectural perspective. For years now I’ve done a quick treatment of user centered design modeling to build a backlog and plan initial product releases.

3. Chunk your design work

Once high level research, modeling, and design is done, it’s time to start construction.

Construction in Agile development is done in small chunks usually called user stories - or bits of functionality valuable and demonstrable from a user’s perspective. This can be a problem for designers in that we’ve got to plan - to decide what those chunks of work are – then later design their interaction and look in sync with development. Hopefully the high level design has left us with a bit of a "sketch" of how thing might look and a high level roadmap of where the software is going functionally. We don’t want to be flying too blind.

The idea of "chunking" or breaking down work into coherent chunks that can be both designed, build, and validated somewhat independently is both art & skill. Some take apart their high level sketches of the software and chunk a screen or screen-component at a time. I work using the "user task" as a building block layering in functional user tasks into the system one at a time. When a system begins to mature functionally, chunks begin to take the form of adjustments and refinements to existing software.

Desiree Sy has written a good description on how she approaches chunking at Alias (now Autodesk Toronto).

I believe most UX practitioners are afraid of building a [[Winchester Mystery House], and it’s not an unfounded concern. But, given a high level sketch of the design, some practice at chunking, and some course correction along the way and things tend to come out OK.

Chunking work into small bits turns out to be difficult for everyone. Developers new to Agile have difficulty with it. Business people or product managers have difficulty in breaking their system down into small parts that can be considered independently. And over time Agile practitioners have recommended breaking functionality down into smaller and smaller parts.

4. Use parallel track development to work ahead, and follow behind

Lynn Miller of Alias, now Autodesk’s Toronto office first described the pattern of parallel track development in her 2005 paper Case Study of Customer Input For a Successful Product. For Alias’s Sketchbook Pro product, she described the emerging pattern of the Agile customer team working ahead one or two time boxes to research or design what will be built in future development time boxes.

It’s actually a bit more complicated than that.

The design team works ahead a development time-box or two. In a given timebox they might be:

• researching work to be done 2 time-boxes from now (T + 2)
• validating design prototypes to be built 1 time-box from now (T + 1)
• being available to collaborate with development to support work done in the current development timebox (T)
• working with customers to validate working software built in the previous time-box (T – 1)

User experience people working on Agile teams become masters of development time travel nimbly moving back and forth through past, present, and future development work.

5. Buy design time with complex engineering stories

At times it takes more time to design and validate a feature. That next Agile development time-box is coming whether you want it to or not. Ideally developers will keep working on software while design and design validation is ongoing. To buy time it’s common to have chunks of work that are technically complex, but trivial from an interaction design perspective. In her paper Lynn describes starting development work on Sketchbook Pro by beginning the construction on the feature that saved drawings as layered PhotoShop files. From the user interface it’s a simple "file, save as" menu choice, but from the development perspective it was heavy lifting. Having the development team work on this very important feature early bought designers time to get ahead.

I used to have a friend in the newspaper business. Newspapers release every day whether writers are ready with stories or not. One of the things she did was keep stories sitting around that she called "evergreens." These were stories that stayed fresh - stories that she could put into the paper at any time to fill space. Interaction designers would do well to identify work that’s "evergreen" - work that can be done any time, but that buys time to do design and design validation work.

6. Cultivate a user validation group for use for continuous user validation

Many times now I’ve seen the pattern of UX people building up a pool of users they collaborate with to validate design before and after it’s built. This pool of users needs to be large enough that the designer doesn’t call on the same user repeatedly every week - but rather talks with them every month or two. My friend Heather described what she calls "development partners" at Elsevier. Desiree Sy describes design partners in her paper on Agile User-centered design.

Organizations like salesforce.com keep a continuous flow of users scheduled to validate design work. I’ve seen many instances of users scheduled for remote usability testing or site visits well in advance of knowing what will be being tested. The one thing you can depend on in an Agile context is that there will always be something.

7. Schedule continuous user research in a separate track from development

For many commercial projects I’m starting to see user research as something that’s considered a separate and continuously running track. User research functions as an independent effort that keeps its activities and schedule transparent. User experience designers working with the development team can help steer future research, and even participate in it in support of what they’re currently working on in development. User research that’s conducted is shared and communicated in lightweight ways with the rest of the team. I’ve seen some UX practitioners using comics to communicate the output of user research rather than dry reports. I commonly see presentations schedule to share-out - or communicate stories about users and research findings with the rest of the team.

8. Leverage user time for multiple activities

In talking with Lynn Miller and Desiree Sy about parallel track development the describe visiting customers and using that time to do some contextual inquiry style observation and interviewing, to then sit down and review a prototype for something that may be built in a future iteration, then to review the working software testing features just built in a previous iteration. The trick here is to leverage that user face time for research and validation. Don’t segregate your work.

9. Use RITE to iterate UI before development

The RITE method - short for rapid iterative testing and evaluation was first described by a game development team at Microsoft. They describe an idea that to many seems common sense.

In more traditional usability engineering it may be common to conduct usability test repeatedly with different subjects on the same software. By doing this you can really prove that usability problems are repeated and actually common across users. The outcome of traditional testing might be a report that communicates problems along with possible solutions.

However, when using the RITE method, we look to get errors out of the software as soon as possible. In between each test the team discusses the problems they observed and how they might fix them. In the original Microsoft report, developers actually went to work correcting problems before the next test subject(s) were brought in. The result isn’t a list of repeated errors and recommendations like standard usability testing, but software that has most of the errors corrected - which is what we wanted in the first placed anyway.

The smart people at salesforce.com have taken RITE and cranked the dials up to 11. They build html prototypes and iteratively test and repair them using remote usability testing. They’ll complete several rounds of this on each chunk of work before it goes into a development time-box. The smart people at Google describe "lunch room testing" - testing paper prototypes of preliminary design during lunch with Google staff at a table setup in the cafeteria.

10. Prototype in low fidelity

When talking to UX practitioners working in Agile environments I continue to hear them describe the lesson learned over spending too much time crafting prototypes. I hear them chant the mantra of staying as low fidelity as possible.

I prefer paper, but I often have the benefit of working with co-located teams and customers/users I can get direct access too. I’ve seen others use PowerPoint and only simple black and white lined wireframes. Some even take the time to use "sketchy" lines to indicate roughness. Other may use html. I recently heard a talk from the designer and owner of dopplr who describes communicating design by sketching on the whiteboard with his developers.

My friend Larry Constantine describes the stuff we build as deliverable and consumable. Deliverable means we have to tighten up and deliver it to someone else. They’ll need to take that work and carry it forward often without us being there. We’re often judged on the quality of that work. But, consumable work on the other hand is the stuff we build for ourselves. This is the stuff we sketch or model to better understand what we’re doing. It only needs to be good enough to understand, to learn, to communicate quickly with our co-workers, then it can be thrown away. In agile development prototypes are consumable.

I also remember a quote from Bill Buxton who said something like: "The idea of high fidelity and low fidelity in prototypes is silly. As far as I’m concerned there’s only right fidelity and wrong fidelity."

11. Treat prototype as specification

In an attempt to travel light, I often hear UX people describe their prototypes as their specification. It’s common to deliver only the prototype, or ideally the prototype + a discussion with the team building the software. During the discussion annotate the prototype by hand if necessary. No need to produce detailed documentation. In Desiree’s paper she points out a number of different documents that were scrapped in favor of discussions over a prototype.

12. Become a design facilitator

Early in this essay I described my early experience as a designer figuring out what the design should be then simply telling others how to build it, or building it myself. As I’ve begun to work within Agile teams, my approach has changed to favor collaborative design. More and more I find myself acting as facilitator to harvest and model information from large groups of people. I find myself working with groups of users and developers to write user scenarios, and sketch user interface design.

Recently I learned about Adaptive Path’s Sketchboarding approach, and Jeff and Jim’s Design Studio approach used at Jewelry TV. In addition to them being great approaches to increase the number of great ideas and the quality of the resulting solution, they both provide a facilitation structure to allow a number of people to participate in the design process.

To allow user centered design activities:

• to move quickly,
• to keep progress transparent,
• to give everyone in the team an understanding of the design, and
• an appreciation for the process and tradeoffs necessary for a good design,

It’s valuable to turn design activities into collaborative activities involving members of the team.

I often hear UX practitioners working in an Agile context explain to me that a big surprise is how much time they spend facilitating. UX people facilitate not just design sessions, but sessions to communicate test results or field visit results. Desiree describes replacing personas and scenarios with stories about real customers and real use.

Two sure fired ways to succeed in software development

I once introduced Jim Highsmith to the CEO of my company with the hopes that my CEO would hire him to help improve things. Jim has been a long time thought leader in the Agile community. He’s one of the original signers of the Agile manifesto. He was one of the consultants that helped Agile-UX experts at Alias get off on the right foot in 2002. He’s got a clear head. My CEO asked him "We can’t seem to finish anything on time – how do we build software and finish sooner?" Jim responded calmly "start sooner."

I overheard Jim in another conversation with someone else years later. They were complaining about "getting all this software built on time." Jim’s response: "build less software."

Let’s recap. Two secrets to success in software development are:

This is sadly simple advice.

Agile development does try to short-circuit elongated research and design phases in favor of beginning sooner and continuing active research and design throughout the development cycle.

This quote from Desiree Sy of Alias/Autodesk says a lot: "For our User Experience Team, Agile user-centered design resulted in better-designed software than waterfall user-centered design. Agile communication modes narrowed the gap between gathering usability data and acting on it." Desiree’s talking about her cycle of being able to talk to end-users, interview them or show them prototypes or working software then take what she learns right back to her development team who takes that understanding and acts on in code that she and her team can see days later. Starting sooner and keeping the customer research thread alive and active pays.

On a current project, I’m lucky enough to work with a very talented team practicing Cooper’s goal directed design in an Agile environment. When it comes time for them to describe what to build to the developers they sit down and talk about it in detail with wireframes and screen mockups in hand. I’ve seen many of these conversations and they all go a bit like this. After explain the interaction design to developers they say "Ok, we can do that. It’s going to take a couple weeks, but I was thinking that we could do something like this instead.." An explanation follows. "if we do that I think it’ll still get at what you intended, but we can do that in a few days."

Building less software does mean keeping your software from being feature rich and quality starved - and it also means collaborating with developers to write a little less code on every single feature you do choose to implement. Sometimes you save a lot of code. Through strong collaboration I see hours, days, and weeks of development time saved every day on healthy Agile projects. For me, that’s the real secret to finishing on time with high quality software.

Don’t neglect ideation

At a recent IxDA conference keynote, Bill Buxton remarked that a problem with Agile development was iterating without ideating. I’m paraphrasing here, but basically Bill asserted that Agile teams lock onto a solution and iterate to refine it without considering that there may be a better solution out there. He’s right, but not just about Agile teams. I see a fair number of designers guilty of the same behavior.

Today it’s easier to respond to Bill’s call to action with some concrete practices like Adaptive Path’s Sketchboarding and Jeff & Jim’s Design Studio approach. Desiree Sy described using interns to prototype 10 or more design solutions to a possible design problem. What’s heartening for me is that Agile development and the desire for collaboration outside the immediate design team has motivated the creation of this type of practice.

Are these Agile UX best practices?

The 12 practices described here are repeated patterns I’ve observed. They’re not hypothetical. They’ve been proven in a number of teams and a number of companies. But, one of the tricky things about any process or practice is that it relies heavily on the context it’s placed in to work. Your mileage may vary with any of these practices.

As I pointed out above, common Agile practice today originated from contexts where UX practice was generally thin to start with. I suspect that it’ll be a few more years before there’s common understanding that we need UX work in every context, some more than others, and that there are common Agile UX practices that should be included in the Agile toolkit.

If you’re a UX practitioner new to an Agile context, I suspect you’ll be feeling some frustration - especially if you’d figured out a process rhythm that was working for you before Agile. Look closely at your company’s reasons for choosing Agile development. With luck they have some good reasons. Then "accept and adapt." Like many UX people that have come before you, I suspect you’ll find that not only is Agile development not as bad as you though, but that it leads you to a level of process improvement you might not have come to otherwise.

To continue the discussion on user experience and Agile development, join Arlen and me in Seattle

I’ll be at the APLN Seattle Leadership Summit July 17th and 18th. Arlen Bankston and I will be facilitating an all day discussion on how user experience practice best fits into Agile development.

Agile Product Development

Is user experience relevant where you work?

How looking at the context your product is targeted for can explain your organization’s support or lack of support for user experience concerns.

Software development and plastics

I remember something Tim Lister said in a talk at a conference a couple years ago now - and I know I’ll paraphrase this slightly wrong - but Tim said something to the affect of:

I hope you get his point - no matter how badly I’ve likely mangled his original quote. Lately I get his point - too often for my tastes.

User experience practice varies wildly by product type

I’ve worked as a consultant for a while. This means I get the opportunity to spend time in a variety of companies. I’ve spent time working in banking, stock portfolio management, medical records, brick and mortar retail, on-line retail, aircraft parts wholesaling, dot-com-search-engine-companies-that-now-do-everything, digital asset management… there’s more, but that’s more than enough to make my point. I’m lucky enough to be involved with all those folks because I know a little about planning, designing, and constructing software in an Agile context. When I appear at an organization, most folks are looking for some answers - better ways to do things.

Because I’ve developed some expertise in user experience practice and Agile development, that’s the particular area I’m often asked to help out in. But, it’s when I talk about user experience design practice that the difference between software development organizations and the products they develop really becomes obvious. Within a few minutes I usually find myself reaching for a whiteboard marker, index card, or sticky note to draw this model:

It’s one of those often overused Gartner magic quadrant diagram. I suspect we can correlate about any two concepts using one of these things - and it may be done a bit too often. It’s important you don’t think about them too hard. I know I don’t. It’s the concept behind them that’s important.

So, let me explain this model.

Look at the source of value for the software

Business value is a big deal in agile development. Sometimes I wish agile practitioners would stop chanting it and start asking where it’s actually coming from. In many companies they seem to value the amount of software built - as if we were selling this stuff by the pound. The real value comes after the software is delivered and/or sold or put into use. Using the software is often the source of the value. - which is an interesting insight I’ll come back to later.

On the bottom axis - the x-axis - is the way we earn return on software after it’s delivered.

To the left is software where we build it for our company’s use - where we as employees of the company are our own customer. We usually build software to make out work more efficient - to save money, to afford us more time to do other more profitable things. That’s where the value comes from.

To the right is software we build to earn money. This includes packaged shrink-wrapped software where consumers buy our software off the shelf, off a website, at big tradeshows with smiling salesmen in logo-ed golf shirts, or from smooth businessmen talking to c-level execs on golf courses. Basically, people pay money for the software. The more they buy, the more our company earns.

Also included here is software that does our selling - such as eCommerce websites where the website sells our products, or even information websites that include advertising where we earn money by bringing eyes to ads and motivating click-throughs.

And finally, I’ll include here consumer facing websites or applications that although they may not be sold commercially, they have direct effect on the brand perception of the organization that created them. I’ll nudge them slightly left towards the middle on the x-axis.

The more direct the connection between your company’s revenue and the software product, the farther right you’ll find the product.

Look at the user adoption model

The vertical axis - the y-axis - is for the user adoption model. Basically do I, as a user, have a choice to use the software or not? Do I have other options?

On the top of this axis I’ll put software products where the user has lots of choice. Something like on-line shopping sites or the software we use to map directions on the internet. I as a consumer have got lots of choices. If I find myself frustrated, I can quickly surf to another url and try something different. And, in practice, many consumers often do try multiple options. For example, if I’m not happy with Google Maps today, I’ll try Yahoo Maps.

Also in the top, but possibly a bit lower are some shrink wrapped consumer products. I get to choose if I want to buy Microsoft Money or Quicken. But, suppose I buy it, take it home, install it, and start doing my home accounting - then decide I don’t like it… now I may have a difficult choice to make. Do I uninstall it? Re-do all this work? Or do I just put up with the things that annoy me? Unmaking my buying decision is a bit tougher than surfing to a different url.

More freedom of choice for the user of the product places them closer to the top of the y-axis.

On the bottom of this axis I’ll put software where use is compelled… where it’s part of my job - something necessary. Around the middle of this axis, possibly a bit below, are things like Microsoft Office. Face it, if you’re in business you don’t have much of a choice. Even tried-and-true Mac users working in business still need to edit and exchange Word, Excel, and PowerPoint files. Bill’s got us.

Lower on the y-axis are the things we use as mandatory parts of our jobs. The point of sale software baristas type orders into at Starbuck’s, the accounting system our company uses, or the hideous time tracking software many of us are forced to use.

I think you can start to see the shape of things here. So, I’ll bring the user experience thread back in.

User experience practice thrives in commercial software where users have options

It’s not hard to imagine why the user experience practice is relevant here. If your company fails at user experience too often here, it’ll be out of business - especially it it’s got competition.

If you’re a user experience person, and you’re choosing a company you’d like to work for - one that really values user experience, you know where to go: high and to the right. As Guy Kawasaki says, "in one of these charts you wanna be like George Bush - high and to the right." (Please remember it’s not me who says that… I’d never say such a thing.) But, what I mean here is that UX people working in one of these companies will likely have peers, co-workers with the same job title - maybe a whole user experience department! The development process the organization uses will factor in UX. There will likely be a fair bit of user research done. There will likely be a fair bit of iteration on design and testing before we release to users - and then we’ll likely be pretty responsive to user feedback. We know that if we don’t, we’ll be in big trouble. We know that if people don’t really love our product, we’ll all be out of a job.

User experience practice struggles for those building internal compelled use products

Contrast those working "high and to the right" with those working on products in the low and left side of this chart.

This is traditional IT where an internal development organization builds or contracts for the construction of products used internally by the company. If there is a user experience person working here, they’re presence is likely sponsored by some well meaning exec who wants things to be a bit better. But, UX people here are lonely, often unheard, often frustrated. The development process they’re involved in likely doesn’t listen to them when they give guidance up front, then calls them in late after the software is done and ugly. They then ask "can you make this look better?" This is commonly referred to as "lipsticking the pig." Some of you might find this humorous - unless of course you’re the UX person in your internal IT department.

This is why when we look at off-the-shelf Agile development practice, there’s often little or no guidance for including user experience practice. Of course, Agile development has spread to be used in all quadrants of this model. But, if you look closely at the user experience practice in Agile organizations building products in those other quadrants, you’ll see user experience practitioners who’ve adapted their process in some innovative ways. (Note salesforce.com’s amped up application of the RITE method.)

UX gets moderate attention where user adoption and/or revenue is at stake

The other two quadrants can go either way. Sometimes products that fit into these quadrants get reasonable attention, and sometimes they struggle.

High and left finds us still in IT - internal development - but working on software where the people working for us aren’t forced to use the software. I see this with company web portals. At my previous employer I could expect a lot more time and effort to be put into the user experience of the company information repository - the one we hoped everyone would use - as apposed to the company time tracking system where I’ll meet Satan ice-skating before I see that thing get serious attention. Of course I’m exaggerating… I suspect the devil ice skates often.

I’ve worked with organizations where their employees are busy high paid executive consultants or agents for athletes, authors, and rockstars. They’ve had to place importance on user experience, or their temperamental employees will storm out taking their valuable clientele with them. These companies have a process that reflects this concern by including user experience people.

Low and right finds us in commercial software - but in the big package stuff. The products where those making the buying decisions likely don’t use the software. Some of them could have been in on the multi-million dollar buying decision… but I suspect the frustrated employee in accounts payable can’t throw up her hands and say "I’m sick of Oracle Financials, I’m off to Staples to pick up a copy of SAP!"

No one expects an identical process to work everywhere

My point with all this explanation is that the process we adopt for requirements, design, development, testing, and end-user validation all reflects the risks involved.

Sadly, internal IT projects with poor user experience aren’t seen as failures. Now if their late and over-budget - that’s failure.

On the flip side, in the commercial world, failure is measured in dollars - and often felt fairly quickly.

I was at a workshop recently at [[CHI] where we discussed user centered design practice in Agile development. At the workshop was Heather, a UX person working in an Agile environment. She talked about an early project were user experience research was not done, but then said "We now do research with every project." "Why"" someone asked. "That product didn’t sell" Heather said.

Of course you know that this four quadrant model only tells a bit of the story. The domain we’re in will have huge impact on our process as well. Developing medical software is very different than developing software for stock traders, or on-line shoppers, or teenage game players. Recently I ranted about this misuse of the term "potentially shippable software" in the standard Scrum model. In many - if not most - software development environments, it’s unrealistic to ship software after one development time-box - one scrum sprint. It takes time and iteration. Rooting around for a Scrum model slide recently I found Mike Cohn’s deck describing Scrum for game development where the term "potentially shippable software" was replaced by "incrementally improved game." I suspect you can’t sell the idea to some game companies that they could "potentially" ship a new version of Half Life after a 2-4 week sprint. Imagine a game where quality of user experience wasn’t critical? Not since Zork have we been able to pay minimal attention to UX in our games.

Best to market trumps time to market only when there is a market

I’ve heard Alan Cooper chant the mantra "best to market trumps time to market" and he’s right, but that only works when there is a market. A large amount of software being created doesn’t really have a market - at least not one that serves users.

Those of you who work inside an organization that values user experience reflect for a moment on your internal systems, your time tracking system, your internal knowledge portals, you internal accounting systems, expense tracking systems… would these be the products you’d buy if you had a choice?

In organizations where there’s a large development effort focused on internal IT support, I’ll often find a traditional requirements gathering process installed and little involvement from user experience design. But, if I walk across the building to the group that maintains the company website, I’ll usually find a designer.

Alternatively, if I walk into an organization that builds high quality commercial software, it won’t take much probing to elicit complaints about the quality of their internal systems. If I walk over to their IT department that builds those systems I likely won’t find user experience staff.

My point here is that we often know exactly what it takes to build high quality software - high quality from a user experience prospective anyway. But, we often don’t do it even when we clearly know better - when our revenue isn’t at stake, when we know people will use the stuff no matter how crappy it is. Often it’s a "cobbler’s children have no shoes" sort of situation.

One size fits one

Lately when I visit companies I spend a lot of time explaining to them why they’re unique - and they are. They all need a process tailored for them - one that’s sensitive to their culture, and their product risks, and benefits. I’ll still go out on a limb as say that it’s crazy-talk to not include some elements of user experience practice even if you are in a traditional IT world. It’ll save you a lot of time and money thrashing over "bad requirements." And, the poor folks using your crappy time tracking software deserve a break.

Postscript: yes this is a retread

I first drew this model in an old blog post describing the difference between buyers and users. Actually I drew it a bit different, but if you squint a bit and think about, you’ll see it’s roughly the same idea. Read the essay since it points out something slightly different - that is the thought process we go through when buying and how it’s different than the thought process we go through when using. Those of you in an Agile customer or product owner role, remember when you’re prioritizing a backlog that you’re a buyer. Your instincts are suspect when you’re in buying mode vs. using mode.

I’m also pulling back thinking on process and the necessity to take into account your context when putting together a process - which is also a stolen idea from my friend Jared Spool who finally picked up the subject again in his battle against dogma in his recent IA Summit keynote. Take a look at that too.

Shameless self promotion

I’ve been a real live independent consultant since January. I’m thankful that I’ve been consistently busy. But, I’m on a crusade to improve software for the people on the lower left quadrant of my model above. To support that I’ve got a buffet of course offerings that allow you to tailor a class to improve your requirements and design practice.

I work on the right side of the model to help UX people working in Agile development adapt. As I mentioned above, Agile development came from the lower left, and doesn’t have much guidance off the shelf to help these folks. I’m happy to charge a high daily rate to be a shoulder to cry on. Wait, that sounds insensitive. There are actually ways to adapt traditional UX practice to be a better fit, as well as ways to help enthusiastic Agile practitioners understand what UX practice is and why it’s important. This blog article is one of the models I use to explain it to evangelistic Agilistas.

If you’re a UX person struggling with Agile, please visit me at UPA 2008 in June, or at Jared Spool’s UI13 in October.

If you’re an Agile person interested in user experience please visit me and the dozens of fabulous agile-savvy UX people at Agile 2008. I’m really looking forward to learning from the fabulous lineup of UX people presenting light weight collaborative UX practice suitable for Agile environments, and any other environment for that matter.

Agile Product Development

Don’t know what I want, but I know how to get it

Why knowing what you want in agile development may be an impediment to getting it.

It all started with one of those weird trains of thought that come to you in the wee hours of the morning when you’re half way between asleep and awake. The first lines of the Sex Pistol’s Anarchy in the UK song were playing in my head. (This may be a hint at both my age, and the type and volume of music I listen to.) On that morning, Johnny Rotten’s words seemed particularly wise - and seemed to precisely describe a recurring problem I’ve had helping people really grok Agile development. Shortly after declaring himself and antichrist, Johnny says:

And, why this is relevant is because I constantly run into a couple problems that make my spider sense tingle. In software development Have you ever heard something like this?

If you felt a shiver, that was your spider sense tingling. The other problem goes a little like this:

In short strokes, I run into situations where people on the specifying side of software development, "customers" or "product owners" in Agile terms, either believe they know what they need, or feel they need to know before we can start development. What’s more, I still run into a number of developers in Agile environments with the same old annoying complaint about "customers not knowing what they want" or "customers always changing their mind."

All these sentiments for me seem to spring from not knowing what "iteration" means, and is used for in Agile development.

Iterating and incrementing are separate ideas

I most often see people in Agile development use the term iteration, but really they mean increment.

By incremental development I mean to incrementally add software a time. Each increment adds more software - sorta like adding bricks to a wall. After lots of increments, you’ve got a big wall.

By iterative development I mean that we build something, then evaluate whether it’ll work for us, then we make changes to it. We building expecting to change it. We never expected it to be right. If it was, it’s a happy accident. Because we don’t expect it to be right, we often build the least we have to to then validate whether it was the right thing to build.

I’ve used the two figures above for a number of years now to help illustrate the concept. Artists work iteratively. They often create sketches, decide to create a painting, create an under-painting showing colors and form, then eventually begin finishing the painting. They stop when it’s "good enough" or they run out of time or interest.

Paint-by-number artists work incrementally. When I was a kid I did some pretty good paint-by-number art. The problem with paint-by-number art was that some real artist had to actually paint the thing, figure out what all the colors were, then draw all the lines and number the areas – which takes more time than just creating the painting. Using a strategy of only incrementing means you more or less have to get it right the first time.

We Iterate for multiple reasons

After talking about iteration during a talk at XP Day 2007, someone correctly pointed out o me that it wasn’t as simple as "changing things" each iteration. He pointed out that:

• we iterate to find the right solution.
• Then given some good candidate solution, we might then iterate to improve a candidate solution.

We Increment for multiple reasons

We add to software incrementally for lots of reasons as well.

• We use incrementing to gradually build up functionality so if development takes longer than we expect, we can release what we’ve incrementally built so far. ("If" is in italics because I honestly can’t remember a project I’ve been on where development took less time than expected.)

• We release incrementally so that we actually get that business value we’re chasing. Because, we don’t really get return on investment till people begin to use the software we’ve built. Until then, the expected business value is just an estimate. And, if you think software development estimation is tough, try estimating return on investment.

We conjoin iteration and incrementing

In Agile development we actually conjoin these two tactics. During a development "iteration" where build several user stories some may be adding new functionality incrementally, others may be iterating to improve, change, or remove existing functionality.

Where things really fall apart in Agile development is when no one plans to iterate.

The gun-shy customer

Perhaps you’ve been on this Agile project:

Customers meet with the team and successfully write a number of user stories. After a lot of conversation between developers and customers, developers estimate the stories. Customers prioritize them, highest value first, and choose the most important stories for the first release scheduled after six iterations.

Development starts, and things seem to go very well. In the fantasy world this story occurs in, all the development estimates were accurate. In the first couple iterations all scheduled stories are finished. But, that’s where things go wrong.

After looking at the resulting software the customer says "Now that I see this, we’re missing a few things. And, although the things you’ve built meet the acceptance criteria, we, well.. uh… weren’t really sure about that acceptance criteria and now that we see it, it needs to change."

"No problem" says the team. "Just write more stories. But, you’ll have to remove some of the others from this release in order to get them done on time."

The customer’s shocked and angry. "What you’re saying is that I needed to get the requirements right up front! This smells just like waterfall - accept without the up front time I’d need to even try to get the requirements right in the first place."

I’ve worked with these teams and customers many times. I know of many organizations where "Agile Development" has been labeled a process that simply doesn’t work and ejected from the organization.

I know of other customers who’ve adapted by spending more and more time on requirements. They’ve introduced prolonged "Iteration 0" or "Sprint 0" phases where they actually write those big requirements. They work 1, 2, or 3 iterations ahead to really craft the details of their stories before they get introduced. They try hard to get them right. And, when inevitably they fail to get them right, they’re deflated, disillusioned, disappointed - and any other "dis" you can think of.

It’s not their fault. They were mislead.

It doesn’t mean what you think it means

There’s a nasty little phrase Agile people often use. They often say "at the end of every iteration you’ll have potentially shippable software." The commonly used Scrum Snowman model that all the tens of thousands of certified Scrum Masters saw clearly says that.

In the movie the Princess Bride one of the villains exclaims "Inconceivable!" each time one of his plans is thwarted by the hero. It happens so often that one of his sidekicks says "You keep saying that word. I do not think it means what you think it means."

For a customer, someone who intends to sell or use the software, shipable means they could actually sell and use the software. This means the minimal number of features all need to be present. The software needs to be useful for it’s intended purpose - at least as useful as the old software or paper process it replaces. The software needs look and behave well - have a high quality of fit and finish - particularly if this is commercial software and you’ve got competitors breathing down your back.

Shippable means done. Completely done and dusted. There’s no need to iterate on something done - really shippable done.

Saying "shippable" to people in the customer role means implies they’d better get the requirements right because that’s the way Agile development works.

Now, I believe Agile people had something else in mind when they said it. I think they mean keep code quality very high. Keep the code supported with unit and acceptance tests. Take steps to validate each and every user story. It tells testers to get involved earlier and more continuously. It tells developers to develop with a higher attention to quality. (Apparently developers would be developing crap otherwise?)

YAGRI: You aint gunna release it

I propose we, the Agile community, clarify what we mean by iterative and incremental. We need to explain to those in customer and product owner role that it’s important to write user stories that they don’t intend to release. To write stories that they intend to evaluate, learn from, improve, or toss out as a failed experiment.

In conversations with my friend Alistair, he proposed writing three story cards instead of just one. The first story card has the actual story on it. The second one is a placeholder for the inevitable changes to the story after we see it. The third for the fine-tuning after we see those changes.

This is an example of planning to iterate. It would relieve a lot of stress from the hand-wringing gun-shy customer worried about getting it right because the story needs to be "shippable."

You can always get what you want, but is it what you need?

Where we can apply Sex Pistols lyrics to software development, we can’t necessarily apply the Rolling Stones.

In software development, sadly if you specify something, and everyone is doing their best, you’ll get what you want - at least what you specified. But, is it what you need? You’ll only know after you look at it and try it.

Don’t listen to Mick.

In fact, try hard to not be too sure about what you want. If you leverage iteration, you’ll get it even if you didn’t know what it was to start with. Johny’s got it right.

"Don’t know what I want, but I know how to get it."

Please leverage the explanation if you’d like

This is a bit of the story I told during my XP Day 2007 talk Embrace Uncertainty. It’s rare when you get to quote Johnny Rotten, Roger Waters, Paul Simon, Pete Townsend, John Lennon, and the Spice Girls in the same talk.

Feel free to download the talk.

Here’s the talk with music clips.

Feel free to use the examples using a creative commons license. Let people know you borrowed them from me.

If you’d just like the Mona Lisa slides, you can grab those here.

The general ideas here are written in a StickyMinds.com article with a little less ranting. You might share that version with your boss.

Stay tuned

If you want to get more about specific strategies for iterating sensibly in Agile development, please visit me at a tutorial I’ll be teaching at a conference. Also pay attention to this site and blog as I resurrect my long overdue book from it’s current purgatory.

Finally, if you’ve read this blog in ThoughtBlogs (and my web analytics tell me most of you do) this may be the last time my blog appears there. Please subscribe directly, or look for me on ThoughtWorks alumni blogs. I’ve had a great time at ThoughtWorks for the last several years, but it’s time to set out on my own.

Thanks for reading.

Agile Product Development

Kicking off the slow software movement

why our rush to build software faster sacrifices quality and treats software like fast food

In August 2007 I was lucky enough to be able to edit an issue of Better Software Magazine. I was able to choose my authors: Alistair Cockburn, Jim Highsmith, and Jim Shore. I did this because I was lazy – I wanted great articles without a lot of effort. Choosing these authors guaranteed it.

In this issue, the Technically Speaking column from the editor allowed me a page to rant about what was important to me – namely the tendency for software development to dig too quickly into building software before being clear about the context their software will be used in, the goals it should reach, and the problems it should solve. I tried to compare too fast software to fast food, and better software to slow food.

The short article describe a conversation I had with a friend, the CEO of a company I spent many years working for. He described the propensity of his people to move fast – to get in, start capturing requirements, and start building software. In a lot of organizations that might be admirable. But for him this was often catastrophic. It resulted in the rapid development of the wrong software, and software that was less than desirable.

What was going on to him seemed like the rush away from building something of high quality to something fast and cheap. Like the difference between fast food, and something you’d pay more for from a fine restaurant.

It’s notable that when I say low quality that I’m not talking about bugs. I’m talking about subjective quality – the stuff that makes us like a product and feel good about our decision to buy it. See my short article on subjective quality on StickyMinds.com.

Immediately after the article was in print, I received a couple odd comments. One from a friend who said “I love this article, but I could never show it to my boss.” He went on to say that his boss would frown on any implication that things should take longer. I also received email from someone asking what the Agile community had to say about this. That’s why I’m writing this. If you’re reading this, you’re either in the Agile community, or listening in on what’s going on there. I’d love it if you could read the article, and give me some feedback.

I know I’m not alone in my sentiment. My friend Alistair recently sent me a link to this old blog entry from Frank Sommers discussing my coworkers Marc McNeill, Dan North, and Martin Fowler. Alistair said “other people are thinking like you.” In the blog entry titled Valuing Outcomes over Features Frank pulls together several conversations to talk about a possible 5th phrase in the Agile Manifesto. The point is that’s it’s a bad idea to focus on the scope we intend to build instead of solving problems, reaching goals, and generally delivering something people want and can use.

This is a “mom and apple pie” statement. No one wants to be responsible for building bad software. But, that said, I often don’t see people putting techniques into practice that stop it from happening.

For example, I teach paper prototyping and lightweight usability testing at a variety of conferences. (I’ll be teaching at OOPSLA and Better Software’s Agile Practices Conference if you can make it.) I know from personal practice that adding the techniqe into an Agile practice ultimately saves time and increases quality. Another friend, Gerard Meszaros, put paper prototyping and lightweight usability testing into practice on one of his projects. In a paper written by he and his customer for the Agile 2005 conference he describes what happened. Basically Gerard says that in addition to increasing customer participation and collaboration, the amount of rework they had on the project to change features went down significantly after the addition of the practice.

By “slow software” I don’t mean wasting time. I mean what’s meant by slow food. Focus on quality of delivery over speed of delivery. Hey, is that a 6th phrase for the Agile Manifesto?

Again, I’d love it if you could read the article, and give me some feedback.

Agile Product Development

Struggling to get into Shu business

Why it’s taken me years to explain simple things simply

Shu Ha Ri explains how we learn

I’ve often found the terms Shu, Ha, and Ri useful to refer to the three stages of learning Alistair Cockburn described in Agile Software Development. I’ll usually explain that to learn something we start at the Shu, or "following" level. Here we are focusing on learning a simple set of instructions for a technique and consider a particular learning experience successful if we can follow that set of instructions and realize success.

Later we’ll learn that simple techniques have limits, and we’ll learn to vary the technique or combine it with other techniques to achieve success - that is in the Ha or breaking away stage.

Finally I explain that many experts find themselves in the Ri stage - or fluency. There they freely improvise, mixing and matching techniques and inventing new techniques - tricks I’ve heard them called - because these invented techniques are tricks until we can explain them and teach them to others as somewhat repeatable techniques.

So, that’s where the complication comes in.

Shu Ha Ri doesn’t help us find those simple steps - it only lets us know we need them

I’m lucky enough to have learned quite a few things from a number of experts. I’m serious about that lucky part. Not everyone gets those opportunities to learn first hand as I have, so I consider myself very lucky. As a result of learning techniques from user-centered design experts, specialists in more traditional requirements work, and Agile experts, I’ve acquired an odd fusion of techniques swimming around in my head. Those, combined with my own unique perspective and experience, have resulted in the tricks that I feel pretty proud of.

Sadly, I have to call many of them tricks, and my tricks in particular, because I struggle to explain them. That is to say, I struggle to arrive at a step-by-step Shu process that I can explain simply, and that people can learn and repeat.

I’ve heard Alistair explain that once you reach fluency level - or Ri level, you sometimes forget how the simple Shu steps worked. But, for these tricks I’m talking about, there never were any Shu steps - at least I never learned any. Because they’re an odd blend of a number of techniques, and a few happy accidents in the context I was working in at the time. I never learned useful language to explain them. And, for the group I was collaborating with at the time, the improvisations seemed obvious. They didn’t need explanation either.

But, I’m happy to report some success.

Finding Shu steps that work takes years of trial and error, but eventually it pays off

I write this at 12:45am from the way-too-crowded waiting area in the Bangalore airport. I’ve just taught a 4 day class to a group of folks here in India - and happily it’s felt more successful than any I’ve taught prior. Nothing feels better than finally getting the language right, coupling that with some simple exercises, and seeing the people in that class really get it. I can see in their words, body language, and movements as they perform the techniques that they’ve gained more fluency more quickly than others that I’ve taught.

Actually, as I write this, I wonder if we should all remember that those in India doing IT work are pretty darn smart. If we want to stay competitive in the US, or wherever we live, we’ll have to do more than simply "be collocated" with the business.

But, back to my story.

For participants, learning a new technique is a deceptively small thing. An instructor explains something. You listen and it makes sense. You try out a technique in an exercise, and it works. What’s the big deal?

But, it’s taken me years - really four years now – to figure this out. I’ll need to write a public apology later to the people who endured some of my earliest classes starting in 2002. Very rough Shu at that time. I almost obfuscated techniques rather then clarifying them.

This time though, I saw my small class of nine pick up the necessity of understanding and modeling business goals. During exercises they poked each other with the "why stick" - to find the real business goal behind the simplistic "build some software" goal usually expressed in software development. Then, they used the goal-question-metric technique to identify meaningful measurements to detect business success.

Those business goals come in handy. Much of what I do is informed by user-centered design thinking and techniques. But, I’ve found that if you don’t balance user-centricity with business-centricity – or person-who-uses-it with person-who-pays-for-it – you don’t succeed in delivering the value you intend to. Well articulated business goals provide goal-alignment for stakeholders and spell out pretty simply to those on a software design and development project where the business value should come from. It seems to help the business people when you can remind them where they believe business value comes from. Software development is tough for everyone involved.

Many in the Agile software development world labor under the false assumption that working delivered software = business value. Actually business value generally comes from using software. Building and delivering it is simply cost until then.

If we know that the best way to get value from software is from using it, then it’s a good idea if we start focusing on users and use as soon as possible. I find my head a little thick at times. I finally really grok Larry Constantine and Lucy Lockwood’s reason for calling their design approach Usage-Centered Design - because it’s the usage (stupid) that earns us the value.

But I digress.

The group I worked with built user models - my neutral way of saying I don’t care if it is an actor, user role, profile, or persona - just figure out who’s using your software and why. Write it down and leverage it.

We moved through modeling use using user tasks and workflow models. They wrote clever user scenarios, and used them as a foundation to create software user interface using paper prototypes. They leveraged the task workflow model and experience from designing Ui to create an incremental release plan. One that delivers something at each release that users of the application can actually use to meet their goals - which of course is mandatory for business stakeholders to reach their business goals.

Honestly - if I hear "prioritize your user backlog by business value" one more time, I’ll scream. It just doesn’t work. It’s not that simple. Usage needs to be more explicitly taken into account. Although simple prioritization sounds like a set of Shu instructions, it isn’t - because the person practicing that approach needs to realize some success. I won’t rule out that this approach has been successful for some - but it just doesn’t work for me. That Shu just doesn’t fit.

I’ve gotten off topic again. I’ll get back to my original rant.

Arriving at Shu instructions for a new technique requires testing

My real point with all this is that identifying, describing, and teaching new techniques is tough. The toughest part, at least for me, has been finding language to explain them. To find that simple language I’ve had to start with a best effort, then try it out. I continue to adjust it until it seems to work. That’s been my process for getting back to Shu.

Shu-Ha-Ri is a useful metaphor for explaining how people learn. But I’ve started to realize that there’s a bit of a process to follow to get to Shu instructions - getting oneself into Shu business so to speak.

If English is your first language, or one you’re pretty strong at, you’ll get the "Shu business" pun. If you’re old enough to know who Ed Sullivan is, you’ll get it twice.

And, here’s the self-serving bits - as if blogs were’t self serving enough.

I’m happy to be teaching a first two-day public course for the Software Productivity Center on Agile Product Management on July 30th and 31st 2007. My goal with the course is to introduce the simple modeling, planning, and tactical Agile software development management approaches that I believe are best practice. It’s not called an Agile requirements class, or an Agile customer class, although you could think of it as both or either.

As I said above, value comes from using software, not just delivering it. Agile development approaches are great delivery mechanisms, the very best as far as I’m concerned. But, it’s the product - the thing we’re delivering that’s the most important. The SPC class focuses on figuring out what to build, and how to work with those who know how to build it. Assuming developers and others in Agile development know how to build software right, this class is about how to build the right software.

I’ve got upcoming half day-classes at Dr. Dobb’s Architecture and Design World, Agile 2007, SD Best Practices, and OOPSLA. If you’re attending any of these conferences, please come say Hi. If you’re willing to start an argument, I’ll pay for coffee while we talk.

(If you want to start an argument and your name is Ron Jeffries, I give up. )

Upcoming teaching events:

Agile Product Development

Dirty hands required

Why user experience people need to roll up their sleeves and get their hands dirty in development to get the best results.

In an email exchange with a friend this morning he was speaking about his involvement on a soon to be released product. He’s been working on that product to design its user experience. He’d recently made a tactical change that I’m beginning to consider critical for the success of a software product.

In support of my friend’s user experience work he’d built user interface prototypes, storyboarded usage scenarios, and validated those prototypes with prospective end users. All seemed well. But something funny happened on the way to implementation. The team members building the product had their own ideas about what the product should be. And those considering the direction of the product believed that a strategy change was in order. If you’re a fan of Garrett’s Elements model, you know that a strategy change changes everything.

There wasn’t time for the team to go back an re-create UI design. The team was composed of really sharp people. I know most of them, and they truly are rocket-science smart software designers and developers. They succeeded in creating a product that was technically sound and innovative, but only sorta usable, and, well, not so sexy.

Meanwhile, the release date looms closer.

Give up and get started

After several rounds of reviewing the product and giving specific feedback on how to change it to make it better, the decision was made to give up.

By give up I mean for my friend to give up on giving feedback, roll up his sleeves and get his hands dirty in the development of the software. For him this meant taking ownership of CSS, directly editing XHTML, and where necessary editing underlying Ruby code. His hands were sorta dirty before - but much dirtier now. "I’m feeling much more positive [about the product]" my friend reports.

My friend’s story is one I’ve hard, seen, and been a direct participant in many times - enough that I’m willing to declare it a critical success pattern for product design and development.

The tactical technical UI design pattern

Where I’ve seem product design be most effective, User Experience types are directly involved with day to day development.

They work on the requirements side to help define what the product does. They collaborate with developers to help them build functionality. No one expects the developers that build functionality to make it work well and look and feel good to use. The look and feel good part is done by a tactical, technical, user experience person.

This tactical technical UE person might be a developer by trade, or UI designer with enough technical background to load up the development environment and directly manipulate code to massage the user interface.

You really gotta do this.

The truth is that it’s tough to predict exactly how the real code will behave once you start using it - see it animated and under your control as a user. Details about what should be in the user interface always arrive late - details that affect the layout and page or screen design. And, there’s always late breaking great ideas. Stuff you thought of in the morning shower, or saw a competitor do in their website. There’s no reason to follow your original design, if you’ve discovered something better.

What’s more, a user interface design conceived by someone without detailed understanding of the tools used to construct it often suggests good, but expensive to implement designs.

There’s a sweet-spot between good, and expensive to build that we’re often looking for. Ideally, we hope to find a design that’s best to use and cheapest to build. Without having a healthy understanding for how to build it, how could one find that spot? One would hope aggressive collaboration - even pairing at the development workstation might help.

The left-brain and right-brain in separate heads anti-pattern

I’ve often observed the "smell" in projects where the abilities to design the outside of the software, and the abilities to manipulate that design in code don’t exist in the same head. I’ve seen lots of hours wasted, and lots of frustration trying to communicate the importance of moving this label 5 pixels to the left. I’ve seen developers angry when a designer asks them to relocate an element of a screen, only to reverse that decision after seeing it.

Sadly, fine grain design, the stuff that really makes a big difference in the perceived quality and actual usability of a product, requires lots of trial and error.

I have seen successful organizations where the designers were indeed separate people from the developers. But, the developers in these sorts of organizations, measured against the average developer, are strong designers. They can make fine-grained - 5-pixel-to-the-left - adjustments without prompting. They understand design well enough to be tolerant of the trial-and-error cycle. They understand design well enough to make changes based on very little guidance from designers. They understand design well enough to bring their own ideas and innovations to the product. While their left-brain may be a bit heavier than their right, their right is strong and fully engaged - and supported by design knowledge, skill and practice.

Good UI design can’t be communicated in a document

I don’t believe a good UI design can be communicated in a document. It may look good on paper. But the translation of that design to a working product will always result in some change - changing not easily predicted and written in the document.

Staff a tactical, technical designer

If you want your product design and development to be effective, put someone on staff who’s both a strong designer and technically capable of making design changes to the product directly.

If you’re a frustrated designer, look for a way to roll your sleeves up and get your hands dirty. Can you learn the technical skills yourself? Can you collaborate with a developer who has the right raw material to become a good designer?

If you’ve had success with your product by having tactical, technical UI designers on staff, I’d like to hear about it. If you’ve had success without them, I’d like to hear about how your process works to accommodate that. And, if you struggle with the connection between UI design and product implementation, I’d love to hear some horror stories. Please get in touch with me.

Agile Product Development

Requirements considered harmful

Why requirements in software development harm collaboration, stunt innovation, and threaten the quality of our products.

I wish I could take credit for first publishing a quote like this. But it was Kent Beck who actually did in his 2nd edition of Extreme Programming Explained.

Let me see if I can explain why the word "requirement" is just plain wrong.

When faced with challenges or in pursuit of a goal we decide on a course of action

I suspect you’re reading this because you’re somehow involved with software development, and Agile development in particular. In software development, we build tools for other people to use. When done well, the software we eventually release will be used by people to overcome a challenge or reach a goal. If the software doesn’t do that, it was unsuccessful.

For businesses, challenges or goals often boil down to increasing profitability. Sometimes that takes the form of building more appealing products to sell, or supporting a necessary process to make it more efficient thus reducing costs.

There’s lots of ways to make money, and lots of ways to reduce cost. So, exactly how we do either takes a bit of thought, deliberation, and finally some decision on a course of action.

The software we eventually build is the compounding of a great number of decisions

If the motivation for building software originated as some business objective, the person with that objective made a decision that placed software somewhere in the solution.

Assume for instance I’d like to build a new consumer product. I’m doing this because I’ve identified a need in the market - a group of users with an unmet goal, or a goal I believe I can meet better. I’m not in this purely for fun, so I need to make money selling software to meet their goal.

The product to build and user constituencies to serve were decisions I made.

I then need to make some decisions about what the product should do. I might look at the products competitors to see what they do. I might look closely at my target audience to understand what their problems are. I want to understand how my software will solve their problems, and very importantly, how many features it needs to have to convince them it will solve their problems - otherwise they won’t buy it. (I tried to get at the idea of features that support buying decisions vs. features that support use in designing software for two-headed people.)

Once I’ve decided on the functionality I need to have, I need to make some decisions about how people might use the software. For the software to support my user’s work well, I need to understand that work, and make decisions on how to organize screens and stuff in screens to best support that work.

My users are a picky lot. They expect the software to be esthetically pleasing, so I’ll make decisions about how the software looks and behaves keeping my customer’s esthetics in mind.

I can’t please everyone, so I’ll have to make some tough decisions. I might have chosen the wrong work to support - and consequently the wrong features. I might have misjudged my customer’s esthetic sense and made choices that make the product not so desirable.

When I eventually sit down to write code, I’ve got to make lots of decisions about programming languages, databases, client type - browser based or rich, etc..

I still need to make lots of fine grain decisions about fonts on screens, text in error messages, and layout for stuff that prints.

I’m getting nervous. I’ve made so many decisions that depend on each other here. Most of these decisions were made on the back of previous decisions. How can I know if they were good decisions?

I don’t intend to develop this software myself, so I’ve hired a development team. The development team asks me for my "requirements." I assume they’re asking me for marching instructions - for me to tell them what to build. They must be referring to all the decisions I’ve made up till now - and I’ve made a lot of them.

"Requirement" is a label we put on our decisions

These were decisions made by fallible humans. The may not be correct.

The reason requirements in software development are hard is because they are decisions. The reason that so often they need to be run by everyone is because everyone wants in on the decision. The reason they’re so hard to validate is because given the same facts, different people make different decisions – let alone given different facts. The reason requirements change is because decisions change. They change when we learn new facts that lead us to believe we could have made a better decision.

Decisions and requirements build on each other

In my story above, my choice of user constituencies was based on the product I believed I could create and the need I believed I could fill. The functionality was based on what I believe those people need. The technical decisions on programming language, browser based or not, database, stuff like that were made based on who I believed my users were, how I believed they’d use it, and the technology choices I understood I had at the time.

In the requirements world functional requirements are based on business requirements, technical requirements on functional requirements.

Whether we’re talking requirements or decisions you can see that if I’ve got a bad one upstream, the one’s based on it will be bad as well.

Giving requirements stops collaboration

Let’s look at the word requirement, and the word decision. I’ll throw them out along with a brainstorming of a number of words that I’d place in the same category.

requirement	decision
mandate	solution
order	problem
necessary	goal
indispensable	creation
non-negotiable	hope
indisputable	idea
contractual	discussed
final	reversible
blame	made by people
irreversible	attribute-able
absolute	explainable
consider as fact	fallible
change is consider to be process failure	change is natural over time

I guess it’s not surprising that the lists vary. Aside from the obvious reason: that I made of the lists and I’ve obviously got an axe to grind, "requirement" and "decision" aren’t synonyms. Both words characterize a piece of information. "Decision" describes an idea arrived at by a person or group usually after some consideration of multiple options, where "requirement" describes a contractual arrangement between one person or group and another.

Requirement is the word I label my decision with when I want to request you do something. By placing the word requirement on it, I let you know that the decision’s been made, and you needn’t concern yourself with it, other than to meet my requirement. Any conversation we have will be about the details of my decision. We’re not working together to make it - it’s been made. It’s a requirement.

But, calling it a requirement gives me an uneasy feeling. These were tough decisions to make, and although I feel mostly confident in them, they could be wrong. By labeling them requirements, I now feel they must be right.

Asking for requirements avoids responsibility

Said by Fred Brooks in his 1987 essay "No Silver Bullet." Notice Fred didn’t say the hardest part was the requirements, but the deciding. He does however say this in a discussion about requirements where he later goes on to say "Much of present-day software-acquisition procedure rests upon the assumption that one can specify a satisfactory system in advance, get bids for its construction, have it built, and install it. I think this assumption is fundamentally wrong, and that many software-acquisition problems spring from that fallacy."

Since it’s difficult to decide what to build, and since decisions are based on decisions, it’s often likely that in a pile of decisions there’s a few that are just bad decisions. Figuring out which are bad is difficult. By difficult, I mean darn near impossible.

Asking for those decisions as requirements avoids the difficulty of figuring out which are wrong by placing all the responsibility on to the person what made them. By calling them "requirements" we ask this person to take full responsibility for their decisions. We’re not collaborators. We’re not here to help.

Taking responsibility for one’s decisions is usually a good thing. However requirements documents have an aptitude for anonymizing decisions - for obfuscating the person or people who made them, and the motivations for those decisions. Sadly, the net result is no one taking responsibility. And often no one to even ask exactly who decided this was a requirement and why.

Even sadder is when the wrong product is built as a result of failing to understand why specific decisions were made.

Adopting a process that uses "requirements" adopts the assumption we could identify correct requirements, and the resulting risks taken on with that assumption. "Requirements" are risky things.

Requirements come from outside, not inside

Now, at times, there is a place for requirements. Healthcare, banking, and a variety of other industries all have regulatory requirements. Not meeting these can result in bad consequences for our product.

But, these requirements are decisions just the same. Just like other decisions sometimes they’re bad. Sometimes they’re reversed. Sometimes they change.

It may not help much to know the basis for these decisions made by others resulting in requirements for you, since you can’t likely changes them if they’re bad decisions. (Or can you?)

These sorts of requirements come from the outside. These are decisions that aren’t easily questioned, explained, or sometimes even understood. They may be a necessary constraint coming into our organization from the outside.

There’s no reason to stand for the same level of inflexibility or lack of collaboration inside our organization.

Stop using that word and start collaborating to solve problems

At the end of the day, it’s not the word "requirement" that does damage to software development, it’s the ideas and process that often come with it.

My concern is the lack of collaboration that often goes into decisions. Or some using the concept to avoid really understanding the people who will use their software and the problems their software solves. Or, evaluating the product you choose to build on the basis of requirements being met, and not on the bases of user or business objectives being met or problems being solved. In the end, everyone in the organization pays.

I often host collaborative requirements worksessions. (Yes I do use the word.) I’ll often start worksession by writing on a whiteboard "requirement = decision." I’ll start by saying "We’re hear to understand out goals, and together make the best decisions on how to reach them."

I’ve seen environments where "requirements" were used in a process-healthy way, where requirements were considered more as decisions, and participants were aware of the details what went into those decisions. But these environments are the exception rather than the rule.

The decisions we make about our software should never reach the state of "mandate." They should never be unquestionable. They should never stop being measured against their ability to effectively solve business problems. They should always be recognized as fallible.

If I had my way, we’d stop using that word. The baggage is too much for me.

But I don’t know what word I’d replace it with. Perhaps user story? But, that uses one of the other unfortunate words in software, "user." And what about all the decisions I make that aren’t so closely related to the person using the software?

I’m using the word "decisions" or the phrase "design decisions." Try out another word around your company and tell me what works for you.

Agile Product Development