Business, stakeholder, system and technical requirements - More thoughts

On the Requirements Engineering Newsletter, Roger L. Cauvin wrote (responding to something I wrote):

Keith Collyer (keith.collyer@uk.ibm.com) wrote:

> System requirements define (but not design) the solution to the
> problem . . . . [T]hese requirements define what the system must
> do to provide the desired capability.

I agree with much of what you wrote in the rest of your message, Keith, but this particular statement highlights again how oxymoronic requirements terminology has become.

How is it possible for so-called "system requirements" to define but not design the solution to the problem?

Let's say one of my problems is that it takes me too long to do my taxes. I challenge you or anyone else to "define what the system must do" to solve this problem without introducing any design choices. Is there anything that the system truly MUST do to solve this problem, aside from create a condition in which it no longer takes me too long to do my taxes?

Here is my response:

the way I think of this is quite old-fashioned, in terms of function, the business requirements define why a system should do something (what I want to achieve), the system requirements define what the system must do to deliver what I want to achieve, and the design defines how the system does it. Of course, quality / constraint / non-functional requirements need to be taken into account throughout.

So, if your business requirement is "reduce the time taken to do taxes", then I would say that you need to do some analysis to understand what this means, still in the business area. Let's say that it breaks down into three activities (please don't argue with these, they are intended just as examples):

• collect income and expense information
• calculate tax
• complete tax form

These are still stakeholder requirements, as they are decompositions of the original requirement and still reflect what you want to achieve, within the context of the environment in which you work (in some jurisdictions you calculate your tax, in others the tax authority does). So now work out which of these is costing you most time and you know where you can save most time. At this point, you are still defining the problem. Say the place you can save most time is in calculating tax. Now I can say what a system must do, it must calculate tax, faster than I could do it, using the collected information. I'm not saying anything at all about how it calculates tax.

This system requirement happens to be a fairly simple restatement of the decomposed stakeholder requirement, changing the focus from what I want to achieve to what the system must do - and that is common, but not inevitable.

Powered by ScribeFire.

Business, stakeholder, system and technical requirements

Terminology bedevils requirements. So here are my views on some common terms.

Business requirements generally define the problem to be solved. The general term we used in Telelogic's Professional Services was "stakeholder requirements". Some people divide these up into levels and differentiate Business and Stakeholder requirements, but they are really just different levels of problem definition. For example, the CEO's view of the problem is necessarily less detailed than that of the department heads. In terms of (at the risk of re-opening a different debate) "function", these requirements define what the business (or stakeholder) wants to achieve - the capabilities. But they must also define all those other types such as performance, security, usability, etc. - and if you don't like the term non-functional for these, then we used "constraint". It's not perfect, but I think it worked better than non-functional without some of the confusing connotations of "quality requirement".

System requirements define (but not design) the solution to the problem. Again, there can be (generally are in a non-trivial system) several levels, for systems, sub-systems, components, etc. In terms of "function", these requirements define what the system must do to provide the desired capability and we were quite happy to call these functions. Again, there will also be constraints in the system requirements.

Stakeholder and system requirements, especially constraints, can be identical. For example, physical constraints, such as a maximum weight, are often identical.

Oh, it is also important to note that you can't always (probably not in general) separate constraints from capabilities or functions; most (but not all) constraints are tied to a single capability (function) or a related set.

Powered by ScribeFire.

Agile versus the traditionalists - Part 1

I have called this Part 1 because I suspect know that this won't be the last entry in this blog on the topic. It's just too big.

Let me make my position clear. I am firmly in the "traditionalist" camp. But, having said that, I think that there is a lot of value in agile approaches. I'm certainly not going to dismiss it out of hand. Of course, some practices associated with agile (like pair-programming) have proved to be less effective than the original proponents claim, but there is a lot to be said, in many environments, for ideas such as test-first design, use of backlogs, user stories, etc. But there is also a lot to be said, in many environments, for having documented requirements specifications.

For an excellent description of what I am talking about, see David Parnas's classic paper "A Rational Design Process, How and Why to Fake It" in (D.M. Hoffman and D.M. Weiss, Software Fundamentals: Collected Papers by David L.Parnas, Addison Wesley, 2001). Even though you didn't really develop your product that way, you should make it look as if you had a chronological flow of development artefacts (for example, stakeholder requirements, system requirements, architecture, etc.). Why is this important? Well, firstly, the development team won't be there for ever. So it won't be possible to just walk over and ask Joe. Joe is now working for your competitor. But someone still has to maintain the system, so you need some way of recording the common knowledge. But we have all our user stories, you say. Unfortunately, a list of user stories does not always give a sufficiently clear overall picture. Sometimes, you need to be able to understand the whole system at various levels and user stories just don't do that.

So it may still be necessary to document requirements. Note that "document requirements" does not necessarily mean the same as "requirements document". But a requirements document does allow an overview of all requirements in a form that is familiar to people.

But isn't there an overhead in doing this? Sure, there is. The question, though, is: does the investment pay off? Well, if you are delivering a "small" system, say up to a couple of hundred requirements, it may well not. Such a system probably only has at most a few tens of user stories, and experienced people can hold these in their heads. This sort of development is ideal for agile. But when the numbers of requirements starts to creep up it becomes essential to be able to see the big picture. Another situation when the need for a requirements document occurs, even with a realtively small requirements set, is where the requirement specification forms part of a contract. Software has the great advantage here of being incredibly malleable, and also usable even if not complete. Hardware systems generally have neither of these characteristics. Even something as simple as a garden shed to hold tools would be useless without one of its major components, without walls the weather and theft protection is lacking, without a door you can't put things in or take them out, without a roof you lose weather protection, without a lock you lose theft protection, and so on.

So where is the gain in creating a requirements document? Simply, it presents the requirements in a consistent, navigable and understandable way. So, if something needs to change in the future, it is easy to see where that change needs to be made. This means that it is possible for people who were not involved on the original project to make changes in the full understanding of what these changes mean.

And more: if you don't just create requirements documents, but use a requirements tool, you can also see what information is linked. So when a stakeholder says a requirement needs to change, you can see what effect that has right down to implementation level. Simply having a set of user stories, without the linking, does not allow this. And naïve use of user stories will make it hard to use common components (I have seen this happen in practice, leading to a system that we estimated at being around three times as large as it needed to be because people re-implemented).

So, what do I recommend? Surprisingly, perhaps, I would say "use agile". But use it with your eyes open. Use it where appropriate. And don't be browbeaten into not documenting your requirements just because that is the fashionable thing to do.

What the agile community doesn't do with requirements

I posted this in a response to a message on the Requirements Engineering mailing list 081012. It has been mildly edited for the blog.

Here are a couple of examples of things that the Agile community tends to ignore (or perhaps does not realise) about requirements:

• Use cases (or user stories) are not requirements. Each can lead to many requirements, many of which are shared by other use cases.
• Users often don't know what they want
• What users want is often not what they need
• What users say they want is often not what they really want
• Users aren't the only stakeholders (nor are customers, the only other group commonly mentioned, nor is the combination of users and customers)
• It is important to record the requirements, why they are there and where they came from. Joe at the next desk who wrote the code based on a user story may be working for a competitor in six months (as could the customer whose user story it was). Then you can't use the "wisdom of the team".
  

There is a lot of good in the agile approach, but we must not forget that it came from the coding and design community, who tend to assume that requirements are things done by other people, if they get done at all. Therefore, not unnaturally, it is light on requirements thinking.

Post in Requirements Engineering mailing list

This is a response to a post on the Requirements Engineering mailing list about the need to understand what the customer wants to do. My comments are embedded:

Roger Cauvin wrote on 01/10/2008 16.08.02:

> "Richard Zultner" (Richard@Zultner.com) wrote:
>
[snip]

>> needs of the customers (or business)? If you don't know,
>> or clearly understand, the customer needs, then you
>> cannot know if you are building the right system -
>> which then makes the technical correctness of the
>> functional spec (what we intend to build) or the
>> design spec (how we think it should work) a moot point.
>
> Yes, the only thing truly "required" is that the system, in conjunction with
> the users interacting with it, solve certain customer problems. Product
> managers and designers spend most of their time designing solutions (such as
> detailed user interactions with, and functional behavior of, the system)
> without ever clearly understanding and specifying what problems to solve.

Roger has hit the nail on the head. Part of our problem, as engineers and software developers, is that we are problem-solvers. We are very bad at problem definition.
>
>> When I review functional specs, the first thing I check
>> is whether customer needs are clearly identified and
>> understood. If they aren't (the usual case), then you
>> can spend days examining one functional requirement after
>> another, and asking, "and why does the user need this?
>> What problem does that solve for the customer?" and
>> finding out that no one, including sometimes the user,
>> knows.

Exactly. I see this often.

> One major reason for this phenomenon is that, when the typical organization
> categorizes functional specifications as "requirements", it lulls its team
> members into believing they've understood what's "required" despite never
> having understood what problems they're solving.

Yes, again

> I propose that every team serious about requirements compose a conceptual
> model of requirements concepts. For most teams, composing such a model will
> reveal numerous ambiguities and contradictions in the way the team members'
> understanding of requirements concepts. See
> http://cauvin.blogspot.com/2006/11/requirements-concepts.html for model of
> requirements concepts that I believe is relatively free of such ambiguities
> and contradictions.

I like Roger's conceptual model, but I think it misses out one vital thing (and my apologies if I am mis-reading it). This is that it does not clearly distinguish between problem definition (stakeholder requirements) and solution definition (system requirements, many, but not all, of which are functional). In my view, it is vital to have stakeholder requirements defined before working on system requirements. And note to the agilists, I do not say that the entire stakeholder requirements specification needs to be defined, just enough to allow the risks to be understood well enough to make it worth spending resource (i.e., money) on defining a solution. And notice that I talk about defining the solution, not designing it. In crude terms, stakeholder requirements are about what they stakeholders want to achieve, system requirements are about what the system must do and the design is about how it does it.

Businness Analysts and design

Sometime since I last posted, so one from the past:

This was originally posted about ten weeks ago on the Requirements Networking Group (RQNG 080320 "Does a BA do design?", http://www.requirementsnetwork.com/node/1117#comment-2132) in response to a post claiming that there is no distinction between requirements and design. I have mildy edited for this blog.

Of course there is a meaningful distinction between requirements and design! That doesn't mean that the distinction isn't sometimes more of a grey blur than a clear line, but it is nonetheless meaningful.

Requirements are all about definition, either of the problem to be solved or the solution. Design is all about, well, design - making decisions about what a thing should be as opposed to what it does. I can define a solution, that is, defining its essential external characteristics, without designing it, which is primarily about internal characteristics.

Do I think that BAs should be involved in design? It depends, at the least they should be involved in reviews to gain confidence that the design will provide the functions, and thus enable the capabilities, that are needed. Does that mean they DO design? No, it does not. If a BA does design, does that mean that requirements and design are not meaningfully distinguished? Of course not.

I suspect that a lot of such thinking comes from the common failure to distinguish between problem and solution definition. Countless times, I see or hear of BAs presenting functional specifications to users without understanding that the functional specification defines the solution, and hence without the actual problem being clearly defined anywhere.

On a related note, should BAs take into account design in the analysis? Of course they should! I often use the example of seats in a car. From a pure problem perspective, you should not include seats in the stakeholder requirements as they are a solution. But no other solution fits the constraints of the problem, so use them.