Requirements

A Requirement is a condition or capability that must be met by a system or component to satisfy a contract, standard, specification, or other form of request.

Holistic Software Development expresses a slightly bigger Requirements stack than is typical for simple agile processes because it covers both product delivery and the strategic direction, portfolio and potential programme management layers that drive product development in an enterprise. Despite that the HSD requirements stack is smaller than traditional formal processes and is intended to be lightweight, only documenting or modeling enough requirements information to direct initial architectural elaboration, high level planning, and early development.

From an enterprise perspective, we can think of Requirements existing in a stack of decomposition. At the top we have few very high level strategic statements, at lower levels we increase both the details and the numbers of requirements. Not all detailed requirements have to exist at the bottom of a number of layers of higher level requirements, it is quite possible to have some low level user stories that directly relate to a strategic goal especially if using the Small Picture of HSD.

Typically, the implementation of lower level requirements will be integrated via Integration Streams leading to higher levels of integration testing and a greater level of acceptance. Each level of integration, and testing activity equates to a Definition of Done and maps to a level of requirements decomposition.

During the early part of a piece of work we go “inch deep, mile wide” across the top level requirements to check we’ve got a good understanding of the system, leaving the depth of detail to later iterations/sprint or Releases. Detailed requirements elaboration is expected to happen iteratively during a project lifecycle with only requirements related to critical business or technical risks being elaborated early in the lifecycle.

Avoid writing large amounts of requirements before architecting, designing, coding and testing the Product. Requirements change, and the feedback from early Releases can significantly change the direction of Product development. Sometimes called “Big Requirements Up Front”, detailed early requirements actually delay risk mitigation in a project. The only meaningful measure of progress is working software in the form of high quality releases, not stacks of paperwork.

Various factors will push teams and organizations towards capturing requirements in a more implicit or explicit fashion:

For low complexity development, more implicit requirements are normal as work is well understood. However, when development becomes more complicated, more detail is often needed and that detail is often best explicitly captured in graphical or textual form. For high complexity systems, perhaps counter-intuitively, explicit requirements become less useful at low levels. When working in high complexity systems, the intent is often more important than the detail. Work is often highly speculative as people are inventing new techniques that may or may not work. Empirical feedback becomes more useful than specification, and due to the experimental “sensing” nature of high complexity work details will change significantly.

Factors such as contractual agreements might mean that we need to elaborate more requirements and especially acceptance criteria to get a firm understanding of broad scope and the required definition of done but these agreements should not specify factors related to the fine grained requirements, only scoping and non-functional requirements.

HSD Requirements Stack

In Holistic Software Development the top of the Requirements stack is derived from Strategic Goals and Customer Requests. Customer Requests are simple requests from Business Customers (internal or external) and are checked for alignment to strategy as part of the Portfolio Selection.

Note that if using the Small Picture of HSD, the Portfolio layer may be implicit and the Programme layer will not exist. In many cases, even when using the Big Picture of HSD, the Programme layer will not exist, Products do not necessarily need to be part of Programmes or other coordinating bodies unless they need to work together to generate Business Value.

Portfolio Requests:

Portfolio Requests are simple expressions of a business requirement related to a Business Case. Portfolio requests elaborate a specific Business Case option with an indication of required funding and technical direction for Solution Architecture within the context of Enterprise Architecture. Sometimes Portfolio Requests will be to change the Enterprise Architecture.

Business Cases and Portfolio Requests are overlapping concepts. Both are essentially arguments for funding. In some organisations these are done via the creation of a Business Case. For very large/complex investments a Business Case may cover a number of Portfolio Requests, or spawn a number of investment options. In simpler cases Portfolio Requests and Business Cases may be the same thing.

Business Cases:

The Business Case represents the justification for purchase or construction of the product. The Business Case describes the problem which will be addressed and the desired outcomes of a solution including how the product will solve the problem or exploit/maximize an opportunity with cost benefits to justify spend/investment.

A Business Case qualifies the expected Business Value of significant investment.

Programme Backlog:

A Programme Backlog is a collection of requirements which are prioritized and split into “must have” and “wish list” items. Backlogs also include non-functional requirements either, as items in the list, or as cross-cutting requirements related to the functional requirements in the list.

Integration Scenarios:

Integration Scenarios describe an end-to-end workflow across a Product or Product Family resulting in Business Value for a Product Customer in flexible format.

Integration scenarios are threads across a number of other requirements such as Features and User Stories that don’t deliver Business Value on their own. They are ideally suited to describing the scope of Releases and providing the input for integration testing, necessary to achieve higher levels of acceptance.

Integration Scenarios can also be used to manage cross-cutting Non-Functional Requirements and drive the discovery of Architectural Mechanisms.

Integration Scenarios provide the context for discussions around User Stories.
Features:

A Feature is a distinct characteristic, property, quality or attribute of a system. A feature is a high level capability displayed by a system representing a distinguishing selling point for a product.

Scoping Diagram:

A Scoping Diagram is used to draw a picture of the primary requirements and users/interacting systems.

Product Backlog:

A Product Backlog is a collection of requirements which are prioritized and split into “must have” and “wish list” items. Backlogs also include non-functional requirements either as items in the list or as cross-cutting requirements related to the functional requirements in the list.

User Stories:

User Stories are fine grained requirements described in a relatively informal simple format, typically as a structured sentence of the form “As a <type of user/system> I want <a requirement> so that <something of value results>“. User Stories normally have attached Acceptance Criteria to help clarify their Definition of Done.
Bugs/Changes:

A Problem is an issue negatively affecting the relationship between a User/Stakeholder and the Product. Problems are often represented as “Bugs” or “Changes”

Non-Functional Requirements

Non-Functional Requirements are requirements specifying general characteristics of a system, not its specific functional behaviour. Typical categories of non-functional requirements are Usability, Reliability, Performance, Supportability, Security, Scalability, Quality etc.

Requirements are sometimes described as being part of the FURPS spectrum meaning Functional, Usability (UX), Reliability, Performance and Supportability. The latter 4 types being collectively called non-functional requirements. The spectrum is usually expanded to FURPS+ since there are several more important categories of non-functionals.

Non-Functional Requirements (NFRs) are sometimes considered to be the most important requirements from an architectural perspective, leading some to mischaracterize them as “architectural requirements”. This is a misleading approach though as a Product must meet its functional and non-functional requirements to provide business value. If functional requirements are ignored, then an architecture with some impressive mechanisms for dealing with non-functional requirements that doesn’t actually do anything is unlikely to be useful as a Product.

Non-functional requirements, sometimes called “-abilities” are requirements that are added on to the functional user needs to represent elements such as (non-exhaustive list):

Usability (UX) – Human factors, aesthetics, consistency, responsiveness, intuitiveness
Reliability – availability, failure extent and recoverability, predictability, accuracy
Performance – speed, efficiency, resource usage, throughput, capacity
Supportability – serviceability, maintainability, flexibility
Localization – language, currency, number formats
Security – information assurance, authentication, authorization, audit, logging
Scalability – elasticity, deployability, packageability
Quality – testability , failure modes, failure logging and reporting, instrumentation, self-fixing software
Packaging – physical marketing material, custom firmware/hardware constraints

Often non-functionals are best expressed using tests rather than positive requirements statements. Called “verification over specification” it can be easier to specify non-functional limits rather than exact requirements, for example:

User Story: “As a customer I want to be able to view all of my previous transactions so I can order again and counter fraud”

Acceptance criteria: “All user history queries must display in under 3 seconds”

In this example the non-functional performance acceptance criteria is fairly broadly defined rather than stating that the query must take exactly 2.5 seconds, it’s also written broadly enough to be used as a cross-cutting NFR.

Non-Functional Requirements are handled in two ways in Holistic Software Development: Embedded NFR acceptance criteria and cross-cutting NFR.

Embedded NFR Criteria

One of the simplest ways of handling non-functionals that apply to specific functional requirements (e.g. a performance requirement on a specific feature) is to embed the non-functional requirement as part of the acceptance criteria. This can be done with any requirement type, as an extra sentence attached to a feature or a set of tests associated with an Integration Scenario or a User Story. Not every functional requirement requires a set of non-functional acceptance criteria.

Cross-Cutting NFRs

Frequently non-functionals are not related to just one requirement but to several, or even the entire system. We call these “cross-cutting NFRs” and represent them in Holistic Software Development as Integration Scenarios, or more specifically as acceptance criteria on Integration Scenarios. This allows for non-functionals to be present on backlogs and explicitly tested and marked as done during integration activities.

Sometimes projects will have significant complexity or risk impact in a non-functional area, such as security. In these cases, we recommend simply adding the high impact non-functionals as non-functional Integration Scenarios or User Stories to the Backlog.

Non-Functionals and Architecture

Non-Functionals are critical in terms of developing an architecture. A very useful tool for understanding complexity across non-functionals (and functional differentiators) is the Architectural Profile.

Architectural Profiling is a method of understanding the relative complexity of different areas of concern for an architecture.

Architecture addresses a number of concerns and so a useful early approach is to consider the profile of the architecture in terms of the relative complexity of each of these areas. This helps to give a feel for the shape of the requirements, architecture and overall solution. We can also identify possible areas in which we can reuse existing components, frameworks or maybe standard packages of requirements, components and tests. Architectural profiles may be useful at any level of architecture (Enterprise, Solution and System) and are useful to understand areas of complexity in the requirements space, especially for understanding the relative complexity of non-functionals.

Although primarily focused on architecture, the Profile can also be used to identify areas for early requirements elaboration based on the complexity of an area, especially if there is a desire to out-source part of the development effort.

Backlogs and (Story) Points

Collections of requirements, at any level from Portfolio Requests to sets of User Stories are called “Backlogs“. A Backlog is a simple list of requirements (sometimes mixed types) that are used for planning. HSD mentions the Programme Backlog and the Product Backlog, but Team Backlogs and others are also common. Indeed, the Portfolio itself is a backlog.

Backlogs are simple collections of requirements that are prioritized by a relevant customer and split into “must haves” and “would like to haves”. Backlogs can contain a mix of functional and non-functional requirements as well as other objectives. They are typically used at team boundaries to manage incoming work. Backlogs can be useful for a variety of delivery methods including waterfall teams (not recommended), agile iterative teams and continuous flow teams.

Backlogs are a prioritised list of things to do which add business value. Backlogs are simply queues of work and as such are useful at any organizational or team boundary as a way of capturing and (re-)prioritizing incoming work. For overloaded teams they also provide a useful visual mechanism for displaying the amount of manageable vs. unmanageable work. From a systemic point of view, establishing pull in an organization can be realized in terms of the division between “must haves” and “nice to haves” in a backlog. Backlogs will often contain other non-requirement pieces of work such as Bugs/Changes or overhead activities.

There are two Backlogs described in Holistic Software Development the Product Backlog and the Programme Backlog, however in a Project with a team of teams structure each team may have a Team Backlog which may be a subset of a higher level backlog or a refinement of some of the higher level elements. The collection of Portfolio Requests and even the collection of Customer Requests can also be considered as a Backlog although the term is less common amongst business managers.

Working with Backlogs

Backlogs aren’t just a collection of requirements, they’re also a statement of scope. The “must haves” in a backlog are used to represent 100% of the current committed scope. Although this will naturally change over time the understanding of scope means that we can measure progress by the amount of the backlog that has been completed to a given Definition of Done. As a result, if we measure completion over time, then we can extrapolate forward to indicate when the must-haves will be complete.

The “would like to haves” become a wish list for the currently committed scope. This means that Backlogs are theoretically endless making multi Release backlogs a little more difficult to work with. Instead of over complicated prioritization schemes we recommend that items in a backlog are first tagged by Release. The immediate Release is the only one that will be further worked with, everything else can be filtered away until we’re almost done with the current Release.

Within the current Release we recommend ordering Backlog items, with Customer involvement and agreement, in terms of a simple prioritization scheme. Either High, Medium, Low or forced-ranking are acceptable. We recommend considering the bottom “chunk” as “on the bubble” meaning that the lowest n-items (as agreed between teams and their Product Customer) are at risk. This is an honest reflection of the realities of software development.

Often in development backlogs Story Points are used as an indicator of the relative size of backlog items to further refine this extrapolation.

Story Points are an abstract, arbitrary number associated with User Stories that indicates the relative effort (not complexity) required to complete the story development.

To manage activities that are necessary but not directly related to delivering business value to the customer we recommend the use of Objectives.

Objectives represent non-requirement activities that must be done to enable successful delivery.

Objectives can be used to represent overhead, setup and team improvement activities on a backlog. These concerns should not be considered as user stories as they add no direct value to a consumer. Objectives, therefore, should not contribute story points to team velocity (for agile/iterative teams) as they are not part of the product scope and will be inconsistent in terms of effort, therefore extrapolation of their completion is meaningless. Instead Objectives are best thought of as overhead. Explicit representation of overhead vs. requirements means that teams can be transparent in discussions with Product Customers regarding the necessity of overhead activities and trade them in/out against requirements.

Non-Functionals

Non-Functional requirements can be incorporated in Backlogs either as embedded NFR critieria on requirements or as cross-cutting NFRs, often represented as Integration Scenarios.

Non-Functional Requirements are requirements specifying general characteristics of a system, not its specific functional behaviour. Typical categories of non-functional requirements are Usability, Reliability, Performance, Supportability, Security, Scalability, Quality etc.

Bugs/Changes

Bugs and Changes often need prioritising alongside requirements and so are often present on Backlogs.

A Problem is an issue negatively affecting the relationship between a User/Stakeholder and the Product. Problems are often represented as “Bugs” or “Changes”

Backlog as WBS (Work Breakdown Structure)

A traditional approach to planning is the creation of work breakdown structures (WBS). A WBS decomposes activities into smaller and smaller units, with dependencies between them creating critical paths for planning. Although a WBS is a useful way of understanding what might need to be done it encourages false accuracy across chains of WBS items and removes responsibility for problem solving from team members into the hands of an elitist planner or Project Manager.

A common dysfunction we have observed in teams adopting Backlogs and User Stories is to adopt the terminology but actually still do WBS planning using these terms. A symptom of this approach is a backlog that is comprised of a hierarchical structure of “Epics” with child “Stories” (sometimes multiple levels of each) and finally child “tasks”. This obscures the purpose and clarity of a genuine backlog and also makes the classic WBS harder to see devaluing both approaches.

Product Backlog

A Product Backlog is a collection of requirements which are prioritized and split into “must have” and “wish list” items. Backlogs also include non-functional requirements either as items in the list or as cross-cutting requirements related to the functional requirements in the list.

A Product Backlog is made up of User Stories and associated Non-Functional Requirements for a Product. Sometimes a Product Backlog may be further sub-divided into Team Backlogs. The elements in a Product backlog are the requirements source for Planning and System Architecture.

Items in a Product Backlog typically trace to a Scoping Diagram item or a Programme Backlog item (e.g. Feature or Integration Scenario).

Programme Backlog

A Programme Backlog is a collection of requirements which are prioritized and split into “must have” and “wish list” items. Backlogs also include non-functional requirements either, as items in the list, or as cross-cutting requirements related to the functional requirements in the list.

A Programme Backlog is made up of Integration Scenarios, Features and associated Non-Functional Requirements for a related set of projects that together deliver some Business Value. Programme Backlogs are used to prioritise high level features and define acceptance critera prior to just-in-time decomposition into finer grained requirements in Product Backlogs. The Programme Backlog serves as the requirement source for Programme Planning and Solution Architecture. Sometimes a Programme Backlog may also contain Scoping Use Cases although we recommend using Integration Scenarios.

These elements frequently exist outside of a Programme Backlog, where a Programme isn’t being used to coordinate multiple products.

User Experience

User Experience (UX) refers to a person’s total experience using a particular product, system or service.

Good user experience is often the difference between a functional product and a successful product. UX is more than just making user interfaces look pretty however, it also incorporates human-computer interaction research, user psychology, accessibility and marketing issues.

The primary goal of good User Experience is to achieve all of the primary functionality without difficulty. This is fairy subjective and so user research is often necessary. User Experience fits into HSD in three ways:

Philosophically
User Experience Driven Design
User Experience Storyboarding

UX Philosophy in HSD

Philosophically, the aims of good User Experience simply align to the aims of good software development which HSD embodies. We constantly recommend that requirements are customer based, and that teams are business value orientated to maximize utility to end-customers.

Not all software is user facing, and sometimes the cost of good user experience is outweighed by other needs (e.g. time critical functionality). However, in most user facing systems, discounting UX in favor of other non-functionals undermines Business Value. Good UX means that users can make sense of a system and use it effectively, maximizing Business Value. For example, a safety critical medical device that is extremely difficult, even counter-intuitive, to use will result in more mistakes than one with good UX.

Our definition of Problems, is UX based as it is concerned with damage to the relationship between users and the system:

A Problem is an issue negatively affecting the relationship between a User/Stakeholder and the Product. Problems are often represented as “Bugs” or “Changes”

User Experience Driven Design

For Products where User Experience is very important (such as public facing apps/services) we recommend user-centric design. UX Driven Design involves doing significant market and user research as part of Scoping a Product. Often a UX Design will be created that deals with human-computer interaction, style, look and feel, content strategy and interaction design.

A UX design this comprehensive often replaces elements of the Requirements stack, and Architectural Overview material, even directing some Architectural Mechanisms. Care must be taken however not to over-detail user experience design, the same principles of the Cone of Uncertainty, iterative elaboration and inevitable change apply. Big Up Front UX is just as bad as Big Up Front Requirements. Instead look to adopt the inch-deep mile wide approach, establishing a common direction for development but letting details change iteratively/continuously.

Not all pieces of work are appropriate for User Experience Driven Design, and because it uses a different set of skills (and therefore probably people) to traditional requirements and architectural elaboration conflict can be caused when UX is new to an organization. Even if not formally adopting UX Driven Design we recommend UX is considered during all requirements and customer interaction/feedback work.

We recommend that UX Design is sketched rather than being too precise as very detailed prototypes and UX designs introduce false accuracy and lead to confusion with users who can think “the work is done” when they see an interface.

User Experience Storyboarding

User Experience Storyboarding is a detailed UX practice that replaces, or complements, Integration Scenarios or large User Stories. Showing in detail a storyboard of UX/system information annotated with notes as a way of communicating requirements.

A storyboard is typically a sequence of graphical designs showing the stages of interaction between users and a Product with annotations and extra information to clarify requirements.

UX Storyboarding can be mixed with graphical and textual Integration Scenarios as suits the set of requirements. As with UX Driven design we recommend storyboards are done in a sketched, rather than polished, finish.

Traceability

Although the Holistic Software Development Big Picture shows links between requirements types, architectural assets and definitions of done we recommend a balanced approach to traceability. Tracking 100% of traceability between requirements artifacts, architecture and design elements as well as tests is usually counter-productive (other than in safety critical systems).

Traceability links between elements of a similar type at a similar level (e.g. between stories in a backlog) might be tactically useful for the team using them but from a systemic point of view vertical links (to higher levels of requirements) and large horizontal links (between requirements and corresponding tests) are far more useful. Vertical links help to ensure that elaborated requirements are focused on delivering business value and large horizontal links help to inform quality confidence and feedback cycles.

Traditional approaches to traceability have had formal links managed, requirements baselined, suspect links investigated and new baselines taken. These formal traceability trees are, in our experience, only used by “requirements engineers”. They are never actually signed off by anyone, or used by development teams. People spend more time managing traceability than talking to customers about their requirements. If the information won’t be used to make decisions, don’t track it.

We recommend a minimalist approach to traceability, create the links that are genuinely useful and no more.

Recommendations for writing good requirements

Clarity

The most important quality of a good requirement is that it effectively communicates the requirement. Requirements should not be written as programming code or using esoteric standards, but in easy to understand plain language.

Customer based

Requirements must come from the customer, if they are generated by the delivery team in isolation then they are not requirements. When working in “green field” or exploratory development there may not be an informed customer, in which case User Experience research is recommended to understand potential customer needs. Risks are carried by the delivery until real customer feedback can be gained.

Testable

Requirements need to be testable, sometimes this is difficult to achieve in an aspiration sentence and so acceptance criteria should be attached to the requirement to make it testable.

Flexible Format

Requirements do not need to be textual; graphical mockups, UX Storyboards, activity diagrams, state diagrams and specification by example are all excellent methods of expressing different types of requirements. We also recommend that organizations are flexible in their approach to the format of textual requirements, not imposing textual templates or patterns to the extent that clarity is diminished. The most effective format for any individual requirement is the one best suited to communicate that individual requirement.

Just Enough (Inch deep, mile wide)

Avoid analysis paralysis and Big Up Front Requirements while still doing enough requirements elaboration to understand the direction of system development. We recommend the inch deep, mile wide approach to start with that involves getting a broad understanding of the scope without creating a lot of detail. Initial architectural profiles can aid identification of areas that may need a little more up front requirements effort.