Bugs and Changes

A Problem is an issue negatively affecting the relationship between a User/Stakeholder and the Product. Problems are often represented as “Bugs” or “Changes”

Bugs (sometimes called “Defects”) and Changes are both ways of representing “problems” where the difference is often described dysfunctionally as:

“Bugs the supplier pays for, Changes the customer pays for” or “Bugs are when the product doesn’t meet the requirements, changes are when the customer wants a change to the requirement.”

The difficulty with the former approach is that it leads to a commercial pressure against bugs, causing suppliers to argue that everything is a change. Indeed, many large software vendors have explicit internal policies to not accept bugs but argue that they are changes wherever possible. This often leads to time-consuming conference calls arguing the details of individual problems – adversarially negotiating the difference between bugs and changes damages the collaborative relationship between suppliers and customers causing transactional behavior.

The latter approach incorrectly implies that a software system can be exactly and perfectly specified, using an up-front requirements approach. We know from the vast weight of industry evidence that this idea is either simply wrong for the majority of software projects or excessively cost-prohibitive and time consuming in predictable cases. There is too much implicit information inherent in requirements, software and user-product interactions to explicitly define everything at this level.

Instead we promote thinking of both bugs and changes in terms of:
“…an issue negatively affecting the relationship between a User/Stakeholder and the Product”

We recommend that Product Customers prioritize bugs, changes and new requirements alongside existing requirements as part of iterative cycles on Backlogs during each Releases. If any bug, change or new requirement is too large to be incorporated without significant change in scope, time or cost then the Product Customer may escalate to the Business Customer.  Where supply is managed by a software development contract we recommend that tolerances are set around changes to scope, time and cost within which the suppliers and customers can make immediate decisions with only very major changes requiring a contractual change.

If a bug, change or new requirements is outside of tolerances and sufficiently large then it can be considered as a new Customer Request for consideration in the portfolio.

We recommend that conflict is avoided by organizations, teams and stakeholders by collectively making group decisions regarding whether a problem is a bug, change or new requirement (if it matters) as well as consideration of whether that problem is within tolerances or not. The earlier problems (of any kind) are identified in development the easier and cheaper they are to fix. For that reason we recommend that quality assurance is built into every activity using feedback cycles rather than left to late lifecycle testing.

In practical terms

Most teams will not be familiar with the term “problem” and so will tend to use bugs (or “defects”), changes (or “change requests”) alongside their requirement types (features, stories etc.) in work item tracking tools. We recommend using these familiar terms without worrying about the differences between them too much. They all represent work that is prioritized on a Backlog and selected for development so the exact difference between a bug and a change or new requirement vs. old requirement is not really relevant.

Measurement driven behavior will often drive items of one type to be turned into another (e.g. bugs into changes to avoid dipping quality metrics). This tends to happen when a type (such as bug) has numerous different contexts such as an internal bug (found by team testing) or an external/escaped bug found in operational usage by customers. Teams generally do not want to mix these different contexts when examining work item metrics as the truth can be easily confused (for example, high numbers of internal bugs is a good thing as it means quality problems are found before the product is released whereas high numbers of escaped bugs are generally considered a bad thing). We recommend using flags in work item tracking tools to differentiate contexts (such as internal vs. escaped bugs) and filtering them to examine metrics.

Is it a bug or an unfinished requirement?

During development, especially when feedback cycles are small, testing activity may identify problems with the current development of a requirement since it hasn’t been finished yet – or more accurately it hasn’t yet achieved its required level of Done. Often a requirement might be thought of as “done” and yet found to be lacking by testing or user acceptance activities. In these cases, teams need to understand whether to raise a bug against the original requirement or re-open the requirement.

Re-opening will typically raise workflow productivity metrics (such as Lead and Cycle Time which can be seen as bad) however it may be the more honest approach for requirements in active development. Raising bugs against a requirement will typically lower quality metrics (such as quality confidence which can also be seen as bad) however this may be the more honest approach for requirements once they have been accepted as “done”. As a result, we recommend that requirements that have not yet hit the highest level of acceptance at the team’s scope have “Changes” against them (since they are not “done” yet) and only once requirements have been accepted as “done” are bugs raised against them.

This means that for development teams delivering a product, once development against a requirement has been accepted by the product customer (reached the “Product User Acceptance” Definition of Done) it should not be re-opened – instead bugs should be raised against the product. Below this level of done the requirement has not been closed or marked as “done” instead more work is required to reach the “acceptance” level of done which are best represented as “Changes”.

For integration teams at a Product Family level whose Product Family has not yet, as a whole, reached the “Product Family User Acceptance” Definition of Done, problems (including unexpected work) are best represented as either Changes or new Integration Scenarios if significant enough. Problems identified with constituent components or systems from contributing teams may be raised against those teams as problems (or bugs/changes). Once “Product Family User Acceptance” has been achieved then “Bugs” can be raised against the Product Family.

Treating problems in this way requires not considering “done” as a simple binary state, but instead adopting a stack of “Definitions of Done”. Most work item tracking tools can do this as either sub-states on “Closed” or by having multiple “Closed” states.

Where software development is managed by a contract this distinction helps drive a need for closer customer collaboration.