Severity, Priority, Impact and Likelihood - Managing Defects and Risks

Defects and Risks are often dealt with in a subjective, emotional way.  That's unfortunate, because among all the things a software development team deals with, those are two that can be handled in a more constructive and empirical way.

First, a couple definitions.
Defect Severity: the degree of impact a defect has on system operation. A defect is something observed, so impact can be empirically quantified.
Risk Severity: the degree to which a hypothetical event, should it occur, would impact system operation. A risk event is something that has not occurred, so impact must be estimated or extrapolated.

The common element in both Risk Severity Assessment and Defect Severity is Impact on Revenue.  The FMEA framework that AKF recommends uses 1 = Low, 3 = Med, and 9 = High, to represent the exponential effect of a high impact risk or defect

Impact on Revenue

  1. No payments being collected and/or payment data security compromised (Critical)
  2. Payment collection being delayed by minutes (Major)
  3. Payment collection being delayed by at least 2x SLA (Major)
  4. Revenue impact is secondary (Minor)
  5. No impact on revenue (Trivial)

Defect Categorization
One of the differences between Defects and Risks, is the number of people affected, which can be ascertained during defect triage.
Number of people affected

  1. Everyone
  2. Most (or, all in a specific geographic region/market)
  3. Many
  4. Some
  5. Few or one
You can add the System Function row to a matrix to help align parts of the system with Revenue Impact.  That gets subjective, but it helps promote discussion and alignment (this table is incomplete and intended only as an illustration):
Defect Severity Matrix
Number of People Affected
Everyone
Critical
Critical
Critical
Critical
Critical
Major

Most (or all in a specific region)
Critical
Critical
Critical

Major


Many
Critical
Critical
Major
Major
Minor
Minor
Trivial
Some
Critical
Critical
Major
Major
Minor


Few or one
Major
Major
Major
Minor
Minor
Trivial
Trivial

Revenue Impact
High  -----------------------------------------------------------------      Low
System function / feature
Checkout / Pay

Login
Shop
Enroll
Commissions
Reports
Email
Etc. . . .

It can be useful to have the Severity Level be derived from a set of variables.  So, you would start with Impact on Revenue, include Number of People Affected, and then add things like Frequency and Available Workaround. When you calculate Severity using multiple variables, then you want to use a weighted scale to assign greater weight to Impact on Revenue (like that 1, 3, 9 scale AKF recommends)
Frequency

  1. Persistent
  2. Multiple times per day
  3. Multiple times per week
  4. Multiple times per month
  5. Happened once; cannot be reproduced
Workaround

  1. None, or very expensive
  2. Complex and totally manual
  3. Complex with systems (e.g., excel, email, fax, etc.)
  4. Complicated
  5. Simple
Risk Assessment
With Risk Assessment, you combine Revenue Impact and Likelihood or Probability:

  1. Certain (observed)
  2. Highly likely (greater than 80% probability)
  3. Likely (at least 50% probability)

Nothing less than 50% probability is worth noting as a risk.


Comments

Popular posts from this blog

Enterprise Agile Framework: The Entrepreneurial Operating System (EOS)

Demand Management Using Agile + Lean