Severity, Priority, Impact and Likelihood - Managing Defects and Risks

Defects and Risks are often dealt with in a subjective, emotional way.  That's unfortunate, because among all the things a software development team deals with, those are two that can be handled in a more constructive and empirical way.

First, a couple definitions.

Defect Severity: the degree of impact a defect has on system operation. A defect is something observed, so impact can be empirically quantified.

Risk Severity: the degree to which a hypothetical event, should it occur, would impact system operation. A risk event is something that has not occurred, so impact must be estimated or extrapolated.

The common element in both Risk Severity Assessment and Defect Severity is Impact on Revenue.  The FMEA framework that AKF recommends uses 1 = Low, 3 = Med, and 9 = High, to represent the exponential effect of a high impact risk or defect

Impact on Revenue

1. No payments being collected and/or payment data security compromised (Critical)
2. Payment collection being delayed by minutes (Major)
3. Payment collection being delayed by at least 2x SLA (Major)
4. Revenue impact is secondary (Minor)
5. No impact on revenue (Trivial)

Defect Categorization

One of the differences between Defects and Risks, is the number of people affected, which can be ascertained during defect triage.

Number of people affected

1. Everyone
2. Most (or, all in a specific geographic region/market)
3. Many
4. Some
5. Few or one

You can add the System Function row to a matrix to help align parts of the system with Revenue Impact.  That gets subjective, but it helps promote discussion and alignment (this table is incomplete and intended only as an illustration):

Defect Severity Matrix

Number of People Affected	Everyone	Critical	Critical	Critical	Critical	Critical	Major
	Most (or all in a specific region)	Critical	Critical	Critical		Major
	Many	Critical	Critical	Major	Major	Minor	Minor	Trivial
	Some	Critical	Critical	Major	Major	Minor
	Few or one	Major	Major	Major	Minor	Minor	Trivial	Trivial
	Revenue Impact	High  -----------------------------------------------------------------      Low
	System function / feature	Checkout / Pay Login	Shop	Enroll	Commissions	Reports	Email	Etc. . . .

It can be useful to have the Severity Level be derived from a set of variables.  So, you would start with Impact on Revenue, include Number of People Affected, and then add things like Frequency and Available Workaround. When you calculate Severity using multiple variables, then you want to use a weighted scale to assign greater weight to Impact on Revenue (like that 1, 3, 9 scale AKF recommends)

Frequency

1. Persistent
2. Multiple times per day
3. Multiple times per week
4. Multiple times per month
5. Happened once; cannot be reproduced

Workaround

1. None, or very expensive
2. Complex and totally manual
3. Complex with systems (e.g., excel, email, fax, etc.)
4. Complicated
5. Simple

Risk Assessment

With Risk Assessment, you combine Revenue Impact and Likelihood or Probability:

1. Certain (observed)
2. Highly likely (greater than 80% probability)
3. Likely (at least 50% probability)

Nothing less than 50% probability is worth noting as a risk.