Maturity and S2Score
Maturity and S2Score are both measurements; however, they don’t measure the same thing and one is more valid than the other.
Maturity
Arguably the most common reference for maturity is the Capability Maturity Model Integration (or “CMMI”) administered by the CMMI Institute, a subsidiary of the Information Systems Audit and Control Association (or “ISACA”). The model was originally developed by Carnegie Mellon University in 1986 as a process level improvement training and appraisal program and has since been modified to fit many
different applications, including information security.
There are five levels defined in the CMMI:
Level 1 – Initial; processes are unpredictable, poorly controlled and reactive.
Level 2 – Managed; processes are characterized for projects and are often reactive.
Level 3 – Defined; processes are characterized for the organization and are proactive.
Level 4 – Quantitatively Managed; processes are measured and controlled.
Level 5 – Optimizing; focus is on process improvement.
Maturity is a common metric, used by many within the information security industry as a measurement of information security program performance and risk. The problems with using maturity as a measurement are defined below:
Maturity, by itself, is an indicator of information security program performance and risk, but it is not a valid measurement. There’s more to information security than maturity, especially when we consider our definition.
Definition of Information Security:
Managing risk to unauthorized disclosure, alteration, and destruction of information using administrative, physical, and technical controls.
Maturity is most often determined and applied subjectively. Subjective measurements are valid ones. Objectivity is the first measurement validity requirement.
The application of maturity as a metric is often inconsistent between organizations and even within an organization from assessment (or measurement) to assessment. This violates our consistency requirement.
SecurityStudio recognizes the value in the CMMI and in using maturity as a risk indicator but it does not use the CMMI as a measurement of information security program performance or risk.
PLEASE NOTE:
For organizations who use maturity as a measurement, SecurityStudio has vastly improved its validity. Within SecurityStudio, maturity is measured objectively and consistently, which makes maturity a valid metric for measuring program maturity; however; it still fails for relevance because of the definition between maturity and information security.
S2Score
The S2Score was built as a measurement of information security program performance and risk. The metric fits our three criteria for measurement (above) and is compatible with traditional maturity measurements.
S2Score uses maturity as it’s intended, as an indicator of performance and risk; however, the S2Score also accounts for our definition of risk:
The likelihood of something bad happening and impact if it did.
Likelihood and impact are derived from threats and vulnerabilities. In this context, maturity applies more to control weakness (or vulnerabilities) as much (or more) than anything else.
Applying our measurement validity test to S2Score:
The measurement must be objective. The S2Score is based on binary data and it is what it is. The measurement cannot be manipulated except through deception (or lying).
The measurement must be consistent. Application of the S2Score is the same from one organization to the next allowing for comparison between organizations, and it’s applied consistently over time allowing for state changes within an organization (improvements, digressions, and external threat factors).
The measurement must be relevant. One part of the assessment (the part the users see) is metrics related to information security control weaknesses (or vulnerabilities), while the other part of the assessment (the part that users don’t see) is related to threats. The result of applying threats to weaknesses is our definition of risk and by proxy information security. The S2Score is completely relevant to what it is we’re attempting to measure.
*S2Score is not the only measurement any more than an inch is the only measurement of distance. It is a valid measurement, and it’s obviously one we use and highly recommend.