Compliance can be appreciated in a number of different contexts. Caetra.io has intentionally left it vague because some of the privacy and monitoring controls are vague and could be met at a policy level while others could actually be automatically managed and thus much more mature. CyMetric does not address maturity (policy, implemented, measured or managed) since it is control based to begin with rather than requirement focused as you might see in some other standards that are built on traditional policy documents. For CyMetric, if the control is working you are already at implemented and if you are doing an assessment you are at measured. So we did not really see a point of putting that on customers to perform when the only issue left was whether the control was automatic or human driven. Thus, we largely incorporated maturity into the assessment and went straight to compliance level which looks to the extent that the control is actually working.
When considering compliance levels, we followed the industry standard of 5 states. Not and fully compliant are pretty easy to understand. Where it gets more dicey is for the middle three and we considered incorporating just one (partially compliant) level because this is generally subjective and the differences for the middle three are nuanced and subtle. However, the goal was to align with industry standards and to deviate from that would have bucked industry norms so we used the 5 point scale.
From a scoring perspective, our suggestion is to not look at each element of the control and see how many are met and use that to calculate the score. If an organization is missing some key element of a control such that all the other parts are rendered useless, then it is probably not compliant or at best somewhat compliant. So there will always be a subjective element to this. But, with that in mind, the chart below summarizes a high-level guideline where each level transitions to the next. We offer both a point value and a range to help quantify these different levels.
Maturity or Compliance Level | Requirements |
Not Compliant (NC) | Very few if any of the elements of the control exist. If you were doing a number score, it would be numeric equivalent of 0% or 0% to 12%(interval estimate). |
Somewhat Compliant (SC) | Some of the elements of the control exist. On a numeric score, it might be 25% or between 13% and 37% if someone were looking at a range. |
Partially Compliant (PC) | About half of the element exist for the control being evaluated. A rough numeric equivalent would be 50% or somewhere between 38% to 62%. |
Mostly Compliant (MC) | Many, but not all of the elements of the control exist and are implemented. A numeric equivalent would be around 75% or between 63% to 87% for those that ball park. |
Fully Compliant (FC) | Most if not all of the elements exist. A numeric equivalent of 100% or between 88% and 100%. |
There certainly are other methodologies that can be used to define compliance levels for the controls but this may give users a starting point for their decision making and scoring approaches.