To PP or not to PP - Too narrow a view?
I had a short conversation with the new NIAP director, Mark Loepker, at the ICCC this September. I explained to Mark the difficulty I have in selling the NIAP policy to vendors who have the mindset of climbing the assurance levels EAL 1, EAL 2, EAL 3 and on up. Mark indicated that I can involve him in the customer conversation, and I will certainly take him up on that offer because we encounter this issue quite often. What remains an issue though, is the fact that our customers still tell us that their customers - the end users - continue to require higher assurance levels for Common Criteria evaluations.
Why is the PP being used as a substitute for the Evaluation Assurance level?
Clearly, as one progresses up the ladder from EAL 1, there is an increased amount of rigor that must be observed. Customers still request evaluations at assurance levels higher than EAL1 or EAL 2. Part of this may be historic:
1. They've always sought to increase the assurance promised by their product
OR
OR
2. They are subject to competitive pressures arising from others in their sector who have a product evaluated at an assurance level equal to or greater than their own
The move to being compliance to a PP only and abandoning the assurance level marker is troubling to many software vendors. In fact, this policy is often a very hard sell when explaining it to them. And it may continue to be a hard sell because the matter of mutual recognition still seems unsettled. That is, currently the CCRA provides for mutual recognition up to and including the EAL 4 level of assurance. But contrast that with what the CCMC says in its September 2012 Vision Statement:
“Mutual recognition should be based on the achievable common level of the cPPs.”
I am not certain what the "achievable common" level translates to - nor how to relay that to our customers. And the CCMC vision statement then further extends the restriction: "No CCRA mutual recognition above the cPP level;" that all cPPs will be at an EAL 2 maximum level unless the technical community can somehow demonstrate that “the activities can be repeated between schemes.”
This effectively negates the already established and agreed-to CCRA arrangement. So I wonder, what are the activities exactly, which cannot be repeated between schemes and why could they not be repeated? Are we fixing the wrong problem?
Also this collaborative PP approach across national boundaries would then assume that the requirements for function and for assurance levels is the same for multiple national interests. Currently, twenty-six CCRA nations must approve a cPP. I doubt that this can be the case because acquiring agencies within one government, let alone several governments, cannot be expected to have the same requirements. Even the risks due to differences in the deployed environments of a single nation would not be the same.
A protection profile can assure more product homogeneity. That is, product A from vendor A will more closely resemble product B from vendor B. Certainly this will make the government or other purchaser’s job of understanding product differentiation much easier. And perhaps it will make product selection easier. But, will it instead change the focus for developers because the overriding factor in decision making will be compliance and not overall product functionality?
At what cost?
Recent developments in the CC community have heightened the interest in protection profiles specifically tailored to categories of products. This emphasis was recently expanded at the 2012 ICCC conference. The CCRA Management Committee Vision statement takes the view that the move toward PP compliance is one that it will endorse. However, there still remains a dichotomy in views between the individual schemes.
For an organization such as ours that spans several schemes, this presents quite a challenge. Developers of software products are thus presented with the choice of being in compliance with a PP and sacrificing the assurance level that they had perhaps attained in the past or, risk not being listed on the NIAP site - or at least not having a fully qualified listing.
Do developers write to the PP or to a product’s functionality?
Does this then suggest that the PP will cause developers to more narrowly develop products to fit the chosen PP? After all, it costs time and money for development. Will this then stifle the development of other functionality as a sacrifice for PP compliance? Perhaps a broader view such as the Department of Homeland Security’s “Build Security In” is what ought to be motivating developers rather than a narrower protection profile view.
by Ken Hake
NIAP CC Laboratory Manager
I right with you, Ken. I was a secure product developer and then evaluator for many years, and I was very unhappy when NIAP started pushing for the new PP model a few years ago. I have nothing against PPs, but NIAP seemed completely unrealisitic in their thinking of how quickly they could put together PPs and get agreement (at various levels) on them. There always seemed to be some products coming our way that did not fit into any standard PP too. I didn't understand why there wasn't an attempt to add specificity to the CEM, make better/more use of the interpretations mechanism, and at least get the US validators to better enforce commonality between (EAL) evaluations. Clearly some products operate in more risky environments or with more sensitive data and therefore need a higher assurance than some peer products.
ReplyDeleteI believe this will drive vendors to have two versions: a CC version conforming to cPP and a more usable functional version.
Delete(cross-posted from the CCUF)
ReplyDeleteI think that before one can talk about "PP or not PP", we need to sort out something that troubled me about Mark's presentation at ICCC. Looking at his slide #10, "EAL vs. PP Example", Mark made the argument that an evaluation driven by a generic EAL package may not match the functional and assurance requirements that are relevant to the product, and therefore it may get a high EAL certification but not cover all of the relevant requirements, while an evaluation driven by a PP that has tailored assurance will cover all of the relevant requirements, and will therefore be more secure even though it has a lower base EAL.
"EAL versus PP" bothers me because EALs and PPs are neither interchangeable nor mutually exclusive! I think there are two independent questions: "EAL versus no EAL" and "PP versus no PP".
Parsing slide #10 in that context, Mark was really saying two things:
1. Tailored Assurance results in better security assurance than generic EAL packages.
2. A Protection Profile, tailored to the technology type, provides better security function coverage than a standalone Security Target because the TOE must fulfill all of the PP's requirements.
I have some thoughts about the first one:
It seems like common sense that tailoring SARs to a technology would be an improvement over using generic EAL packages. After all, we tailor SFRs to a technology. But you lose something if you eliminate or artificially de-emphasize EALs:
As flawed as EALs may be in theory or practice, they provide a simple scale representing a level of confidence that the security claims of an evaluated product are effective and have been implemented correctly. Customers use that scale to help determine if an evaluation is appropriate for their particular use of a product in their particular environment.
So if the problem is that generic EAL packages don't provide appropriate assurance for all technologies, then why not tailor EALs to technologies? In other words, go ahead and tailor assurance requirements, but use the standard EAL characteristics (structurally tested for EAL2, methodically tested and checked for EAL3, etc.) as a guide and then claim equivalence to that EAL. That way, you can improve assurance while maintaining the intent and usefulness of EALs.