The Gift of Guidelines

by Jeremy Powell, CCTL Deputy Manager

NIAP has published several protection profiles in the past year or so that are aligned with their new vision for the Common Criteria. Unfortunately, these profiles appear to be developed in isolated silos without a coherent overarching policy on what the U.S. government wishes to see with respect to information assurance. It is imperative that an assurance policy gets established quickly lest we end up with several protection profiles each with different interpretations of what assurance means and no real argument why this is better than where we started.

As an evaluator, I read protection profiles and often puzzle over the decisions the profile authors made. Why did they reword that SFR when the change appears to be superficial? Why are attacks on entropy considered in scope but we're uninterested in examining undocumented interfaces? I am constantly fretting over what nuances I might be missing. I imagine profile authors go through the same stress. They want to mesh their profile into the framework that already exists but may not quite understand how other technical communities determined the assurance requirements for other profiles. Both authors and evaluators are left to make their own interpretations to the detriment of reproducibility and repeatability.

What's missing in protection profiles (both old and new) is a clear statement about what the assurance objectives are that should be achieved with compliant products. A consumer choosing a compliant product should have a clear view of those objectives allowing him to map them to his own objectives and identify if and to what extent the evaluation addresses them. In old style protection profiles, just one of the assurance levels defined in the Common Criteria were required without justifying it or even analyzing if another combination of assurance measures would be more appropriate for the technology and the overall objectives stated. This mistake is still being made in new profiles. Although there are plenty of objectives for functionality, there are none for assurance.

A central policy needs to be developed to describe in plain words what NIAP and its stakeholders want to accomplish. Maybe the focus is on infrastructure resilience, financial sectors, and national security. If so, then an exposition of each group's information assurance requirements should be included, expressly listing any nuances each one might have. For example, maybe infrastructure (such as energy, utility, etc.) needs to focus on availability, whereas financial and national security groups need confidentiality to be the primary goal. Writing down these assurance objectives makes our efforts more pragmatic and mission-oriented.

Next, the policy needs to specify the actual assurance requirements that profiles would claim and refine. For example, the following assurance requirements might be potential candidates:

NIAP_TST.1 -  The evaluator shall develop and execute a test plan that demonstrates that the TSF behaves as the user guidance describes.

NIAP_VAN.1 - The evaluator shall check that all publicly known exploitable vulnerabilities that violate the security objectives are mitigated or removed from the TOE.

NIAP_ENT.1 - The evaluator, with guidance from the certifying  scheme, shall analyze and determine that the TSF securely generate cryptographic material.

Each of these requirements are very broadly written, but they effectively communicate what an evaluator will do to produce the needed assurance.

Last, the policy should provide guidance on how to adapt the assurance requirement to different technologies. Take NIAP_VAN.1 for example:

NIAP_VAN.1 requires the evaluator to identify publicly known vulnerabilities that affect the TSF. When refining this assurance requirement, the profile author might consider what "publicly known" means. In some cases such as general purpose operating systems, it might be appropriate to point to general vulnerability databases like Mitre's Common Vulnerability and Exposures (CVE) database. In more niche technology areas such as special network appliances, a more specific database may be specified that matches the technology type. It also is entirely appropriate and encouraged for profile authors to iterate and refine this assurance requirement for multiple databases. A profile author may wish to add extended assurance by requiring the evaluator to check the CVE database as well as the Metasploit framework for known exploits.

I'm sure it hasn't escaped the reader that this looks an awful lot like the structure of an assurance package with CEM-like guidance. Well, that's because in a lot of ways, that's a good way to do it. However, in moving toward the vision of technology-specific protection profiles, it becomes necessary for the profile authors to provide technology-specific assurance activities as it makes little sense to require evaluators to follow a vague "attack potential calculation" when the profile developers can tailor the assurance activities to the specific technology. So, we turn the concept of the "Common Evaluation Methodology" into the "Common Profile Development Methodology."

By developing this policy (and supporting methodology), NIAP can provide a common language to discuss the needs of their stakeholders with the vendors and labs. Clearer communication will help the vendors know what their customers need, it will enable the labs to provide more objective and principled evaluations, and it will allow stakeholders to confirm that they received the assurance that they actually asked for. For now, though, both profile authors and evaluators are relegated to making decisions based on guesswork, interpretation, and inference which does not lend our efforts to repeatable and reproducible results.