One year after the Vision Statement – What has happened in the meantime?

 
At the last ICCC in Paris in September of 2012, the CCMC published their " Vision statement for the future direction of the application of the CC and the CCRA. "

Since Dag Stroman asked me to provide comments on this vision statement, I sent him a long statement on October 4, 2012, listing my concerns. I am still waiting for the swift answer he promised and therefore I have decided to make my statement public. Please read my statement here before reading the rest of this blog.

Now it is time for a short analysis of what has happened in this time frame.

As you can see in my comments, I neither rejected (and still do not reject) the idea of cPPs nor do I reject the idea of a new CCRA. Instead I questioned (and still question) if the CCDB/CCMC has provided us with the basis necessary to achieve success in those efforts. My main criticism is that the CC and CEM do not provide the sound basis required for the development of such cPPs and that the management of the CCRA was bound to fail because of a lack of specification of the technical competence required by schemes and labs and because of a lack of elaborated procedures for the oversight of schemes and labs that would ensure comparable results.
This was not the first time I brought up those points. Just have a look at the presentation I gave in April 2012 in Cannes: Quo Vadis Common Criteria.

So, what has been done in the past year to overcome those problems?

I see at the time of writing this, that 119 Protection Profiles are listed on the CC web site. How many of those are “cPPs” developed using the vision statement? 

Not a single one! 

I acknowledge that the development of a good PP takes time, so I may ask how many CCDB accepted technical communities do we have that are close to publishing their cPPs? 

Not a single one that I am aware of (assuming that some time before publishing a cPP it should be out for public review, announced on the CC web site). 

Yes, there has been some effort to develop Protection Profiles by individual nations (especially the U.S.), but those address national requirements and have not been developed in the cooperative way required for international mutual recognition (if I follow the views of the vision statement).

I know that developing a Protection Profile in a collaborative way that also addresses the technology-specific assurance requirements is not an easy task. I have had this experience myself in the last 18 months with the participation in the development of a Protection Profile for General Purpose Operating Systems. While agreeing on the base functionality that needs to be provided was comparatively easy (but excluding nice and useful security functionality not provided by every product in this category), achieving agreement on the assurance aspects and evaluation activities was hard to impossible. Why this? Quite simple: the two operating systems taken as the “guiding examples” target very different customers, have different architectures, have very different development methods and their developers have very different views on assurance. In addition we had the view of a scheme that believes the security assessment of a product should be done by people that first have demonstrated to not have in-depth knowledge of the product (I remember in a discussion we had if in the case of the evaluation of an Open Source product the evaluator would be allowed to look into the source code and the initial reaction of that scheme was “No, this would destroy comparability of evaluations!”). The lesson learned from this experience is: if you do not start with a good common view on the assurance measures required and how they need to be addressed in the evaluation, it is highly unlikely that you will ever come to useful results!

This leads me to my main criticism of the vision statement: the CC and the CEM need to be improved significantly to be a sound basis for the development of cPPs. As I have noted in my comments one year ago: using the current CC and CEM for the development of cPPs is like using rotten building blocks to construct high-rise buildings – it is bound to fail. 

The drawbacks of the CC and the CEM were identified a long time ago and have led to some useful objectives for the development of version 4 of the CC. See the presentation of the CCDB chairperson for those objectives as defined by the CCDB in 2008 – and the difficulties that one has to deal with (mainly: overcoming subjectivity and required evaluator skills). It is interesting to see how the vision expressed in 2008 (which addressed real problems) has changed to a vision in 2012 that avoids talking about problems.

Instead of working to those objectives identified in 2008, the CCDB/CCMC did not move forward with enhancements to the CC, the CEM and the oversight of the CCRA. We still have basically the same basis that even the CCDB has identified as insufficient many years ago. Instead, we have a “Vision Statement”. What we miss are the coordinated actions required to turn this vision into reality, addressing the well-known problems and improve information assurance. Instead we see many individual and uncoordinated activities not addressing the basic problems identified long ago. As a Japanese proverb states:

Vision without action is a daydream. Action without vision is a nightmare.

... and uncoordinated actions that do not contribute to turn the vision to reality are an even larger nightmare.
 

Helmut Kurth