Evaluator Training

by King Ables, atsec evaluator

Common Criteria evaluations are complex projects and require substantial technical knowledge, analytical abilities, and experience. Success also requires an understanding of the Common Criteria standards and methods.

Every scheme has accreditation requirements, derived from the CCRA, relating to the competence of evaluation laboratories. The goal of this is to ensure that evaluations are performed to the scheme's satisfaction, and sets the bar for minimum competence requirements throughout the CCRA by specifying that labs conform with ISO/IEC 17025, ensuring that staff are trained regularly, records are kept and competence is reviewed.

Although, there is a gap. ISO/IEC 17025 is not specific because it is a standard that is not just used for CC labs, but that it provides the general requirements for the competence of all kinds of testing and calibration laboratories.

This leads to some variation as each CC scheme is left to make their own policy about the capabilities required of the people performing evaluations.

For example:

• The U.S. organization, National Information Assurance Partnership Common Criteria Evaluation and Validation Scheme (NIAP-CCEVS), has set policy that the accredited lab is responsible for the capabilities of its individual evaluators.  NIAP have therefore not provided any scheme specific training for evaluators in their policies, methods, or technical skills of evaluators in their scheme.
• The German scheme, Bundesamt für Sicherheit in der Informationstechnik (BSI), not only requires  lab accreditation, but also requires each evaluator attend their Common Criteria training course and pass their exam before performing evaluations for BSI.
• The Swedish Certification Body for IT Security (CSEC) requires lab accreditation, provides Common Criteria training, requires the candidate evaluator pass their exam, and then conducts an IT security competency interview before approving an evaluator. 

(We do not know the current requirements for other schemes, It would be interesting if anyone with knowledge of other scheme requirements could comment on this blog and let us know.)

I joined atsec during a period when no external training from a scheme was available in English.

I was, of course, supervised as I worked with colleagues on their projects to help where I could and to learn by doing. This works, but it is haphazard and leaves gaps in your knowledge that you don't even realize exist. You don't know what you don't know.

Recently, after a year of "learning through osmosis," I attended BSI's Common Criteria evaluator training. The content and presentation was excellent and provided a good overview and summary of Common Criteria.The class nicely filled many gaps in my understanding. A week of well-structured training helped me connect my disparate bits of experience together.
 
BSI offers their training to ensure evaluators understand the CC and know what is expected of them in evaluation projects. However, much of the material provided an excellent high-level view of CC topics and would be applicable to evaluation work with any certification scheme.
 
One of the objectives of the Common Criteria standard is to provide sufficient requirements and a framework for evaluation to allow it to be performed effectively equally by anyone for any scheme. In a field where such specialized knowledge is required, a common educational curriculum could help achieve that objective. When evaluators receive common training from the scheme rather than possibly quite varied training from individuals, labs, or third parties, the various evaluators will operate in more consistent ways. Experience and education are both ultimately necessary to be a successful Common Criteria evaluator.
 
Experience, by definition, varies based on projects and participants. Training is an area where standardization could bring some more repeatability and commonality to the Common Criteria evaluation projects.