Evaluator Training

by King Ables, atsec evaluator

Common Criteria evaluations are complex projects and require substantial technical knowledge, analytical abilities, and experience. Success also requires an understanding of the Common Criteria standards and methods.

Every scheme has accreditation requirements, derived from the CCRA, relating to the competence of evaluation laboratories. The goal of this is to ensure that evaluations are performed to the scheme's satisfaction, and sets the bar for minimum competence requirements throughout the CCRA by specifying that labs conform with ISO/IEC 17025, ensuring that staff are trained regularly, records are kept and competence is reviewed.

Although, there is a gap. ISO/IEC 17025 is not specific because it is a standard that is not just used for CC labs, but that it provides the general requirements for the competence of all kinds of testing and calibration laboratories.

This leads to some variation as each CC scheme is left to make their own policy about the capabilities required of the people performing evaluations.

For example:

• The U.S. organization, National Information Assurance Partnership Common Criteria Evaluation and Validation Scheme (NIAP-CCEVS), has set policy that the accredited lab is responsible for the capabilities of its individual evaluators.  NIAP have therefore not provided any scheme specific training for evaluators in their policies, methods, or technical skills of evaluators in their scheme.
• The German scheme, Bundesamt für Sicherheit in der Informationstechnik (BSI), not only requires  lab accreditation, but also requires each evaluator attend their Common Criteria training course and pass their exam before performing evaluations for BSI.
• The Swedish Certification Body for IT Security (CSEC) requires lab accreditation, provides Common Criteria training, requires the candidate evaluator pass their exam, and then conducts an IT security competency interview before approving an evaluator. 

(We do not know the current requirements for other schemes, It would be interesting if anyone with knowledge of other scheme requirements could comment on this blog and let us know.)

I joined atsec during a period when no external training from a scheme was available in English.

I was, of course, supervised as I worked with colleagues on their projects to help where I could and to learn by doing. This works, but it is haphazard and leaves gaps in your knowledge that you don't even realize exist. You don't know what you don't know.

Recently, after a year of "learning through osmosis," I attended BSI's Common Criteria evaluator training. The content and presentation was excellent and provided a good overview and summary of Common Criteria.The class nicely filled many gaps in my understanding. A week of well-structured training helped me connect my disparate bits of experience together.
 
BSI offers their training to ensure evaluators understand the CC and know what is expected of them in evaluation projects. However, much of the material provided an excellent high-level view of CC topics and would be applicable to evaluation work with any certification scheme.
 
One of the objectives of the Common Criteria standard is to provide sufficient requirements and a framework for evaluation to allow it to be performed effectively equally by anyone for any scheme. In a field where such specialized knowledge is required, a common educational curriculum could help achieve that objective. When evaluators receive common training from the scheme rather than possibly quite varied training from individuals, labs, or third parties, the various evaluators will operate in more consistent ways. Experience and education are both ultimately necessary to be a successful Common Criteria evaluator.
 
Experience, by definition, varies based on projects and participants. Training is an area where standardization could bring some more repeatability and commonality to the Common Criteria evaluation projects.

On information leakage

One of the things we worry about in the security biz is information leakage: information exposed during a normal process that could later be used to compromise security. Some obvious examples of information leakage include:

• e-mail or instant messages accidentally sent to the wrong address
• confidential phone calls or hallway conversations that are overheard
• lost (and unencrypted) laptops or media, like USB sticks
• web site URL arguments containing confidential information (like a username or password)

A less obvious example is the behavior of an algorithm, such as the time required to process an authentication request (known as a side-channel or timing attack). Even when the attempt to authenticate fails, if the process takes more time than was required in a known previous attempt, this might indicate that at least some of the information provided was correct, and that the algorithm had to proceed to further checking before finding the ultimate reason for the failure. Most algorithms of this sort now do the (unnecessary) work of checking all information, rather than stopping at the first mismatch so as not to give any hints about exactly what failed. Ascertaining useful information is often complicated, but sometimes information leakage is neither obvious nor complex.

I recently went to the bank to get cash from the ATM. Rather than $20, $40, or $100, as most people withdraw, I purposely withdraw an unusual amount of money from my account. This way, when I look at my statement, any withdrawal in any amount other than my normal amount(s) attracts my attention.
In an effort to "help" me with my ATM transaction, my bank displays the amount of my previous withdrawal, in case I want to withdraw the same amount again. They do this on the same screen where I am prompted for my PIN after inserting my card. I can type in my PIN and tap the button for my normal withdrawal, all in one action. But all of this information is presented before I have been authorized to access the account. So if somebody gets their hands on my ATM card, they can find out how much money I withdrew the last time I visited an ATM without entering a PIN.

As in so many cases of information leakage, this might not seem very useful by itself. But perhaps it could be used to socially engineer a victim. If a bad guy gets my card, he can look me up in the phone book because my name is on the card. He can call me claiming to be from my bank and tell me my ATM card has been returned. To "prove" he's from the bank, he tells me the last withdrawal amount on my account (and even using my unusual amounts, I may be inclined to accept this as proof). Once he convinces me he really is from my bank, he asks me to confirm my PIN "so he can be sure it's really my card." If I give him my PIN, he can then withdraw all the money from my account, all because my bank is trying to make my ATM transaction easier for me.

Designers of systems should always think about all the information they divulge and remember that information leakage can happen in seemingly innocuous places.

by King Ables