Labgram #078/Valgram #098 - CCTL Evaluation Test Requirements

Validators and CCTLs,

This Labgram clarifies CCTL evaluation test activities as specified in NVLAP Handbooks 150 and 150-20 to ensure all CCTLs perform testing in a manner that maintains the integrity expected from an independent third party.  This includes the determination of the test procedures, the choice of test parameters, the actual performance of the testing, and required reporting of the test activities.

Independent third party testing has been one of the fundamental tenets of NIAP since its inception.   NIAP Protection Profiles place emphasis on objective assurance activities consisting of well defined, transparent third-party testing.   CCTLs must maintain independence and must continually monitor processes to ensure compliance with NVLAP requirements.  Recent trends toward reliance upon vendor-provided test procedure templates, vendor assisted testing, and remote observation of some vendor-conducted tests necessitate promulgation of this Labgram. 

NVLAP References

NVLAP Handbooks 150 and 150-20 include requirements for maintaining the independence and integrity of the CCTLs.  There are specific NVLAP requirements for the performance of testing within the official CCTL Testing Facility and additional requirements for procedures to be followed to ensure comparable control over any alternative test environment, such as a vendor facility.  In particular, NVLAP Handbook 150-20 Sections 5.3.6 and 5.3.7 state the following:

5.3.6 If evaluation activities are conducted outside the laboratory, the management system shall include appropriate procedures for conducting security evaluation activities at customer sites or other off-site locations. For example, customer site procedures may explain how to secure the site, where to store records and documentation, and how to control access to the test facility.

5.3.7 If the laboratory is conducting its evaluation at the customer site or other location outside the laboratory facility, the environment shall conform, as appropriate, to the requirements for the laboratory environment. If a customer’s system on which an evaluation is conducted is potentially open to access by unauthorized entities during evaluation, the evaluation laboratory shall control the evaluation environment. This is to ensure that the systems are in a defined state compliant with the requirements for the evaluation before starting to perform evaluation work and that the systems ensure that unauthorized entities do not gain access to the system during evaluation.

Reporting Requirements

CCTL procedures detailing conformance with the NVLAP requirements are verified during NVLAP assessments.  Additionally, validators will examine the test plan for each evaluation to ensure it describes how the CCTL’s test procedures were implemented. 

The test report for each evaluation must detail the location(s) where testing occurred, who executed the tests, and an explanation demonstrating how the test environment conformed to the CCTL’s NVLAP-accredited test procedures.   The information will also be included in the Validation Report.

As part of evaluation oversight, NIAP validators will verify the testing activities conform to the CCTLs’ procedures.  Any deviation from CCTL procedures will be noted and reported to NIAP, the CCTL, and NVLAP. 

Testing Location

Testing at a vendor’s site is acceptable if the CCTL follows its procedures that conform to paragraphs 5.3.6 and 5.3.7 above.  The non-proprietary AAR must identify the procedures the lab followed in each specific instance to ensure the independence of the testing was not compromised. 

Remote testing is generally not acceptable.  It is very difficult for the CCTLs to ensure proper control over a remote test environment, and difficult for validators to ascertain if the proper control was maintained.  Therefore, remote testing or remote observation of testing being done by someone other than the evaluators is only acceptable on a CCEVS-approved case-by-case basis.

If you have any questions or concerns, please contact us at 410-854-4458 or by email niap@niap-ccevs.org.

Thank you,

NIAP Staff

410-854-4458 office

410-854-6615 fax

Posted on 2015-02-06 by NIAP Staff