PD-0056: Exhaustiveness of ATE_IND Testing

This decision represents a long-term technical decision based on an OD, and may not be the same as the final results of the source OD. With respect to published criteria documentation and scheme documents, it provides suggested guidance on evaluation direction, but is not authoritative. Authoritative decisions are provided through the published criteria documents and published scheme and international interpretations thereof. With respect to published PPs, PDs are authoritative corrections to the PP, based on input from the PP author (if available), that are in force until the publication of the next revision of that PP.

Issue

Do the requirements in ATE_IND.2 mean that the evaluator does not need to exhaustively test the TSF and that the evaluators could target testing to address specific areas of the TSF?

Resolution

The evaluation team does not have to test every function and re-execute every vendor test. The selection of what is appropriate requires the evaluator to fully exercise good evaluator judgment. If possible it is often good and useful to re-run the entire vendor test suite. This is more easily possible for more highly automated test suites. When selecting samples to test, the evaluator should select the tests independently. The vendor should not be allowed to influence that decision. It is within the evaluator's prerogative to re-run up to and including all of the developer tests and independently test every TSF. This is rarely necessary or economical.

Support

The CC states in the evaluator action elements:

• ATE_IND.2.1E.
  The evaluator shall confirm that the information provided meets all requirements for content and presentation of evidence.

• ATE_IND.2.2E.
  The evaluator shall test a subset of the TSF as appropriate to confirm that the TOE operates as specified.

• ATE_IND.2.3E.
  The evaluator shall execute a sample of tests in the test documentation to verify the developer test results.

The "as appropriate" words in ATE_IND2.2E allows the team to test those parts of the TSF for which the team is less comfortable with and that not every function of the TSF needs to be tested. The evaluation team will supplement the vendor's tests and test those functions that were not adequately tested by the vendor. For ATE_IND2.3E, the evaluator can choose a set of tests in the test documentation to re-execute and that the team does not have to pick a sample of tests such that every TSF function is covered.

This guidance is reflected in CEM Part 2 v1.0 paragraphs 1642 - 1645.

Modification History:

2004-08-12

Updated effective date to reflect the date the PD was issued. (August 2004 NIB 6.c.xiv)

References:

• ALFWPPv1.0

Related NIs:

• None

Related CCIMB-INTERPs:

• CCIMB-INTERP-0028: Test Sampling

Source OD: 0017