I-0468: Must Test Setup And Cleanup Code Run Unprivileged?

TYPE:                 Guidance
NUMBER:               I-0468
STATUS:               Sent to CCEVS Management and CCIMB for Review

TITLE:                Must Test Setup And Cleanup Code Run Unprivileged?


RELATED TO:           <None>
CCIMB ENTRY:          CCIMB-INTERP-0243

ISSUE:

Can functional tests written by the developer to satisfy ATE_FUN be privileged to circumvent TSF policy, as long as those privileges are disabled in the code fragments that actually performs the tests? The same question can be applied to FPT_TST.

Privileged software may be needed to set up the initial conditions for a test. In the TCSEC world, we were forced to produce unprivileged test programs separate from the privileged programs (i.e., test harness) needed to set up the test preconditions. Unfortunately the extra synchronization and communication between the harness and the tests led to more complexity, less reliability, and higher costs. The test documentation can show where privileges are disabled and the evaluators can spot check the test code to see that it does what the documentation says. There would also be tests that show that the interface to disable privileges actually has the desired effects.

STATEMENT

It is acceptable for test setup and cleanup code to run privileged, as long as the developer can provide a convincing argument to the evaluation team that the actual test runs in a "normal" mode (i.e., a mode appropriate to the commands and functions being tested). Evaluators should be able to request the implementation of the setup and takedown code so that they can verify the argument, and the setup should do the minimum functions necessary to establish the test conditions.

SUPPORT:

Part 1 of the CEM (v0.6, 97/01/11) enumerated the goals of evaluation as impartiality, objectivity, repeatability and reproducibility, and soundness of results. Automated testing tools are a significant step towards having repeatability and reproducibility. Having an automated mechanism that helps to set up the test will ensure consistency of setup, and thus, ensure repeatability and reliability of the test.

The CEM notes, however, that the purpose of testing (paragraph 609) is "to determine, by independently testing a subset of the TSF, whether the TOE behaves as specified in the design documentation and in accordance with the TOE security functional requirements specified in the ST." Thus, if a mechanism is being tested, one must ensure that privileges to bypass the mechanism are not enabled (unless it is the bypass itself that is being tested).

Work unit ATE_IND.1-1 notes, "The evaluator shall examine the TOE to determine that the test configuration is consistent with the configuration under evaluation as specified in the ST." This can be interpreted to mean that, when the test runs, it must be in the appropriate test configuration. In other words, it is reasonable to permit test setup and cleanup code to do what is necessary to setup and cleanup. However, as for the actual running of the test itself, conditions must be the same as if the setup and cleanup had been done manually. The evaluators will need to examine such test code to ensure this is the case.