General Tips for Lead Evaluator
“What should I look for?” That is the question that the Lead Evaluator (LE) always has in mind when reviewing a workbook. Reviewing a workbook is tedious. It takes up a lot of time and energy especially when the evaluator is new. But reviewing a workbook also is a responsibility. It is an essential part to ensure the technical accuracy and necessary requirements have been met. Besides, you may refer back to the workbook after a couple of months to produce the final report which you may have already forgotten on some issues. This is where a good workbook plays an important role. In this post, you will learn the tips and ideas on what should you, as LE, look for when reviewing a workbook. Or from the evaluator’s perspective, what your LE will look for when reviewing your workbook.
#1. Make sure a complete ‘answer’ in each work unit in the evaluator’s workbook.
The evaluator’s answer must have a clear explanation of their findings and issues (if any). The question is, how clear is clear? One of the tips is to make sure that the answer is complete, where it must consist of Findings, Rationale, and Conclusions.
Incomplete Answer
The following figures show examples of a workbook where the evaluator’s answer is not complete.
Complete Answer
The following figures show a complete answer where the evaluator provides the analysis and evidence to support his/her findings. The evaluator provides a rationale for both Pass and Fail verdicts. This is how it should be done.
Raised Issues
Issues found by the evaluator will be raised in the “Raised Issues” column provided in each work unit. It can be whether a work unit requirement is not fulfilled, insufficient information, require developer clarification, or else. All these issues will be compiled in the EOR (another document) and submitted to the developer for correction. The raised issues must be clearly and concisely described to make sure the message can be conveyed effectively.
Sometimes the evaluator just mentions the main point of the issue but did not explain it further. For example:
- “The rationale for FIA_UAU.1 is misleading”
- “The rationale for FIA_UAU.6 is incomplete”
The evaluator should explain why both rationales are misleading and incomplete so that the developer can identify and fix the issue. Here is the analogy;
Operator 2 raised the issue clearly compared to Operator 1. The customer immediately knows what to fix.
Sometimes the evaluator gives a good explanation of the issue but misses the main point. ‘Main point’ is a summary of the issue in a simple sentence. This helps the developer get the idea of what you will explain on the issue. The main point can be about;
- Inconsistency
- Insufficient info
- Incorrect/ misleading/ missing info
- Out of scope
- etc
#2 – Make sure issues are raised in the right work unit
Content Requirement
Content of evaluation evidence (document provided by the developer) should include the evidence required, what the evidence will demonstrate, and what information the evidence will convey. The content requirement is identified by appending the letter “C” to the element number. You can refer to CC Part 3 to see the requirements for all classes. Below is the example of content to be provided by the developer for ADV_FSP.2 in their Functional Specification document;
Based on the above example, as you can see
- “.1C” is about represents the TSF
- “.2C” is about the purpose and method of use for TSFI
- “.3C” is about the parameters of TSFI
and so on.
Work Unit
A work unit is a task or activity to be performed by the evaluator.
For each content requirement, there are work units. The figure below shows a mapping between the Content and the Work Unit. In another word, it’s like the CC says “Hi evaluator, for this content, this is what you should do”.
Based on the above example for ADV_FSP.2,
- for content .1C, there is 1 work unit for the evaluator; which is
- ADV_FSP.2-1. The evaluator needs to determine whether the TSF is fully represented
- for the content .2C, there are 2 work units for the evaluator; which are
- ADV_FSP.2-2; to determine the purpose of each TSFI is stated, and
- ADV_FSP.2-3; to determine the method of use of each TSFI is given
- and so on
To be continue…..
Good Guidance..terbaikkk