Achieving Consistent Results when Assessing: Some Assessment Rules of Thumb

As we get more experienced in a particular topic, we try to breakdown our experience into some rules of thumb.  Having been involved in setting up assessment systems and assessing since the early 1990s, the team at Polaris Learning wanted to share some rules of thumb around assessment.

Assessment Must Always Be Agreed
We do not insist that organisations use a formal, detailed assessment plan when considering assessing but we do recommend that assessment is planned.  We also always include assessment planning within the assessor training.

The reason that we place such importance on assessment planning is because assessment must always be an agreement between the two parties involved, that is, the assessor and the job holder being assessed.

Why?  This is because assessment planning helps us to ensure the quality of assessment in the following ways:

• The job being used to provide evidence is fit for purpose or valid
• The assessor knows what evidence is being provided, including any witness testimonies or work records
• The assessor and job holder can build up a joint understanding of what is acceptable within the current process

This approach ensures that the job holder does not head off in his or her own direction and stays focused on gathering relevant evidence.  As without guidance, there is a risk that the job holder will start working in the wrong path, possibly collecting evidence that does not meet the requirements.

The job holder may also collect a lot of evidence that a busy assessor may not have the time or the experience to assess. 

Unfortunately, we have seen examples where the job holders have consequently been signed off as competent, when they have do not have the experience or supporting evidence, purely because of the pressure on the assessor.  If this keeps happening, this can result in a loss of trust in the competency management or assessment system, especially as the competence being assessed then appears to be about building portfolios rather than the ability to perform the job itself.

Planning Observations
And what about planning observations?  The idea of formally planning and agreeing to observe someone can be something that assessors are very opposed to.  We can understand this, especially if the reasons for planning have not been explained during the training.  

Some assessors get worried that formal agreements will lead to a very artificial assessment for everyone involved or that they simply do not have the time to do formal agreements.

However, observations do not have to be planned in micro detail.  As well as taking too much time, this may also have too much of an impact on the job holder’s performance, either positively or negatively.  Instead, the assessor can agree a window during which performance can be observed. This approach helps not to overly formalise the assessment while giving the assessor the opportunity to see consistent performance over a period of time.

Carrying Out Observations
Some people still argue that assessments should be done in a covert manner because this is the only way to see the job holder’s performance.  When you discuss and understand their concerns, the issue is mostly where there is a lack of trust with the job holder and that they “can” perform when they want but do not when they are not being supervised.  If there is a genuine concern with a job holder around performance, then there should be an open conversation between the individual and the assessor, and assessment should not start until this concern is addressed. 

Assessment Affects People’s Behaviour
Being assessed can be nerve racking for some individuals, and even the most confident individuals will be aware that they are under the microscope.

An experienced assessor will have learnt how to put people at ease ahead of assessment, enabling them to focus on the job, as opposed to them focusing on the process of assessment.

Communication Skills Matter
Developing the ability as an assessor to get a balanced approach to communication during the assessment is key, including knowing when to keep quiet.

Awareness of this factor and the ability to put people at ease through effective communication is an important part of any assessor training programme.

Assessment Must be Evidence Based
We use our judgement all the time when working with individuals, especially those that report to us.

Some of us are able to do this very objectively, whereas others find this area very difficult and they cannot separate the person from the performance.

Using an evidence-based approach is the most effective way to overcome the problems of subjectivity and bias. Evidence must of course be produced from work that is real and relevant for that position.

A good starting point is to map out what evidence can be produced from the work itself, where it can be supplemented from the person being assessed (including knowledge) and/or witnesses and, of course, include observations or professional discussions from the assessor.

With a few exceptions, assessment on the job is the only way to assure competence and evidence from the work itself must be used within this process.

In Summary
Combining these principles, gives the basis for an assessment process that can produce consistent results in your approach to assessing and the basis for a robust verification process.

For more information, or to talk this through please get in touch with us at [email protected]