Managing Positive and Negative Impacts of Psychometric Assessment

An oft repeated phrase in interpersonal psychology and communications training is to “describe the behaviour not the person”. There are rightly significant concerns about people being categorised in ways that limits them. Psychometrics has been used to lend scientific credibility to such classifications (e.g. 11+) and makes these categorisation more socially acceptable. Both ability testing and personality assessments are big business today. There is no significant commercial pressure on publishers to espouse the efficacy of their products. Salesmen promote “personality reports” as a labour saving tool for recruiters. Teenagers can complete a myriad of surveys online and publicly display their personality profiles on social networking sites. Arguably this exposure leads to less understanding not more. Psychometrics is also been criticised for promoting standardised assessments which in turn is seen are leading to a cramping of individuality in classrooms and a focus on developing only one rigid type of intelligence. Do these concerns outweigh the benefits of more objectivity? Do all ways of assessing people carry their own inherent bias? Can we trust practitioners to use and not abuse psychometrics? In the section below we will look at best practice in using tests for guidance and then for selection.

Using Test in Career Guidance
It is claimed that one of the great benefits of psychometric assessments is there potential for developing self awareness. In 1969, the Institute of Vocational Guidance and Counselling of Ireland published a booklet entitled “The Case for Proper Guidance”. The case was made for a guidance service which promoted “a realistic self-concept, self-determined yet responsible choices at all career stages” and developed “ a well adjusted adult who finds his maximum satisfaction in that way of life and personal satisfaction with life as a whole” (The Case for Proper Guidance, p12). The promotion of self awareness was seen as a challenge for Guidance Counsellors. Today all Irish primary schools are required to administer standardised tests in English and Maths twice during their primary school years (usually first and fifth class). Standardised assessments (e.g. DATS – Differential Aptitude Tests) are also frequently deployed in secondary schools to assist pupils’ subject selection for the leaving certificate.

Despite their widespread use the impact of psychometrics in Career Guidance has been frequently questioned. Best practice requires time to debrief, provide systematic feedback in the context of the whole person and only then to dispense advice. In my experience Guidance Counsellors rarely have this time. Too often psychometric assessments are squeezed into a crowded curriculum and people’s results are taken out of context. Best practice guidelines suggest we must systematically brief, administer and provide feedback to each individual. When this cannot be done it is better not to use the tools at all. Given current Career Guidance Resources best practice suggests that the Department of Education must continue its policy of scepticism / discouragement of the use of personality questionnaires.

Using Test in Selection
In selection situations tests and personality questionnaires are often advocated to improve the objectivity and, it is argued the fairness of the process. However, the issue of group difference is performance is a major concern in ability testing. If you are using ability tests for selection purposes, best practice is to monitor selection outcomes with respect to all of the 9 grounds*. In monitoring assessment outcomes we must watch out for adverse impact. Adverse impact is when one group outperforms another on an assessment. Adverse impact is not unfair discrimination and is not illegal where there are relevant job demands for which one group are more capable and there is no test bias.

* The 9 Grounds feature in The Employment Equality Act, 1998; which states that individuals should not be discriminated against on the grounds of their gender, marital status, family status, race, age, religion, sexual orientation, disability or membership of the traveller community.
Test bias occurs where an aspect of the testing process systematically underestimates the real ability a person has in the area being assessed. (e.g. testing people’s verbal reasoning ability in a language they are not fluent in). Steps are required to minimise test bias. Test bias can be controlled by appropriate test selection, a briefing process that identifies special needs and well managed administration. In summary where adverse impact occurs in a selection situation we must ensure that what we are assessing is relevant to job demands and that the testing process itself is geared to assess what it purports to assess.

In the personality area there is less of a problem with group differences and test bias. Indeed personality questionnaire used appropriately could reduce bias. For example, differences between ethnic groups in terms of personality are very small while interviewers continue to exhibit bias towards their own race. Nevertheless, there remains substantive concern over the ease with which questionnaires can be faked by those who understand how they are constructed. Therefore, the recommendation is made to always use a validation interview to test hypotheses from personality questionnaires and add breadth/ depth to the interview.

Commitment to Best Practice

This programme promotes Best Practice in assessing individual differences. In psychometrics best practice is concerned with objectively and systematically gathering data to minimise error through unreliable administration, design or bias in interpretation. Best practice also means we adopt an approach that maximises benefits while reducing concerns. The growth of psychometrics has helped define a vocabulary of individual differences and provided tools to assess the most salient differences. People professionals have the opportunity to use these tools but they must be used appropriately. Best practice standards provide guidance on how we select appropriate assessments (accepting the limitations of our tools), how we can work in collaboration with clients (informed consent), how to quality assure our administration, interpret results and provide feedback in a professional and respectful way.

At its best psychometrics can promote understanding of difference. This is an area where much work remains to be done. In schools many teacher’s still adopt a one size fits all solutions to the needs of diverse students. Career guidance and development is often based on false assumptions about people’s strengths and limitations. In the world of work too many managers still assume that everyone will act in the same way (carrot / stick theories) despite research evidence to the contrary. In day to day life we seem to fear and criticise others for the ways in which they are different to us while media analysis suggests everyone of a particular background will act and feel the same way

Leave a Reply

Your email address will not be published.