Psychometrics is a systematic approach to looking at individual differences. It is concerned with identifying factors (a) that predict future behaviour AND (b) that we can measure reliably or consistently
In 1905 a French physician named Alfred Binet (1857-1911) published the first ability test, which had been designed to identify students with special learning needs. Binet’s work turned attention to “intelligence” as the key differentiator of individual differences and predictor of performance. Binet’s work not only sparked the great twentieth century debates on “what is intelligence?” but also introduced new levels of objective rigour in assessment methodologies. By emphasising the concept of “reliability” he directed attention to quantitative research methods in measuring individual differences. Intelligence was established as something that was relevant to predicting future behaviour and could be measured with consistency. The standardised testing procedures Binet adopted were designed to “scrupulously control potential bias and error in the assessment procedures”.
Binet proposed standardisation of content- so if you and I were undertaking the same assessment we should be using the same tasks/ questions/ items. He demanded manuals documenting administration procedures. Finally his standardised methods required a transparent scoring system so valid comparisons can be made between the results of different individuals. The potential of standardised methods was identified by the American Military who extended the application of standardised screening to help differentiate potentially unsuitable or overanxious recruits for World War 1. Following the work of Binet in French Education and the American military in WW1, the relevance of standardised testing, to the measurement of both abilities and personality characteristics was demonstrated. These formative projects in the early twentieth century mapped the two domains (ability and personality) that remain the principal concerns of psychometricians engaged in standardised assessment to this day.
Concerns re Test Proliferation
With the advent of the internet there has been a huge upsurge in prevalence of psychometric testing. This has had a substantive impact on access to testing and has changed the rigid standardisation practices, which were so scrupulously controlled in Binet’s time. In response to this change bodies such as the British Psychological Society (BPS), European Federation of Psychological Associations (EFPA) and the International Test Commission (ITC) have come together to advise caution in interpreting tests which are widely accessible. In 2005 the ITC issued guidelines highlighting the need for caution given the diverse challenges associated with 4 newly classified testing modes.
• Open mode – These are the most accessible and are therefore from a security perspective are the least reliable. In this mode any test taker can have direct access to the test materials. In open mode there is no customised briefing from the administrator. Such tests include the books of tests you might buy in the local book shop. Internet-based tests without any requirement for registration are also considered an example of this mode of administration.
• Controlled mode – In this mode there is more security but limited supervision. Although no direct human supervision of the assessment session is involved, security is enhanced because the test is only made available to known test-takers. Access is typically limited to those who have received some sort of customised briefing with individualised username/ password. Internet tests will require test-takers to obtain a logon username and password. These often are designed to operate on a one-time-only basis. Other common controls include restrictions on the location where the test-taker can access it from and the time or date when completion may occur.
• Supervised mode – Where there is a level of direct human supervision over test-taking conditions. In this mode test-taker identity can be authenticated, which addresses substantive security concerns. For Internet testing this typically requires an administrator to log-in a candidate and take steps to ensure that the test had been properly administered and completed.
• Managed mode – Where there is a high level of human supervision and control over the test-taking environment including the equipment being used. In computer based testing this is normally achieved by the use of dedicated testing centres, where there is a high level of control over access, security, the qualification of test administration staff and the quality and technical specifications of the test equipment.
The 4 modes above all have their own benefits so consideration needs to be given to what is most appropriate in specific testing scenarios. In work and organisational settings, specific concerns arise over security and authentication in high stakes selection. Many organisations therefore only use an open or controlled mode for pre-screening recruitment. The assessment of shortlisted candidates can then be validated more stringently. This typically requires supervised or managed modes of delivery.
From a security perspective open and controlled modes have are less problematic where there are collaborative goals such as in a guidance or counselling settings. Nevertheless consideration should still be given to how assessments are delivered and special needs which may distort results in such scenarios.