Moving with the times...
If you applied for a job 10 years ago, the process you went through is likely to be quite different to what you would experience today...
But what journey has psychometrics been on and more importantly, why and how?
Take a step back in order to move forward...
Before we consider evolution, it’s important to first appreciate the origins of psychometrics and the birth of the first psychometric ‘test’. Records suggest that the first published ability test was the work of French physician Alfred Binet (1857-1911), who sought to identify students with special learning needs using standardised content, or what we know as a ‘test’. This was later picked up by the American military that utilised this standardised screening opportunity to help differentiate potentially unsuitable or overanxious recruits for World War 1.
There’s a lot more to the story but essentially, from the work of these two parties, people started to appreciate the relevance and importance of standardised testing in the world of assessment and recruitment more generally and the incentive for using psychometrics has grown as a result.
But where are we today?
Common observations amongst the HR literature are that organisations are constantly on the lookout for how to ensure they attract and retain the best talent, but competition for jobs is fierce for applicants and employers alike.
Employers have a need to get themselves out there, engage applicants and attract people to their organisations, whilst applicants need to sell themselves as a brand and demonstrate to potential employers that they have what it takes to be successful and perform in the role which they are applying for.
Whereas in the past selection was very much depicted as a linear process of advertiseàapplyàhire, many selection processes today now involve a number of consecutive stages. More often than not, a candidate must make it past the application sift, maybe undertake some pre-assessments, perhaps through a telephone interview and then demonstrate the desirable behaviours required for the role at an assessment centre in order to secure a role. It’s a hearty investment for both the employer and the candidate, and one that we need to ensure is both a positive and rewarding experience all round, regardless of the final outcome.
But how has our assessments evolved?
In this article we will be focusing on the evolution of ability tests and in part 2 we will explore the evolution of personality assessments.
Put simply, ability tests are designed to measure either a natural or learnt ability to do something well, whereas personality assessments seek to reveal insights into an individual's character or psychological makeup. But with numerous ability and personality assessments on the market today, along new tests being developed every day, it’s not surprising it feels like a bit of a minefield for employers to know how to assess candidates most effectively...
Deep Dive 1: Ability tests – The opportunity to measure people’s abilities and aptitudes
As the name suggests, ability tests measure either more general ability or a specific cognitive ability to do something; (i.e. numerical, verbal or abstract reasoning). Back in WW1 tests would have been administered in a physical format a.k.a. paper and pencil. One of the first major evolutions for ability tests was the movement to online testing.
Benefit number 1: this made it easier to embed ability tests into the assessment process at an earlier stage as classroom conditions were no longer required.
Benefit number 2: online testing also has the additional benefit of being quick and cost effective to administer, coupled with opportunities to generate instant computerised reports of the results.
Benefit number 3: online testing has the scope to accommodate ‘item banking’, where there is a bank of comparatively similar items that can be chosen from to make up the tests each time so no two tests will be exactly the same.
But what drove this evolution?
...Well essentially it seems to have been driven by the advances we have been able to make with technology. The enhanced ability of computers and computer programming has enabled us to increase exactly what tests can measure and how we can do it.
Whilst these advancements all sound epically wonderful thus far, we should not forget that computerised testing is not entirely free from potential drawbacks. One of the biggest concerns for many is the increased possibility for candidates to cheat with the removal of a supervised environment jeopardising the objectivity, reliability and validity of our test results.
So it must be noted that the old fashioned ‘paper-pencil’ and ‘supervised’ testing formats are not to be done away with just yet as they still hold key roles especially if we want to retest candidates to verify scores.
Aside from moving with the times and the technology, test publishers have also been striving to develop tests that are more reliable and fair for those undertaking them.
In a quest to do just this, we’ve begun to see the introduction of different ways of measuring performance and potential. Historically, many ability tests focused on challenging candidates to be both quick and accurate; placing candidates under significant pressure to complete a large number of increasingly difficult items within a certain time frame. However, studies are now showing there is added appeal to using time recorded tests over time limited tests (Capp, 2017 and Whisen, 2011).
The incentive and appeal behind time recorded assessments, over time limited, is its potential added benefits for minimising adverse impact (when certain groups are disadvantaged in a selection process due to a specific methodology). There is evidence from evolutionary research that men and women respond differently to stress (Maestripieri, 2012; Pilar Matud, 2004); and the stress caused by setting a time limit could favour men, and may lead to bias against women - something we strive to minimise and avoid by trying to make our methodologies fair for all.
On top of technology advances and adjusting test formats, test publishers have also introduced adaptive testing, whereby computerised tests can adapt according to the candidates level of performance. Adaptive testing works on the principle of item-response theory, where in essence, if a candidate gets a question right they progress to a more challenging question and so on and so forth, but when they get a question wrong they get retested at that level or go down a level if required. Think of it like a video game where you have to be successful in order to progress through the levels, the principles here are pretty much the same.
This metaphor of video gaming leads us nicely into one of the latest revolutions for our ability tests, the introduction of game based assessments. Once upon a time people would have been laughed at for suggesting we could use games to assess people, but low and behold test designers are now showing that you can ask someone to take part in a game and objectively assess their cognitive abilities.
This advancement seems to have hinged again on the opportunities technology is bringing us, and our increased focus on customer experience has resulted in us being drawn to increasingly creative and appealing ways of assessing.
The reality is, regardless of scepticism around the use of ‘games’ for assessment, game-based ability assessments have been shown to provide a great candidate experience and acceptable levels of validity, especially for select applicant groups (i.e. apprentice and graduate roles where candidates tend to be more tech savvy).
However, as stated before, we must keep in mind whether game-based assessment is going to be the most appropriate choice of methodology. To do this we should be considering to what extent the game is truly assessing the abilities we want to measure, that the game content is relevant and applicable to the level of role being assessed and going to be a fair assessment for test-takers from different cultures.
So, what does all this mean?
Well it seems the common thread amongst all of the developments for ability tests so far has circulated around advances in technology. So, with more on offer, in terms of both what can be assessed and how, it is the role of our Occupational Psychologists to help ascertain which specific assessment is most applicable for the needs of that unique assessment situation. Technology has its benefits but it is down to us to optimise this.
Read part 2 here.