By Chris Baraniuk
Candidates hoping to land their dream job are increasingly being asked to play video games, with companies like Siemens, E.ON and Walmart filtering out hundreds of applicants before the interview stage based partly on how they perform. Played on either smartphones or computers, the games’ designers say they can help improve workplace diversity, but there are questions over how informative the results really are.
To the casual observer, many of the games might seem almost nonsensical. One series of tests by UK-based software house Arctic Shores includes a trial where the player must tap a button frantically to inflate balloons for a party without bursting them. In another, the candidate taps a logo matching the one displayed on screen, at an ever more blistering pace.
Afterwards, a personality profile is built using data on how someone performed, says Robert Newry at Arctic Shores. They claim the traits that can be measured include a person’s willingness to deliberate, seek novel approaches to tasks and even their tendency for social dominance. “What we are measuring is not your reaction skills, your dexterity,” says Newry. “It’s the way you go about approaching and solving the challenge that is put in front of you.”
Using the games, Siemens UK doubled the proportion of female candidates that made it past the initial stages of graduate recruitment than in the previous year, according to data recently released by Arctic Shores. Another company that makes such tests, Pymetrics, says its assessments have boosted recruitment of under-represented groups, with one financial services firm increasing the number of minority candidates offered technical roles by 20 per cent. However, it’s not clear if the boost could simply be down to an increased focus on or awareness of diversity in the workplace.
The games are meant to offer a form of psychometric testing and are based on techniques developed for measuring personality traits. But whereas in academic research the tests are generally calibrated the same way for all participants, when used for recruitment they are often tweaked depending on how existing employees at a company play them.
“We go into these companies and say, ‘Your individuals may be different. Let’s use your high performers to put together a data set,’” says Frida Polli, co-founder of Pymetrics. In other words, if your gameplay matches that of someone already at the firm, you’re more likely to advance to the next stage of recruitment.“You can develop a game-based assessment as rigorous as any traditional psychometric assessment,” says Richard Landers at Old Dominion University in Virginia. “But I don’t know how many companies actually succeed at that.” This is because it takes time and money to show that any assessment’s measurement of a given trait is statistically reliable.
Landers performed an independent review of game-like intelligence tests by Australia-based firm Revelian and says the results were reliable. Arctic Shores have also ran a study with around 300 participants to validate their games.
Caryn Lerman at the University of Pennsylvania has studied brain-training apps and says that although people’s improved performance at these can be tracked over time, they generally have no observable impact on cognitive ability in the real world. She is sceptical that playing the games well corresponds to ability to do a good day’s work in the office.
Although the game-based tests are mandatory, a company’s decision to interview someone may be based on other factors as well, such as their academic record. But in trying to find new ways of shortlisting the best of the bunch, companies risk alienating unsuccessful candidates, says Margaret Beier at Rice University in Texas. They might even expose themselves to lawsuits. “If I apply for a job, play games that seem totally unrelated to it and then don’t get that job, I might have a lot of questions about that assessment,” she says.