The myths and realities of psychometric testing Selecting the best psychometric testing tool for your organisation from the many available can be fraught with difficulty. Dr Paul Englert, Sarah Burke and Teresa MacGregor uncover the myths surrounding testing and outline the realities. sv chomeric testing (personality and ability testing is increas select the right candidate fora jab, Ince fection practices in Personnel Payholegy Found in New Zealand fs greater than in all but three Companies distributing ps ales techniqu Many of the ue of thelr products, 1 to improve the perceived jo not addres the true issues that en-us another. Tis article uncow ing the assessment testing industry and provides HR professional of curcing oF purchasing testing materials from the ‘Row operating in New Zealand. ging, and numerous ‘ssessment tests in their organisation, the quality of « psychometric test is, however, a difficult without specialist taining inthe area of ps ing services may have quality and differentiating test providers, Myth 1: Good quality psychometric tools must be expensive Reality: Psychometric tools are often expensive simply because in the past there has been litle competition in the New Zealand novewotr 05 employment today. 26‘marketplace. There is no reason why psychometric tools need to be expensive. Once developed th overhead for assessment testingisnot large and the cost of ongoing development requires only time and tho employment of suitably skilled individuals to continue research. IE we lok beyond Nevw Zealand at more competitive markets the costs for developing psychometric tools ae recouped primarily by longevityandusage volumes nothigh priestructores. Any company whose prices are well and truly above those of the competition is likely to have been a monopoly player in the past. twill continue to phy from this positon until competition slowly erodes the market advantage it had from being fist ofthe block (Myth 2: Being a dominant player in the market means a test pro Vider isthe best. Reality: Dominance does not, in itself, guarantee the best quality (Often it has come about because a company was the frst to market and its name has, accordingly become synonymous with psycho metric testing. A dominant player in the market means subsequent ‘competitorshave toovercome twointerrelated obstacles toconsumer behaviour inertia against change and market share. Psychological testing is lie banking, people change their provider only rarely. ‘The customer will complain about the costs, level of service and Ihidéen fees but, in reality, i's often too much trouble to change, Potential new customers, however, do not have to overcome a re sistance to change and are more open to decisions based on merit and value. Citing market share isa ploy to sell a product and it ‘works on the philosophy of If its good enough for them, then it is good enough for me" “Myth 5: Predictive validity studies demonstrate the usefulness of a tol fo ll organisations Reality: Predictive validity studies—refering to the ability of a psychometric tool to predict some future measure such as job per- formance—primarily demonstrate the usefulness of a tool forthe particular organisation in which they were conducted, Meta-analyses are the most powerful demonstration of general usefulness. ‘The question to ask is: “How do we use predictive validity to assess the usefulness of a tool for our organisation?” To demonstrate the usefulness ofa test, it is important to evaluate it against your ‘owneriteria, rather than bindly accept the validity studies produced inside other organisations that may have litle similarity to your ‘own, Some test producers use dubious predictive statistics primarily because the people they are selling to are not statisticians and there- fore have dificulty determining the value of the information they ave reading, Issues with respect to sampling errr, the application of significance testing, andthe promulgation of misleading corelation data all contribute tothe weakening of predictive data for any one 30 employment today woveweet 06 organisation, To ensure predictive validity studies apply to your organisation, you need to: 1. Check that a test has some relationship to performance. This is a starting point, like a tick ona cheeks. To be of value a test used for selection must predict the world of work, Also, check any meta-analysis research that combines single tests into joint studies, Does your test eat to the types of ests that were used inthese studies? Ifso, wecan infera degree of predictive validity for the tool, given it has been constructed well 2. Develop your own in-house study. This can be done by using a testing system that slores your data and enables it to be cor- related with job performance 3. Develop 2 multiple regression equation. This will give you ‘weightings to apply to test scores so that you can make predic- tions of performance 4. Plug the test scores for one individual into the regression equa- tions that come out and you will have a useful guide for your selection. All of this, by the way, can be done in-house in less that five minutes with the appropriate assessment software that allows you to make strategic rather than transactional decisions 6on the basis of assessment data, ‘The fllowing quote from an informed test u predictive vaiity should be used for: "Predictive valde shown by test procucers are anly good for demonstrating that the est has some utility for predicting work in general. It is a starting, point ‘ot an end point for HR professionals. It provides the neoessary support to then tral the tool in your organisation to demonstrate its predictive validity for you" demonstrates what Myth 4: Psychometric testing is a transactional sevice. Reality: Psychometric testing isa strategic initiative. Its value comes Primarily from having collected a lot of data and then using that data to model performance. Test providers have done a disservice to HR professionals by not giving them the capability to collect their own candidate data for later use. In real terms, this means that most organisations have litle empirical data about what predicts performance inside their organisation—despite paying thousands of dollars for assessment testing services over many years. This is, an injustice Iti equivalent to having a cash register that only holds the last transaction and des not feed a database of sales over the last month or year, ete. How useful is that? “Myth 5 Psychometric testing isthe domain of psychologists whose ‘main interests in furthering the discipline of psychology. Reality: Psychometric testing is too often the domain of business, people interested solely in making a profit In recent yeas, we have seen an infiltration of organisations funded not by psychologistsbut by capital venture firms and large corporate hackers. These stakeholders know litle about testing and, while they may get a few quick wine on the board, they tend not to translate thei in vestment into long-term returns. Indeed, the paychologiss in those teams may even be the casualties. Issues of psychometric robustness are slowly being erexed as tests are supplied over the intemet to tion data available without suitable training, or produced without any valida Myth 6: Good psychometric tests are made by psychologists, Reality: Psychologists use the tools but good payehometsic tests are made by psychometricians—i, those people whose have the neces ‘sary skills to make meaningful psychological measurement possible, Be aware that anyone ean call themselves a psychologist. Only the term ‘registered psychologist” indicates a person whose actions are governed by the Psychologist Act of 1981. Moreover, psycholo: sists qualified in one field can switch to another, eg, occupational psychology ane HR, without retraining or any’ in-depth induction They can do this because the legislation still does not differentiate between types of psychologist, Myth 7 Pating testo the internet i fcuk and that is wy few people offer Reality: Pating 9 test up on the internet is eayy. The internet provides convenience in many situations and has obvious poplar appeal ta: Some test provers have prospered thee-commerce environment and some have not. Psychometric test interpretation relies on standard esting conuliions. Insiruments use forthe purpose of selasensment by the individual have been relatively sy to mount onthe internat (e, career or welbeing assessment) In af selection, however, standardnation means that a mone con ‘role testing environments required and range of ober factors ree to be addeesed, sch as the Kentiy ofthe respondent and the lke, Other issues such as restricted acess and data storage also need to be addressed fone wishes 0 it kesing from a transac tinal toa sete evel authorise acento tet data for ait analysis ea pricy. These are but a few ofthe issues that ned to be adresse for reliable, valid, ethical peychometrc testing over the intemet. Tha 6 why dsceming providers have been careful and deliberate in their preparations and processes bre making prychomotic tests avaiable on the web. 5 Part of tis article wil appearin the nent issue of Employment Todey. ‘Paul Englert. Serah Burke are rectors and Tvesa MacGregor Js the Sout sand manager for Opra Consulting Group (espa cone) SUCCESSION PLANNING ee ee RCL) time to do the right thing for your business... or your client’s business” ING SucceSSION PLANER Available tober business is somewhere onthe succession pat Ly eee eee, Ce een ee a ene ae eee CU eee Bee Ouest en - read practical guide which will help you identify where Se ee eed Pere ce te eee Can Nar Tae Coes SE) See Nee eee) cc Secs ec ee eet Going to market Negotiations and settlement ee eS nee Do the right thing! CS aa ees oreo rens A eee re Gee Rr Email: serv LOTT ay et Novewaee 2006 employment today 31