Scientific Approach

Psychological evaluation

A valuable element of risk management and a contributor to economic success

In the aviation sector human error can have costly, sometimes even catastrophic consequences. At the same time hand humans are still an irreplaceable resource when it comes to solving complex problems.

A psychological assessment contributes to the selection of suitable operators and helps tailor individual training. Selecting suitable staff for the flight deck significantly reduces the risk of adverse events - provided the assessment methods are reliable and predictive!

In aviation extensive research, job analyses and validation studies have led to the identification of factors that largely determine the probability of a person with or without flight experience to successfully undergo flight training and become a safe professional pilot. Airlines have early adopted a strategy to invest only into applicants who succeed in their selection schemes in order to safe millions in extra training costs for less able employees.

Even for flight training organisations there is not only a safety but also a financial aspect that speaks for the application of psychometrics. A school's ability to attract students, in other words: her economic success, is related to her graduates’ results in airline selection schemes as well as their performance in the cockpit. In addition to acquiring a reputation for providing excellent training methods, being particular about student selection will also contribute to a flight school's competitive advantage on the market. Provided the student selection is based on reliable and predictive assessment methods, accepted students will perform better during training and in subsequent jobs as professional pilots.

Furthermore, having a differentiated inventory of accepted students' potential will help instructors and teachers tailor the training to the specific needs of their students.

Testing and coaching

Unsystematic test preparation by commerical preparation service providers impairs test validity!

Most assessment systems do not take into account what is very well documented: test scores increase with repetition. Not linearly, but there are tremendous effects for the first couple of training runs.

This inevitably challenges validity! Take for example a score in the lower reagion of the accepted range: Are you looking at a talented but unprepared test taker? Or is this an untalented but highly prepared test taker?

MOLLYMAWK psychometrics is unique in its aim to provide optimum test preparation for all. As a consequence both false positive and false negative decision are less probable!

As much as we all welcome the benefits and opportunities of the dramatic progress in computer and information technology, for experts working in psychological evaluation it has also brought disadvantages and new challenges: Nowadays most tools used for personnel selection or potential analysis are discussed in internet forums or news groups, preparation courses and books. Perhaps more than for other jobs any method used in aviation for differentiation of individuals immediately draws the attention of the community of (aspiring) pilots. The reasons are diverse: most airline pilots receive a good salary and the profession in general has a favourable reputation with the general public. While pilots are often highly motivated individuals who are enthusiastic about their job, many who already acquired a licence have additional pressure of the need to pay back a loan they took for their flight training. These factors combined have facilitated the evolution of a busines sector that copies test principles and sells preparation material, software or seminars. These offers are available only for those who have the information and who are willing or able to pay a sometimes extortionate amount of money.

Preparing well for an important assessment should not pose a problem for the integrity and validity of the procedure. On the contrary: a careful preparation for an important step in one's career is indicative of a strongly motivated individual and should be valued. However, many psychological methods cease to measure what they intended to as soon as a considerable part of the item pool has leaked to the public. For example: doing knowledge based tests with candidates who have prepared by memorizing specific question-answer-combinations leads to scores that do not correlate with the relevant skills any more but instead reflect memory, motivational and other effects. To a certain degree, such a test score may reflect the extent of an individual’s financial resources (how much money he/she was able to invest in test specific preparation material) or social network (the opportunity to talk to people who did the test before) rather than the specific skills that the test originally was designed to quantify.

Basing one's personnel decisions on such test scores may lead to accepting persons who in a real life situation turn out to have problems because memorizing specific question-answer combinations for an assessment usually does not mean that an individual can apply the memorized information in a different context. Only the 'true skill' would.

Similarly, the results of tests for basic cognitive aptitudes are subject to a learning effect. I.e. the test scores usually improve with training, even though the underlying aptitudes remain rather stable. If a subgroup of candidates has the opportunity to practise extensively with more or less authentic material the variation in a test's results no longer reflects the variation in the underlying aptitude but rather the training level and the variation of test specific skill. Again, basing the selection on such test scores may lead to accepting students or pilots who had excellent preparation conditions but not necessarily the ones who have a true talent in addition to a solid motivation.