Psychometrics get a bum rap

I got stabbed in the heart the other day on Twitter. The knife was taken out and plunged once again in my gut as I re-read the tweet. It made my heart bleed. I reeled from the damage and had to take a few days thinking time off. My body needed serious recovery, and I sought some internet counselling. What was it that caused such pain and hurt to my person?

“I’m about to go through some Myers Briggs training. Give me strength.”

Oh my eyes! Oh my soul! Please excuse me while I go cry for a moment.

I asked the question, what is it about going through the training they weren’t looking forward to. I was told it was because it’s like reading a horoscope, it could be applied to anyone.

Sadly, this is true.

Except.

A clinical trial takes years of development before it is passed and allowed to be used by medical professionals to treat patients. Thousands of people go through the trials, and a complete data set is produced that helps doctors to understand under what circumstances should a particular drug be used. What medical condition should the patient be presenting that suggests a drug should be administered? What population is it meant for? What age range is it most appropriate for? What are the limitations of the drug? What are the possible side effects of the drug? When will you likely see the benefits of taking the drug? Who commissioned the testing, the development and the production? Are there ulterior motives?

So by the time the doctor says you should take this drug, this many times a day, for this period of time, it’s with a strong degree of confidence that the doctor thinks it will help improve your condition.

And at the same time, the doctor could just as well prescribe a placebo, and there’s a fifty/fifty chance it could help improve your condition because you believe it will.

That drug is meant for millions. There is no guarantee it will help, just a very educated guess.

There’s a large number of people who will accept the drug because they trust the doctor, and they’ll trust it will help improve their condition. And there’s a smaller number of people who will scrutinise to the nth degree the prescribing of drugs, because they think they know better than the doctor. Only a very small number of people actually do.

Which is interesting then that there’s such reluctance to consider the usefulness of psychometric testing. In and of themselves, psychometrics are designed to provide insights on quite specific things. They’re not meant as a one size fits all. So I wonder why there is reluctance about using them. For example, the 16PF measures 16 factors of personality. The Myers Briggs Type Indicator measures 4 pairs of preferences. The Belbin Team Roles measures 9 different preferred team roles.

They’ve all gone through quite rigorous testing and development which allows the facilitator a degree of confidence that they’re going to help the person understand something better about themself. The person needs to be open to this, and open to what it might raise for them. It’s that self-awareness which is important because it helps us to be able to decide how to act. It doesn’t have to come via a psychometric, and I’ve written before about the association people have with profiling. It could come via coaching, or via mentoring.

I feel psychometrics get a bum rap because of bad experience with them. A facilitator didn’t help the person gain something. A facilitator fumbled their way through the tool and it wasn’t a useful exercise. A facilitator was inexperienced in the tool and created more uncertainty about the value of the tool.

Or possibly the person receiving the feedback has passed on their own cynicism about the tool. Or their bad experience of it.

Psychometrics get a bum rap. What do you think?

Advertisements

It’s all about the profile, isn’t it?

I’ve always been fascinated by horoscopes. That there is some power in being able to look at the stars, read some writing done about zodiac signs, and devise a short paragraph which is allegedly prophetic in nature. I mean it’s all complete nonsense, but fascinating all the same – and even more surprising is the industry that has been built up around it. That the likes of Russel Grant and Mystic Meg have made their fortunes through such activities is both a farce and a stark look at how susceptible people can be to suggestion.

And it’s not people who aren’t clever, it’s very regular people who have had a good education, with a critical media around them, with the likes of Derren Brown showing just how much is down to chance. But people want to believe. Regular Joe has a need to invest something of them into this tom foolery.

But to what end?

The attraction from these types of Barnum statements is that they can apply to anyone, but they’re couched in such ways that the person reading them thinks it’s only written for them. Forget the millions of others who share the same period of the year in which you were born, it’s all about you.

Which presents some real challenges and problems when it comes to trying to profile people using personality tests.

In the corporate world, psychometrics are used in a variety of ways from personal development, to recruitment and selection, to management and leadership development, to team development. They’re used because they’re designed to not fall foul of the Barnum effect.

A well developed and designed psychometric will have some core features which offer it a level of confidence that the profile produced can be trusted.

It will be a reliable test. That is, if you take one tomorrow, and then a year later, it should yield the same result.

It will be a valid test. The questions should produce a specific response every time – it measures what it’s meant to.

There will be a standardised set of instructions. This mitigates for personal influences you might have of, and biasing the results in a certain way.

There will be a set of normative data produced. That is, you can reliably look at your profile against a cross section of the population in different categories and have reasonably confidence about how you compare to them.

These things are important in giving a person confidence that the profile produced is as accurate as it can be.

The challenge with profiling is that people don’t like to be categorised as X or as Y. They want to be unique. They want to be Bob, not Bob Who Shares a Similar Profile as Millions of Other People. They want to know they can act and behave in ways that no one else does.

Except we all act and conform greatly. Pretty much every act we do in day to day life is an act of conformity and accepting a set of norms to adhere to.

So when those of us trained in these tools work with people in helping them to understand their profile, there’s a lot to contend with. In my experience I’d say it’s a 60:40 split – 60% will go along with what you’ve described, and what the profile suggests. The other 40% will need convincing.

And considering a lot of people buy in to horoscopes which is about generalisms and the majority, it’s interesting that in corporate life we don’t want to trust a set of tools which at their very heart are about helping an individual.

The science of… Psychometrics

Yesterday I started a series of posts on: The science of… Occupational Psychology. Today I continue with talking about psychometrics.

You mention psychometrics and people immediately think about profiling, being boxed in, being classed as unsuitable, and a host of other negative associations. It’s all hogwash of course. These things are spouted by those who have zero concept about how psychometrics should be used, their value and the insight they provide.<

Personality Theory

Where do we start? Well the first thing to understand is that psychometric tests are all about providing an easy to understand frame of reference for personality. This frame of reference is often steeped in two schools of thought. They are either based in trait theory or type theory.

Trait theory is about a scale of behaviour. The theory argues that we all have a range of behaviours, and we will exhibit various strengths of those behaviours. For example, we all have the capacity for ‘social boldness’ but we may differ the extent to which we display that behaviour. We can have a strength in this behaviour or it can be a weakness. The most popular psychometric that uses trait theory is the 16PF personality questionnaire – distributed by OPP Ltd in the UK.

Type theory is about either exhibiting a behaviour, or not. The theory argues that we will all have preferences for behaviour, and this is the place we will default to in any given situation. We might be able to learn the opposite behaviour, but this does not mean we can do both at the same time. It means that we develop a maturity in our understanding of behaviours and are able to exhibit both types. Thus, we may be extrovert by preference, but equally able to exhibit introvert behaviours when appropriate. The most popular psychometric tool that uses type theory is the Myers Briggs Type Indicator – distributed again by OPP Ltd in the UK.

Types of Psychometrics

As well as personality based psychometrics, there are also other types of psychometrics which are very commonplace – the biggest distributor of which is SHL in the UK.

Aptitude tests and ability tests measure your ability to do a certain task e.g. analytical skills, inference skills, deduction skills, critical reasoning.

Verbal reasoning tests measure your ability to understand verbal instruction.
Numerical reasoning tests measure your ability to understand mathematical problems

Construction of Psychometrics

The key thing that sets psychometrics apart from other questionnaires such as Belbin team roles or the Honey and Mumford Learning Styles, is that there is rigorous construction of the questionnaires. Every psychometric developed goes through a process of being validated.

This means it has to show to be reliable. That is, if you retake the questionnaire, your answers will be consistent.

It has to also show to be valid. That is, a set of or bank of questions measure what it purports to measure.

A set of norms is produced to enable a benchmark from the results. That is, whatever your results may show from a psychometric, you are measured against an appropriate norm group, and as such your results interpreted appropriately.

Standardised administration is a key part of psychometrics. Instructions on how to complete a questionnaire must be understood by anyone undertaking the test.

Feedback and Interpretation

The most important part of completing the questionnaires is receiving feedback from a fully qualified person. Qualification means they have attended a training programme where they learn about all the things I’ve mentioned above. Any person claiming they are qualified will have 2 certificates to prove this. One is the ability to administer and feedback results – a Level A qualification in occupational competence. The second is the ability to use, administer and interpret a specific personality tool. This is the Level B qualification in occupational competence.

A qualified person will be able to take your results and provide insight to you based on the answers you’ve provided. At no point should this be judgemental or profiling. Instead it should be only about feedback and insight.

Once you’ve received feedback you should always receive a report that explains the results.

Myths about Psychometrics

There are those who will tell you that you can fake a test, or answer it in your favour. The likelihood of you being able to do this is seriously slim. The construction of psychometrics means that the questions are designed to not be faked. that’s why you’ll often find that the same question seems to be asked several times in different ways. That’s done so you answer consistently. You might be clever, and you might think you can fake it, but you can’t. Trust me.

There are those who will tell you that you can’t change once you’ve been profiled. Oh that’s just nonsense. First you’re not being profiled. You’ve provided a set of answers and based on the information you’ve provided a set of results are produced. It’s totally based on the information you’ve given. Second – and importantly – you can change your behaviours. Significant life changing events can have profound impacts on us and they do. Death, birth, job change, redundancy, divorce, marriage, all have profound effects on our condition. And they can influence and change your behaviour. It does normally have to be something quite significant though in order for your behaviour to change.

There are those who will claim they can exhibit all behaviours all times of the day. Idiots. As I’ve explained above, you can learn behaviours, but that takes time and you will default to a way of being in most situations. You can and will learn how to act differently, but this will often be in relation to and dependent on the situation you are in.

Posts in this series:

The science of… Assessment Centres
The science of… Competency Frameworks
The science of… Ergonomics
The science of… Appraisals
The science of… Learning and Development
The science of… Occupational Psychology