Psychometrics get a bum rap

I got stabbed in the heart the other day on Twitter. The knife was taken out and plunged once again in my gut as I re-read the tweet. It made my heart bleed. I reeled from the damage and had to take a few days thinking time off. My body needed serious recovery, and I sought some internet counselling. What was it that caused such pain and hurt to my person?

“I’m about to go through some Myers Briggs training. Give me strength.”

Oh my eyes! Oh my soul! Please excuse me while I go cry for a moment.

I asked the question, what is it about going through the training they weren’t looking forward to. I was told it was because it’s like reading a horoscope, it could be applied to anyone.

Sadly, this is true.

Except.

A clinical trial takes years of development before it is passed and allowed to be used by medical professionals to treat patients. Thousands of people go through the trials, and a complete data set is produced that helps doctors to understand under what circumstances should a particular drug be used. What medical condition should the patient be presenting that suggests a drug should be administered? What population is it meant for? What age range is it most appropriate for? What are the limitations of the drug? What are the possible side effects of the drug? When will you likely see the benefits of taking the drug? Who commissioned the testing, the development and the production? Are there ulterior motives?

So by the time the doctor says you should take this drug, this many times a day, for this period of time, it’s with a strong degree of confidence that the doctor thinks it will help improve your condition.

And at the same time, the doctor could just as well prescribe a placebo, and there’s a fifty/fifty chance it could help improve your condition because you believe it will.

That drug is meant for millions. There is no guarantee it will help, just a very educated guess.

There’s a large number of people who will accept the drug because they trust the doctor, and they’ll trust it will help improve their condition. And there’s a smaller number of people who will scrutinise to the nth degree the prescribing of drugs, because they think they know better than the doctor. Only a very small number of people actually do.

Which is interesting then that there’s such reluctance to consider the usefulness of psychometric testing. In and of themselves, psychometrics are designed to provide insights on quite specific things. They’re not meant as a one size fits all. So I wonder why there is reluctance about using them. For example, the 16PF measures 16 factors of personality. The Myers Briggs Type Indicator measures 4 pairs of preferences. The Belbin Team Roles measures 9 different preferred team roles.

They’ve all gone through quite rigorous testing and development which allows the facilitator a degree of confidence that they’re going to help the person understand something better about themself. The person needs to be open to this, and open to what it might raise for them. It’s that self-awareness which is important because it helps us to be able to decide how to act. It doesn’t have to come via a psychometric, and I’ve written before about the association people have with profiling. It could come via coaching, or via mentoring.

I feel psychometrics get a bum rap because of bad experience with them. A facilitator didn’t help the person gain something. A facilitator fumbled their way through the tool and it wasn’t a useful exercise. A facilitator was inexperienced in the tool and created more uncertainty about the value of the tool.

Or possibly the person receiving the feedback has passed on their own cynicism about the tool. Or their bad experience of it.

Psychometrics get a bum rap. What do you think?

Advertisements

Published by

Sukh Pabial

I'm an occupational psychologist by profession and am passionate about all things learning and development, creating holistic learning solutions and using positive psychology in the workforce.

16 thoughts on “Psychometrics get a bum rap”

  1. Hi, i think they do get a bum wrap, and often amongst the people who would benefit most. I think a lot of larger companies use them and people are happy with them as they have been used both for recruitment and development so mostly people have had an ok experience with them.
    I think the challenge is amongst smaller companies where there is a different view. My thoughts are that there are two experiences. Been for a job, done a psychometric and “failed” so they have possibly discounted the psychometric, rather than getting some feedback around “so you were not right for that job, and lets look at your strengths….”
    There are also some poor facilitators too who give the profession a bad name.
    I recently attended a local chamber of commerce event, where a guy started talking about how psychometrics did not work, how MBTI was poor and other tools too. He then went on to explain that he had spent three days training in the US with some guy (checked him out as a motivational speaker) and was licensed to use this tool which was the answer. The tool was a version of Social Styles/Bolton and Bolton which describes 4 communication styles (driver, expressive, amiable and analytical). Shocking, even worse he was telling people he would help with recruitment!!
    My other story, loads to get off my chest here, was someone locally had developed a take off of Belbin. Very clever, and nicely packaged around a theme of dogs. Using it in schools, which I was ok with until I heard that a head was using it for recruitment, and that some teachers whilst helping understanding peoples better were possibly then “boxing” in the kids. As my wife says “just cause I am a completer finisher it does not mean I am not creative”.
    So what I am trying to say.

    There are a lot of poor experiences out there, also not helped by people using stuff that they are not qualified to use, or using them for purposes they never were intended for, or valid for. No thats a bum rap!!

    1. Wow, this is a good set of stories which share some of the experiences around bad use of psychometrics.

      Where there are facilitators who misuse certain tools that’s a real challenge in the profession. I’m keen on rooting these people out and showing them up for what they are – shams! It takes time, but I hope with conversations like this we can get there.

    1. Thanks for the link sharing your experience of the development of bad tools in India. I had no idea that there was such bad practice and how prevalent it is.

      You’re right, it raises important concerns for the profession. I’m glad you’re able to know what good development and certification means.

  2. Heh heh – these things get a bum rap because they’re over rated, mostly by the people who inflict, sorry I mean use them on, sorry I mean with, others.

    Malcolm Gladwell wrote, ‘If I happen to remember my first boss, then I come out as a Thinker. If my mind is on my second boss, I come out as a Feeler. When I took the Myers-Briggs, I scored as an INTJ. But, if odds are that I’m going to be something else if I take the test again, what good is it?’

    http://www.gladwell.com/2004/2004_09_20_a_personality.html

    I’m gonna misquote Carl Jung, but when he saw what Myers and Briggs had done to his research – he said, ‘man this sucks innit, unless you is gonna use it as a game, like.’

    http://www.guardian.co.uk/commentisfree/belief/2011/jun/27/carl-jung-psychological-types

    Yes I accept that these tests can be helpful, but personality metrics are about as scientifically meaningful as star signs.

    Old wine – new bottles.

    http://stopdoingdumbthingstocustomers.com/learning/mbtwhy/

    1. Appreciate the comment here, Doug. Your comment about being as scientifically meaningful as star signs is a point I made in the post I link to above about profiling.

      You and I fundamentally disagree with the use psychometrics, and I’m cool with that.

      I am in accord with the argument that it can be nothing more than a parlour trick, and that’s the challenge of the facilitator to help people see they can be more useful than that.

      It’s something OPP are quite hot on, and they produce a lot of material to demonstrate the validity and reliability of the MBTI in particular.

      Professionally, I’ve always been mindful that I’ll only suggest the use of a psychometric if I think it can offer something extra to the solution I’m suggesting. I find that when you put it in the right context and it’s used in the right way it makes it a much more useful tool than could otherwise be the case.

  3. Interesting stuff! I get the sense that you are taking someone’s comment out of context and using it to make a more general point – that kind of misses their point. What kind of ‘training” is it that the commentator was so dreading and what kind of employer makes him or her go on such training? Surely those tools that seek to address such deeply personal issues as a person’s personality or core behavioural traits cannot be trained into people against their will, or can they?

    I have been MBTI trained and I think it is a fundamentally bad tool, bad in its conception, its provenance, its routine and common abusage and its reinforcing of a paradigm that at best miseads and at worst positively harms people.

    You compare MBTI to prescribed medicines, but so far as I can tell it has not been rigorously, “scientifically” tested in the way that pharmaceuticals must be before being licensed. I have read studies purporting to ‘prove’ its legitimacy, but they have seemed partial and unconvincing. MBTI’s creators claim it is based on Jung’s work, but Jung refused to associate himself with it, despite Its creator, Katherine Briggs, asking him repeatedly to endorse it. He said it was “futile to stick labels on people, nothing but a parlour game”.

    Like many others, I admire Jung and greatly value his insights into the human condition, which are mostly not ‘scientific’ in the sense of objective and rational (nor claimed to be), but are nevertheless profoundly sensible, I feel. Much of his insight and wisdom is based on his own introspection, coupled with his psycho-analytical observation of deeply troubled patients who came to him for help.

    I wonder if MBTI is popular partly because it meets a modern cultural want to categorise, box and label? The Briggs’ mother and daughter who founded and developed the tool meant well and hoped their tool would lead to greater peace and harmony, but did not see how it played to a 20th Century management Tayloristic paradigm in which employers (especially big corporate/government employers) came to view employees as disposable standardised machine parts to be fitted and kept in their place by management in the sole and constant pursuit of more profit for the shareholders. It seems to me that all psychometrics are deeply culturally embedded, which is rarely apparent within that culture, but startlingly apparent to those from different cultures.

    I do not believe that personality is fixed, static or unchangeable and independent of context. I think that people are constantly adapting and becoming, constantly learning and unlearning. They may find recurring patterns in their behaviour that help them, or not, and they may find great value in coaching to help them to raise their self-awareness, but I don’t think that MBTI does much to help that and in the wrong hands, or in the wrong context, or used in the wrong way, MBTI can and does do a great deal of harm, all either with the best of intentions or without much mindful consideration of its impact on the subjects of its impact.

    As for 16PF, it is not only that it claims to measure 16 Personality Factors as that is claims to measure the 16 Personality Factors. Arthur Cattell claimed his tool was comprehensive. I reject that absolutism and I feel antipathy (and pity as well) to Arthur Cattell who developed the 16PF tool. I am also troubled by reading that Cattell enthusiastically promoted eugenics alongside his research into his 16 factors, eventually creating a new religion, Modernism, that sounds like the ultimate dystopian extension of Taylorism from management theory to religious dogma (“a New Morality from Science”, 1972) .

    I think that people are far greater, richer and deeper than any model can possibly portray. I share Carol Dweck’s beliefs that our talents and our personalities are not fixed or given, but capable of, desirous of continual development. I do not believe that any purportedly ‘psychometric’ tool is capable of measuring immensurable psyches and that their use is part of a much deeper social malaise that Iain McGilchrist explores so powerfully in “The Master And His Emissary”, the essence of which is that in he last hundred years or so, we have fatally focussed on disembodied, spuriously precise and apparently objective measurement at the expense of exploring how things, including people, interact, develop and become.

    Do psychometrics get a bum wrap. Not as much they dispense them day in, day out around the corporate world, I think!

    1. Jonathan, thanks for this very thoughtful and considered response. It’s clearly hit a note with you and is a topic you’ve clearly discussed in other forums.

      I’m afraid I can’t accept that the MBTI positively harms people. In the same way that I disagree with Doug about the use of psychometrics, I think it is clear you and I will disagree on this too.

      With any psychometric tool they only ‘harm’ people when they are used against their design. The purpose of any psychometric is to provide a framework, or a structure within which to raise self-awareness. There are people who appreciate this, and how it can help them.

      I take the point that effective coaching can do this, and it is certainly one of the solutions available to professionals in our field.

      I am in absolute agreement with you that no psychometric can fully capture the human condition. We are far too complex and unique to be whittled down to a set of 16 factors or 4 preferences or 9 team roles.

      It seems to me that you adamantly refuse to use any psychometrics for your practise. Is this the case, or am I reading too much into your comment?

      First and foremost we should try to understand what help the person needs. I’m completely open with myself about my limitations and my strengths. I’m open to using tools where I can see how they will help, and I’m confident in my own abilities to help others through good dialogue.

  4. A random selection of thoughts.
    There was another blog about MBTI recently (the cynical girl) and the majority of the people who disliked MBTI were INTJs I seem to remember, with a few INFJ’s thrown in. I found that interesting!
    I’m like to think I’m interested, I’m curious, I’m open, I avoid dogma and being dogmatic, except I’m dogmatic about that. Having a framework to explore self and relationships can be very useful but nothing is everything about us. We are not defined by these things. I’m a Leo according to star signs, an ENFP according to MBTI, an activist according to H&M, and other things according to other frameworks too. There are some overlaps between them all. But I’m not one thing.
    I think there can be a reliance on tools and techniques of all sorts that is not healthy, and an insistence on a “truth” that is attached to selling something that can be arrogance. Systems thinking for example *ducks*; I agree with pretty much most of what they promulgate, but they (and they are a they, because they think the rest of the world are inferior in some way in their thinking) separate and divide working communities because of their insistence of theirs being the way.
    I have met so many people who have been “mbti’d” that have the impression that their report defines them, that they are an x rather than “here’s something interesting that describes preferences and you can play with this knowledge to increase insights” etc.
    Good coaching is grounded in some foundation school of thought, it ain’t a random ability to be a great coach, there are frameworks in abundance across the social sciences to helps us understand each other.
    I don’t have to be right, you don’t have to be wrong.
    I think I’m rambling but I like the cut of your blog jib Sukh.

    1. No need to duck in these here parts, Meg. It’s all good discussion.

      I think you’ve got it right where you say that you are not defined by these things. Perhaps that’s the piece I haven’t been clear about myself.

      I’m certainly not defined by these things, and don’t wish to be. I use them for myself and with others because I see the potential they can help to realise.

      Appreciate the comment, thanks.

  5. Did you see this, Sukh: http://www.guardian.co.uk/science/brain-flapping/2013/mar/19/myers-briggs-test-unscientific

    I remember doing the MBTI and in the process challenging the fact I did not sit well with binary opposite options.

    I then got my number and heard nothing more from my manager, the team or the business. As far as I could see it had no relevance to me and my development in the company. MBTI is a multi-million dollar industry – is it a good investment?

    Interestingly with the analysis of DNA we should see far more personalised drug therapies in the future based on DNA/medical problem for each individual – no one size fits all approaches. That’s interesting and if we were to take that concept into personality testing in the workplace then it could take such testing in quite a different direction.

    I’m not sure the likes of MBTI can ever be that helpful. Wouldn’t self-awareness, mindfulness and being aware of your biases be more useful.

    I guess Taylor attempted to make workplace efficiency a science – and from that has grown ‘worker science’. The only problem is that humans are messy beings. If we took that as our starting point then our approach might be a bit more art as well as science.

    1. Thanks for the link, Martin. It’s a good read and mirrors the criticisms from Doug and Jonathan above.

      I’ve always been mindful that I don’t force people to accept their type. It’s a set of preferences at the end of the day, and we learn through life how to adapt to different situations and express different preferences.

      In organisations I’ve seen good usage of the tool too. It’s been regularly used as part of management meetings as opposed to a one off affair. It’s been used as a way to help teams talk to each other and give feedback. It’s been used as a way to share insights about team members in an open and transparent way. These were all ethical and good practices – nothing underhand about the way they were carried out.

      What’s interesting is that the bad experiences like you experienced shows how it can be used in some areas and nothing more is done with it which makes people question its value.

      As with any tool, the value we gain from it arrives from the continued practise of the tool. As a one off it will always raise questions about value. Used over time, it can be seen how it helps organisations.

      It should never be the sole way organisations interact or engage with each other, and should always be part of the mix of engagement solutions.

      I could equally challenge the validity of coaching and mindfulness as approaches to gaining self awareness. They have their place as with most things, and we need to use them in the right ways.

      This is the challenge we face in L&D, that we have to offer solutions which work, and which offer value. We can only do this through understanding the models and tools we come across and being critical of them.

  6. Some great and diverse comments.

    I do think its all about intent of how we use such tools. I personally like using MBTI and other psychometrics. Why, because I feel it stimulates a conversation and that is my intent. Normally, its not threatening, there is no right or wrong or even binary answer. We are all shades of everything, and if as a minimum it gets people to reflect and think about themselves then it opens up all sorts of opportunities and learning.
    A slight aside, on the odd occasion I do time management training, I start with the question are you planned or spontaneous? Depending on your preference might impact how you receive some of the training and choice to apply it.
    Personally with things like MBTI I don’t like to label or box people,if people are just able to reflect on the 4 dichotomies and use them to understand themselves and others then I consider that useful and helpful. Am I an ENTJ or an ISTJ I would question how useful that is! If it gives some a useful framework to plan for a conversation or reflect on a situation then I feel thats ok, but context, the situation and the people involved add another level of complexity and personality tools should therefore be taken as some kind of magic bullet.

    Bad practise I think is an issue.
    I have done numerous MBTI sessions where the person has never done the self awareness conversation either individually or as a group and been given a computer print out of their 4 letter type. They neither understand the score or helped to understand what it might mean.

    Ultimately, what ever we do, should be of value and in service to our clients who we are trying to help. Tools or not, don’t think it matters as long we equip them in a robust and sustainable manner.

  7. I am with you Mr P in appropriate use and context and careful considered handling. I have seen and had feedback that people were visibly moved to something better in their thinking and acting (their words) and that has to be good whatever your belief of their validity and usefulness. You could argue they could have been moved by their “numbers” or having their “colours” done. If the impact of use is a helpful, positive and potentially lasting one then great.

    I know lots of people take offence at the MBTI boxes element. And whilst being of a punk/non conformist persuasion I should hate that. But I found comfort and intrigue in my box and enhanced understanding, appreciation and similarly intrigue about people in their boxes.

    Gospel truth? Nope. Harmful? Nope. Do others categorise me because of MBTI type (ENFP btw). Maybe. But they might not like my eye colour or musical tastes or report drafting style. I can deal with that and I won’t default to “well that is because they are an INTJ”.

    Whatever people feel about them (psych tools) where they are used in pursuit of greater understanding in the right way/place/time then we have nothing to worry about.

    Maybe controversially, I feel it isn’t psychometric tools that damage people, other people do.

  8. Love this debate.

    Personally, I’m a fan of profiles/psychometrics but (a) as part of wider processes, (b) not as an exclusive, conclusive judegement or (c) anexcuse to do other things properly (eg manage performance)

    The fact that the focus here is on MBTI may explain some of the problem, in as much as people may not look at which tool is most appropriate for the expected outcome – which needs to be clear at the outset.

    To those who are dismissive I always say that the ‘tools’ are just that – put a hammer into the wrong hands and it can reek havoc, but give it to someone who uses it appropriately, eg with a chisle and you can really sculpture something. I’ve recently spent time with a head-hunters psychologist as part of a recruitment exercise and everyone involved found the validated analysis very useful.

    Used properly the insights phychometrics provide (for discussion) over-ride the issues, and it seems to me the real problem here is about poor practioners/poor communication rather than the tools themselves.

Say something...

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s