Reliability and Validity
Like you, we are driven by reliability and validity, hence the test is now in its third iteration.
We invest in frequent independent statistical checking from world leaders in Psychology and Psychometrics at
Goldsmith’s College, London.
Psychologist and psychometric expert, Gorkan Ahmetoglu recently reported:
“The Psychometric Test™ is the result of 12 years of development. The measure has gone through
several
revisions and the latest version shows the measure has excellent psychometric properties.“
Reliability
Reliability refers to the consistency of a measure. It indicates the extent to which observations (scores)
are dependable. Reliability indices of The Psychometric Test™ show that a majority of the facets score
substantially above the standard (which is a score of 0.7). The average reliability of the 7 scales as
measure by Cronbach’s alpha is 0.74.
Social desirability
In psychometrics, social desirability checks are made in order to assess whether the items (questions) of
the questionnaire may be framed in a way as to engender socially desirable responding (sometimes called
“faking good”). All dimensions of the The Psychometric Test™ variables are normally distributed
suggesting that the items of the test do not engender socially desirable responses. There is also an
in-built test that helps indicate of a person’s responses appear high, relative to the norm.
Validity
Validity essentially refers to the usefulness of the test. Most notably, validity assesses whether one can
predict
performance outcomes based on scores on the psychometric tests.
Analyses show that all facet scores of The Psychometric Test™ are linked to important performance
outcomes. These include the budget and the duration of the project a leader will be in charge of, as well as
the experience they will have.
Incremental validity tests are made in order to examine whether the
personality
traits as measured by The Psychometric Test™ are important for performance after age and experience of
leaders
is
taken into account.
Our analyses show that several facets apparently predict the performance of project leaders, even after the
gender,
age, and experience of the leader are taken into account. In simple terms, the higher a leader scores on The
Psychometric Test™, the bigger the project they lead -
regardless of their gender, experience, or age.
he data also shows that some facets are important for all performance criteria, while others are only
related to
specific criteria (e.g. the duration of the project). This stresses the importance of considering specific
personality traits of leaders depending on the objectives of the project.
Ahmetoglu concludes:
“Overall, the The Psychometric Test™ is a reliable and valid measure that is not subject to
socially
desirable
responding [faking good results], making it an excellent tool for businesses to select and develop
top-level
project leaders. Given the relative lack of reliable and validated tools developed for these purposes,
The
Psychometric Test™ is likely to add substantial value to businesses and individuals alike.”
Dynamic norm
The norm group, against which each respondent is compared, is live: constantly evolving according to new
responses.
This adds an important dynamic component that many "static" tests don't have. In some other tests you may be
compared with generic "management" people who were operating many years ago in a very different world!
Part of the assessment process
Ultimately, all psychometrics present a useful starting point for discussion. The resulting data provides a
basis
against which people can be compared objectively and with which respondents can be scrutinised in a specific
and
deeper way.
For example if a respondent scores relatively high or low on “stability” you may want to focus questions
around
evidence of working under pressure, or design assessments that investigate this specific facet.
No self assessment should ever be used or claim the ability to predict human behaviour accurately on its
own. What
we are trying to do with our tool is to provide something specific to those in a programme and project
management
environment to
improve the consistency and accuracy of recruitment and personal development in
the field.