Reliability and Validity
Like you, we are driven by reliability and validity, hence the test is now in its third iteration.
We invest in frequent independent statistical checking from world leaders in Psychology and Psychometrics at Goldsmith’s College, London.
Psychologist and psychometric expert, Gorkan Ahmetoglu recently reported:
“The Psychometric Test™ is the result of 12 years of development. The measure has gone through several revisions and the latest version shows the measure has excellent psychometric properties.“
Reliability refers to the consistency of a measure. It indicates the extent to which observations (scores) are dependable. Reliability indices of The Psychometric Test™ show that a majority of the facets score substantially above the standard (which is a score of 0.7). The average reliability of the 7 scales as measure by Cronbach’s alpha is 0.74.
In psychometrics, social desirability checks are made in order to assess whether the items (questions) of the questionnaire may be framed in a way as to engender socially desirable responding (sometimes called “faking good”). All dimensions of the The Psychometric Test™ variables are normally distributed suggesting that the items of the test do not engender socially desirable responses. There is also an in-built test that helps indicate of a person’s responses appear high, relative to the norm.
Validity essentially refers to the usefulness of the test. Most notably, validity assesses whether one can predict performance outcomes based on scores on the psychometric tests.
Analyses show that all facet scores of The Psychometric Test™ are linked to important performance outcomes. These include the budget and the duration of the project a leader will be in charge of, as well as the experience they will have.
Incremental validity tests are made in order to examine whether the personality traits as measured by The Psychometric Test™ are important for performance after age and experience of leaders is taken into account.
Our analyses show that several facets apparently predict the performance of project leaders, even after the gender, age, and experience of the leader are taken into account. In simple terms, the higher a leader scores on The Psychometric Test™, the bigger the project they lead - regardless of their gender, experience, or age.
he data also shows that some facets are important for all performance criteria, while others are only related to specific criteria (e.g. the duration of the project). This stresses the importance of considering specific personality traits of leaders depending on the objectives of the project.
“Overall, the The Psychometric Test™ is a reliable and valid measure that is not subject to socially desirable responding [faking good results], making it an excellent tool for businesses to select and develop top-level project leaders. Given the relative lack of reliable and validated tools developed for these purposes, The Psychometric Test™ is likely to add substantial value to businesses and individuals alike.”
The norm group, against which each respondent is compared, is live: constantly evolving according to new responses. This adds an important dynamic component that many "static" tests don't have. In some other tests you may be compared with generic "management" people who were operating many years ago in a very different world!
Part of the assessment process
Ultimately, all psychometrics present a useful starting point for discussion. The resulting data provides a basis against which people can be compared objectively and with which respondents can be scrutinised in a specific and deeper way.
For example if a respondent scores relatively high or low on “stability” you may want to focus questions around evidence of working under pressure, or design assessments that investigate this specific facet.
No self assessment should ever be used or claim the ability to predict human behaviour accurately on its own. What we are trying to do with our tool is to provide something specific to those in a programme and project management environment to improve the consistency and accuracy of recruitment and personal development in the field.