Calibration analysis within the cognitive and personality domains: individual differences in confidence, accuracy, and bias

Calibration analysis within the cognitive and personality domains: individual differences in confidence, accuracy, and bias

[Abstract]
Calibration research is concerned with the accuracy of confidence judgments made by individuals when responding to various cognitive tasks. Individuals are scored as accurate or inaccurate based on the objective criterion of whether their responses are correct. Within the personality domain, judging the accuracy of trait self-ratings is more complicated than in the cognitive area as there are no perfect criteria for evaluating the accuracy of these types of judgements (Colvin & Funder, 1991).

Because cognitive calibration research findings formed the anchor for the current studies, the decision was made to initially scrutinize the cognitive domain for mis-calibration, and whether individual differences in gender, age, personality, and ability, influenced cognitive confidence and bias scores. In order to achieve the aims, of this dissertation, three studies were conducted with a total of 831 individuals being tested. To determine accuracy within the personality domain, the current studies constructed a situation wherein Big Five personality assessments could be scored as accurate or inaccurate. Results showed that when consistency measures were used, accuracy scores for each Big Five trait were reasonably high across Studies 1, 2, and 3. Prior to the studies conducted in this dissertation, no techniques using calibration procedures had been established to assess Big Five confidence or bias.

Within the cognitive domain, calibration research has demonstrated the existence of a trait of self-confidence that appears to be independent of the type of activity being investigated. This result was replicated in Studies 1 and 3. However, the generality of this trait across other domains, such as personality assessments, remains largely unexplored. Two measures were designed to obtain confidence ratings in relation to Big Five personality judgments. Results from three studies showed that the benchmark for peoples‟ confidence in Big Five judgments was around 80%. Data from Studies 1, 2, and 3 also demonstrated a one-factor solution when confidence scores for each Big Five trait were factor analysed. In studies 1 and 3 the factorial structure of cognitive and Big Five confidence scores was examined and both studies produced a two-factor solution.

The calibration paradigm also uses bias scores as a measure of how well calibrated individuals are when self-monitoring their performance on various cognitive tasks. Whether people are well-calibrated within the Big Five domain has not been investigated by previous researchers. The current studies examined whether people were mis-calibrated when making Big Five judgments about themselves. The data from two studies indicate that people were well-calibrated for each of the Big Five traits. Factor analyses of Big Five bias scores revealed a one-factor solution. When study 3 examined the factorial structure of cognitive and Big Five bias scores, the analyses showed that bias across these domains were separate but correlated processes. Across all three studies, individual differences in gender and age did not influence Big Five confidence, Big Five accuracy or Big Five bias scores.

One of the most significant implications of this dissertation, for calibration researchers, who are striving to understand the mis-calibration phenomenon, was that the structural analyses of cognitive and Big Five bias scores yielded a two-factor solution (i.e., Personality and Cognitive Bias), that was moderately correlated. Also, in the current studies, simple methods were used to obtain Big Five confidence ratings. These procedures could now be used to investigate the factorial structure of confidence in much more detail, and across other domains such as interests, attitudes and values. Practical implications of the current research within the field of clinical psychology were also discussed.