Psychological causes of intransitivity in pairwise comparisons: a qualitative study

Abstract

Ordinal, or rating scales are popular in psychology. The use of these methods assumes that respondents can order psychological attributes preserving transitivity (A ≥ B, B ≥ C, and A ≥ C). However, according to some research, about 30% of respondents violate transitivity when ordering, raising questions about the applicability of ordinal methods in psychology. The present study aims to discover psychological causes of intransitivity in pairwise comparisons (a standard procedure for assessing transitivity). We hypothesize that intransitivity results from variability in respondent priorities during pairwise comparisons. So, the purpose of the study is to identify respondents’ actions that may lead to variability in priorities. We conducted a qualitative study in which students (N = 29) did pairwise comparisons, choosing more important motives for attending university classes. A semi-structured interview about the experience of performing pairwise comparisons was following. Results showed that 45% of respondents ordered motives with transitivity violation. Content analysis of the interviews allowed us to identify respondents’ actions leading to variability of priorities in pairwise comparisons, among which was the revaluation of motives’ importance, change of choice strategy, adherence to a special strategy in difficult choices, change of choice context, and randomness.