Ratings or pairwise comparisons? An experimental study on scale usability

Malgorzata KARPINSKA-KRAKOWIAK

University of Lodz, Poland

Abstract

A series of experiments was run in order to evaluate the usability of two different measurement approaches: ratings and ranks (pairwise comparisons). Respondents were asked to assess perceived characteristics (i.e. height and length) of different physical objects by using either a rating or a ranking scale. An artificial neural network model was built to analyse the ranks and standard statistical tests were applied to analyse the ratings. The results were then statistically compared with actual (real) characteristics of objects (i.e. their real height and length). Both systems for measuring values were found equally valid in projecting reality. Such findings offer some methodological and epistemological insights, as they provide information on the measurement power of each scale in terms of approximating real-life phenomena.

Keywords:

rating scale, rank scale, pairwise comparisons, scale usability