Assessment with construct-response items: validity, reliability, comparability, and fairness

Abstract

Large-scale assessments may guide important decisions depending on the area in which they are applied. In educational exams, objectives may focus on individual differences, monitoring the performance of students in different contexts, as well as on the assessment of educational programs or projects, supporting or justifying actions in the political sphere. The validity of the measures and their interpretation are of paramount importance, as their consequences may affect the population involved and even the whole society. The key issues for large-scale assessment are validity, reliability, comparability, and fairness. These terms should be considered whenever value decisions are made based on the assessments. This article discusses the concepts of validity and reliability, as well as the relationship between them. The comparison of assessments with construct-response items is currently an issue of great concern to experts, due to the increased use of shared reference matrices developed to guide curricula at all educational levels in several nations. This article also discusses fairness in evaluations, which is related to the requirement to ensure equal conditions to all participants. Quality assessment should provide all with opportunities for responses which ensure correct inferences about their performance in relation to the construct measured. The aim of this article is to describe the main theories present in large-scale assessments, providing information for the correct interpretation of the concepts involved in their processes.