Quality analysis of a Mathematics test in National Secondary Education Exam

Abstract

The National Secondary Education exam (Enem) is large-scale evaluation with strong influence on educational policies, on the curricula of several levels of schooling as well as an impact on the future of the individuals taking the test. These factors show the importance of looking into a number of variants involved in the preparation, the application and the score of such evaluation. By means of studies, it is possible to identify steps in the process that are not working as expected and also to validate the steps whose results are adequate. In both cases, the intent is to improve the process in every edition of the exam. In this study, analyses were conducted for mathematics test in Enem 2015, by means of two model from the Item Response Theory (IRT), the logistics of three parameters with studies on discrimination, difficulty and probability of casual hits and the model of nominal response, with studies on the difficulty and behavior of each response option in every item. Approximately one third of the items showed parameters within the quality standards. Regarding difficulty, the easiest item had only 64.1 percent of hits. In addition, only two items had the difficulty parameters lower than the mean point of the scale and approximately 23.4 percent of participants had their skills estimated below the value corresponding to the easiest item in the scale. This indicates that these individuals could not answer any of the 45 items. By utilizing several methods, it was found that the mathematics test in Enem 2015 shows a quality level much lesser than desirable.