The influx of technology in education has made it increasingly difficult to assess the validity of educational assessments. The field of information systems often ignores the social dimension... Show moreThe influx of technology in education has made it increasingly difficult to assess the validity of educational assessments. The field of information systems often ignores the social dimension during validation, whereas educational research neglects the technical dimensions of designed instruments. The inseparability of social and technical elements forms the bedrock of socio-technical systems. Therefore, the current lack of validation approaches that address both dimensions is a significant gap. We address this gap by introducing VAST: a validation framework for e-assessment solutions. Examples of such solutions are technology-enhanced learning systems and e-health applications. Using multi-grounded action research as our methodology, we investigate how we can synthesise existing knowledge from information systems and educational measurement to construct our validation framework. We develop an extensive user guideline complementing our framework and find through expert interviews that VAST facilitates a comprehensive, practical approach to validating e-assessment solutions. Show less
Blankenstein, F.M. van; Trutescu, G.; Rijst, R.M. van der; Saab, N. 2019
The learning benefits of peer assessment and providing peer feedback have been widely reported. However, it is still not understood which learning activities most facilitate the acquisition of... Show moreThe learning benefits of peer assessment and providing peer feedback have been widely reported. However, it is still not understood which learning activities most facilitate the acquisition of feedback skills. This study aimed to compare the effect of a modeling example, i.e., a model that demonstrated how to give feedback, on the acquisition of feedback skills. The participants were second-year bachelor students in pedagogical sciences (N = 111). They were assigned randomly to a practice condition, in which they practiced giving feedback on oral presentations, or a modeling example condition, in which a teacher demonstrated how to give feedback on a good and a bad presentation. Students then gave feedback to a presenter in a video (direct feedback measure). One week later, they gave each other peer feedback on oral presentations (delayed feedback measure). On the direct feedback measure, students in the modeling example condition used assessment criteria more often in their feedback, and produced significantly more overall feedback, and significantly more positive and negative judgments than students in the practice condition. There was no significant difference in the amount of elaboration and feed-forward between the two conditions. On the delayed feedback measure, there were no significant differences between the two experimental conditions. The results suggest that, at least in the short term, a modeling example can stimulate the use of assessment criteria and judgments in feedback. The results and implications for future research and practice are discussed. Show less
Espin, C.A.; Saab, N.; Pat-El, R.; Boender, P.D.M.; Veen, J. van der 2018
Curriculum-Based Measurement (CBM) is a system for monitoring the progress of and evaluating instructional program effectiveness for students with learning difficulties. Although a large amount of... Show moreCurriculum-Based Measurement (CBM) is a system for monitoring the progress of and evaluating instructional program effectiveness for students with learning difficulties. Although a large amount of research has been conducted on CBM, little has focused on the interpretation and use of the data for instructional decision-making, despite the fact that it is data use that leads to performance gains. In this study, we examine factors affecting the interpretation of CBM data. Specifically, we examine the effect of CBM graph patterns on ease of graph interpretation. Thirty college/university students completed a two-part study in which they viewed various slope-to-goal and slope-to-slope patterns of CBM-graphed data and answered decision-making questions. Response times and accuracy were measured. Results revealed that graph patterns differed in terms of ease of interpretation. Differences depended on the type of question. Implications for interpretation and use of CBM data for decision-making are discussed. Show less
Tio, R.A.; Schutte, B.; Meiboom, A.A.; Greidanus, J.; Dubois, E.A.; Bremers, A.J.A.; Dutch Working Grp Interuniv 2016
Opportunities for negotiation in formative assessment may benefit teachers' professional development. Detailed analysis of nine assessment dialogues involving pairs of nursing teachers in secondary... Show moreOpportunities for negotiation in formative assessment may benefit teachers' professional development. Detailed analysis of nine assessment dialogues involving pairs of nursing teachers in secondary vocational education showed that the amount of negotiation was limited. Assessment dialogues provided ample opportunities for negotiation (based on expressed disagreement), but more than half of these opportunities were neglected and hence not used to negotiate learning implications. Participants found it difficult to confront a colleague. Possible reasons for this, also methodological, are discussed with a view to future training, formative assessment and research. Show less
The literature indicates that teacher professional development and learning may be improved by using formative assessment procedures. This thesis focuses on a specific form of formative assessment,... Show moreThe literature indicates that teacher professional development and learning may be improved by using formative assessment procedures. This thesis focuses on a specific form of formative assessment, negotiated assessment, which is characterised by the exchange of views between assessor and assessee and the extensive involvement of the participants (assessees) in their own assessment procedure. The participants negotiate with their assessor about several aspects of the negotiated assessment procedure. Assumptions in negotiated teacher assessment are that the opportunities for negotiation might promote the development of a sense of agency in terms of teachers feeling in control of their learning and assessment processes and feeling able to pursue their learning objectives. This thesis covers four studies in which different aspects of negotiated assessment were examined. Its aim was to gain more knowledge about a negotiated assessment procedure to promote teacher learning in the area of stimulating students__ reflections. The results show that very little negotiation took place during the dialogues. Despite the lack of negotiation during the assessment meetings, the teachers reported a strong sense of agency. Overall, the teachers__ opinions indicated that the elements of the negotiated assessment procedure facilitated their professional learning. The contribution was particularly seen in change in their knowledge, beliefs, attitudes and practice. Teacher professional development; Teacher learning; Formative assessment; Negotiated assessment; Negotiation; Feedback; Agency; Reflection Show less
This dissertation focused on how the social context in which peer assessment takes place is related to the learning effects of peer assessment. We wanted to identify interpersonal beliefs in peer... Show moreThis dissertation focused on how the social context in which peer assessment takes place is related to the learning effects of peer assessment. We wanted to identify interpersonal beliefs in peer assessment settings that influence the peer assessment process on the one hand, but could also been seen as predictors of the effect of peer assessment interventions. Four studies are presented that question parts of the interpersonal context of peer assessment: one systematic literature study and three empirical studies. In the literature study, a structural model of analysis was presented indicating different interpersonal beliefs, as well as structural features of peer assessment. The results of the empirical studies deliver content for the empirical discussion of peer assessment: peer assessment as such seems to stimulate the beliefs in certain interpersonal variables. Show less
