In this topic, three dimensions of research on assessment are distinguished: scope (micro – macro); function (AoL – AfL) and use of technology (none – full). The topic members are currently working on research on quality of assessment in general, the relation between self-assessment, peer assessment and self-regulation, the effects of different types of feedback on attention and learning, the use of (video-enhanced)rubrics in education, ensuring authentication and authorship in e-assessments and large scale assessments in Massive Open Online Courses (MOOCs). The topic members provide workshops and training on these themes in primary, secondary and (vocational) higher education. They are teachers in the Educational Science program of the Open university and work in national and international teams.
Watch the presentations at the conference on e-assessment on 17 May 2019
Background
The importance of assessment quality in higher education is emphasised by research and by practice. Looking at the research, three dimensions can be distinguished.
Figure 1. Dimensions for assessment research
Firstly, assessment quality can be evaluated on a micro level (the individual test for personalized assessment) or on a macro level (the introduction of national attainment tests in primary education). Research of the topic members contributes to assessment quality in different ways. For example, a framework for quality of assessment is given by the quality pyramid for contemporary assessment (see Figure 1). Each level of the pyramid implicates issues that influence assessment quality.
Figure 2. Framework for assessment quality (Joosten-ten Brinke, 2011; Sluijsmans, Peeters, Jakobs, & Weijzen, 2012)
The second dimension refers to the balance between assessment for learning and assessment of learning. AfL is, theoretically, strongly related to social interaction in an educational relationship between student and teachers or students themselves as well as to self-regulated learning. The research here focuses on effective methods, like rubrics, feedback or self and peer assessment. This also includes learning strategies for effective preparation for an exam (testing effect). For example, we investigate how teacher feedback is used by students in improving their writing products. In the Viewbrics project we study the effect of rubrics with video modelling examples on the quality of formative feedback and mental model formation while practicing three complex skills (collaboration, information literacy and presentation skills) and on final mastery of these skills.
Thirdly, with the increased use of technology in education paper-and-pencil testing is more often replaced by technology enhanced assessment (TEA). This allows the use of multimedia (e.g., videos, animations, etc.) and the implementation of adaptive learning (i.e., tailored to specific needs and level of the student). Research related to this dimension has various foci. From an educational perspective there are still many open questions on how to optimally design technology enhanced assessments (Jarodzka, Janssen, Kirschner, & Erkens, 2015). Another focus is on how to design non-obtrusive assessments in technology enhanced learning environments, for example in serious gaming. Besides, we investigate the use of state of the art technologies like face and voice recognition, keystroke dynamics, and forensic analysis for the purpose of assuring authentication and authorship in e-assessment scenarios.
Leading research questions
The leading questions in this theme are:
- How can assessment optimally foster learning (i.e., which factors are true determinants of learning; balance between AoL and AfL; quality of assessment)?
- How can we best develop and implement effective, efficient and enjoyable technology enhanced assessment in the learning situation?
Projects
- SOONER Project (Julia Kasch)
The aim of the Sooner project is to get insight into the possibilities of extending rich learning activities to high student numbers. A growing challenge in (open online) education is to facilitate an increasing number of students with (personal) feedback and support. Within online education, Massive Open Online Courses (MOOCs) are a great example of courses that are challenged by high student numbers when trying to support them. How can MOOCs provide high quality learning activities to high student numbers without increasing teacher time and without cutting down on quality? What are the possibilities of MOOCs to provide students with high complexity learning activities and assessment at large scale? - TESLA Project (José Janssen, Henry Hermans)
The TeSLA project provides to educational institutions, an adaptive trust e-assessment system for assuring e-assessment processes in online and blended environments. It will support both continuous and final assessment to improve the trust level across students, teachers and institutions The system will be developed taking into account quality assurance agencies in education, privacy and ethical issues and educational and technological requirements throughout Europe. - Testing and Learning Processes (Kim Dirkx)
In this research-line, different themes such as the testing-effect, the effect of testing on monitoring and the effects of feedback on learning are examined. Master students are very welcome to submit a proposal in this line of research. - TEA4ALL (Desirée Joosten-ten Brinke)
TEA4You addresses the need for highly qualified learning and computer scientists to deliver new ICT systems for summative and formative eAssessment with training 15 ESRs by international experts with outstanding reputations in eAssessment. It offers a unique research environment and strong industry-academia partnership for future scientific leaders in Technology Enhanced Assessment (TEA). Our ESRs will learn to develop a variety of cutting edge TEA systems such as 1) the specification of item quality and competence models, 2) development of analysis frameworks for heterogeneous data using cloud computing, implementation of mechanisms to generate TEA artefacts, and 3) through working on interactive feedback strategies be among the first to integrate formative feedback into computerised adaptive testing (CAT). (project in development)
Dit is TeSLA
Zo gebruik je TeSLA
Publications
Key publications:
- Arts, J., Jaspers, M., & Joosten-ten Brinke, D. (2016). A case study on written comments as a form of feedback in teacher education: so much to gain. European Journal of Teacher Education, 39, 159-173. doi:10.1080/02619768.2015.1116513
- Dirkx, K. J. H. (2014, 11 April). Putting the testing effect to the test. Why and when is testing effective for learning in secondary school. Unpublished doctoral dissertation. Heerlen: Open University of the Netherlands.
- Hummel, H. G. K., Nadolski, R. J., Joosten-ten Brinke, D., Baartman, L. (2016). Content validity in a serious game assessment: Development of an E-assessment for ICT-managers in training. Technology, Pedagogy and Education.
- Geitz, G., Joosten-ten Brinke, & Kirschner, P. A. (in press). Sustainable feedback: students’ and tutors’ perceptions. The Qualitative Report, 21(11).
- Joosten-ten Brinke, D., Schultz, N., & Platjouw, R. (2015). The success factor in assessment: motivation or playing? A case study in gamification. In E. Ras & D. Joosten-ten Brinke (Eds.) Proceedings Computer Assisted Assessment conference, (pp. 40-46). Heidelberg: Springer.
- Meusen-Beekman, K. D., Joosten-ten Brinke, D., & Boshuizen, H. P. A. (in press). De retentie van zelfregulatie, motivatie en self-efficacy in het voortgezet onderwijs na formatieve assessments in het basisonderwijs. Pedagogische Studiën.
- Ras, E. & Joosten-ten Brinke, D. (Eds.). (2015). Proceedings Computer Assisted Assessment conference, Heidelberg: Springer.
- Rusman, E., Nadolski, R., Boon, J. & Ackermans, K. (accepted). Viewbrics: Formative Assessment of Complex Skills with Video-Enhanced Rubrics in Dutch Secondary Education.
- Ackermans, K., Rusman, E., Brand-Gruwel, S., & Specht, M. (accepted). A First Step Towards Synthesizing Rubrics and Video for the Formative Assessment of Complex Skills.
- Rusman, E., Martínez-Monés, A., Boon, J., Rodríguez-Triana, M.J., & Villagrá-Sobrino, S. (2014, 30 June-1 July). Gauging Teachers’ Needs with Regard to Technology-Enhanced Formative Assessment (TEFA) of 21st Century Skills in the Classroom. In M. Kalz, & E. Ras (Eds.), Computer Assisted Assessment. Research into E-Assessment. Proceedings of the International Computer Assisted Assessment (CAA) in Conference Communications in Computer and Information Science Proceedings 439 (pp.1-14). Zeist, The Netherlands: Springer.
Books:
- Sluijsmans, D., Joosten-ten Brinke, D., & Van Schilt-Mol, T. (Red.). (2015). Kwaliteit van toetsing onder de loep. Handvatten om de kwaliteit van toetsing in het hoger onderwijs te analyseren, verbeteren en borgen. Antwerpen, België: Garant.
- Van Berkel, H., Bax, A., & Joosten-ten Brinke, D. (2014). Toetsen in het hoger onderwijs. (3e geheel herziene druk). Houten: Bohn, Stafleu Van Lochem.