Sessions / Testing and Evaluation (TEVAL)
Peer feedback is a popular method for involving students in the formative assessment process, yet students often struggle to provide adequate feedback for their partner(s). One reason for this is that generating feedback is a skill requiring training and practice to improve. This workshop will demonstrate how the web-based application Pear Deck can be used to train students to generate productive feedback for their partner(s) and improve the pedagogical benefit of the activity.
In this presentation we first consider the role of assessment in the context of an educational curriculum. We then consider principles that can guide classroom assessment, including assessing learners in ways that achieve greater fairness, using assessment results to guide future learning, involving learners in the assessment process, and relating assessment to student motivation. Finally, we provide an example of how the principles can be applied to a second language university reading course.
This paper outlines findings from an ongoing mixed-methods study on rater leniency in L2 speaking tests. The research finds a small but statistically significant relationship between two rater characteristics, “Agreeableness” and “Experience”, and “Rating Leniency”. Raters higher in agreeableness and/or experience give more lenient scores. The study then thematically analyses rater commentary to triangulate these findings with qualitative evidence.
The findings of a one-year study will be presented in an attempt to discover how Japanese university EFL students feel about peer assessing their classmates’ oral presentations. Data was obtained through a survey that was administered at the end of the first semester as well as at the conclusion of the one-year course to determine student perceptions of this important form of formative assessment.
This survey study examined fairness perceptions of three assessment types (norm-referenced, criterion-referenced, and individual-referenced) for 281 EFL junior high school students in Macau. Results revealed that students viewed criterion-referenced assessment as the most fair, individual-referenced assessment as fair, and norm-referenced assessment as unfair. These findings reinforce the importance of using criterion-referenced assessment to measure achievement in the classroom, while the low opinions of norm-referenced assessment affirm that it should be avoided.
Developing assessment methods in EFL is a daunting task, but in this workshop, an explanatory and integrative set of nine principles for rubric design will be described. They are: exhaustiveness, deterrence, representativeness, customizability/adaptability, preemptive normalization, justifiability, transparency/communicability, accountability and formative feedback. Using as an example a set of rubrics that were custom-designed for speaking tests, these principles will be explained to assist language teachers in the design/customization of assessment criteria.
This presentation will deal with the role of vocabulary in EFL essay writing. When assessing vocabulary in learners’ essays, raters and the instructors should take into consideration the relation between passive and active vocabulary knowledge. Based on the result of data analyses, the presentation will discuss the construct reflecting the continuum of these two types of vocabulary knowledge and the possibility to convert vocabulary use from passive to active.
The presenters will describe a research project they undertook with four classes of students at a university in Japan. Although the hypothesis was not supported by the results, separate analyses for each class showed that there were opposing outcomes between classes. The presenters will discuss what they learned from the collaboration, how working together improved the quality of the research, and share advice for other teachers who wish to conduct research with their colleagues.
This workshop will examine the challenges faced in measuring spoken production and look at how these challenges were overcome at a Japanese tech company by creating an internal speaking test used for both English and Japanese . By using systematic approach to evaluate spoken production we are able give accurate and actionable feedback to learners enrolled in language programs as well as track the progress over short and long term timespans.
This study examines scores of the IELTS and TOEIC (L&R and S&W) and investigates test-taker reactions to the tests. The IELTS, TOEIC, and a questionnaire were administered to 84 university students, and interviews were conducted. The test scores were analyzed for descriptive statistics and correlations, and the questionnaire and interview results were examined with special attention to the speaking and writing tests, the formats of which are different for the IELTS and TOEIC.