Presenter: Dr. Gary Ockey

Concerns about the need for assessing multidialectal listening skills for global contexts are becoming increasingly prevalent. However, the inclusion of speakers on listening assessments who have diverse speech varieties may threaten test fairness because test takers’ listening could be affected differently depending on their experience with certain speech varieties. To shed some light on this conundrum, this study aimed to determine the degree to which strength of accent can be measured reliably and the extent to which accent strength and familiarity affect comprehension on a test of L2 listening.  

In stage one of the study, a strength of accent scale was developed. After numerous tryouts of the scale and resulting revisions, a study was conducted to investigate its reliability. Twenty English speakers, two of whom were believed to have accents representative of the local speech variety (Standard American English) and 18 of whom were believed to have accents that differed from the local speech variety, were recorded giving short lectures. Two 20-second audio clips for each of the speakers were taken from the lectures. These 40 audio clips were then played to 69 L1 and 31 L2 listeners. The listeners used the strength of accent scale to judge each of the audio clips. A Many-Facet Rasch Measurement analysis indicated that the Strength of Accent scale led to ratings which distinguished the various speakers, the two different speech segments led to similar estimates of strength of accent, and L2 listeners were slightly harsher and more variable in their strength of accent judgements than L1 listeners.

In the second stage of the study, the accent scale developed in stage one was used to select one US, four Australian, and four British English speakers of English. TOEFL test takers (N = 21,726) were randomly assigned to listen to a common lecture given by one of the nine selected speakers, and respond to six comprehension items and a survey designed to assess their familiarity with various accents. The results suggest that strength of accent and familiarity do affect listening comprehension, and these factors affect comprehension even with quite light accents.

 

Reception to follow

 

Gary Ockey, (M.A., University of Utah, Ph.D., University of California, Los Angeles) is an Associate Professor at Iowa State University. He investigates second language assessments, with a focus on the use of technology and quantitative methods to better measure oral communication (listening and speaking). He recently led the development of the Iowa State University oral communication placement test, an example of a test which combines strengths of humans and computers for test delivery and scoring. His co-authored book: Emerging issues in the assessment of second language listening, is expected to be published by John Benjamins in 2018. He has published in various journals, including Applied Linguistics, Language Learning, Language Assessment Quarterly,Language Testing, and Modern Language Journal. He has served as the Editor of the TOEFL Research Report Series, and is currently an associate editor of Language Assessment Quarterly.