I recently got back from a training session at Educational Testing Service that took me deep into the workings of TOEIC and other common standardized workplace English tests. A lot of interesting material was covered, and TOEIC representatives from around the world had great opportunities to share their experiences and “war stories.”

One of the funniest stories I heard was about a competitor test that shall remain nameless but that continues to be used worldwide.

When this competitor test was introduced, it became popular with a particular country’s call center businesses because it was quite inexpensive and gave immediate, computer-scored results. Sales slowed to a crawl, however, as news spread of a creative way to “game” results. When test takers didn’t understand questions, they found they could still get a decent score by responding with English-language songs.

This bizarre strategy worked because current computer-scored tests must focus primarily on “prosody,” that is, on the way language sounds. Today’s technology is not sophisticated enough to fully judge meaning.

The TOEIC Speaking Test differs in that it is human-scored and task-focused. The standardized rubrics the scorers use focus primarily on whether the test taker understands and can effectively respond to a given request. Intelligible pronunciation is of course essential, but it is only one element in the effectiveness of the response.

The task-based nature of TOEIC tests became very clear when we listened to actual examples of questions and responses and guessed what score the person would receive on the question. One of the examples was a telephone message in which the test taker was asked to provide a strategy to deal with a team scheduling problem. The test taker responded with language that was wonderfully clear in its pronunciation but that was very vague. She said things like “I want to help with your problem…let’s meet about your problem…maybe we can discuss your problem over lunch….” Nothing more solid. She hadn’t understood the question.

What score did she get? Contrary to many of our expectations (including my own), she received the lowest possible proficiency level above a complete zero. Despite her fluency, she could not comprehend or adequately address the question. And looking at the rubrics the scorers use, we could see that she should indeed have gotten that low score.

Not everyone cares as much as one would hope about meaning. In fact the same TOEIC representative who told the song story complained that many call centers mainly wanted a “warm body in a seat” (preferably with an American accent) who could read a script. They didn’t care about ability to answer off-script questions. (Maybe this is why so many of us get frustrated by call centers!)

If you do care about your employees’ ability to respond to client needs and complete real-world communication tasks effectively, however, then TOEIC is the workplace English test for you! 

Learn more about giving TOEIC at your institution.

—Lia Nigro, TOEIC USA Team