Using Google Translate for Malay translation of Cochrane plain language summaries: a randomised controlled trial (RCT)

Session: 

Oral session: Linguistic translation of Cochrane content

Date: 

Sunday 16 September 2018 - 14:40 to 15:00

Location: 

Carrick 3

All authors in correct order:

Lee ZH1, Chew JE1, Ng CA1, Lee XX1, Tan ML2, Ho J2, Lee HF1
1 Junior Research Fellowship, Penang Medical College, Malaysia
2 Cochrane Malaysia, Penang Medical College, Malaysia
Presenting author and contact person

Presenting author:

May Loong Tan

Contact person:

Zcho Huey Lee
May Loong Tan
Abstract text
Background:
As part of Cochrane's Strategy 2020, plain language summaries (PLS) are currently translated into 14 languages, including over 1300 Malay PLS. The translations are done by volunteers using Smartling, a translation management platform. It has its own built-in machine translation tool (MT), which translates individual word strings. However, Google Translate (GT), a freely available online translation tool, produces a more natural translation as it "translates whole sentences at a time rather than just piece by piece". We conducted a RCT to compare the two.

Objectives:
To compare the efficiency, accuracy and quality of PLS translation to Malay using GT and MT in Smartling.

Methods:
Two people not involved in the intervention selected 16 PLS. Matched PLS of similar length were paired and randomly assigned to either GT or MT. Each of the eight pairs of PLS were translated by a volunteer. Errors in vocabulary, prefix, suffix and punctuation, number of sentences requiring rewriting and time to task completion were recorded. The translated PLS were then randomly assigned to two blinded, experienced editors for quality assessment of completeness, accuracy of terminology, grammatical accuracy, readability and overall satisfaction using a scoring system (total score 100%). The results were analysed in Stata/MP13.0 for Windows.

Results:
There was no difference in time taken to complete translations between the groups. The percentage of vocabulary errors was lower in the GT group compared to the MT group (MD 5.48, 95% CI -10.48 to -0.49). The number of errors in prefix, suffix and punctuation and sentences requiring rewriting were similar. Editors' score was higher for the GT group (MD 8.56, 95% CI -7.01 to 24.20). Multiple linear regression showed that accuracy of terminology, readability and overall satisfaction positively influenced editors' score.

Conclusions:
With this small sample we found that GT reduced vocabulary errors and might result in better quality translations. It did not reduce the time needed to complete a translation. GT might help inexperienced translators with word choice. This study could serve as a basis for power calculation in future studies.

Patient or healthcare consumer involvement:
A more efficient and accurate translation tool can greatly expand the accessibility of Cochrane evidence to consumers.

Relevance to patients and consumers: 

To support consumer involvement in their own healthcare decisions, it is important that reliable evidence is available in a language they can understand. Identifying a more efficient and accurate translation tool, would mean that a greater body of up-to-date evidence is made available in their language in a timely manner. We found from our study that Google translate helps improve accuracy in Malay translation. We can consider inviting consumer volunteers on to our translation team if we use Google translate.