Nothing is more magical than getting back results to a closed-ended question from 10 different languages and being able to instantly analyze the results! Nothing is more frustrating than realizing that cost overruns have left you with no money to translate all of the verbatim responses.
International surveys may be the type of survey-research projects most likely to go over budget, costing you more dollars, pounds and Euros than you planned for. As I think back over 22 years of research studies, the majority of project overruns involved fielding a survey in multiple countries. If your client (internal or external) makes any of the following statements, prepare to be in the red!
“I thought we could use free software to do the translation.” You could, but many of your questions would be ungrammatical gibberish. While Star Trek may have had a Universal Translator, we’re going to be able to beam ourselves through walls before a device translates from any language to any language with high accuracy. You need humans to create accurate translations, which brings us to the next sentence you’ll dread hearing.
“We’re going to have our local offices do the translation.” Yes, coworkers in international offices qualify as human, and they certainly understand their local market. What they do not necessarily understand is the art of translation or the specialized language of surveys. They may also have great intentions, but have difficulty getting the translation done in a timely fashion. The ideal translator is a professional with extensive experience translating questionnaires, a background in the industry being studied and knowledge of the local market. Since this ideal is often outside the budget, the next-best solution is to pair the translator with your in-country coworkers.
“We don’t have the time or money to re-translate the translated questionnaires back into English.” The time to catch a mistake is before you field the survey, not after you get back results that you don’t understand. David R. Morse, president of New American Dimensions, gave one example at the MRA First Outlook Conference. David once was surprised to see in a survey that 100% of Spanish-speaking respondents disagreed that “the Internet is color blind.” Rechecking the Spanish translation, he found that it had been translated literally: “the Internet has red-blue-green color blindness.” English questionnaires are often informal and use idioms and expressions that do not easily translate; English, like most languages, has words with dozens of different meanings, making a question that wasn’t ambiguous to you ambiguous to your translator, who misinterprets it. You need to budget for back-translation or risk refielding portions of your research.
“I can’t imagine it taking more than six weeks.” Let’s say your international survey project involves six different markets. That provides six times the opportunities for delays. I’ve seen delays because of the variability in the availability of skilled translators for a particular language, as well as delays in fielding due to country-specific holiday schedules (especially over the summer in different European countries). It is easy to run over budget when you dramatically underestimated the time required.
“I didn’t really expect to have all these verbatim responses.” Design the questionnaire up front to minimize open-ended questions. Nothing is more magical than getting back results to a closed-ended question from 10 different languages and being able to instantly analyze the results! Nothing is more frustrating than realizing that cost overruns have left you with no money to translate all of the verbatim responses.
“Wow! I can’t believe we got that done under budget.” I confess that I have never heard those words used to describe any international survey research. Prepare your budget conservatively and factor in what the French call faux frais, “items one overlooks when making a budget.” Then you might have some extra dollars, pounds or Euros on hand when the project ends.
- What has been your experience working with international surveys? Have you had similar experiences with clients when working in different countries or has your experience been completely different?
- Have you tried using translation software? Has your experience with the software been good or bad?
- If you are a translation software company, why do you think people should still consider using software?
- For our international members, what are your thoughts on English speaking researchers doing research in non-English speaking countries?
Yes, same experience, especially regarding the differing holidays and expectations/standards. Based in Curacao we do research across the Caribbean, a region with many languages. Luckily, in Curacao, we are multilingual and have rather easy access locally to native speakers/translators for all the regional languages(French, Creole, Dutch, English, Caribbean English, Spanish, Papiamentu) and cultures. It works quite well. Sensitivity to the fact that there are cultural and market differences and an ability to relate to these is more important than language skill. But, questionnaires and focus groups must be in the respondents' native language.
- Tamira La Cruz 01/06/2010
All Good points. Can I just add based on years of experience (but no current interest in running such projects!) - expect to pay for expertise and coordination and value it. Running international research studies (well) requires a lot of operations skills, knolwedge of local cultures (both consumer and the MR environment) and lot of diplomacy to make things happen. Especially in developing markets it is easy to get some data back, but far harder to ensure it is quality and useful data. Many of the issues are below the surface and may not be apparent. Remember that if you get a result that seems "screwy" from a survey in California it will probably ring warning bells (you will have a feel for the market, or back-data etc.). Unless you are P&G or Unilever etc., the same is unlikely to be true for Thailand, Korea or Egypt -- it's harder to assess believability. MR professionals specialiing in international MR projects often face a thankless task and often, rather than being prepared to pay extra for their expertise, clients ask for discounts for commissioning work across several countries. This is usually short-sighted.
- Alastair Gordon 01/17/2010