As mobile devices continue to become a frequent, if not primary, means of Internet access, researchers and marketers struggle to maintain respondent engagement in online activities.  How can we adapt and evolve traditional models to continue to facilitate meaningful, rewarding interactions between brands and consumers, while leveraging the unique possibilities that mobile presents? Many researchers struggle to commit to a mobile-first strategy and continue to rely on existing online and in-person methods or attempt to replicate them on mobile without accounting for the vastly different user experience.

In many ways, the slow adoption of mobile techniques mirrors the caution around online methods a decade ago: “Who are these people? What about data quality? Bias?” For example, the 2014 GRIT report showed that only 27 percent of researchers used mobile surveys in 2013, compared to 42 percent who had said they planned to. 1 The marketing research industry must address this gap sooner rather than later. 2 Currently, around 25 percent of global Web usage comes from mobile; 3 soon, it will be difficult to capture representative samples without including mobile.

So what is the key to mobile engagement? Innovative technologies, device-agnostic platforms, and best-practice design can all help to increase participation, deepen engagement, and reduce quality and bias concerns. However, none of these matter if consumers aren’t motivated to give you their time and energy in the first place. Therefore, building long-term, trusting, collaborative relationships with consumers may prove to be essential to unlocking the true power and potential of mobile.

The Accidental Mobile Challenge

Online research methods, surveys in particular, are garnering increasing rates of mobile participation, whether or not they were designed with a smaller screen or an on-the-go user in mind. Meanwhile, fewer consumers have extended time and attention to give, yet many surveys take 20 minutes or longer. These two factors help to explain why so many researchers currently struggle to engage mobile respondents, bumping up against increased drop-off rates, declining engagement, and serious questions about data quality.

A recent report by Decipher found some startling trends from mobile participation across their vast survey history.4 Most of their clients are not currently fielding surveys optimized across devices. Survey completion rates go down along with screen size, with smartphone users nearly twice as likely to abandon as desktop users (41 percent to 24 percent). Grid, open text, and open numeric questions are all likely to prompt dropouts. However, they also found that mobile-friendly designs (e.g., large buttons and text, less clutter, and shorter scales) could help to mitigate many of these challenges.

In Communispace’s online communities, we experience a fair amount of unintentional mobile participation. That is, we don’t explicitly ask people to use a mobile device, they simply use that device to access the Internet. We currently see, on average, 15 to 20 percent (and rising) of community traffic coming from mobile devices, split evenly between smartphones and tablets.

While we have our own proprietary platform 5 that is designed for an optimized experience across methods and devices, we wondered how survey responses would differ (even with a consistent experience) both qualitatively and quantitatively, by device. We designed a 19-item questionnaire (moderate length relative to a typical client survey) on a universal topic (the generation gap) and fielded it to 10 communities with above-average mobile participation. We did not provide any direction regarding device usage. The result was a sample of 1224 members, 76 percent of whom took the survey on a desktop or laptop, 13 percent smartphone, and 11 percent tablet.

Within this sample, we observed distinct patterns of mobile use across communities, devices, and segments. In general, Millennials, African Americans, and Hispanics were most likely to participate via smartphone, while B2B (those participating in their professional capacity) and Boomer members over-indexed on tablet use – patterns that reflect mobile trends observed by Pew and others. Women were also 50 percent more likely than men to access the survey on any type of mobile device (30 percent versus 19 percent). When planning research projects, understanding mobile usage among your target audience can help to anticipate the amount of mobile access you will see.

Making Mobile Surveys Work

The results of our survey were both surprising and illuminating. In the context of a private, online community, we generally saw little difference in both responses and response metrics across devices:

  • Drop-off rates uniformly low: Incompletes were rare (3 percent), and relatively even across devices: 3 percent of desktop/laptop users dropped out, followed by 4 percent of smartphone users, and 5 percent of tablet users. These abandonment rates compare very favorably to Decipher’s numbers for their non-optimized surveys, and are about half those found by Marketing Strategies International for mobile-friendly surveys in 2012.6 Survey length not significantly affected: Desktop and tablet users took almost exactly the same time to finish the survey (just over 13 minutes), while smartphone users took about 20 seconds longer. However, given the lower character counts on mobile (see below), this would suggest that the mechanics of taking a survey on a smartphone (e.g., loading, scrolling, and zooming) may add a bit of time to the experience.
  • Survey results consistent across devices: We found that survey responses were strikingly similar across devices – same winning answers, same top-ranked items, same qualitative themes in open-ended questions. We saw no evidence of straight-lining, order bias, or any other data quality issues attributable to device usage, with one notable exception: a multi-select grid question. In our mobile-friendly format, these types of questions are presented as a series of multi-select voting questions, so each rating item is evaluated separately. Out of 30 possible selections, we found that desktop users made significantly more selections overall (13.2 versus 8.5 for tablets and 9.3 for smartphones). We hypothesize that respondents are more discriminating when presented with items one at a time, giving each one more consideration and leading to fewer selections overall.
  • Universally positive user experience: When we asked members about their experience completing the survey, the vast majority reported that it was “somewhat” or “very easy” regardless of device (84 percent desktop; 78 percent tablet; 85 percent smartphone). Furthermore, most say that same device is their preferred platform (though smartphone users, at 55 percent, were least likely to say this). This indicates that optimized designs vastly improve the mobile survey experience, even as some members acknowledge that it’s not ideal.
  • Image viewability remains high: Concept, package, and creative testing are a big part of the research business, and this often involves the detailed evaluation of images. In our survey, we asked members to view an infographic and provide their impressions. While smartphone and tablet users were more likely to say they could not view the image (7 percent versus 2 percent for desktop), the vast majority of members were able to view and respond to the stimuli, regardless of device.
  • Open-ended character counts vary, but content is similar: Unsurprisingly, character counts across four open-ended questions were lower on mobile devices, with tablets (68) falling in between smartphones (58) and desktops (85). However, we have seen evidence that even these shorter mobile text responses are more verbose than other sources: In a separate survey, our mobile open-ends averaged 94 characters while Google Consumer Surveys’ averaged 53. Beyond length, responses were qualitatively and thematically similar, but contained fewer adjectives, details, and emotions; a “reduction in articulation,” as one of our consultants put it.

This last point is especially significant within online communities. We have come to expect rich, descriptive text answers from our members as one of the benefits of having an ongoing dialogue with consumers, but shorter answers don’t mean mobile text responses do not provide value. While open-text questions will never be ideally suited to the mechanics of mobile, it’s important to think about the context and purpose for asking such questions. If focused, in-the-moment feedback is the goal, then length and depth may not be as important as capturing an unfiltered response in real time.

These findings emphasize the importance of matching method with research objective and anticipating the impact of planned or unplanned mobile access. They also show the advantage of a respondent pool that is dedicated, highly engaged, and “trained” over time to provide high-quality responses. Just as Decipher found that panel respondents’ experience with long, complex surveys made them less likely to start one on a smartphone, long-term collaborators know what’s expected of them. Therefore, when they choose to participate in surveys with their mobile device, and when provided with an optimized experience, those high-engagement behaviors translate naturally.

Best of all, because these community partnerships can last for months or years, they cultivate intimacy and trust over time, giving brands the right to ask for more and not just surveys, but a range of activities and tasks that are possible only through mobile.

Beyond Surveys: Playing to Mobile’s Strengths

While mobile survey methodologies and platforms will need to evolve to continue gathering reliable quantitative data, to focus only on surveys is to ignore the true power of mobile devices: They are with us all the time, enabling the type of real-time, in-the-moment insight that often gets lost through recall and self-report.

Researchers must leverage a range of technologies (e.g., mobile ethnography or virtual shop-along) to connect with consumers on the go through text, photo, video, and audio. These tools provide the opportunity to conduct a range of explorations, from focused, mission-driven tasks to ongoing, ethnographic discovery work. By using these activities as a complement to ongoing community engagement, we can gain a deeper, more holistic view of each customer while leveraging the strengths of both in-depth and in-the-moment interactions. The key is knowing when to employ each methodology to best address the learning objective while optimizing the participant experience.

Established relationships and trust really come into play when we ask consumers to do things like download a mobile metering app that will track what they do on their smartphone, record a video of themselves trying on blue jeans in a dressing room, or take us on a remote shop-along playing the role of part-tour guide, part-investigative journalist. By enlisting these consumers as ongoing brand advisors, they have come to truly value the opportunity to share their opinions and their lives. Without that existing connection to the brand, they may not accept these missions, or care enough to do a good job.

In many ways, mobile leads research away from the disintermediation of online and back to the interpersonal methods of focus groups and IDIs. On devices that have become extensions of ourselves, interactions feel one-to-one, even if they’re not, and it matters who’s on the other end. As researchers look to leverage capabilities such as geolocation and biometric sensors, new personal boundaries will come into play. Given the uniquely intimate nature of these devices, it will become ever-more critical that consumer research move from a series of one-off encounters to a long-term, mutually rewarding collaboration.

Mobile is no longer the future and we need to start fitting research into reality. Truly embracing a mobile-first mindset means shifting your priorities and your perceptions of how research should work: speed over perfection; brevity over detail; immediacy over purity; and relationship overreach. Though the transition may be challenging, and a little scary, the rewards more than justify the effort. Employed thoughtfully and carefully, mobile research can both enable and reinforce deeper, more intimate connections between brands and their customers.



Get the PDF of this article.