I was recently called by a large national organization who wanted to ask me a few survey questions. This group is one of the largest political action committees in the United States, and for the most part I support their cause although I do take issue with some of their stances.
Statistically speaking, my gut told me this was not going to be reliable research from the onset. I am not a member, so I’m assuming that they were using some sort of RDD sampling, but I honestly do not know based on what follows. It wasn’t a blind research call as they identified themselves, it didn’t use any sort of demographic screening questions and I was asked to listen to a message from their president prior to moving into the survey questions.
What followed was a pre-recorded, heavily biased diatribe, which was clearly intended to sway anyone who might be on the fence about the topic and further whip into frenzy even the most ardent supporter. If you were against the topic, I’m sure this message would have made most people hang up.
When I was transferred back to the “interviewer,” he once again read an intro statement, pejoratively named specific political foes that almost always vote against their issues and asked his first question. I was honestly without words. I informed this person that although I am generally a supporter, I was going to refuse to answer any questions that they might ask me. I let them know that as a market researcher by profession, I was embarrassed with what they were doing and that if they were going to ask the questions in this manner it was no different than just making up the results.
The interviewer thanked me and hung up, but I continued to wonder how they were going to use this research. In general other than supporting their obvious position with overwhelming and biased numbers, trying to gather even more donations and memberships along the way and continuing to berate the “other side of the aisle,” I don’t see much further use.
Sadly, news organizations will receive a press release and potentially report the results as true public opinion without once investigating the sampling credibility or ask to review the questionnaire that was used. But this isn’t surprising since there are nightly polls conducted by the same networks that are fully a self-selected sampling, completely unscientific and passed off as legitimate data
What do you think? How can we create the change that is needed with these sorts of practices or are they a fait accompli? How would you have reacted to the type of call that I received?
Photo Credit: Nelson Davis, taken at MRA ISC 2014