1:14 a.m. - 2002-11-13
Survey Says?
I got a call around dinner time asking if I would be willing to participate in a public opinion survey. Depends on my mood whether I'll respond or not. I feel sorry for the surveyors because they're just trying to put food on the table like the rest of us. Cold calls to a cranky public aren't the easiest way to make a buck and most of them wouldn't be doing it if there were other options available or if they didn't need the money. Most of the callers are people with issues at home that make it difficult for them to work regular 9 to 5 hours - you know a sick parent or small child at home, where there always needs to be one adult available, or a young person trying their first job.
That being said, most polls are commissioned by some body or organization that needs "proof" that they are doing the right thing or that the majority of citizens approve of a certain line of thought upon which they wish to act. Surveys are carefully crafted to elicit the response that the commissionee pays for. It isn't just the way the surveys are worded either, it can also be in the sequencing of questions where the choices become more and more subtly circumscribed so that one is actually left to choose among responses where, under normal circumstances, one wouldn't consider any of the choices acceptable because the question is unacceptable in it's make up or subject matter. For example, on an issue where you absolutely object, you might be asked to choose one of two lesser evils for a question phrased as support for the issue. Sometimes the questions, especially if they are commissioned by a political body, are testing the emotional response of the public to volatile issues. Trial balloons. They don't really care what you think about the issue, they just want to know how far they can push the envelope before the public reaction threatens their time in office.
The second part of the manipulation is in the interpretation of the "statistics" gathered, seemingly from a "random" population. Yet, before they start asking you survey questions, the surveyor will ask a few personal questions. They're checking to see if you're likely to give the responses required and if you are in the "target demographic" - the group the commissionee wants to convince. The survey company will release the results of their random, unbiased poll without, of course, ever disclosing the full text of the survey or how many people declined to be involved, complete with "expert, objective" commentary by some pundit or other. Hence the quote "There are lies, damned lies, and statistics."
How do I know this? Number two son worked for one of the major polling outfits in Canada. He finally quit, because he couldn't, in good conscience, administer some of the surveys. The agency always allowed staff to opt out on surveys they found offensive, but my son also objected to the process he witnessed them undertake - focus groups and test runs of the questions - to develop the surveys so that they were certain the statistical outcome could be manipulated to be what the commissionee had paid for.
In addition, many of the volunteer projects I worked on, especially where there was government involvement, required a "needs and preference" survey before we were allowed to proceed. For example, with the regional park we helped build, we were required to survey residents to determine what kinds of features and activities the majority of the community wanted or didn't want. Fair game and a fascinating process to boot. To develop that survey, I was given the opportunity to work with specialists in that field who demonstrated to me how difficult it was to create a survey that didn't have bias. They did this through ensuring none of the practices mentioned above were used, even unconsciously, or that no one chose language that would elicit certain responses. An example would be, wording that evoked strong subconscious emotional reactions. There were test runs to see if the questions were worded clearly. It was amazing to receive back a dozen test papers, each of which had entirely different interpretations of the same question, meaning very different answers were supplied. Respondents interpret the language they see based on their subjective values and the experiences they have had that relate in their minds in some way. If one leaves room for comments or clarification after each question, those thought processes or values are often revealed. Next, they taught me how simple things like grouping the responses together in different ways could affect the statistical outcomes. Verbal sleight of hand.
Anyway, I did answer the survey tonight partly because it was on the issues I've been writing about the past month or so, partly because I was cranky and wanted to vent, especially on those particular issues, and partly because I was told all additional comments about the survey and its'questions were being recorded. I could critique both the issues and the survey questions to my heart's content - and I did. An hour and a half later the patient surveyor said good-bye. She got her commission and I was a lot less crabby. Win, win. Can't wait to see the stats reported on this one.
�
�
