Ten Questions you Need to Ask your Market Research Vendor
GIGO—Garbage In, Garbage Out. Well, people aren’t garbage, but data from an unqualified sample might as well be. Be sure and challenge your market research vendor on the following ten questions when qualifying respondents:
1. Will the sample be drawn from a market research panel or an email “blast”? Panels created for the purpose of market research offer more accurate sampling and predictable response rates than email lists. The cost for using a panel may be higher, though email blasts often require repeated efforts, raising their cost — both in terms of money and time.
2. How is the sample validated? The professional qualifications, demographics and contact information must be individually checked for accuracy from the beginning of list creation through opt-in registration, as well as upon completion of the survey.
3. Will the results be statistically significant for the market to be studied? A detailed understanding of the total number of individuals in the group to be studied is essential to determine the number of responses necessary to achieve statistical significance.
4. How do you minimize sample bias? Bias can only be minimized through a deep knowledge of the market’s demographics, pre-testing and weighting of the responses.
5. How many different search criteria can be used to “pull” the sample from the database? The depth and breadth of the search criteria must reflect the demographics of the population and be robust enough to support the detailed sample requirements plan.
6. Will the number of responses needed to achieve statistical significance be guaranteed? Achieving the pre-determined response quota per segment is a far more important consideration than overall response rates.
7. Will the response quota be large enough to permit cross-tabulations and other statistical analysis of important groups of respondents? It is important to decide what statistical models will be used in the analysis phase so that over-sampling of certain groups will ensure sufficient responses from groups of high interest.
8. What quality control steps are taken to ensure sample integrity? Carefully defined sample parameters, use of screening questions and thorough inspection of the raw data must all be used to ensure the sample is representative of the population being surveyed.
9. What incentives will be offered to respondents? Monetary and non-monetary incentives are proven to increase response rates, but balancing the two is important to ensure thoughtful responses and to control costs.
10. How do you prevent “professional” survey takers? Respondents whose only interest is receiving an incentive can be detected and removed from the dataset by manual inspection of statistical variances and automated time-based integrity checks.
And of course, you can challenge us on these points any time!
