MarketTools Blog

This is the third installment in an ongoing series looking at the invaluable list of 26 Questions to Help Research Buyers of Online Samples assembled by ESOMAR, the global non-profit market research organization.  You can review the other entries in the series here.

This time around we’ll explore the policies of panel companies for blending sample: “Do you supplement your samples with samples from other providers? How do you select these partners? Is it your policy to notify a client in advance when using a third party provider? Do you de-duplicate the sample when using multiple sample providers?” (Question #20 on the list.)

Ultimately I think it’s fair to say that most panel companies have at some time engaged in the practice of blending their samples with those of other sample providers (or will in the future).  Reading between the lines, you might realize that while there are four separate questions above, they really boil down to a single question: “Is this ok?”  The answer, in my opinion, is “Yes, it can be,” but I’m getting ahead of myself.  Let’s tackle each question in turn and dig into the key issues related to each and I’ll show you how I reached my conclusion.

1.    Do you supplement your samples with samples from other providers?
Make sure you get a clear “yes” or “no” response – anything else is already headed in the wrong direction.  There are a couple of key reasons why you might want your panel provider to blend sample:
-  You need them to deliver sample from countries where they don’t operate their own panels.
-  You are conducting studies that require low incidence respondents that really cannot be delivered from a single panel.

This should be viewed as a value-added service that a panel provider is engaging in on behalf of their clients.  Many clients do not have the procurement infrastructure to use multiple providers to fulfill their sample needs, or do not have the volume of business to engage with multiple vendors and still keep their operations efficient and cost-effective.  Where low incidence studies are concerned, the ability to blend sample can be viewed as an insurance policy that gives clients the confidence that their survey can still be completed even if there are changes in scope or incidence while fielding the study.

2.    How do you select these partners?
This is a much more critical question, and starts to get into the more important aspects of blending sample – the practice does require that sample providers be highly selective in which partners they work with and what they require of them.  At a minimum, look for a firm that conducts extensive reviews of the industry (50+ sample vendors) to both stay on top of best practices, and to make sure that they select only the best partners to blend their sample with.  In addition, they need to implement and mandate participation in an industry-leading data quality validation program (such as TrueSample).  With these requirements in place, clients can have confidence that their providers have their best interests in mind.

3.    Is it your policy to notify a client in advance when using a third party provider?
Transparency is a key issue in this situation, and the answer again should be a clear yes or no.  Ideally a panel provider and their clients will work diligently to establish a candid and honest relationship.  Ultimately, the provider’s responsibility is to make sure that the research they are delivering to an end client is of the highest quality and as accurate as humanly possible.  Our industry does not have a place for antiquated ideas of secrecy.

4.    Do you de-duplicate the sample when using multiple sample providers?
Yes, and only yes, is the appropriate answer to this question.  The technology exists to de-duplicate sample, and it’s a critical process that relates to the quality of the data being supplied.  Panel and sample suppliers must have proactive and clear policies on how they identify and manage overlap across panels as well as analyze panelist behavior within a survey.

In conclusion, blending sample with other providers is an acceptable practice, and it's also a value-added service that drives real value to clients – provided that the sample supplier has taken the proper steps to identify and select the best supplemental sources, and implements an automated technology-driven data quality solution.