Breaking Down Balanced Clicks Studies
A researcher might use a balanced click sampling strategy when researching new products/brands, to understand the market landscape, or to determine the right audience for the brand and its competitors. For established products or brands, a researcher would use the balanced clicks approach to highlight fluctuations in the landscape or audience. This is done typically on tracking projects, on pre/post-marketing campaigns, or after major events in the world that influence purchasing behavior.
Researchers sometimes use phrases like, “I don’t know my target audience – that’s what I’m looking for” or “I need a true natural fallout from the general population to assess my audience.” The more detail the researcher can provide upfront, the better the sample provider understands the research objective.
LET'S BREAK IT DOWN
- The world of sampling has evolved from offline, to CATI, to double-opt-in panels, river, and aggregated methodologies. When talking about river and aggregated sample, the concept of balancing does not exist anymore since it requires control of invite outgo and knowledge of response rates – neither concept exists with this type of sample, creating a gap in communication.
- Some sample providers have adapted and are able (via technological innovation) to set quotas on clicks that make it to the main survey. By controlling the invitations from double-opt-in panels and managing the flow of respondents on various demographics from river, some sample providers are able to set demographic quotas and control the respondents that get to the research survey, thus delivering the needed results.
HOW DOES IT WORK IN THE SAMPLING WORLD?
- The easiest to manage sources on balanced clicks studies is when using double-opt-in panels. With full control over the number of invitations sent at any given time and statistical predictability of response rates, sample providers using this type of sample can control the balancing throughout fieldwork.
- River sample cannot be balanced. These respondents are not sent invites, they come in when they want, and the concept of response rate does not apply to them — there is nothing to respond to! However, most sample providers that have river capabilities have used them enough to understand the flow capabilities and can control this via quotas on their end. This type of sample greatly expands reach, especially for audiences that are hard to maintain in an online panel.
WHY DO SOME SAMPLE PROVIDERS FEAR USING THIS SAMPLING STRATEGY?
- First, the risk of miscommunication is extremely high. This can result in completely missing the goal of the project. A sample provider should facilitate clear and manage nuanced communication.
- Second, with double-opt-in panels, feasibility becomes limited, so using multiple panels to complete a survey results in deduplication which lowers the conversion and thus could increase the sample provider's cost if they lack the experience in managing these types of studies.
- Third, with different types of sample, a sample provider has a much more labor-intensive task at hand which requires a high level of attention to detail.
BALANCING VS. QUOTAS VS. WEIGHTING
Balancing sample is done at the invitation level. It is done during the sample outgo design phase. It is only applicable to DOI panels. There are 2 ways to balance:
- On invites. This is when you control how many invites are sent for each control group (demographic). This way of sampling is not precise, as the speed of answering and actual response rates vary from demographic to demographic. For example, young males will be slower and less likely to answer surveys vs. older age groups.
- On clicks in the survey. Also referred to as sample outgo, this approach means the sample provider will automatically adjust the number of invites sent to less responsive demographics to ensure the profile of respondents clicking into the survey matches the required profile. This is the strategy that should be used for balanced clicks studies since the analysis of the market landscape is done on the penetration of completes vs. clicks. The inbound clicks are what need to match the required profile (i.e. gen pop).
Quotas are typically used for completed surveys. When using quotas on completes, the researcher knows the targeted audience and sets targets according to the research goals to have enough respondents with a specific demographic profile for analysis. “Balancing on completes” means setting hard quotas on completes.
Weighting happens post-data collection when fielding has finished. This is a mathematical algorithm that can offset the lack of respondents in a specific demographic by counting their answers with a higher weight during analysis versus those where we have higher numbers. Each company uses different acceptance rates for weighting, but it is generally considered that a weight of 2 is the maximum acceptable.
Based on the research objective and the sample size needed for analysis, managing a balanced clicks study becomes more of an art than your “average everyday study”. Given all the recent challenges in the world, we can anticipate changes in consumer attitudes, behaviors, and brand perception. In turn, there will be more demand for brand/product penetration studies.
At Paradigm, we are ready to be your go-to resource! We will work with you to ensure that click balancing is the correct approach – and we will be ready to implement to make your project a success.
With results-oriented researchers just like you in mind, we created this guide to help you design and field a successful study. So what are you waiting for? Click below and let our team of experts guide you through the complete process.