Have you ever stopped to wonder how a company can accurately measure consumer behavior without knowing what every single person is doing? If so, you’re not alone. In fact, we see questions about this very topic all the time. After all, with more than 325 million people living in the U.S., it’s understandable that people might be skeptical about our ability to measure media engagement and shopping habits without checking in with each and every person in the country.
For many reasons, it’s not practical or necessarily a good idea to try measuring consumer behaviors by engaging with as many people as possible. In other cases, it’s simply not feasible. Some people choose to opt out of measurement—whether by choice or by their circumstance. And this can skew the resulting data to focus only on those groups that are most likely to participate.
The good news is that it is possible to understand the behavior of a very large group without engaging with each person individually. People are essential to person-level measurement, and panels made up of key groups of people that are selected to represent a larger universe of people can provide insight into the behavior of the larger population. But how exactly is that possible, especially when panels include far fewer people than the larger groups we’re measuring?
Because our panels provide person-level insight into true consumer behavior, we use rigorous data science and statistical sampling methods to ensure that the population we measure through panels and surveys accurately represents the greater population. Generally speaking, sampling is a statistical means of measuring a given population with the intent to accurately extend the measurement to represent a larger population. But just like measurement, not all sampling methods are created equal.
Sampling methods typically fall into one of two categories: probability- and non-probability-based. In probability-based sampling, every person in a population has the same chance of being selected to participate. In non-probability sampling, it’s possible that some people have a greater or lesser chance of being selected. So in order to ensure equal representation, some form of probability-based sampling must be used.
But there’s more to sampling than picking between these two different methods. Specifically, sampling processes can vary in complexity and execution. Said another way, the parameters for carrying out a sample are fairly general, and the integrity of any given sample depends on how thorough it was prepared. For radio measurement, we apply more than 100 critical processes that drive and enable our Portable People Meter (PPM) and Audio Diary measurement services. We perform these processes on a daily, weekly, monthly, quarterly and annual basis to ensure that Nielsen produces the most representative sample possible.
So what does the overall process look like? At the onset, we leverage U.S. Census data to estimate the demographic (age, gender, race, ethnicity, language preference, etc.) and geographic composition of the population we’re measuring, ultimately focused on establishing sampling targets for our Nielsen products. Based on historical data and predictive models, we then forecast the demographic makeup of a randomly selected household and the likelihood that it will participate. We run these models at a granular geographic level (such as parish, city or county), and we constantly monitor these models to maintain a representative sample and make updates as needed.
When it’s time to enlist households for our panels, we use various recruitment methods across Nielsen as a whole. For radio measurement, for example, we contact sample vendors that help us establish the primary contact details for a wide range of potential panelists within a measured area. This becomes our starting sample. Once we have stored the contact information for our starting sample, our team begins the sample selection process, ultimately prompting various downstream recruitment operations (e.g., recruitment materials are mailed, phone interviews are conducted, membership representatives visit households, etc.).
Across the entire process, there are many factors that determine whether a household will be given the opportunity to participate in our panels and surveys. These factors are directly connected to our ability to achieve a representative sample of the population that we measure.
It’s true that we’re living in a world in which technology, cookies, return-path data, likes, playlist algorithms and myriad other digital markers leave “footprints” wherever our online lives take us. But much of that data isn’t complete. Much of that data carries bias. Much of that data wasn’t meant to be used for measurement. Person-level information, however, remains the definitive source for true measurement. And sampling is the key to unlocking person-level, representative measurement.
