Introduction
Product testing is like painting the Golden Gate Bridge – it’s an ongoing task that is never really finished. Products need to evolve with the ever-changing needs of consumers, so product testing and needs assessments must be done regularly to ensure relevance.
Between bringing new products to market and monitoring the performance of established products against ever-changing conditions, most companies have at least one product testing program underway at any given time.
Depending on where you are in a product’s lifecycle, there may be multiple teams with a vested interest in the research outcome. And if the product or product line in question anchors the brand, then the stakes are high because change is a risk (Just ask Coca-Cola.)
Digital ethnography is a research method growing in popularity that facilitates innovation and de-risks product development. The value of digital ethnography comes from its ability to capture human insights that allow product teams to better understand their customers as people and how they interact with products in the real world. This enables businesses to be empathetic in their design, leading to experiences, products, and features that customers actually need.
We’ll discuss this type of research in more detail but first, a quick review of what product testing typically seeks to accomplish and where the main challenges lie.
Product testing can have a lot of moving parts
The punch list for a specific project will vary depending on where it falls on the product roadmap. Concept testing seeks to capture customer reactions to a new idea, a new product, or a new twist on an existing product. Concept testing is often conducted using digital mockups and provides a valuable “reality check” before investing in the development of an untried idea.
- Does it elicit measurable positive interest from the target audience?
- Which aspects drive the appeal?
- Are there any red flags, potential weaknesses, or pitfalls that could undermine success?
The next step is often testing the product as a prototype. Again, the objectives are to gauge interest, check for “turn-offs,” and identify refinements that could enhance appeal or value. Then the final version of the product is often test marketed to manage risk before committing to a full-scale launch. And there may be separate but related research conducted around packaging, pricing, and POS tactics, to name a few.
Expensive, time-consuming, logistically complicated
Central location testing (CLT) has long been the backbone of product testing research. Researchers recruit participants to visit a focus group facility or mall to try out a new product in a controlled environment. While this approach enables the research team to eliminate some sources of bias by controlling the conditions in which participants engage with the product, it has certain drawbacks. Mainly, the “controlled environment” likely bears no resemblance to the circumstances in which consumers actually use the product in “real life.”
The CLT approach is also time-consuming and expensive. Sampling multiple markets require setting up the test in several locations. Depending on the product and testing scenario, there is likely a limit on how many people can feasibly participate in a given session. This means a broadly sampled test across multiple markets could take weeks to plan and field, and then you need to consider the time required to analyze and report.
Furthermore, no matter how carefully a CLT project is designed and implemented, some segments of the market will inevitably be excluded, e.g., those with physical, economic, or logistical barriers to traveling to the research location.
The alternative is to send the product home with participants and have them report back about their experiences. The main challenges with how this is traditionally done are tied to the reliability of self-reported quantitative data and the researcher’s inability to understand the context surrounding product usage. There’s also the loss of real-time interaction to ask follow-up questions and probe for details.
If only we could test products in real-world conditions with the power to observe
If you ever watched a nature show where trail cameras record animal behavior up close and in the animal’s natural habitat, you might have – like me – thought, “Wow, wouldn’t it be great if we could do that with consumers!” Well, now we can with digital ethnography tools.
Sticking with the animal analogy, digital ethnography is like studying animals in the wild compared to at the zoo. When you take animals out of their natural habitat, they’re bound to change their behavior, and you lose the context of how they interact with other animals and the environment. To understand an animals’ true habits and behaviors, it must be studied in its natural habitat.
Digital ethnography basics
Digital ethnography uses an online platform that enables participants to engage in research via their smartphones and complete specific tasks set up by the moderator. In a typical project, participants will upload videos and/or photos, keep a daily diary, answer open-end questions, and perhaps engage in one-on-one chats with the moderator. The moderator may also invite a subset of participants to a real-time or asynchronous group chat, sort of a micro focus group within the larger project.
The moderator’s interface is a dashboard where they can review participant entries, monitor activity, and interact with other research team members. The moderator can see who is falling behind in the study and quickly issue reminders to participants as necessary. Researchers can also pose follow-up questions to everyone in the project or individuals as needed, in real-time or asynchronously.
Engaging via smartphones creates a connection between participants and researchers that can be intimate without being intrusive. It’s a natural way for respondents to participate in a study as they already take videos and pictures and message friends and family on their phones in their everyday lives. Using smartphones for data collection allows participants to capture experiences as they go about their regular routines. They can stay in their comfort zones, encouraging candor and forthright sharing, even about private or sensitive topics.
Taking product tests into the wild
Understanding how users engage with a product when nobody’s watching is the holy grail of product testing, and digital ethnography can take this area of research to an entirely new level. From initial ideation to finding new feature opportunities and line extensions, real-world feedback keeps product and marketing teams aligned with the needs of their target audience.
Because it scales easily and is more cost-effective than CLT and other traditional product testing approaches, digital ethnography is better for agile research. A single digital ethnography project can sample consumers across the country and even globally while tapping demographic segments based on income, gender, household makeup, category usage, etc.
The CLT method of placing test users in a controlled environment creates consistency, but it is artificial and eliminates unknown swaths of information about how customers use products in the real world. Digital ethnography captures the context, giving researchers a detailed, rich perspective of usage patterns and experiences. Participants share in real-time what they are doing along with where, when, how, and why. To participants, it feels like a one-on-one conversation with the moderator, but at a distance, so they are typically more forthcoming than they would be in person, let alone in a group setting. Plus, nobody has to depend on after-the-fact recall. Experiences get documented in the moment.
Whether your product team needs new ideas, feedback on concepts, or to confirm needs with established product users, digital ethnography offers a fast and efficient way to capture authentic feedback from your target audience.