Turn a new page with better research. Diary studies are making a comeback in User Experience Research (UXR). Once seen as insightful but an extremely difficult practice, diary studies have benefitted from the transition to an increasingly connected environment via texting, email, and purpose-built research platforms.
What exactly is a diary study?
A UX diary study is a research method that revolves around collecting qualitative data about user behaviors, activities, and experiences over a defined period of time.
Diary studies can run as long as a few days, to months, or longer. During this defined time period, users (participants) are asked to keep a diary to log specific information about the study’s focus (often task-oriented activities like brewing coffee).
Benefits of UX diary studies.
Easy to execute
Diary studies shouldn’t be conducted in a vacuum, they need to be carried out within a user’s reality in order to capture the context. Studies like these enable researchers to see the full picture and grasp how the participant’s environment and real-world circumstances impact what they are studying.
Quickly scalable
Due to their low cost and low resource utilization, diary studies can easily be scaled to include more participants or to widen the focus of the study without corrupting research.
Minimize biases
Diary studies happen in or close to the moment and do not necessitate real-time observation. Due to this – framing effects, hindsight bias, and performance bias can be reduced simply due to the solitary nature of this method.
Affords flexibility
Researchers have the ability to work remotely, away from the users that they are studying, allowing them to increase their geographic reach. Researchers can also evaluate data as it comes in or they can wait until they have time to do so.
Increases diversity
Diary studies are an adaptive methodology that can be used by participants near and far. In doing so, researchers benefit from seeing activity among a variety of user types to glean even greater insights.
Big Picture
Diary studies enable researchers (and companies) to see the whole picture: the data, the people, the activities, the feelings and thoughts, and the possible solutions. Diary studies enable researchers to finally share the story behind the data.
When to conduct a UX diary study.
Diary studies are a powerful method for researchers who want to:
- Know what comes before the big decision like buying a house
- Know what motivates users to do something
- Understand engagement
- See usage in a variety of contexts
- How differing lifestyles impact users’ relationships with products
- Ascertain customer loyalty and thoughts on competitors
- Don’t want to influence users with the presence of a researcher
Common UX diary study scopes
Product/ Website: To understand user interactions with either
Behavior/ Thoughts: To gather general information on user behavior
General Activity: To acquire insight on how people complete general day-to-day activities
Specific Activity: To collect information on how users complete specific activities
Methodologies
In-Situ (In-Situation)
In this method, users are asked to log information about experiences relevant to the study in the situation in which they occur. This is key to this method. Users will be logging information within mere moments of, or during the experience itself. This method comes close to “in-the-wild” studies, so it is important to minimize the amount of time a participant spends logging data.
Important considerations:
- Data being captured should be minimal to not disrupt a user’s day
- Providing data should be quick and efficient for users
- Use alerts to remind participants in the moment
Tools: EthOS, email, webform questionnaires, digital log, audio diaries, and video diaries
Snippet Technique
A popular and less intrusive method that has users record snippets of information during an activity or shortly thereafter. Users then come back later (within the same day) to elaborate on the snippets, providing essential details about the activity.
Important considerations:
- Less disruption to a user’s life
- Participants must remember to come back to elaborate on the initial snippets
- The more time between the snippet and a user providing detail, the more potential insight lost
Tools: EthOS, email, text, chat, Twitter, Facebook, video diaries, audio logs, digital logs
Conducting UX diary studies
Before you get started
Plan and prepare for your next UX diary study by defining:
- the timeline of the study;
- the tools that will be used;
- the study’s focus;
- what you will ask of participants.
Additionally, you will need to recruit participants and prepare instructions and support materials.
During this phase, it is advisable to conduct a pilot study to ensure the study plan, materials, and researchers involved are prepared to run with real participants. This dry run of the diary study eliminates potential gaffes that could make a study unusable.
Conducting orientations
Researchers will conduct orientations with users to walk them through the logistics of the study:
- the reporting period schedule;
- expectations of participants;
- how to log information;
- what exactly to log
Researchers should provide sample logs so that users understand what information researchers are expecting to gain from each log. It is important to encourage variability outside of the samples so that users provide the full experience of their interactions. Create clear and direct logging instructions for users.
Let the logging begin
Once users start logging experiences, ensure each participant is provided a very simple framework to submit feedback within. This framework should require minimal disruption to their day to ensure real, usable insights are gained. Encourage candid responses from users during the logging period.
Keep users motivated by offering an incentive. Encourage participation by paying the incentive in installments over the study period.
Researchers should recognize organized and insightful participants. Motivate users that aren’t providing insights in a timely fashion or providing limited insights with alerts and clarifying questions.
UXR teams should be looking at the data as it comes in to begin evaluation. By sinking into data early, researchers can ask probing questions about specific logs while they are still fresh in the mind of the user. This is why text messaging features are extremely valuable to diary studies.
Exit interviews
After the study, evaluate all the information from each participant. Conduct exit interviews to gain further insights on the overall experience as well as feedback on the study process.
Data Analysis
After revisiting the initial research questions, UX researchers can begin to weave qualitative and quantitative information into full-fledged stories that speak to the need for a change or a new solution. The data analysis will be key to evangelizing stakeholders behind a solution.
In short,
Diary studies can be extremely effective and efficient. In UX research, it is incredibly important to keep the user in the central focus, which more complex methods can lose sight of. UX diary research doesn’t have to be hard or complex to be insightful. Often the simplest methods are the most telling.