This is a cross-posting of a post originally published on the IDX Backstage Blog.
It was prepared as a “leave behind” resource for participants at the 2014 Design for Social Innovation conference who attended the speed teaching session I hosted on mobile diaries.
In the spirit of Legible Practice I wanted to document in a bit more detail some of the aspects of what was discussed in those sessions. I hope this is a useful resource for participants and those who weren’t able to attend but are interested in the method. I’d be delighted to hear any feedback you might have…
Header image: janitors @ Flickr
Why mobile diaries?
One of the key tenets of design (in all its guises—design thinking, service design, etc.) is a desire to better understand the world of the people we aim to serve as a key input into the design process.
There are numerous reasons and benefits for this: building empathy, ‘zooming out’ to understand context of use, understanding motivations/frustrations, gaining insights into barriers that might impact the behaviours/attitudes etc. we aim to influence or ‘change’. (At the conference, keynote speaker Gerry Greaney used the term “BACKS measures”—Behaviour Attitude Condition Knowledge or Status—which I think is apt description of the various elements we may wish to impact through the design process applied to social innovation.)
There are many ways that we can seek this greater understanding. Surveys, interviews, analytics, and more can be employed. Each has its place in the design researcher’s toolkit. The key is to tailor the method to the nature of the questions we are seeking to answer.
For example, while interviews (in isolation) can yield great results, they do present some challenges: asking someone directly about their needs/wants/experience/etc. can result in people saying what they think we want to hear. Also, the nature of an interview provides insight into what they say, not what they (actually) do (and you may be familiar with the oft-recognised gap between these two things for many people). Some methods are good at uncovering explicit knowledge—the things that people are aware of and think are worthwhile or important to point out. But not (necessarily) tacit knowledge—the stuff that’s highly personal, difficult to formalise, heuristics etc.—that is often the proverbial ‘gold’ of design research.
“Getting out of the building” and observing people’s behaviour is one way we can seek to address this. Design thinking toolkits (like those outlined in IDEO’s Method Cards) can lean towards observational research methods for this reason. Lean Startup suggests that “learning doesn’t happening in the office”—highlighting the value of getting out “into the field” to learn. Shadowing, contextual inquiry, trace analysis, and user testing sessions are all examples of observationally-based research methods.
In my experience, a key aim of research is to frame questions in such a way as to minimise bias. Different observational techniques offer differing degrees of “intrusion”, which may influence a participant’s behaviour and/or what they say/do. If our aim is to get natural and “least biased” responses, this may be key consideration when evaluating different research methods.
What are mobile diaries?
I was first introduced to mobile diaries by Penny Hagen and Natalie Rowland. They have written a terrific primer for Johnny Holland online magazine that I think is a great starting point: Mobile Diaries: discovering daily life. Rather than try to reproduce/recreate their excellent work, I’d recommend reading that post first—my hope is that this post builds upon that and adds some further practical tips that may be useful to others considering this design research method.
The way I would characterise mobile diaries is a mashup of cultural probes and traditional forms of participant diaries (i.e. notebooks). A more practical description might be: using social and mobile technologies—e.g. smartphones and low-cost video cameras—to enable self-reporting of behaviours, context and other factors with rich media.
The basic gist: provide participants with a smartphone and/or video camera (or have them use their own if appropriate) and a series of activities to capture elements of their world.
Typically we provide a “pack” that goes out to participants with the core equipment provided, instructions for use, and a series of cards that outline the activities to be undertaken.
An example pack sent to participants containing activity cards and other materials. Image courtesy Natalie Rowland (used with permission).
As is common practice with interview methods, the activities would start by exploring the topic at hand more broadly, and then hone in on more specific questions over time. These questions would be “answered” and explored using video, photos and other capabilities of the smartphone/devices provided.
Depending on the technology available and context, some studies will see these materials being relayed to the researcher in real-time (e.g. posted to a blog at the time of a photo being taken). In other studies, the equipment (and thus materials) are sent back at the end of the study period.
The studies I’ve been involved in have used a variety of different technologies—from MMS and SMS, to WordPress-powered (and customised) blogs, to dedicated app platforms such as Revelation. We’ve also used a variety of devices—the (now defunct) Flip cams to iPhones.
When the technology allows, it can be fruitful to augment the pre-defined activities with researcher-instigated prompts to the participant, asking questions or following up on insights gained in near real-time.
Mobile diaries don’t exist in isolation: we typically pair the diary process with “exit” interviews with participants after initial analysis of the data they’ve self-collected. This provides an opportunity for the researcher to dig into areas of interest—unusual or unexpected things they did, specific comments, to understand the “why” or to have participants reflect on patterns that may not have been immediately obvious to the participant during the process.
What are the benefits?
In some respects a mobile diary study is similar to participant diary methods that have been used for many years. However, using social and mobile technologies brings with it some additional benefits.
Personal: The (typically) personal nature of a persons relationship to their smartphone or mobile device can be leveraged to create a degree/level of comfort that is difficult to achieve when the researcher is physically present. This can help improve the quality of the data captured, as the participant may be more comfortable sharing information like they might with a friend via SMS or a social network.
At hand: A smartphone will often be present in environments that would otherwise be inaccessible to a researcher. For example, private homes or homes of friends, public transport, workplaces etc. The portable nature of the device also means that it can be used to capture impromptu or unexpected happenings at the time they occur that may not be captured/remembered in a traditional journal approach.
Temporal: Methods like shadowing require the researcher to be present for the full duration of observation. Using mobile technology as the “proxy” means that the researcher can still connect and prompt the participant, but doesn’t necessarily have to be present at all times. This means that you can capture experiences over a longer time period (e.g. a week or two) rather than a more compressed timeframe (e.g. a day) for a similar cost.
Rich: The rich media captured by these devices can prove tremendously valuable, not only to the researcher (providing visibility and insights that would be unavailable using other methods), but also in communicating research results to teams. At the outset I mentioned “empathy” as a key goal—a video segment or photo can bring to life anecdotes, insights, stories and narratives in a way that words on a page (e.g. from a transcript) simply can’t achieve. I find it makes learnings more tangible—they feel more real when you can actually hear the voice of the participant.
Interactive: depending on the technology used, the researcher may be able to ask questions and provide prompts during the study using the device (as noted above). This can be really important in terms of sustaining momentum, building rapport and also being able to elicit more details. It also means that the researcher can “check in” on how things are going and ensure that the data being collected is appropriate (e.g. quality, quantity, matching intent etc.)
The technique is possibly most effective for questions that require reflection on behaviours or actions that are ‘captured’ through the process.
There are probably other benefits as well, but these are the highlights from my perspective.
What are the downsides?
As with any research method, there are downsides that need to be considered.
Discomfort with being ‘on camera’: While the use of a mobile device can increase comfort levels of participants (see above), it is important to recognise that many people don’t like to be “on camera”. So when asking for video responses, it can be useful to arrange activities in a way that they are not in the spotlight (e.g. talking from behind the camera as they video an object or scenario, providing spoken narrative).
Challenges with capturing public spaces: Often we might ask participants to video parts of their daily life. Often these will be in public or shared spaces. It’s important to include guidelines (e.g. getting permission from people depicted). Taking video or photos in public spaces can also create a degree of discomfort for participants in the research, so trying to come up with activities that can occur in more private settings can be helpful.
Self-reporting: It is a self-reporting technique, which has its positives and negatives. On the plus side:
- You are hearing directly from the participant
- The participant is empowered to shape how their ‘story’ is represented
On the down side:
- It’s not a “direct” observation of behaviour—that same positive opportunity to “shape” the story may distort or move away from the impromptu or reflexive responses/actions (leaning into the ‘telling us what they want us to hear’ territory)
- While there is an element of observation involved, the researcher is still somewhat restricted to seeing what the participant feels is important
Challenges & tips
As with any research method it takes some time and practice to get the most out of it. Here are a couple of tips that come to mind that I’ve picked up working with mobile diaries.
Provide clear instructions: We usually provide instruction cards that provide a clear and concise description of the task we’d like the participant to complete. Be sure to test your instructions on someone who is unfamiliar with your project to make sure they are as clear as you think they are.
Don’t overload the activities: Have one clear output/outcome per activity. We’ll often provide one activity card per day over a study period.
Regular checkins: It’s important that the researcher remain in touch with participants throughout the research period to ensure there are no barriers or blocks, or that the person fall too far behind. Where possible, structure activities in a way that if one activity is missed, it doesn’t blow out subsequent activities. This can also reduce the number of ‘drop-outs’ from a study (which is always a challenge/risk with self-reporting studies).
Easy technology is important: If the instructions for how to use the technology can’t fit on one A5 card with large type, it’s probably too complicated. Try to choose technology that is super-easy not only to use, but to share media. e.g. an app that does the capture and uploads immediately with minimal configuration or intervention. Don’t assume that just because something is ‘point and shoot’ that you don’t need to provide directions. A simple visual depiction of the key elements of a piece of technology can be really helpful at removing roadblocks to participation.
Build confidence: Your directions and pack should start with some broad, easy activities that help participants build confidence in using the technology. For example “take three photos of XYZ” before you get into harder questions that require people to take video footage or narrate to camera etc.
Hone in on key questions: Building on this last point—structure your activities in a way that you start broadly around the topic of interest, and get more pointed and directed in your questioning over the study period. This helps build comfort and confidence of the participant, and has the potential to uncover context/tacit knowledge that may prove quite valuable.
Video (and rich media) is important: Avoid activities or technology choices that lean towards your mobile diary being a ‘survey filled out by smartphone’. Be explicit in your asks around video and photo materials. e.g. “Take a video showing us XYZ and explain why that’s important to you”; “Take 3 photos today when you notice XYZ in your workplace.”
Value of “dedicated device”: While it can be handy to rely on people using their own smartphone (or similar) device, it’s not always the best thing to do. Providing a “dedicated device” such as a low-cost video camera (but one that has sufficient quality and handles low-light well) can:
- create a prompt for participants (reminding them of the day’s activities);
- create a sense that “I have been handed this specific device to document specific things about myself and my life”. It’s a little hard to describe, but there is something about the device having a specific purpose, a sense of importance attached to it, that can aid in encouraging participation (especially in the capturing of richer data, rather than text-based responses).
- make it easier to use and reduce opportunities for tech (or user) failure—a simple “point and press the button” interface of a video camera may be a easier for someone to use than a smartphone (which has multiple apps, a touch screen, etc.)
- make it easier for participants to share their data (“just post the whole unit in this pre-paid envelope at the end of the study”). This can be particularly helpful if the participants are not confident with technology and aren’t comfortable with uploading etc.
- enable the researcher to “pre-configure” the device, making it easier to use/upload etc.
There is a downside to “video only” devices, as suggested above, in that you can’t see what they have captured until the end and can’t ask questions throughout (which, as noted above, can be very powerful).
In summary
Mobile diaries can be a powerful tool in the design researcher toolkit for observing research participant behaviour. The personal nature of mobile technology can provide specific benefits in achieving our research objectives. At its best it can provide access to contexts that would be otherwise inaccessible and uncover tacit knowledge and behavioural patterns over time that can be a challenge to identify using other methods. But as with all research methods, consideration of the downsides and potential pitfalls is important. It is a self-reporting method which has positives and negatives, and the technology itself can be both a positive but also presents its own challenges.
I hope that this overview provides some practical tips that may be useful to others considering this design research method. As noted at the outset—I’d love to hear your thoughts…
Update 2015-10-05: Further reading—I just came across this great writeup of a video diary study from Nick Bowmast. Well worth a read.
