Archive for category User & Design Research
Validity and Think-Aloud Protocols
Posted by Todd Toler in User & Design Research on August 5th, 2009
First adapted from the work of experimental psychologists – most notably, Ericsson and Simon’s landmark 1993 work Protocol Analysis, – think-aloud protocols are the de-facto standard for usability research in both the lab and field settings. If you’ve seen or given a usability test before then you know what this is, it’s when the moderator tells the respondent to use a website or other application and then says “Hey, tell me what you are thinking.” Jakob Nielsen and other HCI researchers were quick to trump the merits of this technique for uncovering usability problems with sample sizes as small as four people. Why is the technique so effective? Well, it’s validity stems largely from the fact that it’s a direct measure of what’s happening in a subject’s short-term memory. Other examples of direct measures of human cognition are hard to find… in fact, the two others that are primarily used are response tests (e.g. reaction time indicators) and MRI brain scans! So to have a direct measure that is cheap and easy to administer and also provides qualitative insights into the user experience is powerful indeed! But if the interview is poorly moderated, or descends into a Q & A session between moderator and respondent, then this validity flies out the window… so let’s look at the issue more closely.
10 Focus Group Moderation Tips
Posted by Todd Toler in User & Design Research on August 3rd, 2009
Photo Credit: curiouslee
Focus groups are probably the most difficult moderating challenge. And I know this because I was terrible at conducting them until I learned a few tricks…
Practical Tips for Fielding Design-related Ethnography
Posted by Todd Toler in UX-Driven Company, User & Design Research on August 1st, 2009
Ethnographic Insight: Manhattan Vets Store Their Extra Prescription Pet Food Where They Can
It’s been a great week in the field conducting ethnographic interviews with veterinarians and vet techs. Mostly we’ve been in Manhattan, where food storage in the animal hospitals is a major issue. We’ve seen bags of prescription chow stacked in hallways and offices, hidden in unused cages, stacked on surgical operating tables and x-ray equipment, and almost always taking over the waiting rooms and foyers of the client greeting area. Someone should devise a drop-shipping or home delivery scheme wherein vets can earn their margins re-selling the supplies but don’t have to receive and store the inventory.
I doubt this idea will be used by my employer, which for this job is our scientific, technical, and medical publishing division, Wiley-Blackwell. I’m lucky to work for a company that is committed to understanding its customers with this kind of research before embarking on a product idea. My favorite study design of late for the early, product discovery phase of user research is to do 10-12 remote contextual inquiry interviews via phone and web conferencing software, followed up by 5-6 in-home or in-office ethnographic visits. It’s a large enough of a sample to cover a couple of key audience segments and really learn their unmet needs, but still quite cost effective. And even just a few visits out to a respondent’s real environment crystallizes the findings and brings the research to life. We’ve refined the mechanics of fielding a study like this without breaking the bank, so let me share a few pointers with you.
Read the rest of this entry »
Crafting Simple, Effective User Surveys
Posted by Todd Toler in User & Design Research on July 10th, 2009
Photo Credit: dgilder
Without a doubt, online surveys are the cheapest, easiest way to reach out to a large number of a website’s customers to gauge their opinion- but if not crafted correctly, they can result in misleading information. You do not need to be a research expert to field a successful survey, but knowing the limitations of each option in your research toolkit is the only way to get accurate, impactful, and persuasive results. There are many issues to cover within this topic, but let’s begin with the basics by crafting a survey that successfully answers three key questions:
- Who is using the site?
- What do they think of it?
- Are they accomplishing what they want to?
If we can get accurate answers to these questions, then we can deduce what needs to change about the site in order to become a greater success.
Recent Comments