The Independent Film Channel's home page used to be one big "Dead Zone" before they redesigned it.

Ad placements are, by definition, dead zones.   These are specific, predictable sections of a website’s screen real-estate that are subconsciously tuned out by the user as unrelated to the page’s main content and functionality.  But designers unintentionally create dead zones of their own all the time.    A classic and well understood example of a dead zone is “right-rail” blindness.  Content and features below an ad – such as in the right-hand column of a typical two or three column layout – are tuned out as ads on the assumption that everything from an ad down is also an ad.   According to Nick Gould, CEO of the design and research firm Catalyst Group, the evidence of this phenomenon goes well beyond the anecdotal.   “There is no question that right-rail blindness is a phenomenon we’ve observed in both eye-tracking and usability testing.  This is of course mainly due to the ingrained expectation that ads live there.”   And it’s not just a matter of positioning elements in a layout.  The manner in which a page element itself  is designed can greatly amplify or lessen the dead zone effect, in the worse case scenario unintentionally deactivating important content areas and features from the user’s attention.   Often this comes from trying so hard to make an element “pop” visually, that the reverse effect occurs.   “The dead zone effect is obviously exacerbated if elements below ads are ad-like in their design,” Gould says.  ”Furthermore, promotional elements that have standard ad dimensions and contain images are frequently mistaken for ads.”

Eye-tracking data shows that the "right rail" has little visual salience with users.

There is strong evidence now that users have specific expectations about where to find the most relevant information on a web page before having seen the page.  In a recently published eye-tracking study observing  usage of a broad range of web sites for both page recognition and information foraging tasks, the right-rail blindness phenomenon was dramatically confirmed.  (see Bushcer, Cutrell, & Morris 2009) “It is striking that the entire right side above the fold is neglected for both information foraging and page recognition tasks.”    The eye tracking data reveals that users are looking at the right rail, but the “fixation impact value” (a computation that correlates fixation length with the amount of visual information processed) approaches zero.  According to Gould, “This all relates to visual salience – which is more about an unconscious assessment of what’s relevant and what’s not that occurs pre-cognition.  A variety of design, placement, and relative size/shape issues can contribute to items on the page not being deemed salient.”

Netflix consistently avoids "Dead Zones" with careful use of white space and "Functional" looking design cues.

As a predominantly qualitative researcher, these eye-tracking findings do not surprise me.  I’ve frequently observed that small differences in visual design and placement can have a dramatic effect on a user’s perception of whether they should pay attention- not to mention interact- with a page element.  Most of us who work on commercial websites have to balance a colorful, branded look while simultaneously creating a usable interface.   I’ve observed a few tricks over the years.  For instance, promotional elements that have an interactive “call to action”, such as a blank zip-code field in a registration opt-in promo, will outperform a straight image & copy version of the same message.  Netflix.com are masters of working with a heavily branded design full of bold, contrasty elements, yet systematically avoiding the creation of dead zones with careful use of white space and  “functional looking” design cues.

Original page layout (with CramCast ad) on cliffsnotes.com

I find myself conflicted (not to mention confronted by other stakeholders) with a desire to make something “pop” and the counter-intuitive, opposite notion that an integrated look is the key to discoverability.   We recently ran a multi-variate test on www.cliffsnotes.com to test the hypothesis that elements with a more “integrated” versus “ad-like” design perform better in right rail placements.  It was my first dabbling with the free tool Google Website Optimizer (GWO) – which turned out to be one of those “Where have you been all my life?” moments.   In the Cliff’s Notes test, we took an existing internal promo on the site for a podcasting series called “CramCast,” which was designed as a standard 300 by 250 ad, and redesigned it in a style more consistent with the existing functional right-rail modules on the site.   We made no changes to the copy or the photo between variants.  The positioning on the page was exactly the same.  The only difference was the look and feel.  Then we ran both versions in a split-traffic test on the live site.

The integrated look outperformed the original "Ad-Like" look by a narrow margin of 6%.

The integrated version did outperform the ad-like version, but not by a huge margin – only a 6% improvement.  (We got about 30 more clicks per 200,000 visitors with the Integrated look.)  It was not worth our time to redesign the creative material for that campaign, but certainly good counter-evidence the next time someone says something doesn’t pop enough.   What we couldn’t get at with GWO was whether the integrated look, since it replaced the traditional ad placement and cast less of a shadow on the user’s salient zone,  helped to activate the entire right rail.  This is the more interesting question.  After all, the worst dead zone problems are when multiple ad-like elements conspire to deactivate entire regions of a page – such as the old homepage design on IFC.com (pictured above.) In this design,  multiple ad-like features are interpreted sub-consciously by the user as one big ad area, and important content features (such as the programming schedule) are overlooked.  In our  next A/B test here, we’ll have to monitor the clicks outside our variants as well, perhaps with the advice of someone who is an expert with these tools – like Avinash Kaushik or Marko Hurst.    If anyone has done any research on this issue, or can point me towards soemthing else to share here – please comment on this post!

What are Anti-Patterns?

(Definition taken from Looks Good Works Well) Anti-patterns, also called pitfalls, are classes of commonly-reinvented bad solutions to problems. They are studied as a category so they can be avoided in the future, and so instances of them may be recognized when investigating non-working systems. The term originates in computer science, apparently inspired by the Gang of Four’s book Design Patterns, which displayed examples of high-quality programming methods. - Wikipedia Anti-pattern

If a subject is steadily verbalizing while performing a task (e.g. concurrent verbalization), they are assumed to be speaking from short term memory.  Dumas & Redish (1993) conveniently summarize three levels of think-aloud protocols commonly referred to by HCI researchers:  Level 1 verbalizations:  where the emphasis is on pure thoughts with no or minimal explanations; Level 2 verbalizations:  same, but when the participant is dealing with non-verbal information, like shapes, which must be internally “coded” in order to be articulated verbally, and Level 3 verbalizations:  or “thinking plus explanations.”  The latter are also referred to as retrospective reports, because the respondent is recapping and opining about what they actually did earlier.  Dumas summarizes the distinction between Level 3 and the other levels as the stage where the researchers are “no longer getting a read out of short-term memory… rather it is the interpretation of the process they are using or the reasons they have selected a strategy.”  Retrospective reports are not useless to the user researcher, in fact they are necessary to clarify a respondent’s statements and actions, but they are far less valid.

This leads us to the question of active vs. inactive moderation.  In an inactive moderation scenario, the emphasis is on experimental control and creating a unified experience for all test subjects.  This is the old-school style, where the researchers stand behind the glass and the respondent sits in the room by themselves talking out loud like a crazy person.  Participants, faced with the unnatural task of constant verbalization, are typically “coached” on how to deliver a think-aloud protocol.  This often takes the form of a warm-up exercise where the participant and experimenter practice thinking aloud with non-related stimuli, preferably a simple game such as tic-tac-toe (in order to place emphasis on cognitive strategy.)  During the actual experiment, the moderator prompts only when the participant ceases to verbalize: “Please keep talking,” they say.  And that’s all they say!

In a high moderator intervention scenario, the experimenter employs probing questions to focus the participant’s attention on particular features or to elicit and clarify subjective explanations of their behavior.  Moderators are skilled in asking neutral, non-leading questions to minimize bias.  In addition, active listening techniques are employed to emulate the clinician’s empathic stance. This implies paraphrasing of a speaker’s comments to ensure them that they have been listened to, noted, and understood as well as other verbal and non-verbal forms of caring, non-judgmental acceptance.  Mike Kuniavsky, whose work is mentioned frequently in these pages,  lays out the guidelines for “non-directed” interviewing:  questions should be concentrated on immediate experience, nonjudgmental, focused on a single topic, open-ended, and non-binary (e.g. yes-no, true-false).

In a quick look at the literature, you’ll see that Taylor and Dionne (2000) suggest that probes are best deployed for collection and verification of data in retrospective reports, and that they have a detrimental impact on validity if used during concurrent think-aloud protocols.  Preece (1994) suggests that the role of the moderator on the participant is both interruptive and imposes additional cognitive load.  Nielsen (1993) is pragmatic on the subject, suggesting the moderator intervenes as little as possible, yet directing the flow and direction of the interview to maximize the number of usability issues found.   Nielsen is the spiritual father of the commercial usability field, so it’s no surprise that most usability is performed with this degree of pragmatism.  I think it was Jared Spool who once commented that you can only watch participants avoiding clicking on the red button so many times before you are compelled to intervene and ask why.   Plus, most clients of usability research do not share the academic’s interest in validity.  They want enough validity to feel good about the process and the results, but ultimately they want their specific questions answered for a reasonable amount of time and expense.  In Nielsen et. al.’s (2002) interpretation, The human is a psychological being engaged in a psychological interaction, which cannot be reduced to that which is concurrently verbalized.

Some usability researchers, characterized by Whiteside, et. al. (1993), have also posited that observing a user’s behavior is not enough to understand what is happening in terms of higher order thinking and cognitive strategy.  If you set up a priori conditions then you are bounding yourself to learning only what falls within those conditions.  In this view, subjective experience is the most comprehensive criterion for understanding usability.  Concerns of generalizability are side-stepped: the goal is to obtain rich, experiential data.  Since specific questions are the only effective way to elicit and clarify mental models, the researcher must come to terms with at least a partially subjectivist stance. (Tamler, 2001)  This approach values reflexivity:  the subject is full participant in the study, leading the research into relevant areas for exploration as well giving the subject the ability to respond to the researchers interpretations. It is not a search for “truth,” per se, as a philosophical underpinning of this method of inquiry.  The assumption is that an expansion of perspectives leads to the exposure of more aspects of learning.

Here’s some References (I know, for a blog post, it’s over the top… but this stuff’s important!)

DUMAS, J.S. (2001) “Usability Testing Methods:  Think-Aloud Protocols,” in Design by People For People:  Essays on Usability, UPA,  pp 119-129

DUMAS, J.S. & REDISH, J.C., (1993)  A Practical Guide to Usability Testing.  Norwood, NJ, Ablex Publishing Corp.

ERICSSON, K.A. & SIMON, H.A. (1984, 1993) Protocol analysis:  Verbal reports as data (Rev. ed).  Cambridge, MA:  MIT Press

KUNIAVSKY, M. (2003)  Observing the User Experience: A Practitioner’s Guide to User Research,  San Francisco: Morgan Kaufmann Publishers, Inc.

NIELSEN, J., (1993) Usability Engineering.  Chestnut Hill, MA:  Academic Press, Inc.

NIELSEN, J., CLEMMENSEN, T., & YSSING, C., (2002)  “Getting access to what goes on in people’s heads? – Reflections on the think-aloud technique”, paper presented to NordiCHI, Arhus, Denmark, October 19-23

PREECE, J. (1994), Human-Computer Interaction, Addison-Wesley, England

TAMLER, H. (2001) “How (Much) to Intervene in a Usability Testing Session,”  in Design by People For People:  Essays on Usability, UPA,  pp 165-171

WHITESIDE, J., BENNETT., J.L., & HOLTZBLATT, K., (1988)  “Usability Engineering: Our Experience and Evolution,” in Handbook of Human Computer Interaction; edited by Helander, M.  New York, NY:  Elsevier Science Publishers