Verner Von Croy mentors Lara Croft directly within the main game play of Tomb Raider: The Last Revelation. Most games have dedicated training modules and are therefore less compliant with cognitive apprenticeship theory.

In leading up to my presentation at IA Summit 2010, “Think Like an Instructional Designer,” I’m posting on the important learning theories that any interaction designer would be well served to know the basics of.

Theory #3 – Cognitive Apprenticeship Theory

Cognitive Apprenticeship Theory can be a powerful instructional framework for interaction design, in fact it’s one of my favorites to think about, but it’s best not to take the theory too literally.  Collins and Brown, most closely associated with the theory, were writing some 20-odd years ago and they did not have computerized environments in mind at the time (they were mostly interested in classroom pedagogy.) Their genius lay in the recognition of a theoretical gap between students’ learning to integrate sub-skills and conceptual knowledge. Despite the educator’s best intentions, when the two were unintegrated, the information remained inert.  They started to notice that the most successful in-school learning had very similar characteristics to out-of-school learning (most notably the concept of “apprenticeship”.) They observed a strong interplay between observation, scaffolding, and increasing amounts of independent practice.  And while many before them had emphasized the power of conceptual learning and independent practice (see Lave), they thought more about how to provide  “internalized guides” during periods of relatively independent practice.   CAT is an extension of situated learning theory, but rather than leave things as a purely sociological construct (e.g. Lave & Wegner’s “communities of practice”) they placed a strong emphasis on methods (modeling, coaching, scaffolding, fading, articulation) and sequence (global before local, increasing complexity, increasing diversity.)

The reciprocal teaching method in reading instruction is the most famous example of Cognitive Apprenticeship Theory in action.

Its weaknesses lie in the limitations of its description: dated, hard to extrapolate beyond well-structured domains, overlapping with similar-sounding theories, and too committed to it’s inspiration within the apprentice-master models found in non-formal instructional environments (you know, traditional crafts like blacksmithing).  Unfortunately, the theory is not well applied to technological environments and modern notions of interactivity.   When it has been applied, it’s been done so mostly in classrooms in highly rigid lesson formats such as those used in Palincsar and Brown’s reciprocal teaching methods for reading instruction. But the theory does an excellent job of abstracting successful non-formal (or “out-of-school”) learning attributes into a set of principles for instructional designers to work with.  In fact, the theory does exactly what Collins and Brown claim a good educator should do – “make the invisible visible.”  It is also one of the only formal learning theories that strikes the right balance between discovery learning and structured learning – the sort of balance that video game designers have intuitively found but educational designers seem woefully behind in.  What attributes does a learning environment have if it’s influenced by cognitive apprenticeship theory?

• Expert modeling (particularly the notion of “distributed expertise” – or multiple mentors to learn from)
• Combination of scaffolding (adding help when needed) + fading (removing help gradually)
• Reflection on performance (e.g. replay and abstracted replay)
• Articulation (student demonstrations of expert performance)
• Exploration (relatively independent practice)

.

A diagram of a someone's possible schema for the concept of "egg." Source: P.Davis 1991

Schema theory is a foundational element in almost all cognitivist descriptions of learning, and this one will likely make immediate sense to user experience practitioners because it is tightly entwined with the familiar concept of mental models.  The theory emphasizes the role of prior knowledge and provides a robust explanatory framework for how expert performance is attained.  Ok, so what’s a schema?  Piaget defined the term in 1926 as a mental representation of an associated set of perceptions, ideas, and/or actions.   Think of a schema as a network of connected facts and concepts into which any newly-formed structures can be fitted.  Then think of your brain as a bigger network of overlapping schema and sub-schema.  The schema themselves are a markup language for the brain, cognitive XML if you will.  Schema theory explains why we remember things so subjectively (In Bartlett’s 1932 research on people’s memory of stories such as the “The War of the Ghosts,” he found that in reconstructing a story they added elements of their own culture.  This is famously considered evidence of schema theory’s existence.)   More interestingly to designers, the theory can be exploited to provide more effective instructional materials.

Learners generally want to read for “gist” and “theme.”  Why? So they can incorporate the new knowledge within their existing schema framework. When learners lack specific schema-based prior knowledge, they tend to apply general problem solving strategies in inefficient, even unsuccessful, ways. (see Driscoll’s 1994 book on this topic)  Worse, a lack of a proper conceptual model can transform learning into the ‘rote’ memorization of a seemingly arbitrary series of steps.  While this is an effective method in learning to tie one’s shoes or to master the alphabet, it leads to problems when things get more complex – as in, say, learning how to use a modern software application.  Donald Norman (1998) has called this disconnect between what we can conceptualize and what we are being asked to learn the “bane of modern existence.” hmm… to make a new layer in Photoshop, just press CONTOL-ALT-SHIFT-N.

Of course, the rub is that it is difficult to determine what schema a potential user already possesses and to develop an instructional method that guides them towards the task at hand.  According to Driscoll, users arrive at most learning opportunities with an “imprecise, partial, and idiosyncratic” set of mental models.  This is where schema theory dovetails nicely with situated learning theory – the more grounded the instruction is in authentic contexts and shared cultural references, the more likely it is to activate prior-knowledge and allow for the acquisition of new schema.   If you strip situated learning theory from it’s socio-cultural agenda of “authentic thoughts in authentic contexts” (I don’t believe that “situated” learning is the opposite of “symbolic-computational” learning, as some do), the two theories can work nicely together.  Pull the conceptual world into your designs by providing context.  Context can include both the “atmosphere” of the learning environment and the “background events.”  To make a design better at activating prior knowledge in learners:

• Understand and ’segment’ your audience
• Draw upon information that is likely to be familiar to learners
• Contextualize material with analogies and background events (for “gist” and “theme”)
• Develop a sense of “situational intent” (focus on benefits, outcomes, applications, examples)


Photoshop Unsharp Mask Tutorial – Typical Example. Layers Magazine provides beautiful tutorials of how to use the software but nothing about the concepts behind the tool.




Photoshop Unsharp Mask Tutorial – Schema Building Example. Cambridge in Colour takes what could be a description of an arbitrary software feature and embeds it with conceptual meaning and background information.

In the Diamond Age, Stephenson imagines the ultimate discovery learning environment.

Let’s look to the fictional near-future of Neil Stephenson’s The Diamond Age in order to set the tone.   Nell is the novel’s young protagonist.   She is born of limited means to a lower-class single mother named Tequila, but then rises to be a free-thinker and a leader who transcends her class with the help of a nano-technology powered instructional aid, the “The Young Lady’s Illustrated Primer.”  The Primer is state-of-the art interactive technology. A fairy tale book, of sorts, but one with amazing properties. First of all, it talks – and not in that robo-voice of the Kindle 2’s text-to-speech feature, but in an uncannily human neo-Victorian contralto.   The Primer not only recognizes the user and the details of her environment, it can actually work those into the narrative flow.  When Nell wonders aloud during one story “What’s a Raven?”-  the book stops and explains it to her – then it gives her a brief, age-appropriate quiz on how to spell the word.  It is, in other words, a rich, engaging, and perfectly scaffolded learning environment sensitive to the needs of the individual learner.

But the Primer has a key limitation, even in this speculative future of unlimited processing power.   The Primer is commissioned by a Bill Gates type kajillionaire who spares no expense on it’s development, yet the “designed” part of the interactive experience is not so fully dynamic or fluid that it replaces the need for human teachers.  Nell begins to suspect that there is a human intelligence behind her interactive book – which is, of course, exactly the case.   Behind the Primer, is actually a full time “‘ractor” (interactive actor) acting out some of the characters in the stories.  There are other characters modeled fully by the AI (or Turing machines, in the narrative) and Nell is able to fool one of these into revealing their true nature.  A major theme of Stephenson’s book is the rejection of the idea of Artificial Intelligence – favoring the term “psuedo intelligence,” and in doing so he also dismisses something that is the pot of gold at the end of any technology-oriented instructional designer’s rainbow – the automated yet fully individualized discovery learning environment.  The central theme of The Diamond Age is that these sort of designed environments will always have their limitations – even in a future where nano-technology makes diamonds cheaper and more widely available than glass.  A non-subtle illustration of this theme occurs in the novel when an army of lower-class Han Chinese girls who get a cheaper, fully automatic pirated version of the Primer (with no human ‘ractors behind the scenes) turn out to be efficient, devoted and somewhat mindless automatons.

Configuring instructional materials in such a way that they can be traversed with infinite flexibility – depending on the needs of the learner – is not possible in today’s technological landscape (nor will it be ever, if you subscribe to Stephenson’s philosophy and his take on the Theory of Computation).  Therefore strategies must be deployed to balance the desire for dynamic discovery with the practical need for pre-defined content structures and manageable levels of algorithmic complexity.  In large part, the efforts to define and test such strategies comprise the modern field of educational technology.  But the debate is not merely a technological one – it is pedagogical as well.

A Skinner Teaching Machine from the 1950's, which he was fond of saying was just as good as a private tutor. (image source: The Archives of the History of Psychology @ University of Akron)

It is useful to think of structured vs. discovery learning environments as being on a continuum.   On one side lies the extremely structured, which saw a high-point in the instructional trend of programmed instruction .   Behaviorists like Pressey and Skinner devised teaching machines that leveraged the principles of operant conditioning.   The emphasis was on logical presentation of content, and a strict system of rewards and punishment as a learner progressed through their lessons like so many laboratory rodents.   On the other side of the continuum lies the extremely un-structured, where the learner has almost complete freedom to engage with the material in an order and at a pace that suits their individual learning style.  Of course, un-structured is a misnomer – what we really mean is flexibly structured to support complex linear branching based on user input (what has been coined fractal narrative by the literary scholar Marie-Laure Ryan) and alinear hypertext relationships.  Examples at the extreme end of this continuum include Nell’s Primer, of course, or for an example more grounded in reality – a video game such as Spore, which has been used to teach concepts in astrobiology.

Most instructional designers work somewhere in the middle, varying the degree of emphasis on discovery learning depending on the nature of the material itself and the pedagogical goals for the project.   1950s style behavorialism is not fashionable among educators these days, who are dismissive of anything that smacks of rote learning.  Today’s educators want to create conceptual thinkers, who are facile in deploying metacognitive strategies to solve diverse types of problems.  This same forces shape the instructional designer’s objective.  The ID is designing not only for retention of the material but for transfer, the ability to apply the concepts in new types of challenges going forward.   Traditionally,  some educational domains are considered virtually impossible to learn by letting individuals freely explore the materials.  Take Accounting, for example, an applied field in which nearly every concept builds on the one learned before it – necessitating an ordered march through the content.   Yet it is precisely in such traditional bastions of highly structured domains like math, music, and language that the biggest educational revolutions are taking place.   The New Math controversy in the 1960s has flared up again as the New, New Math – a constructivist take on teaching math in which the traditional order of abstract math education is abandoned (and even entire areas of math such as factoring polynomials, which is considered too theoretical to be of much use) in favor of a highly situated, case-based approach.   The “creative spelling” and “whole language” movements in primary education are boldly attacking one of the most traditionally structured domains of all – first language acquisition in children.

A classic 'worked example' of the kind advocated by John Sweller

The academic literature provides ample support for both discovery and structured learning – often in complex combination with one another.    One of the more interesting findings to emerge from research in the field is something called the worked-example effect, advocated most notably by the educational psychologist, John Sweller.   Worked examples are step-by-step demonstrations of how to solve a problem.  Highly structured, worked examples lay the information for the leaner out all at once, or present it a little at a time to facilitate learning. But there is generally a high degree of guidance and modeling provided as part of the main instruction, and a minimal amount of trial and error on the learner’s part. In certain types of domains (math and physics content are often the subjects used in this type of research), worked examples are proven to facilitate learning, particularly from a cognitive load perspective.   Yet, paradoxically, a worked problem can completely backfire when the learner has a degree of prior knowledge and some of the information becomes redundant.   (This is called the “expertise reversal effect.”)  Still, in 2006 Sweller and his colleagues went so far as to challenge the entire concept of ‘minimal guidance’ and claim that the constructivist, inquiry-based, problem-based, and experiential theories of learning flat out don’t work.   This is bold refutation of what has been the single biggest intellectual trend in education dating back to John Dewey.

Molecules & Minds: the direct-symbolic version of the simulation on the gas laws. Note the step-by-step guidance on the left side. Also, in this version, the contols for such variables as temperature and pressure moved on their own as an animation.

Of more applied interest to web interaction designers, a project at NYU’s CREATE lab in 2009 called “Molecules and Minds” sought to directly compare the learning benefits of the discovery vs. worked approach in interactive learning simulations.  As research stimuli, the M & M team developed online simulations across a variety of concepts in Chemistry – such as the gas laws, kinetic theory, and equilibrium.  The same material was designed in multiple versions across two key variants – direct vs. indirect presentation of the material, and iconic vs. symbolic representation of the material.    The first variant is of particular interest to us here.  In the “direct” mode, the researchers essentially provided a ‘worked example’ by animating use of various sliders and user-controlled inputs to demonstrate the chemistry concepts.  In the “indirect” mode, the researchers let the users play with the sliders and inputs on their own until they felt like they had learned the concept.  The M & M team also experimented with providing step-by-step instructions in some versions (the “worked” versions), and none in other versions.    Another varied element included the ‘advance organizer,’ or problem explanation itself, which varied from the more explicitly stated to the more metaphorical.

The findings of the study were interesting – and not nearly as cut-and-dried as the findings in the more static, text-book style content used in Sweller’s studies.   For instance, the worked versions with animations and step-by-step textual descriptions provided the least effective instructional environment of all due to something called “split attention effect,” a cognitive load inducing phenomenon related to trying to read and follow something else at the same time.  The study also linked low prior knowledge to great difficulties in using the exploratory environment.  For instance, low prior knowledge students could play with the variables and watch how the simulation changed but struggled greatly in comprehending the role of the plotted points on the adjacent graph.  They just weren’t familiar enough with the basic variables the simulations were based upon and didn’t have enough access to guidance.   Asking them to start forming understanding of relationships to drive transfer learning was too much.  Adding scaffolding (in the form of contextual hint overlays) helped leverage both the engaging qualities of the exploratory with the instructional benefits of the worked. The study went on, over 3 years, to find that there was no simple answer to the direct vs. indirect question.  In general, the study seemed to conclude that the design aspects matter most for students with lower prior knowledge and/or lower executive function – and that there is much to be gained by continued research into how to get the right combination of exploratory + scaffolding.  Another of the most significant findings of the Molecules and Minds project, that the icons helped learners with low prior knowledge and significantly raised levels of engagement, will be the topic of a future “Think Like An Instructional Designer” post on the role of icons.

Instructional designers seek to measure not only retention, but also "transfer." The Molecules & Minds project measured both near-transfer (highly related to the instruction) and far-transfer (only conceptually related), represented here by the aerosol can of air-freshener.

Teens work on their hand-eye coordination playing Dragon's Lair - circa 1983. Photo credit: The Tribune News

I’m from the earliest generation of gamers.  The first, really.   I played table-top Pong when I was 7 or 8, even though it was only available in bars (parenting was more relaxed then.)  I played Missile Command, despite the fact that it was kinda boring.   Robotron was an obsession.  I played Zork, with it’s command line interface, on my Apple IIc -  drawing my own map.   In college I got sucked into Myst, and Super Mario Bro.’s and there were dozens of others along the way.   And then, like nearly everybody else of my generation, I quit playing video games.   Why? In a word, guilt.  Games were considered indulgent, addictive, violent -something for man-boys.  Certainly not suitable terrain for serious people.  But throughout this entire period of moralization against gaming there was always a bit of pop cognitive science floating around in defense of video games.   Games build hand-eye coordination, people would say -everybody would say.  After a 7 hour stint on the couch I’d think, well, at least I’ve got that going for me.  In graduate school I studied cognitive science and learning theories and even video games- and never once encountered the phrase hand-eye coordination.   So I set myself to wondering – what is it? is it important? and does playing video games improve it?

First of all, the terms hand-eye coordination and eye-hand coordination seem to be used interchangeably.  But the former (HEC) is far more common in the pop-culture definitions of the term, and the latter (EHC) is more used in the academic and scientific literature – so I’ll maintain that distinction here.  Eye-hand coordination is a  neuroscience concept which links eye movement to hand movement (with and without the use of vision.)  You need eye-hand coordination to use a fork, pick up a book lying on a table, and punch your secret code into the ATM.   Your eyes provide spatial information and feed it to a part of the brain called the parieto occipital junction where it is combined with moto-sensory input and then communicated to the hands.   But since we also use non-visual sensations known as kinesthesia to control limb movement, the picture is a bit more complex.   In fact, the process involves such a wide-ranging combination of systems (the central nervous system, vision, rapid eye movements known as saccades, and multiple brain functions) that it is one of the more mysterious and studied aspects of brain science.

THe occipital-parietal cortex plays a key part in controlling eye-hand coordination.

EHC can be impaired but can it be improved? Both normal aging and a variety of disorders that cause damage to the occipital-parietal cortex, including Alzheimers, will impair EHC.  One’s performance in EHC tasks are known to be explained rather tidily by Fitt’s law – a favorite principle of HCI and cognitive science researchers who are exploring point-and-click computer interfaces.  Fitt’s law predicts that the time required to rapidly move to a target area (particularly when pointing) is a function of the distance to and the size of the target.  Clearly the environment itself and the way it’s designed can improve or lessen a healthy person’s eye-hand coordination.  But little in the literature suggests that someone’s general abilities can be improved over time.   A person’s improvement in a specific physical/kinesthetic task over time is a well understood area of learning theory and is closely tied to the principle of automaticity – e.g. the ability to perform an action without occupying the mind with it’s low-level constituent tasks over time.  It is even known that eye-movement differs by it’s very nature throughout the process of mastering a specific task.  But an improvement  in one’s overall EHC would imply that long-term exposure and expertise in one area (say, video games) is transferable to another (say, playing the piano, competing in sports, or drawing).

Importantly, eye-hand coordination is also defined by what it is not.  A person’s fine motor skills, visio-spatial ability, and visual acuity are separate phenomena.  Fine motor skills are learned series of movements, and more specifically, refer to the coordination of small muscles such as those in the (you guessed it) hands.  Children rapidly improve their fine motor skills in their early years as they experiment with activities such as drawing and sculpting play dough.  Motor skills improve naturally, except when they don’t – in which case an occupational therapist is brought in to work with the child.  Visio-spatial ability is a concept linked to working memory referring to one’s “inner sketchpad.” Visio-spatial ability is crucial in one’s ability to visualize objects and tasks, coordinate multiple tasks, react and reason in three-dimensions.  It’s useful in some kinds of math problems.  Males typically are born with more of it, and one study has even linked male-oriented action shooter  games to it’s improvement.  Visual acuity is simply the sharpness and accuracy of our vision, which is mostly related to the health of the nerve cells in the eyes.   To what extent it can be improved is likely to come from something ingested (foods and vitamins) and certainly not by staring at a video monitor for hours on end.  Why go into all this? 1) because cultural definitions of HEC commonly blend these areas, and 2) because studies on the benefits of video gaming often make their correlations to these other phenomenon, not EHC.

Kido Rings - one of the thousands of toys and games on the market that make claims of improving hand-eye coordination

On the web, the term hand-eye coordination  is thrown around with comic imprecision – which matches the anecdotal evidence I’ve collected myself over the years.    A site called bodyomics.com promises that it is great for sports and can be improved by playing the Milton Bradley game, Connect Four.  eHow has an article on how to improve HEC – play catch, eat zinc – and states that by doing so your performance at everyday actions like driving a car and playing with your kids will be improved.  (I wonder how much research the freelancer did on this piece given that they were likely paid around $20 for it – but that’s a whole different story.) The concept is quoted frequently in parenting articles – usually under the pretense that certain kinds of activities and the use of certain toys will increase it, and that increasing it will lead to positive outcomes for the child.  In fact, improving hand-eye coordination seems to have sprouted into a cottage industry.  There are memory games, exercise regimens, special toys, books, and dozens of websites devoted to it.  All this despite the fact that very little is understood about linking eye gazing to performance in visually guided manual tasks (see Sailer, et. al 2005).   Hand-eye coordination therefore exists mainly in the minds of optimistic parents, wanna-be atheletes, and the entreprenuers that pray on them.

Meanwhile, I’ve started gaming again.   My wife bought me a cheap used X-box on ebay and I started catching myself up on the last 15 years of video game evolution – including modern classics like Halo, Medal of Honor, Max Payne, and Metal Gear Solid.  I did this because I am an interaction designer, and games seem to be where it’s at these days in terms of pushing the limits of interface design.  I also did this because I read James Paul Gee’s What Video Games Have to Teach us About Learning and Literacy, an amazing book cataloging the true benefits of gaming.   Gee cites 36 principles behind why games affect learning and development in those who play them, including the principle of projected identity, the self-knowledge principle, and the practice principle.   Unsurprisingly, hand-eye coordination is not among them.

Rounded Corners Are Cognitively Cheaper, Yet Facebook Drops Them Anyways: Image Source: UI & US

Yesterday Facebook announced that it decided to drop all rounded corners in their latest UI refresh, sacrificing those cheerful corner radii on most of its interface modules for the more severe but coder-friendly squared off look. It’ s almost like they’re declaring the end of web 2.0, once and for all. Keith Lang’s UI & Us has consolidated a terrific history of where this rounded rectangle thing started in the first place… with the original Macintosh apparently. Folklore has it that Steve Jobs pointed out to Bill Gates that the real world is full of rectangles and squarish shapes that have rounded corners -stop signs, coffee tables, beverage coasters – so why not user interfaces? Right angled shapes are computationally efficient to draw, but let’s face it – you can put an eye out with one of those things if you’re not careful. And that’s exactly the kind of human-centric thinking that has Apple rounding the corners on everything from your iPhone to those error message pop-ups that you’re getting in iTunes. Interestingly, Lang makes a cognitive processing argument for the benefits of the roundedness, quoting author Jurg Nanni. “A rectangle with sharp edges takes indeed a little bit more cognitive visible effort than for example an ellipse of the same size. Our fovea is even faster in recording a circle. Edges involve additional neuronal image tools. The process is therefore slowed down.”

Linking aesthetics to cognition is not a stretch. We often express a qualitative preference for the look of something that is driven by something more elemental in our biology and our evolutionary need to survive. For instance, we see and process information better when it’s portrayed in color combinations that are found together in nature. I quote a good piece from 2003 by Luke Wroblewski. “[colors] found in nature are often less saturated and more pleasing to the eye than their artificial counterparts. As a result, they allow users to focus on interactions, and not be distracted by overly bright hues.” In fact there is a whole book about this called Evolutionary Aesthetics, which makes the case that our aesthetic preferences are not only driven by evolution, but that there are huge differences in what we visually respond to based on whether we’re male or female and in a precopulatory or postcopulatory state. I’m sensing a whole interesting new category of design research on the horizon here.

Trendy No More - Say Goodbye to Your Rounded Corners in Facebook

So CSS3 is including a property for rounded corners and I’m sure browsers will support this soon. This will relieve the technical “expense” of how they are rendered in browsers now (via the use of image files.) But do we want them? Are they really there to provide a more “organic context” (as argued in this piece four years ago on basement.org), or are they just trendy? I’m sure bell-bottom trousers were more efficient for us in terms of neuronal image processing but that didn’t keep them around any longer, did it? Personally I think there’s something else going on here. We have clearly established a design idiom over the years that elements with rounded corners are things you are supposed to interact with… e.g. rounded corners are a functional cue. Functional cues help us put some space between the kind of design choices we make for an interface versus those we make for print. Think about the kinds of things that consistently don’ t have rounded corners on the web – ads and photographs. Now think about the kinds of things that do – buttons, modules containing links and form elements, navigation menus, and tabs. We don’t want our users tuning out areas of the screen just because it doesn’t look like something that is interactive. For designers, rounded corners have played a tiny role in managing that issue. Plus, they just look so damn cool in that Web 2.0 way, don’t you think?

Well, typically this would lead to a discussion of bad, or extraneous, cognitive load and how to avoid it in the design of multimedia materials.   Attention is a resource, and a limited one at that.   When users strain their ability to actively process material, they are forced to make decisions about what they do and do not pay attention to.  If you ask the user to process more than a few chunks of information simultaneously, working memory is easily overloaded.  Designs which add to this effect are thought to actively generate extraneous load.   You can control for this by following a few basic principles, most notably those outlined in Richard Mayer’s seminal 2001 book, Multi-Media Learning.   An example of a design principle that minimizes bad cognitive load is spatial contiguity.

Mayer’s Spatial Contiguity Principle – Student’s learn better when corresponding words and pictures are presented near rather than far from each other on the page or screen.

Integrated (top) vs. Separated Captions (bottom) in Multi-media Instructions. The Integrated Approach is Said to Reduce Extraneous Cognitive Load

There are many more of these principles in the book and all of them are worth reviewing.   Curiously, what Mayer doesn’t explicitly lay out as an anti cognitive-load principle is one you hear all the time from clients if you are in the design game – make it “less busy,” “clean,” “less cluttered,” more googley.  Surely unsightly clutter must increase extraneous cognitive load, so why doesn’t he mention this?   Because reduction of complexity is not, in itself, a sound instructional design principle.   Some material is just more complex – it has more elements and more types of element interactivity within it.   This inherent complexity is a fact of life and is thought to generate intrinsic cognitive load.    Think of this as cognitive load that the designer inherits as a baseline, only to add to it by breaking Mayer’s rules in their design process.

And now for my favorite part.  Designers can actually create good, or germane cognitive load.   Germane cognitive load “enhances” learning rather than interferes with it; this may be attributable to effects like motivation or increases in effort that can increase the amount of cognitive resources devoted to a task.  You can tell if you are getting germane cognitive load when you have a high degree of intrinsic cognitive load but learners stay within their working memory capacity due to the intelligence of the way the materials are designed.   In other words, a lot of meaningful learning activity would be impossible without cognitive load – it’s just a tougher design challenge.   Mayer’s research was focused on reducing overall cognitive load, with the assumption that mostly it was extraneous load that was effectively isolated in his study design.  But researchers still are looking for ways to measure the different types of cognitive load, so at the moment it’s unclear exactly what can be done to increase good cognitive load (see Kalyuga, 2009).

Edward Tufte Has Inspired A Generation to Create "Germanely" Loaded Data Displays

So why did I go into all of this?  Because I think people shouldn’t complain about clutter and busy-ness in web designs without pausing to think about the benefits of considered complexity.    As Don Norman has said – Simplicity is Highly Overrated.   Of course, in this essay he’s talking about products mostly – and the fact that people value and are willing to pay for complexity and additional features.   But simplicity is overrated in design circles too… complex things are beautiful and persuasive.  Nature is complex.   Sometimes the user’s task is aided by complexity in the interface.  For instance, pattern recognition is a common task for a user on a product website’s list pages, which can be  aided by having more information on a single screen, not less (particularly when it’s optimized for comparison tasks such as the famous Orbitz grid design for listing airfares, now universally imitated.)    We may not know exactly what generates germane cognitive load, but I have a strong hunch that the work of Edward Tufte does exactly that.   So let’s do both – reduce the bad cognitive load and seek to add the good stuff – and we’ll find yet another reason why we should think like an Instructional Designer.

Everybody Loves Puppies, but The Instructional Potential of This Image is Zero

Careless use of imagery – particularly photographs – is often the difference between a dull “brochure-ware” site and a persuasive information experience.   When we redesigned www.dummies.com, we initially populated our design mock-ups with the type of stock photography that professional designers mostly use… only to find that the otherwise attractive imagery added a level of generic-ness that felt downright off-brand.  Then we created a style guideline only to use imagery that had true instructional value, whenever possible framed and centered on the action that was being discussed in the content.  This applied not only to imagery embedded in the how-to articles, but also the landing page teaser thumbnail photos for the articles and videos.  This is a policy the mega-successful book line has had for years, but it wasn’t immediately obvious to us on the web design side.  Once we made this change the site felt instantly better.  The difference?  We thought like an instructional designer.

Interaction designers, graphic designers, and anybody else involved in the ongoing production of commercial websites should pay a lot more attention to instructional principles – even if the site is not overtly instructional.   What makes material good for learning also makes it good for other conversion goals – such as explaining products, services, and brand building.

Dummies.com Uses Explanatory Imagery Even in Thumbnails

In fact, why not audit your site’s imagery right now for instructional potential?  If you find that most or all of your visuals are falling into the “decorative” category – unless of course it’s fluffy puppies or blondes on the hoods of sports cars (two time-tested exceptions) – then it’s time to make a change.   Fortunately, the educational psychologist Richard Mayer has conveniently classified types of imagery in terms of their instructional benefit and effectiveness for learning.  The good folks behind the Wiley Visualizing series, which is a textbook line that is highly committed to applying cognitive theories of learning to the design of their products, put together the following summary table, which I’ve adapted with examples that might apply to a typical web designer’s challenge…

Types of Visuals Commonly Used in Instruction

Category of Visual	Instructional Potential	Examples
Decorative - visuals used for aesthetic appeal, entertainment, or to stimulate interest.	None	Caption: America's 5 Dog Friendliest Cities
Representational - visuals that portray a single object or piece of information in isolation from related objects or pieces of information.	Minimal to None	Caption: San Francisco: A Dog-Friendly City
Organizational - visuals that display relations, structures or organization among multiple objects or pieces of information.	High	Caption: America's 5 Dog Friendliest Cities
Explanative - visuals that explain how systems, processes, or cause/effect sequences work.	High	Caption: What Makes A City Dog Friendly?

If a subject is steadily verbalizing while performing a task (e.g. concurrent verbalization), they are assumed to be speaking from short term memory.  Dumas & Redish (1993) conveniently summarize three levels of think-aloud protocols commonly referred to by HCI researchers:  Level 1 verbalizations:  where the emphasis is on pure thoughts with no or minimal explanations; Level 2 verbalizations:  same, but when the participant is dealing with non-verbal information, like shapes, which must be internally “coded” in order to be articulated verbally, and Level 3 verbalizations:  or “thinking plus explanations.”  The latter are also referred to as retrospective reports, because the respondent is recapping and opining about what they actually did earlier.  Dumas summarizes the distinction between Level 3 and the other levels as the stage where the researchers are “no longer getting a read out of short-term memory… rather it is the interpretation of the process they are using or the reasons they have selected a strategy.”  Retrospective reports are not useless to the user researcher, in fact they are necessary to clarify a respondent’s statements and actions, but they are far less valid.

This leads us to the question of active vs. inactive moderation.  In an inactive moderation scenario, the emphasis is on experimental control and creating a unified experience for all test subjects.  This is the old-school style, where the researchers stand behind the glass and the respondent sits in the room by themselves talking out loud like a crazy person.  Participants, faced with the unnatural task of constant verbalization, are typically “coached” on how to deliver a think-aloud protocol.  This often takes the form of a warm-up exercise where the participant and experimenter practice thinking aloud with non-related stimuli, preferably a simple game such as tic-tac-toe (in order to place emphasis on cognitive strategy.)  During the actual experiment, the moderator prompts only when the participant ceases to verbalize: “Please keep talking,” they say.  And that’s all they say!

In a high moderator intervention scenario, the experimenter employs probing questions to focus the participant’s attention on particular features or to elicit and clarify subjective explanations of their behavior.  Moderators are skilled in asking neutral, non-leading questions to minimize bias.  In addition, active listening techniques are employed to emulate the clinician’s empathic stance. This implies paraphrasing of a speaker’s comments to ensure them that they have been listened to, noted, and understood as well as other verbal and non-verbal forms of caring, non-judgmental acceptance.  Mike Kuniavsky, whose work is mentioned frequently in these pages,  lays out the guidelines for “non-directed” interviewing:  questions should be concentrated on immediate experience, nonjudgmental, focused on a single topic, open-ended, and non-binary (e.g. yes-no, true-false).

In a quick look at the literature, you’ll see that Taylor and Dionne (2000) suggest that probes are best deployed for collection and verification of data in retrospective reports, and that they have a detrimental impact on validity if used during concurrent think-aloud protocols.  Preece (1994) suggests that the role of the moderator on the participant is both interruptive and imposes additional cognitive load.  Nielsen (1993) is pragmatic on the subject, suggesting the moderator intervenes as little as possible, yet directing the flow and direction of the interview to maximize the number of usability issues found.   Nielsen is the spiritual father of the commercial usability field, so it’s no surprise that most usability is performed with this degree of pragmatism.  I think it was Jared Spool who once commented that you can only watch participants avoiding clicking on the red button so many times before you are compelled to intervene and ask why.   Plus, most clients of usability research do not share the academic’s interest in validity.  They want enough validity to feel good about the process and the results, but ultimately they want their specific questions answered for a reasonable amount of time and expense.  In Nielsen et. al.’s (2002) interpretation, The human is a psychological being engaged in a psychological interaction, which cannot be reduced to that which is concurrently verbalized.

Some usability researchers, characterized by Whiteside, et. al. (1993), have also posited that observing a user’s behavior is not enough to understand what is happening in terms of higher order thinking and cognitive strategy.  If you set up a priori conditions then you are bounding yourself to learning only what falls within those conditions.  In this view, subjective experience is the most comprehensive criterion for understanding usability.  Concerns of generalizability are side-stepped: the goal is to obtain rich, experiential data.  Since specific questions are the only effective way to elicit and clarify mental models, the researcher must come to terms with at least a partially subjectivist stance. (Tamler, 2001)  This approach values reflexivity:  the subject is full participant in the study, leading the research into relevant areas for exploration as well giving the subject the ability to respond to the researchers interpretations. It is not a search for “truth,” per se, as a philosophical underpinning of this method of inquiry.  The assumption is that an expansion of perspectives leads to the exposure of more aspects of learning.

Here’s some References (I know, for a blog post, it’s over the top… but this stuff’s important!)

DUMAS, J.S. (2001) “Usability Testing Methods:  Think-Aloud Protocols,” in Design by People For People:  Essays on Usability, UPA,  pp 119-129

DUMAS, J.S. & REDISH, J.C., (1993)  A Practical Guide to Usability Testing.  Norwood, NJ, Ablex Publishing Corp.

ERICSSON, K.A. & SIMON, H.A. (1984, 1993) Protocol analysis:  Verbal reports as data (Rev. ed).  Cambridge, MA:  MIT Press

KUNIAVSKY, M. (2003)  Observing the User Experience: A Practitioner’s Guide to User Research,  San Francisco: Morgan Kaufmann Publishers, Inc.

NIELSEN, J., (1993) Usability Engineering.  Chestnut Hill, MA:  Academic Press, Inc.

NIELSEN, J., CLEMMENSEN, T., & YSSING, C., (2002)  “Getting access to what goes on in people’s heads? – Reflections on the think-aloud technique”, paper presented to NordiCHI, Arhus, Denmark, October 19-23

PREECE, J. (1994), Human-Computer Interaction, Addison-Wesley, England

TAMLER, H. (2001) “How (Much) to Intervene in a Usability Testing Session,”  in Design by People For People:  Essays on Usability, UPA,  pp 165-171

WHITESIDE, J., BENNETT., J.L., & HOLTZBLATT, K., (1988)  “Usability Engineering: Our Experience and Evolution,” in Handbook of Human Computer Interaction; edited by Helander, M.  New York, NY:  Elsevier Science Publishers