Solid State UX

In the Diamond Age, Stephenson imagines the ultimate discovery learning environment.

Let’s look to the fictional near-future of Neil Stephenson’s The Diamond Age in order to set the tone.   Nell is the novel’s young protagonist.   She is born of limited means to a lower-class single mother named Tequila, but then rises to be a free-thinker and a leader who transcends her class with the help of a nano-technology powered instructional aid, the “The Young Lady’s Illustrated Primer.”  The Primer is state-of-the art interactive technology. A fairy tale book, of sorts, but one with amazing properties. First of all, it talks – and not in that robo-voice of the Kindle 2’s text-to-speech feature, but in an uncannily human neo-Victorian contralto.   The Primer not only recognizes the user and the details of her environment, it can actually work those into the narrative flow.  When Nell wonders aloud during one story “What’s a Raven?”-  the book stops and explains it to her – then it gives her a brief, age-appropriate quiz on how to spell the word.  It is, in other words, a rich, engaging, and perfectly scaffolded learning environment sensitive to the needs of the individual learner.

But the Primer has a key limitation, even in this speculative future of unlimited processing power.   The Primer is commissioned by a Bill Gates type kajillionaire who spares no expense on it’s development, yet the “designed” part of the interactive experience is not so fully dynamic or fluid that it replaces the need for human teachers.  Nell begins to suspect that there is a human intelligence behind her interactive book – which is, of course, exactly the case.   Behind the Primer, is actually a full time “‘ractor” (interactive actor) acting out some of the characters in the stories.  There are other characters modeled fully by the AI (or Turing machines, in the narrative) and Nell is able to fool one of these into revealing their true nature.  A major theme of Stephenson’s book is the rejection of the idea of Artificial Intelligence – favoring the term “psuedo intelligence,” and in doing so he also dismisses something that is the pot of gold at the end of any technology-oriented instructional designer’s rainbow – the automated yet fully individualized discovery learning environment.  The central theme of The Diamond Age is that these sort of designed environments will always have their limitations – even in a future where nano-technology makes diamonds cheaper and more widely available than glass.  A non-subtle illustration of this theme occurs in the novel when an army of lower-class Han Chinese girls who get a cheaper, fully automatic pirated version of the Primer (with no human ‘ractors behind the scenes) turn out to be efficient, devoted and somewhat mindless automatons.

Configuring instructional materials in such a way that they can be traversed with infinite flexibility – depending on the needs of the learner – is not possible in today’s technological landscape (nor will it be ever, if you subscribe to Stephenson’s philosophy and his take on the Theory of Computation).  Therefore strategies must be deployed to balance the desire for dynamic discovery with the practical need for pre-defined content structures and manageable levels of algorithmic complexity.  In large part, the efforts to define and test such strategies comprise the modern field of educational technology.  But the debate is not merely a technological one – it is pedagogical as well.

A Skinner Teaching Machine from the 1950's, which he was fond of saying was just as good as a private tutor. (image source: The Archives of the History of Psychology @ University of Akron)

It is useful to think of structured vs. discovery learning environments as being on a continuum.   On one side lies the extremely structured, which saw a high-point in the instructional trend of programmed instruction .   Behaviorists like Pressey and Skinner devised teaching machines that leveraged the principles of operant conditioning.   The emphasis was on logical presentation of content, and a strict system of rewards and punishment as a learner progressed through their lessons like so many laboratory rodents.   On the other side of the continuum lies the extremely un-structured, where the learner has almost complete freedom to engage with the material in an order and at a pace that suits their individual learning style.  Of course, un-structured is a misnomer – what we really mean is flexibly structured to support complex linear branching based on user input (what has been coined fractal narrative by the literary scholar Marie-Laure Ryan) and alinear hypertext relationships.  Examples at the extreme end of this continuum include Nell’s Primer, of course, or for an example more grounded in reality – a video game such as Spore, which has been used to teach concepts in astrobiology.

Most instructional designers work somewhere in the middle, varying the degree of emphasis on discovery learning depending on the nature of the material itself and the pedagogical goals for the project.   1950s style behavorialism is not fashionable among educators these days, who are dismissive of anything that smacks of rote learning.  Today’s educators want to create conceptual thinkers, who are facile in deploying metacognitive strategies to solve diverse types of problems.  This same forces shape the instructional designer’s objective.  The ID is designing not only for retention of the material but for transfer, the ability to apply the concepts in new types of challenges going forward.   Traditionally,  some educational domains are considered virtually impossible to learn by letting individuals freely explore the materials.  Take Accounting, for example, an applied field in which nearly every concept builds on the one learned before it – necessitating an ordered march through the content.   Yet it is precisely in such traditional bastions of highly structured domains like math, music, and language that the biggest educational revolutions are taking place.   The New Math controversy in the 1960s has flared up again as the New, New Math – a constructivist take on teaching math in which the traditional order of abstract math education is abandoned (and even entire areas of math such as factoring polynomials, which is considered too theoretical to be of much use) in favor of a highly situated, case-based approach.   The “creative spelling” and “whole language” movements in primary education are boldly attacking one of the most traditionally structured domains of all – first language acquisition in children.

A classic 'worked example' of the kind advocated by John Sweller

The academic literature provides ample support for both discovery and structured learning – often in complex combination with one another.    One of the more interesting findings to emerge from research in the field is something called the worked-example effect, advocated most notably by the educational psychologist, John Sweller.   Worked examples are step-by-step demonstrations of how to solve a problem.  Highly structured, worked examples lay the information for the leaner out all at once, or present it a little at a time to facilitate learning. But there is generally a high degree of guidance and modeling provided as part of the main instruction, and a minimal amount of trial and error on the learner’s part. In certain types of domains (math and physics content are often the subjects used in this type of research), worked examples are proven to facilitate learning, particularly from a cognitive load perspective.   Yet, paradoxically, a worked problem can completely backfire when the learner has a degree of prior knowledge and some of the information becomes redundant.   (This is called the “expertise reversal effect.”)  Still, in 2006 Sweller and his colleagues went so far as to challenge the entire concept of ‘minimal guidance’ and claim that the constructivist, inquiry-based, problem-based, and experiential theories of learning flat out don’t work.   This is bold refutation of what has been the single biggest intellectual trend in education dating back to John Dewey.

Molecules & Minds: the direct-symbolic version of the simulation on the gas laws. Note the step-by-step guidance on the left side. Also, in this version, the contols for such variables as temperature and pressure moved on their own as an animation.

Of more applied interest to web interaction designers, a project at NYU’s CREATE lab in 2009 called “Molecules and Minds” sought to directly compare the learning benefits of the discovery vs. worked approach in interactive learning simulations.  As research stimuli, the M & M team developed online simulations across a variety of concepts in Chemistry – such as the gas laws, kinetic theory, and equilibrium.  The same material was designed in multiple versions across two key variants – direct vs. indirect presentation of the material, and iconic vs. symbolic representation of the material.    The first variant is of particular interest to us here.  In the “direct” mode, the researchers essentially provided a ‘worked example’ by animating use of various sliders and user-controlled inputs to demonstrate the chemistry concepts.  In the “indirect” mode, the researchers let the users play with the sliders and inputs on their own until they felt like they had learned the concept.  The M & M team also experimented with providing step-by-step instructions in some versions (the “worked” versions), and none in other versions.    Another varied element included the ‘advance organizer,’ or problem explanation itself, which varied from the more explicitly stated to the more metaphorical.

The findings of the study were interesting – and not nearly as cut-and-dried as the findings in the more static, text-book style content used in Sweller’s studies.   For instance, the worked versions with animations and step-by-step textual descriptions provided the least effective instructional environment of all due to something called “split attention effect,” a cognitive load inducing phenomenon related to trying to read and follow something else at the same time.  The study also linked low prior knowledge to great difficulties in using the exploratory environment.  For instance, low prior knowledge students could play with the variables and watch how the simulation changed but struggled greatly in comprehending the role of the plotted points on the adjacent graph.  They just weren’t familiar enough with the basic variables the simulations were based upon and didn’t have enough access to guidance.   Asking them to start forming understanding of relationships to drive transfer learning was too much.  Adding scaffolding (in the form of contextual hint overlays) helped leverage both the engaging qualities of the exploratory with the instructional benefits of the worked. The study went on, over 3 years, to find that there was no simple answer to the direct vs. indirect question.  In general, the study seemed to conclude that the design aspects matter most for students with lower prior knowledge and/or lower executive function – and that there is much to be gained by continued research into how to get the right combination of exploratory + scaffolding.  Another of the most significant findings of the Molecules and Minds project, that the icons helped learners with low prior knowledge and significantly raised levels of engagement, will be the topic of a future “Think Like An Instructional Designer” post on the role of icons.

Instructional designers seek to measure not only retention, but also "transfer." The Molecules & Minds project measured both near-transfer (highly related to the instruction) and far-transfer (only conceptually related), represented here by the aerosol can of air-freshener.