Catalyst Group, who published a comparative usability study of eReaders in 2009, knows a lot more about eBook user experience than I do.

Tomorrow I’ll be participating in a panel discussion for Digital Book World called Reader Experience and eBooks: What UX Experts can Teach Digital Publishers. (A free webcast!) One of the points I’ll be making is that eBooks are only a part of the digital publishing landscape, even a fairly minor one up until now.  On balance, web publishing has been a far more important source of digital revenue for Wiley, and also is generally perceived as the area in which to evolve whole new product lines and revenue streams away from the print business. This seems to be changing as the eBook technologies and web technologies are converging, but still web publishing gets the lion’s share of investment in the sort of customer experience work that I do.  In fact, those of us with technologically focused jobs in the publishing business are practically mandated to break apart the very concept of the book.  It is an informal mantra around here that we don’t just want to put the book online, we want to create a value added service, get deeper into the customer’s workflow.  It comes up on every project.   We are not in the book business anymore, we are in a broader content and services business.  It is the status quo way to talk about digital if you are a person of ambition in the book business.

I’m the Director of User Experience at a major book publisher and have never been involved in design or user research directly related to an eBook. Perhaps it is because eBooks are quite unembarrassed about being based on books, and I’ve been conditioned not to think about books.  This simple association (the word “book” is in the title, afterall) may have caused me to pigeonhole eBooks as a format (think PDF) and not a platform for innovation (think, the Internet or the iPad).  I was impressed with my fellow panelist Brett Sandusky’s description of how the book proposal process is evolving over at Kaplan. He writes, in a UX Matters article he posted back in January, “Our book proposals look nothing like book proposals.  In fact, each day they resemble functional specs more and more.”   According to Sandusky, the teams at Kaplan are interdisciplinary and intra-departmental from the earliest stages in the publishing workflow, from when the book ideas themselves are coming together.

We create our fair share of functional specs at Wiley, not to mention wireframes and other UX deliverables, but I don’t think I’ve ever seen this process done for a book idea.  Usually the book is planned, produced, and sold and the digital ideas that hit our central technology group (where I work) are either completely separate or are created downstream as a form of “line extension” of the print product.   The exception to this, and it is a very long-standing exception, is when the editorial content of the book has a technological component or supplement, in which case the functional bits are planned along with the print product itself.   You’ve seen this sort of thing in textbooks for the past quarter of a century, for instance in the form of a CD-ROM with some extra problems or animations or a quiz on it that comes shrink-wrapped with the book.   This type of development is generally handled in-house by a special “media editor,” who sits in the editorial division but works with a wide variety of technology and design specialists, both internal and external, depending on the nature of the development work.  The CD-ROM has gradually morphed into a “book companion website,” and, now, book companion websites are morphing back into the book in the form of “enhanced” eBooks.

In the central technology group, it is rare that I or my team work on an individual title.  Usually we work on a broad platform, on a capability for an entire imprint or product line, or some other digital business idea that encompasses a wider swath of business than a single book title. This is partly a question of revenue (most books have small budgets), and partly a question of where we think our UX design efforts will have the most impact.

So why has there not been a lot of thought put into eBooks so far from the in-house UX team? One reason is that the eBook formats themselves are quite limited in what can be done with them.  It would be difficult to enforce a design principle uniformly across all of the competing device formats such as Kindle, iOS, Nook, not to mention legacy desktop eBook technologies like those from Vitalsource, the company behind Wiley Desktop Editions, and DRM controlled Adobe PDFs.   The various formats have nowhere near the level of standardized presentation conventions as web browsers do, and this means a publisher has to give up a fair amount of control over the reading experience itself in exchange for having their content available in as many locations as possible.   This is a fundamental trade-off.  In a Frommers travel guide, for instance, the print books make thoughtful use of wayfinders, icons, color photos, call-out boxes, and annotated maps – most of which are not available in a reflowable eBook such as those that appear on a Kindle or Nook.  But this is changing as I type this, with significant development efforts underway to add enhanced features to increase the usability of this type of content.

A Frommers book makes extensive use of color photographs, wayfinding devices such as the colored side-tabs, and interactive maps and icons.

Frommers content in a reflowable eBook format. Stripping out the content's design has made it less usable. Managing this balance between content presentation and the affordances of the eReader devices is at the heart of the eBook's future user experience.

This leads to the obvious question of why be on a platform where we can’t control the user experience?  The answer is equally obvious: a publisher is generally not looking to limit the formats a book is available in – the more, the better.  As people’s reading habits change (going from print to Kindle, for instance), it’s important to keep your content and your brand where your customers want to consume their information.  And customers who are excited about their devices are surprisingly forgiving of a challenging user experience – at least at first.  As these devices evolve and get more and more capabilities, publishers will have to invest more in usable electronic content.

It is such a daunting challenge to think about developing and distributing a consistent user experience across all these formats, in fact, that most publishers are content to resign this work to their print production people – not user experience designers.    The basic workflow is such.  Manuscripts, in the form of MS Word documents, are laid out in InDesign with specific instructions to the typesetter for use in preparing the book. The typesetter delivers web-optimized ePDFs that preserve the exact layout of the print book.  Once the print PDFs are approved and sent off to the printer, the typesetter then bundles them up in a variety of standardized XML formats for distribution across the various electronic channels – Wiley’s own XML for the web, MOBI and ePUB for eReaders, and proprietary formats such as the one Amazon uses for the Kindle.

Depending on the affordances of the particular electronic format, the publisher can include or exclude properties that the print book design may have had.  And this is the rub.  For instance, most eBook formats are “reflowable,” which means the book’s pagination and page layout is not applicable.  Some “enhanced” eBook formats, on the other hand, are quite elaborate, like the Apple iPad edition of Ryu Murukami’s book, which was released straight to the iPad in app form, jam packed with cool transitions, sound effects, arty layouts and DVD-style extras. (This event triggered many resounding predictions of death to traditional publishers as a result, despite that fact that it is in Japanese language only and contains what is essentially a straightforward paginated eBook at its heart.  The potential shake-up is, of course, that Murukami didn’t need a big publishing company to release his book to the iPad, only Apple’s approval.)

Murukami's The Singing Whale was released straight to the iPad.

What would I focus on if I did work on eBooks?  Well, lets talk a little about the usability of eBooks themselves.   For a certain type of experience, a linear text-based one, eBooks are downright dreamy.   “The technology is built for romance novels,” a veteran publisher told me.  “They do a great job of serving up row after row of text.”  This may explain why some of the Kindle’s initial demographic success was not with your typical high-tech gadget early adopter crowd.   Anecdotal evidence would point more towards the moms-on-vacation set.  50% of Kindle users were over 50 in 2009, and 70% over 40.   The electrophoretic, e-Ink displays are great poolside, even in bright sun.  And besides connoting fabulous retirements revisiting the classics on the beach, e-Readers are also great assistive technology, providing tremendous benefits for arthritic hands and impaired eyes.

But the eReader is more mainstream technology every year, and if you count the electronic reading that occurs on phones and tablets, it’s not long until they reach near ubiquity.  It is the obvious benefits of not being in print that are driving electronic book sales – the ease of getting updated content, the ability to annotate and take notes, the fact that it’s often cheaper than the print version, and the most obvious benefit of all, portability (note: the combined physical weight of just two of the books currently on my iPhone, Atlas Shrugged and The Wealth of Nations, would be 4.7 pounds if I were to carry even the smallest paperback editions and not the Kindle editions.)

eReaders, with their resizable text and text-to-speech features, are brilliant assistive technology.

From a user experience point of view, the benefits more or less end there.  For one thing, the underlying concept still has a sizable army of detractors.  eReaders give many people the feeling of putting just too much technology in their lives.  When a book goes electronic, it feels like the electronics are intruding into a sphere into which they don’t belong.   New York interaction designer, Chris Fahey, writing about he and his wife’s resistance to digital books back in 2007, mused “We both grew up surrounded by the printed word — looking at them, feeling them, smelling them — and we intend for our family to continue in that tradition.”

And Fahey makes his living designing user experiences for digital products.   “I’m not going to say much about Kindle,” Fahey wrote. “As an iPhone owner, I find both the device and the service colossally dumb.”

Collegehumor.com mocked the very idea of refactoring books on electronic devices with their spoof ad for a (then not yet released) Kindle 3, in which the device basically plays the Hollywood movie version of a book, complete with crisp color video and Dolby surround sound (the signature line, uttered from a headphone wearing kid over the explosive sounds of The Fast & The Furious:  “I can’t hear you. I’m reading!”)

The Library of Chris "I don't need no stinkin' Kindle" Fahey. Fahey's point is that books aren't just a technology. A person's library reflects their lives and values.

But those predictions of the Kindle’s failure that were written circa 2007 and 2008 sound pretty hollow now.  In 2009 Amazon heavily promoted the news that their digital sales had eclipsed their print sales.  And by 2009, the Kindle was a clear success with estimated sales close to half a million units and projected sales in 2010 of almost a billion dollars.  This NYTimes essay attributed the Kindle’s success to making it really, really easy to use.  But is it?

Nick Gould released some eReader usability research that his firm, Catalyst Group, conducted in 2009.  The Kindle 2, which has been replaced by newer models such as the Kindle 3G since the time of this study (although the interface hasn’t evolved much,) was considered better off usability-wise than some of it’s competitors like the Sony eReader.  But it still had loads of usability issues, mostly as a result of a lot of peculiar and specific (to them) navigation conventions.   For instance, reflowable eBooks don’t have page numbers, they have “locations,” an abstraction which makes little sense to most readers and presents the rather significant problem of orientation within the linear experience of a book.  The hard keys on the Kindle (necessary on e-Ink displays unless a separate touch-screen panel is added, which is the approach taken by the original Nook.  Note the color Nook uses an LCD display, not an electrophoretic one) require a significant amount of getting used to, such as the large, redundant “next page” buttons that appear on both the left and the right side of the device (versus having the left-hand one devoted to “previous page” and the right-hand one devoted to “next page” which most users in the Catalyst study expected.)  Their are other peculiarities, such as trying to find the “text-to-speech” feature of the Kindle behind the text-resizer button and not in the main menu, which is entirely devoted to shopping.  The software versions of the Kindle reader, like the iPhone app, have a few issues of their own.  There is no back button, despite the fact that there are multiple internal linking conventions, such as end-notes, that can cause a reader to jump out of their current position.

What makes the Kindle easy to use, of course, is the ease of acquiring content.  Similar to how iTunes cracked the digital music nut, the Kindle team invested most of their energy in having a wide range of titles with unbelievable ease of purchasing and transferring content to the device.   But the real underlying issue affecting eBook user experience has far more to do with how the content itself is packaged, reflowed, and displayed – not any device conventions.  Expect to see real changes in the next few years as eBook producers develop their own design vocabulary and UI conventions.  One thing is for sure: electronic reading is here to stay.

Go out and grab a coffee when the NMR guy is refilling the liquid helium, unless you are willing to risk quick freezing of body parts or catching shrapnel from a surprise tank explosion. (image source: Dephologisticated)

Most of an organic chemist’s physical work appears to the naked eye as an interchangeable set of clear liquids and white powders (that is to say, if they are lucky enough in the lab not to produce brown sludge.) This is because atoms, even entire molecules, are too small to be seen through the lens of a microscope, so chemists must deduce their shape and structure indirectly.  This is achieved with a variety of instrumentation and analytical techniques, most of which output data in the raw form of spectra, wavy lines that with a little experience can be used to paint a high-resolution image of the unseen.  Because atoms and molecules, even gigantic ones such as a protein or enzyme, are smaller than a wavelength of light, they appear under even the most powerful electron microscopes as a nothing more than a fuzzy blob.  Because it’s not part of our human perception, interpreting spectral data is a difficult challenge that chemists face every day starting when they are undergraduates.  Operating the obscure equipment, and the hardware and software interfaces that this entails, is its own sort of challenge.

There are several types of spectroscopy, which is a broad concept that describes any kind of radiation of energy as it passes through a given material.   Mass spectroscopy or Infrared spectroscopy is widely used in organic chemistry, but is mostly good for identifying mixtures.  For instance, a winemaker might use one of these techniques to understand levels of eugenol in their chardonnay and therefore determine how long to toast their French oak barrels  (eugenol is a compound from oak which gives the clove-like aroma and flavor to wine). Ultimately the Mass and IR techniques are too low in resolution to do what most organic chemists really need to do, which is to confirm if the thing you think you made in the lab is what it is supposed to be. Step in, NMR.  Nuclear Magnetic Resonance, the work horse tool of the organic chemist, and therefore the only one I’ll get into much detail with here.  It is said that if the NMR machine is shut down for some reason, then the organic chemist goes home for the day.  (So in my world I guess that makes it a bit like a Starbucks.)

The ability to detect the resonance of magnetized nuclei and its implications as an imaging technique was one of the great post-war scientific breakthroughs.  What is resonance? Think of how sound resonates from a crystal glass when you run a wet finger around its rim.  A proton makes a similar effect when you hit it with a radiowave, and a  scientist can “listen” for how it responds. Well, that is, some protons do.   To be useful for NMR, an atom must have protons with a property called “spin,” which means their magnetic fields line up cleanly, sort of like a dipole magnet.  These atoms (13C and 1H are the most commonly used by far) can get disrupted and relax back to their normal state without too much asymmetric rotation going on and distorting things, which makes them great for the technique of NMR.

An NMR machine is a complicated instrument. It consists generally of a large thermos bottle that contains a layer of liquid nitrogen surrounding another layer of liquid helium.  This set-up acts as a large superconducting magnet.  A sample of the chemical to be analyzed is inserted down into the middle of the thermos and the giant magnet causes the protons to all line up in the same direction.  Then from this state, the molecule is disrupted with radio waves (or RFs, radiofrequency pulses) and the instrument records the time it takes for the protons to relax back into their normal state.  This time to re-equilibriation (T1) is one of the most telling variables about a molecule’s composition and arrangement of atoms.  For instance, if a proton is shielded because it is densely surrounded, it doesn’t move around much when blasted with an RF pulse. That sort of information helps chemists put a picture together of the molecule’s structure.  The larger the NMR machine(typically expressed on a range of 300-600 megahtz), the greater the resolution is.   And if you are a chemist working on a large molecule, say a protein with 500 hydrogen atoms in it, you need a lot of resolution to separate the signals.

An NMR machine consists of a sample, inserted via the probe, which sits within a magnetized chamber. A layer of liquid helium surrounded by a layer of liquid nitrogen turns the machine into one big superconducting magnet. Source: Rsc.org

The physical operation of the NMR process is fairly straightforward, with a few minor quibbles.  Since these are big-ass magnets, one might need a ladder to insert the sample. Also, it is advised not to wear a pacemaker in the general vicinity.  And, importantly, try and step out for coffee when the liquid helium guy comes to refill the machine.  (Quick-freezing of body parts from escaped vapor, or catching some shrapnel from a highly pressurized tank explosion is no fun.)  Most of the usability pain of an NMR is in the proper calibration and operation of the instrument via its companion software interface and the subsequent interpretation of the results – also done on a computer.

Chemistry labs are still places full of Rube Goldberg contraptions. The NMRs, particularly older ones, are no exception.

A 500 Mhz NMR spectrometer, like the one they have at Rutgers University, is large enough to require a ladder in order to insert a sample.

Try not to trip on the air hose attached to this monstrous NMR in the basement of Bagley Hall at The University of Washington.

Despite a gradually growing commitment to ease of use by the industry (primarily through automation, not necessarily design), NMR machines are a long way from “point and shoot.” Some of them can auto-lock and auto-acquire the sample.  Some of them have auto-feed magazines, which can rotate in multiple samples overnight while the chemist steps out to the nearby brew pub.  Some of them have web-based clients for remote analysis directly from the machine.  But ultimately, there is still a lot of old-timey, goldbergian aspects to the electronics and mechanics involved.   Step on the air line that is draped across the laboratory floor from the nearby air compressor, slightly choking the air flow but maybe not enough to notice, and the finicky machine will start giving odd readings with very little feedback as to what went wrong.  The attached PCs run homegrown software provided by the manufacturer and there is little standardization across the industry in terms of file formats, navigational idioms, or iconography.

“They have the worst interfaces” says Dr. David Flanagan, a former Umass polymer scientist and now Editor in Chief of the journal Advanced Functional Materials.   “They still had green screen, command-line interfaces when I went to grad school.  And this was well after Windows 95 had been released, so the general public was used to a windows based graphic interface by then.   I don’t think it’s changed much since.”   According to Flanagan, the NMR software culture borrows its style cues from early silicon valley, back in the GNU era, when the place was still dominated by bearded, slightly crazed, anti-social types.  NMR is more Woz than Jobs, sort of a UNIX nerd service at the heart of the larger nerd-dom of organic chemistry.  “It’s all very retro, very black arts,” he told me.

In this excerpt from a user guide for Topspin (from major NMR maker Bruker), one gets a taste of the sheer number of steps and arbitrariness of how the features are named, grouped, and sequenced. If you want to adjust the number of scans, simple - just type "eda" into the command line or click on the tab called AcquPars. Got that?

The single most onerous part of getting a good reading from an NMR machine is in controlling the smaller magnetized elements that are wrapped around the sample.  These are used to fine tune the way the sample is calibrated, a process called shimming.  It’s the equivalent of focusing the lens of a camera in order to get a clear image – but an almost infinitely more abstract concept that can be the most difficult aspect of mastering these machines.    Originally, the shimming process involved a cabinet with a bunch of knobs, cryptically labeled, that the operator tweaks while staring at the feedback coming back from an oscilloscope.   Long symmetrical waveforms on the scope mean the machine is picking up the resonance clearly, sort of like tuning a guitar with a chromatic tuner.  Flanagan reminisced about the university NMRs he had encountered in his student days.  “The controls are not labeled clearly enough for a noob grad student.  It’s a little intimidating at first.   A few knobs invariably had ‘don’t touch’ signs on them.”

To shim the control magnets, an operator sits there and tweaks the knobs in trial-and-error fashion until something like a trumpet appears on the oscilloscope.  If you don’t get your trumpets, you’ll get a pretty messy looking spectrum and it will be a problem if you are trying to get your results published in a top journal.  The NMR guy in the lab might write down the best shims of the day nearby to help people out.   The knobs may have been replaced with sliders on a computer interface in a modern spectrometer, but the challenge is more or less the same.  The operator is asked to adjust controls with cryptic labels like X, Y, Z, Z1, & Z2, all of which are variables in mathematical formulas used to understand the homogeneity of magnetic fields. This requires some substantial mathematical modeling skills in order to understand, and using them is not intuitive for less experienced operators. Spectrometers with auto-lock and auto-acquire features are starting to make this process a little less onerous.  The NMR technicians, or facility managers, do a lot of day-to-day coaching to help chemists figure out what they have.  “Having good relationships with these guys is invaluable to a grad student’s success,” says Flanagan.  “The real usability issues start to come when things go wrong. The machines don’t provide a lot of feedback or guidance in troubleshooting or recovering from mistakes.”

The spectrometer blasts a sample with (typically 16) radiofrequency RF pulses which display on the oscilloscope as "resonances."

The computer attached to the instrument then transforms the resonances into a useful spectrum using a mathematical process called a Fourier Transform.

A properly calibrated, or "shimmed," machine will give off nice little trumpet shapes.

The shimming control unit on a Bruker Avance DMX-500 spectrometer, conveniently located behind all the dangling cables. Fortunately, the shims are controlled via the computer on this model.

Hard key shim adjustments (left) and the software versions (right) of the same controls. While the machines are more automated these days, understanding how to properly shim the smaller, controlling magnets around the sample is one of the hardest things to master in NMR usage.

Assuming you can get things set up and you’ve fired your RF pulses and you’ve got nice little trumpets coming up on your oscilloscope and the NMR machine hasn’t vibrated itself off of its pedestal,  the battle is still nowhere near over.   It is time to analyze the results, which is another pain point in the chemist’s workflow.  The traces of each proton’s activity after receiving the RF pulse show up in the spectrum itself as a series of peaks, whose areas are proportional to the number of protons they represent.  The bottom axis of an NMR spectrum (usually expressed as a dimensionless unit called “ppm”) is an abstraction of the difference in chemical shift position between atoms in the molecule, which differs based on how well a proton can absorb electromagnetic radiation in its position within the compound, as well as the resolution of the machine (expressed in MHz). What is interesting is that the chemical shifts of protons in organic compounds fall into predictable ranges on the ppm range based on their type, which is what makes NMR so powerful as an analytic technique.  In fact, it is possible for a chemist to interpret an NMR spectrum by a simple visual inspection once they master the basic theory. (Note: this isn’t true of IR and Mass spectroscopy)

A typical NMR spectrum. This one is from the compound strychnine. The numbered peaks from the spectrum itself (bottom part of image) can be used to draw the chemical structure (top part of image) which is used to identify the compound.

NMR spectra are an efficient way of identifying a compound and drawing a chemical structure. Since types of elements tend to appear in predictable ranges based on their chemical shift properties, it's possible to visually inspect the spectra and piece together the compound's structure. This chart shows where certain classes of elements tend to show up on an NMR spectrum, with alkanes and alcohols in the upper right region and aromatics and aldehydes in the lower left. A diagram like this helps to interpret an actual compound's spectrum like the one that appears in the preceeding figure. Source: "NMR Spectroscopy," MSU faculty website.

Upon closer inspection of a "peak," a chemist will find that they are split into particular patterns of height and spacing called "multiplets." These patterns are caused by the magnetic fields of the adjacent protons (2 adjacent protons will split a peak into three peaks with a heights ratio of 1 to 2 to 1, or a "triplet.") This phenomenon gives the chemist a great deal of information about the substance's exact chemical structure.

Interpreting multiplets of spectral peaks is an inexact science, as can be seen around 2.9 on this proton NMR spectrum. The peaks themselves are sort of a blob, but an organic chemist would infer that the short height of the lines and location of this particular cluster means that this signal is coming from a highly shielded proton (e.g. this proton is surrounded by a lot of other hydrogen protons). A chemist therefore can use several different clues to deduce the likely chemical structure of the substance.

Spectral analysis on a computer typically requires a lot of zooming. The macro display (left) needed to see the entire spectrum is too low in resolution to do proper peak analysis needed for identifying the compound. So the user mouses over an area, blows it up (left), but loses the wider context.

NMR spectra are analyzed in 1D or 2D, but typically not 3D. This creates the problem of figuring out whether certain shapes are overlapping peaks (such as those occuring around 7.5-7.8), or just strong couplings. Source: P2C2E

There is such a thing as an analytical chemist, one who works with organic chemists in the same way that a radiologist might work with surgeons – partnering with them as a diagnostic specialist and expert on the equipment.  Big pharmaceutical companies have these people around. The analytical chemist is particularly adept with the equipment , and can roll up their sleeves and work with the algorithms and pulse sequence formulas that are behind the scenes of how spectroscopy works.  In drug discovery, where the organic chemists are cranking out multitudes of new molecules on a daily basis chasing marketplace gold, there is considerable need for help with the identification process of novel compounds. But most organic chemists, especially in academia, have little choice but to use the machines themselves.  It’s the only way a chemist can makes sense of what they’ve done in the lab.  So therein we introduce the heart of the usability problem,  a mainstream technology right in the middle of the experimental workflow turns out to be difficult to both learn and master.   The machine needs to be operated correctly and the results need to be interpreted correctly.   It turns out neither of these are very straightforward.

We NMR spectroscopists have to accept that for a majority of the scientists in todays chemistry/biology research, NMR is a black box that’s neverless – and “unfortunately”- absolutely necessary. They like to use software that seems to generate listings and plots without requiring knowledge by the operator. We still try to teach our own students about the innards of NMR-experiments. But the reality is that black-box attitude and the trend for automation are increasing all the time.  – From Italian NMR Spectroscopist and Blogger, “Old Swan.”

The data itself is often transported to a chemist’s personal laptop computer for analysis, typically with a thumb drive.  When I asked several academic chemists why they don’t keep the NMR PCs on the network for easier file access, most were concerned less by security (it’s not top secret what most university labs are working on) than they were by the intense desire to keep the expensive, and fickle machines operating.   If the NMR machine was on the network, most felt, it would probably eventually stop working.  Some spyware or virus would screw things up.  And since the machines are already perceived as being finicky and expensive, this is a risk most just aren’t willing to take.   But this introduces some real pain in the workflow.  Files need to be transferred in physical space and then analyzed in separate desktop software.   In other words, these machines are a long way from point, shoot, then see instantly what you have.

Getting spectral data into a manuscript for journal publication can require many manual steps. The mnova spectral analysis software lets you drag and drop data, such as a peak list, directly into a document from the analyzer itself. The left side of the screen shows a spectrum (with the multiplet analyzer feature toggled on), the right hand side is a printable document view where a user can drag and drop a variety of information about the spectrum as well as other text and graphical objects, such as a structure drawing from Chem Draw.

One way chemists can learn to interpret NMR is by doing exercises. The "Spectral Game" is one such example. Analyze the peaks using a java spectral analyzer, then guess which compound it is from the choices below.

Other Reading on Solid State UX:

The User Experience of Organic Chemistry Part 1 – A Chemical Language

Other Reading from Around the Web:

What is NMR?

The Basics of NMR

Fire and Spice

Topspin (NMR software) User Guide from Brown University’s Department of Chemistry

Proton NMR (RSC Video on YouTube)

Brief Tour of a Bruker Avance 500MHz Spectrometer

Proofiness is the mathematical version of "truthiness." It lurks in business, politics, media, and yes - user research.

If Charles Seife’s Proofiness has a lasting contribution to offer those in the fields of user experience, design, or even business, it will be in the elegant branding of its own subjectivist epistemology.  This, in itself, is no small victory.  It involves taking a complex debate on the origin of knowledge and in a single catchy word, turning it into a meme.  Picture a future where somebody whips out a clever piece of marketing research in a design or business meeting, something with lots of correlations and a confident sounding sample.   Maybe there is a scientific looking visualization,  like a scatterplot diagram with one of those Jackson Pollack splatters of microscopic pinpoints, something that screams data was collected here.  Its presenter starts speaking with the cajoling air of someone trying to impress the truth, with a capital ‘T,’ upon their audience.  And then suddenly, the attendees stand in protest and accuse their tormenter, in unison, of proofiness.

Then try to grasp what a profound departure that is from today’s climate of info digestion, where almost nothing is spit back if it smells and tastes like it was cooked up from numbers.   Jakob Nielsen, the founding father of pragmatism in HCI research, has condemned number fetishism in our field periodically since he came to prominence in the early 1990s – most exasperatingly in this 2004 post, The Risk of Quantitative Studies.   He writes, “…most statistical research is less credible than qualitative studies.  Design research is not like medical science.” In a 2009 post, five years later, little has changed.  He writes, “People still pay far more attention to questionable quantitative studies than they do to simpler qualitative studies that have much greater validity.”

If those of us in the innovation game are struggling with number fetishism, that is saying quite a lot. User and design research is about human cognition and emotion, pretty squishy topics really.  And design, which is generally carried out in the service of business, is an applied field.  What business person has the time to wrestle with taking the uncertainty factor below a certain threshold of instinctive confidence?  (The genius of Nielsen’s discount usability method was in demonstrating that closing in on the uncertainty factor has a long, expensive tail as it approaches zero – which, of course, it never does.  n=3 represents a lot of bang for the buck, and n=2000 a lot less so.) Why, then, are we so persuaded by numeric evidence?

Unlike the output from qualitative research studies, usually called insights, numbers have a primordial pull on us.  They are more valuable entities in social exchange, a fact which our lizard brains are fully aware of.  Numbers can be transmitted with less nuance, that is to say, less noise, and are therefore choice currency.   Seife provides evidence that human beings are capable of reacting to numbers from somewhere biologically deep, processing them somewhere in the muscles and tissue and nerves but not necessarily with the intellect.  “No matter how idiotic, how unbelievable an idea is, numbers can give it credibility.” (p.8)  He cites an example of MSNBC host Deborah Norville reporting with a straight face that 58% of all exercise done in America is broadcast on television.  3.5 billion situps were done in 2003, she reported.  Two million and 300,000 of those on exercise shows.  “The numbers had short-circuited Norville’s brain,” Siefe notes, “rendering her completely incapable of critical thought.”

Our lust for numbers is like our lust for sweet and fatty foods. Two million years of evolution have taught us to crave them.  But in the modern world, where they are not only in abundance, but easily manipulated and processed with the explicit goal of satiating us, they are dangerous.   The author lays out, in clear language, the heart of his epistemic argument.  Once a number tries to describe the real world, or “acquires a unit” in Seife’s language, it loses its purity.   There is always a measurement bias of one sort or another and therefore it can no longer inhabit the “platonic realm of absolute truth.”(p.10) He writes this without apologizing for it or acknowledging its provocativeness. But this is no mainstream view. This is a rejection of the very idea of structure or universal truth, a return to the Dionysian notion that the sublime lies in closeness of experience and not in critical distance. This is the sort of postmodern thinking that was radical in intellectual circles as recently as the 1960s.  This is Piaget.  This is Derrida.  And maybe because of that inaccessibility, this sort of thinking is still far from being absorbed into the fabric of our daily thoughts and culture in business life.

Enter proofiness.  A great term, a self-descriptive masterpiece of nomenclature. Its dubious etymological structure (with its comic closeness to being a real word, a science word) carries its actual semantic argument.  Of course, the author, Charles Seife, had some help in this.  The word is a clear homage to television comic, Stephen Colbert’s, famous neologism “truthiness.”  But it is Seife who brings it to the concept of proof, not truth.  Truthiness, which applies to information that has the patina of truth, is pure social criticism.  Proofiness, which also applies to information that has the patina of truth, is an attack on status quo views of ontological reality.  Truthiness, like the Bushism ‘strategery,’is aimed at easy targets and misanthropes- liars, manipulators, unilateralists, oversimplifiers.  Proofiness is aimed at all of us with the instinct to prove, and therefore at the natural condition of mankind itself.  “Proofiness has power over us because we’re blind to this impurity (of numbers).  Numbers, charts, graphs all have an aura of perfection.”  Seife’s argument, in short, is the more important one.

The book’s tagline, “The Dark Arts of Mathematical Deception,” implies that the book is about a sort of numeric version of truthiness, some sort of number trickery done with intent to mislead.  In fact, the description of various mathematical anti-patterns ends by about page 40.  These are interesting in their own right.  Potempkin numbers are those which are built out of data that only looks like real data, data based on nonsensical or made-up measurements, such as IQ or the crowd estimates taken at free outdoor concerts.   Disestimation is the act of taking numbers more literally than the uncertainty surrounding them would seem to warrant.  (Seife’s example is the museum docent who dates the brontosaurus bones to 65,000,038 years. What’s the point of including the final 38 years when the error margin on such an estimate is likely in the tens of millions or hundreds of millions of years?)  By page 26, we’re on to fruit-packing and cherry-picking, which refers to the act of ignoring or obscuring the data that fails to support a hypothesis or argument.  The book is already drifting away from mathematics and is onto basic research ethics with 275 pages to go.

Despite the fact that there are the usual warnings here about the pitfalls of overlooking co-variants and making specious correlations that one would see in any tome on statistics, the book isn’t about math.  It’s about numbers. And to Seife, numbers represent powerful disinformation, unfairly persuasive rhetoric.   There is very little statistical depth or discussion of mathematical techniques in this book, other than to say, what is the point of doing statistical slicing and dicing on something that is fictional to begin with?  When Seife explains the concept of margin of error in political polling, which uses statistical formulas to harness some pretty nifty laws of nature in order to determine the amount of random weirdness likely to be in any sample, he doesn’t criticize the math, or even accuse the journalists who cite the polls of being innumerate.  He is more offended by those who accept the mathematical veracity in such a way that they don’t question basic systematic errors in the logic behind the poll’s construction and execution, such as who was sampled or what were they asked and in what way?(p.102)

Particularly revealing of Seife’s lack of concern with actual number manipulation is in his principle of causuistry.  “Casuistry – without the extra ‘u’ – is the art of making a misleading argument through seemingly sound principles.  Causuistry is a specialized form of casuistry where the fault in the argument comes from implying that there is a causal relationship between two things when in fact there isn’t such a linkage.”  He breaks this out as a separate idea from the statistical concept of regression analysis, used to prove causation between multiple independently correlated values. When Seife accuses someone of causuistry he isn’t concerned by the bad math on display, he’s offended by the sheer nerve that someone would try and build an argument out of it.

This gets back to my opening statement.  If the book is to make a lasting contribution outside of journalism or civics (Seife teaches journalism and the book is unfortunately heavy-handed with examples about electoral polling, elections, law and politics.  There are virtually no examples from the world of business or private life), it will require people to start taking this proofiness thesis to heart.  The concept instantly resonated with me.  Numbers, the coldest, hardest facts of all, are twisted and manipulated in order to add an air of proof to some act of data collection.   This touches right to the core of what are the least obvious but most insidious sources of user research failures: those that are related to epistemological hubris and the act of using research to validate ideas rather than to enrich them.

Where research fails most often is when the intent of it is misused.   The more you know about research methodologies, the more aware you are of their inevitable flaws.  The bad researchers are invariably the objectivists, the ones who arrogantly presume to be reporting on reality.   It is in this spirit of hubris that virtually all unforgivable research mistakes are made.     For instance, if you have just conducted a survey and concluded that 85% of potential customers liked your idea for an online information website, I will tell you that you have probably wasted your time.   Most likely your survey was flawed, your sample was flawed, or you created some other sort of systematic error and you cannot make that claim.   You have committed an act of proofiness.   And in the meantime, instead of using that opportunity to talk to your customers to learn something that might enrich your idea (which you always knew you were going to do anyways), you have squandered it trying to convince yourself and others that the idea is less risky than it probably is.   Concentrate on delivering insights, not validation.

Once, I worked as a usability researcher for two arch competitors in the same retailer category at roughly the same time.  Their contrasting styles will always stick with me.  One had a usability manager that emphasized insights.  The other had an old school HCI guy that emphasized Truth (capital T is no accident here) and experimental control.   When interviewing for the first client, I would tell the participant that I didn’t know where this interview was going but I was interested in partnering with them to try and understand what it is like to use this website, how it fits into their life, and how it could be improved.  When interviewing for the second client, I would give the participant a pre-filled out card with a task on it and watch them try and complete the task while somebody from the retailer’s staff timed them in the back room.    I barely opened my mouth for the second client, lest I contaminate the experimental conditions.    The first client had a progressive stance about the research, understood its limitations and was focused on collecting inspiration to refine their design in ways that might resonate with its customer base.  The second client wanted validation only for its existing “agreed upon” design, and set up a priori experimental conditions to get at it.   It is worth noting that for reasons unrelated to this particular usability study, the second client unexpectedly and spectacularly went out of business several years later.  The first one is thriving.   I think it goes to show that proofiness never prospers.

The Independent Film Channel's home page used to be one big "Dead Zone" before they redesigned it.

Ad placements are, by definition, dead zones.   These are specific, predictable sections of a website’s screen real-estate that are subconsciously tuned out by the user as unrelated to the page’s main content and functionality.  But designers unintentionally create dead zones of their own all the time.    A classic and well understood example of a dead zone is “right-rail” blindness.  Content and features below an ad – such as in the right-hand column of a typical two or three column layout – are tuned out as ads on the assumption that everything from an ad down is also an ad.   According to Nick Gould, CEO of the design and research firm Catalyst Group, the evidence of this phenomenon goes well beyond the anecdotal.   “There is no question that right-rail blindness is a phenomenon we’ve observed in both eye-tracking and usability testing.  This is of course mainly due to the ingrained expectation that ads live there.”   And it’s not just a matter of positioning elements in a layout.  The manner in which a page element itself  is designed can greatly amplify or lessen the dead zone effect, in the worse case scenario unintentionally deactivating important content areas and features from the user’s attention.   Often this comes from trying so hard to make an element “pop” visually, that the reverse effect occurs.   “The dead zone effect is obviously exacerbated if elements below ads are ad-like in their design,” Gould says.  ”Furthermore, promotional elements that have standard ad dimensions and contain images are frequently mistaken for ads.”

Eye-tracking data shows that the "right rail" has little visual salience with users.

There is strong evidence now that users have specific expectations about where to find the most relevant information on a web page before having seen the page.  In a recently published eye-tracking study observing  usage of a broad range of web sites for both page recognition and information foraging tasks, the right-rail blindness phenomenon was dramatically confirmed.  (see Bushcer, Cutrell, & Morris 2009) “It is striking that the entire right side above the fold is neglected for both information foraging and page recognition tasks.”    The eye tracking data reveals that users are looking at the right rail, but the “fixation impact value” (a computation that correlates fixation length with the amount of visual information processed) approaches zero.  According to Gould, “This all relates to visual salience – which is more about an unconscious assessment of what’s relevant and what’s not that occurs pre-cognition.  A variety of design, placement, and relative size/shape issues can contribute to items on the page not being deemed salient.”

Netflix consistently avoids "Dead Zones" with careful use of white space and "Functional" looking design cues.

As a predominantly qualitative researcher, these eye-tracking findings do not surprise me.  I’ve frequently observed that small differences in visual design and placement can have a dramatic effect on a user’s perception of whether they should pay attention- not to mention interact- with a page element.  Most of us who work on commercial websites have to balance a colorful, branded look while simultaneously creating a usable interface.   I’ve observed a few tricks over the years.  For instance, promotional elements that have an interactive “call to action”, such as a blank zip-code field in a registration opt-in promo, will outperform a straight image & copy version of the same message.  Netflix.com are masters of working with a heavily branded design full of bold, contrasty elements, yet systematically avoiding the creation of dead zones with careful use of white space and  “functional looking” design cues.

Original page layout (with CramCast ad) on cliffsnotes.com

I find myself conflicted (not to mention confronted by other stakeholders) with a desire to make something “pop” and the counter-intuitive, opposite notion that an integrated look is the key to discoverability.   We recently ran a multi-variate test on www.cliffsnotes.com to test the hypothesis that elements with a more “integrated” versus “ad-like” design perform better in right rail placements.  It was my first dabbling with the free tool Google Website Optimizer (GWO) – which turned out to be one of those “Where have you been all my life?” moments.   In the Cliff’s Notes test, we took an existing internal promo on the site for a podcasting series called “CramCast,” which was designed as a standard 300 by 250 ad, and redesigned it in a style more consistent with the existing functional right-rail modules on the site.   We made no changes to the copy or the photo between variants.  The positioning on the page was exactly the same.  The only difference was the look and feel.  Then we ran both versions in a split-traffic test on the live site.

The integrated look outperformed the original "Ad-Like" look by a narrow margin of 6%.

The integrated version did outperform the ad-like version, but not by a huge margin – only a 6% improvement.  (We got about 30 more clicks per 200,000 visitors with the Integrated look.)  It was not worth our time to redesign the creative material for that campaign, but certainly good counter-evidence the next time someone says something doesn’t pop enough.   What we couldn’t get at with GWO was whether the integrated look, since it replaced the traditional ad placement and cast less of a shadow on the user’s salient zone,  helped to activate the entire right rail.  This is the more interesting question.  After all, the worst dead zone problems are when multiple ad-like elements conspire to deactivate entire regions of a page – such as the old homepage design on IFC.com (pictured above.) In this design,  multiple ad-like features are interpreted sub-consciously by the user as one big ad area, and important content features (such as the programming schedule) are overlooked.  In our  next A/B test here, we’ll have to monitor the clicks outside our variants as well, perhaps with the advice of someone who is an expert with these tools – like Avinash Kaushik or Marko Hurst.    If anyone has done any research on this issue, or can point me towards soemthing else to share here – please comment on this post!

What are Anti-Patterns?

(Definition taken from Looks Good Works Well) Anti-patterns, also called pitfalls, are classes of commonly-reinvented bad solutions to problems. They are studied as a category so they can be avoided in the future, and so instances of them may be recognized when investigating non-working systems. The term originates in computer science, apparently inspired by the Gang of Four’s book Design Patterns, which displayed examples of high-quality programming methods. - Wikipedia Anti-pattern

If a subject is steadily verbalizing while performing a task (e.g. concurrent verbalization), they are assumed to be speaking from short term memory.  Dumas & Redish (1993) conveniently summarize three levels of think-aloud protocols commonly referred to by HCI researchers:  Level 1 verbalizations:  where the emphasis is on pure thoughts with no or minimal explanations; Level 2 verbalizations:  same, but when the participant is dealing with non-verbal information, like shapes, which must be internally “coded” in order to be articulated verbally, and Level 3 verbalizations:  or “thinking plus explanations.”  The latter are also referred to as retrospective reports, because the respondent is recapping and opining about what they actually did earlier.  Dumas summarizes the distinction between Level 3 and the other levels as the stage where the researchers are “no longer getting a read out of short-term memory… rather it is the interpretation of the process they are using or the reasons they have selected a strategy.”  Retrospective reports are not useless to the user researcher, in fact they are necessary to clarify a respondent’s statements and actions, but they are far less valid.

This leads us to the question of active vs. inactive moderation.  In an inactive moderation scenario, the emphasis is on experimental control and creating a unified experience for all test subjects.  This is the old-school style, where the researchers stand behind the glass and the respondent sits in the room by themselves talking out loud like a crazy person.  Participants, faced with the unnatural task of constant verbalization, are typically “coached” on how to deliver a think-aloud protocol.  This often takes the form of a warm-up exercise where the participant and experimenter practice thinking aloud with non-related stimuli, preferably a simple game such as tic-tac-toe (in order to place emphasis on cognitive strategy.)  During the actual experiment, the moderator prompts only when the participant ceases to verbalize: “Please keep talking,” they say.  And that’s all they say!

In a high moderator intervention scenario, the experimenter employs probing questions to focus the participant’s attention on particular features or to elicit and clarify subjective explanations of their behavior.  Moderators are skilled in asking neutral, non-leading questions to minimize bias.  In addition, active listening techniques are employed to emulate the clinician’s empathic stance. This implies paraphrasing of a speaker’s comments to ensure them that they have been listened to, noted, and understood as well as other verbal and non-verbal forms of caring, non-judgmental acceptance.  Mike Kuniavsky, whose work is mentioned frequently in these pages,  lays out the guidelines for “non-directed” interviewing:  questions should be concentrated on immediate experience, nonjudgmental, focused on a single topic, open-ended, and non-binary (e.g. yes-no, true-false).

In a quick look at the literature, you’ll see that Taylor and Dionne (2000) suggest that probes are best deployed for collection and verification of data in retrospective reports, and that they have a detrimental impact on validity if used during concurrent think-aloud protocols.  Preece (1994) suggests that the role of the moderator on the participant is both interruptive and imposes additional cognitive load.  Nielsen (1993) is pragmatic on the subject, suggesting the moderator intervenes as little as possible, yet directing the flow and direction of the interview to maximize the number of usability issues found.   Nielsen is the spiritual father of the commercial usability field, so it’s no surprise that most usability is performed with this degree of pragmatism.  I think it was Jared Spool who once commented that you can only watch participants avoiding clicking on the red button so many times before you are compelled to intervene and ask why.   Plus, most clients of usability research do not share the academic’s interest in validity.  They want enough validity to feel good about the process and the results, but ultimately they want their specific questions answered for a reasonable amount of time and expense.  In Nielsen et. al.’s (2002) interpretation, The human is a psychological being engaged in a psychological interaction, which cannot be reduced to that which is concurrently verbalized.

Some usability researchers, characterized by Whiteside, et. al. (1993), have also posited that observing a user’s behavior is not enough to understand what is happening in terms of higher order thinking and cognitive strategy.  If you set up a priori conditions then you are bounding yourself to learning only what falls within those conditions.  In this view, subjective experience is the most comprehensive criterion for understanding usability.  Concerns of generalizability are side-stepped: the goal is to obtain rich, experiential data.  Since specific questions are the only effective way to elicit and clarify mental models, the researcher must come to terms with at least a partially subjectivist stance. (Tamler, 2001)  This approach values reflexivity:  the subject is full participant in the study, leading the research into relevant areas for exploration as well giving the subject the ability to respond to the researchers interpretations. It is not a search for “truth,” per se, as a philosophical underpinning of this method of inquiry.  The assumption is that an expansion of perspectives leads to the exposure of more aspects of learning.

Here’s some References (I know, for a blog post, it’s over the top… but this stuff’s important!)

DUMAS, J.S. (2001) “Usability Testing Methods:  Think-Aloud Protocols,” in Design by People For People:  Essays on Usability, UPA,  pp 119-129

DUMAS, J.S. & REDISH, J.C., (1993)  A Practical Guide to Usability Testing.  Norwood, NJ, Ablex Publishing Corp.

ERICSSON, K.A. & SIMON, H.A. (1984, 1993) Protocol analysis:  Verbal reports as data (Rev. ed).  Cambridge, MA:  MIT Press

KUNIAVSKY, M. (2003)  Observing the User Experience: A Practitioner’s Guide to User Research,  San Francisco: Morgan Kaufmann Publishers, Inc.

NIELSEN, J., (1993) Usability Engineering.  Chestnut Hill, MA:  Academic Press, Inc.

NIELSEN, J., CLEMMENSEN, T., & YSSING, C., (2002)  “Getting access to what goes on in people’s heads? – Reflections on the think-aloud technique”, paper presented to NordiCHI, Arhus, Denmark, October 19-23

PREECE, J. (1994), Human-Computer Interaction, Addison-Wesley, England

TAMLER, H. (2001) “How (Much) to Intervene in a Usability Testing Session,”  in Design by People For People:  Essays on Usability, UPA,  pp 165-171

WHITESIDE, J., BENNETT., J.L., & HOLTZBLATT, K., (1988)  “Usability Engineering: Our Experience and Evolution,” in Handbook of Human Computer Interaction; edited by Helander, M.  New York, NY:  Elsevier Science Publishers